1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-10-31 18:30:33 +03:00
Commit Graph

6 Commits

Author SHA1 Message Date
mariadb-AlanMologorsky
a76e153a1d feat(upgrade): MCOL-6028 upgrade MVP with repo-managed flow, prechecks, and other enhancements
Implements the initial upgrade capability across CMAPI and the CLI, including
repository setup, package operations, environment prechecks, and coordinated
cluster steps with progress reporting.

Details:
- CMAPI upgrade manager:
  - Add `cmapi/cmapi_server/managers/upgrade/` modules:
    - `repo.py`, `packages.py`, `preinstall.py`, `upgrade.py`, `utils.py` and `__init__.py`
  - Extend endpoints and routing to expose upgrade operations and status:
    - `cmapi_server/controllers/{endpoints.py, dispatcher.py, api_clients.py}`
    - `cmapi_server/managers/{application.py, process.py}`
    - Add improved constants and helpers for upgrade flow
- Backup/restore and safety:
  - Add `cmapi_server/managers/backup_restore.py`
  - Fix pre-upgrade backup regressions (due to `mcs_backup_manager.sh 3.17 changes`)
  - Improve cluster version validation; add `ignore_missmatch` override
- CLI enhancements:
  - Progress UI and richer feedback (`mcs_cluster_tool/tools_commands.py`, `README.md`, `mcs.1`)
  - Add steps to start MDB and start MCS during/after upgrade
  - Improved error surfacing for version validation
- Platform and packaging:
  - Ubuntu and Rocky Linux support
  - RHEL/DNF dry-run support
  - Distro detection and platform-dependent logic hardened
  - Logging improvements
- Updater service:
  - Add `cmapi/updater/cmapi_updater.service.template` and `cmapi_updater.sh` to make CMAPI update itself
- Docs:
  - Update mcs cli README and mcs.1 man file
  - Add `cmapi/updater/README.md`
2025-09-30 18:48:32 +04:00
mariadb-AlanMologorsky
927eb4b2bd feat(cmapi, mcs): MCOL-5941 Improve mcs cli backup\restore wrapper.
* fix(mcs, wrapper): default is None for --backup-location, --backup-destination, --storage, --parallel, --highavilability, --skip-save-brm, --skip-polls, --skip-locks, --skip-mariadb-backup, --skip-bucket-data, --name-backup, --quiet, --no-verify-ssl, --poll-interval, --poll-max-wait, --retention-days, -scp, -bb, -url, -f, -m, -aro, -li and most of arguments in backup_commands.py and restore_commands.py
* fix(mcs, helpers): cook_sh_arg parser function now detects None as a value
* fix(mcs, wrapper): list -> li in typer command argument name for both backup_commands.py and restore_commands.py
* docs(mcs, wrapper): --parralel arg help message was edited to simple and readable one
* fix(mcs, wrapper): removing -no- prefixed flag variants for all bool arguments in backup_commands.py and restore_commands.py
2025-04-18 18:51:59 +01:00
mariadb-AlanMologorsky
215e4eea4d feat(cmapi): MCOL-5019: distributing cskeys secrets file, move cskeys and cspasswd functions to mcs cli.
[add] distribute .secrets file to all nodes while adding a new node
[add] encrypt_password, generate_secrets_data, save_secrets to CEJPasswordHandler
[add] tools section to mcs cli tool
[add] mcs_cluster_tool/tools_commands.py file with cskeys and cspasswd commands
[add] cskeys and cspasswd commands to tools section of mcs cli
[mv] backup/restore commands to tools section mcs cli
[fix] minor imports ordering
[fix] constants
2025-04-08 14:56:06 +03:00
mariadb-AlanMologorsky
91e82f4d15 fix(mcs): MCOL-5618: change list option to --list and refactor.
fix(mcs):  list option to -li/--list for mcs backup, restore and dbrm_restore commands
fix(mcs): add missed option "aro"
fix(mcs): resort options for backup and dbrm_backup to keep same ordering as in original scrypt
fix(mcs docs): updated README.md + mcs.1
2025-03-28 13:02:19 +03:00
mariadb-AlanMologorsky
271e3d5956 MCOL-5618: Sync with PR3410.
[add] list option but in argument notation "list" for backup, restore, dbrm_restore
[fix] help messages for --config-file
[fix] -p/--path > -bl/--backup-location option for dbrm_backup and dbrm_restore
[fix] -d/--directory > '-l', '--load' option for dbrm_restore
2025-03-28 13:02:19 +03:00
Alan Mologorsky
c11cadeef5 MCOL-5618-related: Duplicate to stable 23.10 (#3223)
* feat(package_manager): improved cs_package_manager functionality for better UX

- add apt support for community and enterprise install
- add enterprise_staging install
- filter check results down to os/arch
- add --token flag for enterprise (#3002)

* Update cs package manager dev ubuntu (#3060)

* cs_package_manager.sh - fix ubuntu dev installs

* cs_package_manager.sh - remove some useless comments

* debian/ubuntu dev install to use drone repo + remove wget dependancy

* + exits if repo files missing

* auto install aws cli for dev builds

* removed commented section from cs_package_manager

* feat(tooling): add rhel support in cs_package_manager.sh (#3105)

* added dbrm_restore for LocalStorage & S3

* small comment cleanup

* resolved PR comments for cs_package_manager.sh & mcs_backup_manager.sh

* added dbrm_restore for LocalStorage & S3

* Added backup retention flag -r and incremental auto_most_recent option to mcs_backup_manager.sh

* resolved PR comments for cs_package_manager.sh & mcs_backup_manager.sh

* add comment to auto_select_most_recent_backup_for_incremental in mcs_backup_manager.sh

* feat(cmapi,cli): MCOL-5618: Add backup, restore, backup-dbrm, restore-dbrm commands for mcs cli tool. (#3130)

* MCOL-5618: Add backup, restore, backup-dbrm, restore-dbrm commands for mcs cli tool.

[add] mcs_cluster_tool/helpers.py file with cook_sh_arg function inside
[add] MCS_BACKUP_MANAGER_SH to mcs_cluster_tool/constants.py
[add] backup and restore commands to "mcs" Typer APP

* MCOL-5618: Move mcs_backup_manager.sh to cmapi/scripts.

* MCOL-5618: Install mcs_backup_manager.sh with CMAPI package.

* MCOL-5618: Testing fix + script minor fixes.

- [fix] wrong cooking of .sh invoke string
- [fix] restore_dbrm -> dbrm_restore, backup_dbrm -> dbrm_backup to use same naming in both python wrapper and mcs_backup_manager.sh
- [fix] mathod of checking cmapi online or not in mcs_backup_manager.sh

* MCOL-5618: Minor improvements to sync with PR 3144.

[add] -ns option for dbrm_restore.

* MCOL-5618: Minor fixes to sync after #3147

[add] -r option for backup command

* MCOL-5618: Minor fixes to sync after PR #1398.

[add] -ssm argument for dbrm backup
[add] -sdbk -ssm arguments for dbrm restore
[add] silenece annoying pylint warning unused argument for bachup_commands.py and restore_commnads.py

* update to the latest mcs_backup_manager + cs_package_manager

* Intermediate commit to move changes from old filepath to the new one.

* remove old file

* MCOL-5618: Fix packaging.

[mv] extra/cs_package_manager.sh -> cmapi/scripts/cs_package_manager.sh to pack and deliver the script with cmapi package
[add] cs_package_manager.sh to cmapi package
[fix] download paths for mcs_backup_manager.sh and cs_package_manager.sh

* MCOL-5618: Fix after testing

[add] -P to restore help in mcs_backup_restore.sh
[fix] -f argument default value in both backup and restore wrapper commands
[fix] -i argument behaviour and type + default value
[fix] -P argument position and help for both backup and restore wrapper commands
[fix] disable some pylint warning for both backup and restore wrapper files

---------

Co-authored-by: Allen Herrera <82840027+mariadb-AllenHerrera@users.noreply.github.com>
Co-authored-by: Allen Herrera <allen.herrera@mariadb.com>
Co-authored-by: Allen Herrera <allen@Allens-Black-MacBook-Pro.local>
Co-authored-by: Leonid Fedorov <79837786+mariadb-LeonidFedorov@users.noreply.github.com>
2024-06-27 17:23:13 +04:00