1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-11-02 06:13:16 +03:00
Commit Graph

16 Commits

Author SHA1 Message Date
mariadb-AlanMologorsky
a76e153a1d feat(upgrade): MCOL-6028 upgrade MVP with repo-managed flow, prechecks, and other enhancements
Implements the initial upgrade capability across CMAPI and the CLI, including
repository setup, package operations, environment prechecks, and coordinated
cluster steps with progress reporting.

Details:
- CMAPI upgrade manager:
  - Add `cmapi/cmapi_server/managers/upgrade/` modules:
    - `repo.py`, `packages.py`, `preinstall.py`, `upgrade.py`, `utils.py` and `__init__.py`
  - Extend endpoints and routing to expose upgrade operations and status:
    - `cmapi_server/controllers/{endpoints.py, dispatcher.py, api_clients.py}`
    - `cmapi_server/managers/{application.py, process.py}`
    - Add improved constants and helpers for upgrade flow
- Backup/restore and safety:
  - Add `cmapi_server/managers/backup_restore.py`
  - Fix pre-upgrade backup regressions (due to `mcs_backup_manager.sh 3.17 changes`)
  - Improve cluster version validation; add `ignore_missmatch` override
- CLI enhancements:
  - Progress UI and richer feedback (`mcs_cluster_tool/tools_commands.py`, `README.md`, `mcs.1`)
  - Add steps to start MDB and start MCS during/after upgrade
  - Improved error surfacing for version validation
- Platform and packaging:
  - Ubuntu and Rocky Linux support
  - RHEL/DNF dry-run support
  - Distro detection and platform-dependent logic hardened
  - Logging improvements
- Updater service:
  - Add `cmapi/updater/cmapi_updater.service.template` and `cmapi_updater.sh` to make CMAPI update itself
- Docs:
  - Update mcs cli README and mcs.1 man file
  - Add `cmapi/updater/README.md`
2025-09-30 18:48:32 +04:00
Alexander Presnyakov
08d46a9e70 Pass git revision from the build process -> cmapi - Sentry (to be able to understand which revision caused some error) 2025-09-03 20:32:03 +04:00
Alexander Presnyakov
9b98c5c20a Created a separate package for tracing-related stuff
Added mirroring of spans into Sentry
Tracer is a facade that redirects actions to tracing backends
2025-09-03 20:32:03 +04:00
Leonid Fedorov
46922f8d6a chore(build, cmapi): fallback of packaging to os-release if lsb-core is missing 2025-08-28 03:45:46 +04:00
Leonid Fedorov
6db2dc668f stubs and cmake formatting 2025-05-20 18:22:59 +04:00
mariadb-AlanMologorsky
2c0367ea2b feat(mcs): MCOL-5300 review/finetune log collection tools.
* chore(mcs, scripts): extra/columnstore_review.sh with scripts/columnstore_review.sh with 1.4.13 version
* feat(mcs): add review command to the Tools section. It's the wrapper for columnstore_review.sh
* feat(mcs): add review command implementation to tools.py file + constansts.py
* chore(mcs): add separator argument to cook_sh_arg function
* docs(mcs): updated README.md and mcs.1 man file
2025-05-02 16:12:14 +03:00
Timofey Turenko
26acfc055d Recognize lsb_release 'RedHatEnterprise' as rhel 2025-03-12 14:31:57 +04:00
mariadb-AlanMologorsky
f2e9ceacb4 MCOL-5526: CLI rename "mcs CLUSTER" commands to just mcs.
[add] now all 'cluster' subcommand commands working without using 'cluster' subcommand itself
[fix] 'cluster' command hidding its help message
[add] explicit names for commands
[add] help-all command to list all nested help messages in man page view
[add] rich colored tabled help
[add] table panels for cluster commands
[upd] typer dependency version and some extra indirect dependencies
[add] README.md for mcs cli tool with all command reference
[add] mcs.1 man page
[add] README_DEV.md for mcs cli tool
[fix] cmapi/CMakeLists.txt to install mcs.1 man page
2025-03-11 17:26:16 +04:00
Alan Mologorsky
c11cadeef5 MCOL-5618-related: Duplicate to stable 23.10 (#3223)
* feat(package_manager): improved cs_package_manager functionality for better UX

- add apt support for community and enterprise install
- add enterprise_staging install
- filter check results down to os/arch
- add --token flag for enterprise (#3002)

* Update cs package manager dev ubuntu (#3060)

* cs_package_manager.sh - fix ubuntu dev installs

* cs_package_manager.sh - remove some useless comments

* debian/ubuntu dev install to use drone repo + remove wget dependancy

* + exits if repo files missing

* auto install aws cli for dev builds

* removed commented section from cs_package_manager

* feat(tooling): add rhel support in cs_package_manager.sh (#3105)

* added dbrm_restore for LocalStorage & S3

* small comment cleanup

* resolved PR comments for cs_package_manager.sh & mcs_backup_manager.sh

* added dbrm_restore for LocalStorage & S3

* Added backup retention flag -r and incremental auto_most_recent option to mcs_backup_manager.sh

* resolved PR comments for cs_package_manager.sh & mcs_backup_manager.sh

* add comment to auto_select_most_recent_backup_for_incremental in mcs_backup_manager.sh

* feat(cmapi,cli): MCOL-5618: Add backup, restore, backup-dbrm, restore-dbrm commands for mcs cli tool. (#3130)

* MCOL-5618: Add backup, restore, backup-dbrm, restore-dbrm commands for mcs cli tool.

[add] mcs_cluster_tool/helpers.py file with cook_sh_arg function inside
[add] MCS_BACKUP_MANAGER_SH to mcs_cluster_tool/constants.py
[add] backup and restore commands to "mcs" Typer APP

* MCOL-5618: Move mcs_backup_manager.sh to cmapi/scripts.

* MCOL-5618: Install mcs_backup_manager.sh with CMAPI package.

* MCOL-5618: Testing fix + script minor fixes.

- [fix] wrong cooking of .sh invoke string
- [fix] restore_dbrm -> dbrm_restore, backup_dbrm -> dbrm_backup to use same naming in both python wrapper and mcs_backup_manager.sh
- [fix] mathod of checking cmapi online or not in mcs_backup_manager.sh

* MCOL-5618: Minor improvements to sync with PR 3144.

[add] -ns option for dbrm_restore.

* MCOL-5618: Minor fixes to sync after #3147

[add] -r option for backup command

* MCOL-5618: Minor fixes to sync after PR #1398.

[add] -ssm argument for dbrm backup
[add] -sdbk -ssm arguments for dbrm restore
[add] silenece annoying pylint warning unused argument for bachup_commands.py and restore_commnads.py

* update to the latest mcs_backup_manager + cs_package_manager

* Intermediate commit to move changes from old filepath to the new one.

* remove old file

* MCOL-5618: Fix packaging.

[mv] extra/cs_package_manager.sh -> cmapi/scripts/cs_package_manager.sh to pack and deliver the script with cmapi package
[add] cs_package_manager.sh to cmapi package
[fix] download paths for mcs_backup_manager.sh and cs_package_manager.sh

* MCOL-5618: Fix after testing

[add] -P to restore help in mcs_backup_restore.sh
[fix] -f argument default value in both backup and restore wrapper commands
[fix] -i argument behaviour and type + default value
[fix] -P argument position and help for both backup and restore wrapper commands
[fix] disable some pylint warning for both backup and restore wrapper files

---------

Co-authored-by: Allen Herrera <82840027+mariadb-AllenHerrera@users.noreply.github.com>
Co-authored-by: Allen Herrera <allen.herrera@mariadb.com>
Co-authored-by: Allen Herrera <allen@Allens-Black-MacBook-Pro.local>
Co-authored-by: Leonid Fedorov <79837786+mariadb-LeonidFedorov@users.noreply.github.com>
2024-06-27 17:23:13 +04:00
Timofey Turenko
d3896efb0f Add dependency on libxcrypt for Rocky/RHEL9 2024-05-28 15:47:48 +04:00
Alan Mologorsky
14e936976d MCOL-5610: Aliases on internally dependent tools. (#3026)
[add] mcs_aws and mcs_gsutil template files
[fix] cmapi CMakeLists to add mcs_aws and mcs_gsutil tools to usr/bin folder
2023-12-26 17:01:30 +03:00
mariadb-AlanMologorsky
2ccfc149c6 MCOL-5595: Wrong metadata in CMAPI packages.
[fix] cmapi packages metadata version format for debian systems and version + release in RHEL systems
2023-10-27 05:34:54 +03:00
mariadb-AlanMologorsky
d2aaf857f7 MCOL-5548: Fix cmapi package filenaming.
[add] dynamic year to cmapi CmakeLists
[add] package description to cmapi CmakeLists
[add] use MDB ES repo cmake dir
[fix] cmapi package filenaming
[fix] .drone.jsonnet to do cmapi build step depending on clonning mdb step
[add] cmapi_misc.cmake file
[add] lsb-release package to build deps in drone.jsonnet file
[fix] wrong debian and rpm requires curl placement
[fix] SET(CPACK_PACKAGE_FILE_NAME
[add] redhat-lsb-core package for RHEL systems
[add] SERVER_DIR argument for cmapi CmakeLists.txt
[add] get_linux_lsb_release_information() for both deb and rpm builds
2023-09-08 16:33:12 +03:00
Alan Mologorsky
b2a573cf82 MCOL-5521: Use Columnstore VERSION file for CMAPI versioning. (#2885)
[fix] columnstore_version.cmake setting extra variables for CMAPI versioning
[fix] CMAPI CmakeLists.txt using columnstore_version.cmake file
[fix] CMAPI VERSION file to template file due to using it inside CMAPI applicatio
n
2023-06-29 15:47:55 +03:00
mariadb-RomanNavrotskiy
1eca477485 MCOL-5496: Fix CI adding CMAPI steps.
[fix] deb packages deps commands
[add] several additional local variables
[fix] packages url
[fix] smoke step
[fix] mtr step
[fix] regression step
[add] cmapipython, cmapibuild, cmapitest and cmapilog steps
[fix] dockerfile step
[fix] build step to include cmapi package on repodata creating
[fix] multi_node_mtr step
[fix] pkg step
[add] cmapi steps to pipelines
[fix] cmapi/CMakeLists.txt to prevent cmake byte-compile .py files for rpm packages
[add] setup-repo.sh file
[fix] now use packages from the repos in tests steps
2023-06-07 10:00:16 +03:00
mariadb-AlanMologorsky
a079a2c944 MCOL-5496: Merge CMAPI code to engine repo.
[add] cmapi code to engine
2023-06-07 10:00:16 +03:00