1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-07-30 19:23:07 +03:00
Commit Graph

46 Commits

Author SHA1 Message Date
449029a827 Deep build refactoring phase 2 (#3564)
* configcpp refactored

* chore(build): massive removals, auto add files to debian install file

* chore(build): configure before autobake

* chore(build): use custom cmake commands for components, mariadb-plugin-columnstore.install generated

* chore(build): install deps as separate step for build-packages

* more deps

* chore(codemanagement, build): build refactoring stage2

* chore(safety): Locked Map for MessageqCpp with a simpler way

 Please enter the commit message for your changes. Lines starting

* chore(codemanagement, ci): better coredumps handling, deps fixed

* Delete build/bootstrap_mcs.py

* Update charset.cpp (add license)
2025-07-17 16:14:10 +04:00
05bbfcae57 One more try to fix autobake build 2025-07-16 06:59:55 +04:00
dcdfb9d018 add verbose optio in testS3Connection 2025-06-18 18:29:08 +04:00
aa7e0fb9b4 Deep build refactoring phase 1 (#3562)
* configcpp refactored
* logging and datatypes refactored

* more dataconvert
* chore(build): massive removals, auto add files to debian install file
* chore(codemanagement): nodeps headers, potentioal library
* chore(build): configure before autobake
* chore(build): use custom cmake commands for components, mariadb-plugin-columnstore.install generated
* chore(build): install deps as separate step for build-packages
* more deps
* check  debian/mariadb-plugin-columnstore.install automatically
* chore(build): add option for multibracnh compilation
* Fix warning
2025-05-30 14:05:21 +04:00
a1019b7c0e chore(build): refactor main CMakeLists.txt (#3543)
* chore(build): refactor main CMakeLists.txt

* chore(build): fix boost version for packages, set clang-20 only for amd and arm

* chore(build): boost 4 sm

* chore(build): boost dep for rowgroup

* chore(build): toolset for boost

* chore(build): suppress clang warnings for boost

* chore(ci, build): use ASAN for unittest on ubuntu 24.04 only, added custom cmake flag option for bootstrap, custom params for new and existing pipelines

* chore(build): sort bootstrap flags

* chore(CI): remove publish pkg step, adding clickable link instead to publish steps, fix customenv
2025-05-20 05:00:48 +04:00
60dc7550f1 fix(group by, having): MCOL-5776: GROUP BY/HAVING closer to server's (#3371)
This patch introduces an internal aggregate operator SELECT_SOME that
is automatically added to columns that are not in GROUP BY. It
"computes" some plausible value of the column (actually, last one
passed).

Along the way it fixes incorrect handling of HAVING being transferred
into WHERE, window function handling and a bit of other inconsistencies.
2024-12-20 19:11:47 +00:00
4aa281645e feat(SM): MCOL-5785 S3Storage improvements
Update libmarias3
fix build with the recent libmarias3

feat(SM): MCOL-5785 Add timeout options for S3Storage

    In some unfortunate situations StorageManager may get stuck on
    network operations. This commit adds the ability to set network
    timeouts which will help to ensure that the system is more
    responsive.

feat(SM): MCOL-5785 Add smps & smkill tools

    * `smps` shows all active S3 network operations
    * `smkill` terminates S3 network operations

    NB! At the moment smkill is able to terminate operations
    that are stuck on retries, but not hang inside the libcurl
    call. In other words if you want to terminate all operations
    you should configure `connect_timeout` & `timeout`

Install smkill & smps

Add install for new binaries
2024-08-21 20:45:38 +04:00
9935431b31 Remove blank lines 2022-11-17 12:02:31 -06:00
37fd915a08 Serg`s patch for develop-6 revised for develop https://github.com/mariadb-corporation/mariadb-columnstore-engine/pull/2614 2022-11-09 22:41:38 +00:00
58f96ac9ae Sanitizers are not ready for all Distros 2022-09-19 19:26:17 +03:00
51a04ba960 Fixing some test for StorageManagerm commenting all broken rest 2022-09-16 18:51:10 +00:00
7e17d88254 cmake version set to 3.10 2022-09-01 16:03:29 +03:00
726cc3684b Proper external_boost for storage manager, cmake version REQUIRES 2022-08-30 22:41:44 +00:00
a7a9ccf889 Serg dev (#2504)
* more build dependencies

* fix for cmake < 3.11

It cannot do ADD_LIBRARY(... ALIAS ...) on IMPORTED targets

* another fix for cmake 3.10.2

It doesn't know about CMAKE_CXX_STANDARD=20,
let's add the correct flag manually

* gcc 8 on aarch64

utils/common/simd_arm.h:241:16: error: need ‘typename’ before ‘simd::TypeToVecWrapperType<T>::WrapperType’ because ‘simd::TypeToVecWrapperType<T>’ is a dependent scope
2022-08-15 13:35:30 +03:00
c25ae4f378 Use external boost 1.78 2022-05-02 18:23:37 +00:00
02dd4f6ab6 MCOL-3889: use ENGINE_DATADIR cmake value for storagemanager.cnf paths 2020-12-03 16:32:00 -06:00
b8caaaa760 Rebased MCOL-3836 on top of e7bc7e93. 2020-05-12 15:56:10 +00:00
98abf95eae MCOL-3991 MCS is now single package and properly uninstalls 2020-05-12 13:36:24 +00:00
6987ac40e1 MCOL-3982 Avoid using autotools for libmarias3 and remove libxml2 from libmarias3 dependencies. 2020-05-06 12:45:34 +00:00
75d1dbc592 MCOL-3494: initial commit of test program to verify storagemanager.cnf during postConfigure. 2020-04-30 16:59:56 -05:00
90d4b03fdb MCOL-3743: cmdline params to postConfig to configure SM
Made some changes.  It's working the way it should.
2020-03-17 13:55:08 -04:00
7489d0bfd0 MCOL-3625 Rename packages
Rename packages to MariaDB-columnstore-engine, MariaDB-columnstore-libs
and MariaDB-columnstore-platform.

Also add the "columnstore-" prefix the the components so that MariaDB's
packaging system understands then and add a line to include them in
MariaDB's packaging.

In addition
* Fix S3 building for dist source build
* Fix Debian 10 dependency issue
* Fix git handling for dist builds
* Add support for MariaDB's RPM building
* Use MariaDB's PCRE and readline
* Removes a few dead files
* Fix Boost noncopyable includes
2019-12-04 11:04:39 +00:00
a8cd34f86d Add support for building from server 2019-12-03 21:21:28 +00:00
efe829784d MCOL-3624 Move jemalloc to an LD_PRELOAD
Linking will become an issue for the unified build and it is an issue
for jemalloc 5.x. Instead we will LD_PRELOAD on the forked ColumnStore
specific processes.
2019-12-03 21:21:28 +00:00
5bbd21b8e0 MCOL-3577: Add functionality to sync S3 storage for suspendDatabaseWrites. 2019-11-21 15:19:03 -06:00
b9708530e2 Fix library and storage manager install paths 2019-11-13 23:34:20 +00:00
0a175f0052 Merge pull request #871 from mariadb-corporation/cpimport-s3
MCOL-3514 Add support for S3 to cpimport
2019-09-25 12:13:47 -05:00
59d1caae02 Fix CMake dependencies
Some odd quirks with CMake mean that marias3 has to be in the top level
CMake. Also made submodule checkout happen at cmake time.
2019-09-25 11:30:32 +01:00
70f4ffe8c2 MCOL-3436: have sighandler function call crashtrace handler. 2019-09-24 11:16:02 -05:00
fe8228e50a Fix build order so ms3 is checked out 2019-09-24 16:47:00 +01:00
8633859dd4 MCOL-3514 Add support for S3 to cpimport
cpimport now has the ability to use libmarias3 to read an object from an
S3 bucket instead of a file on local disk.

This also moves libmarias3 to utils/libmarias3.
2019-09-24 10:31:22 +01:00
97bda78c3b Move config files
This patch:

* Moves config files from /usr/local/mariadb/columnstore/etc to
ENGINE_SYSCONFDIR/columnstore (ENGINE_SYSCONFDIR is /etc by default)
* Sets a define called MCSSYSCONFDIR whic contains the
ENGINE_SYSCONFDIR compile time setting
* Modifies scripts and code to use the new paths
* Removes a whole bunch of files we don't use
2019-09-09 14:13:56 +01:00
89eff8bc45 MCOL-3455: link to jemalloc. Fix packaging issues for RPMs and other compile issues encounter. 2019-09-03 14:16:34 -05:00
9e1282e437 put unit_test exe in correct place 2019-08-29 17:35:58 -05:00
fc8f68e391 A couple random things
- Re-added the StorageManager/Enabled key in the default config file.
  Thought I had switched all that stuff to only look at DBRootStorageType
  but I was wrong.
- Made the unit_tests bin for SM stay in the SM dir
- Collapsed one level of build dirs for SM.  For some reason it was
  putting test data in storage-manager/storage-manager/testData
- Limited a single read() call to 100MB as a paranoid safety measure.
  Normal usage won't read that much at once anyway.
2019-08-28 12:56:52 -05:00
6454b53572 Fix SM_TRACE flag. 2019-08-21 15:59:29 -05:00
fa0da4b9bb Merge pull request #834 from pleblanc1976/develop
Bring in the fixes to make S3 lib work with google cloud storage.
2019-08-21 15:50:27 -05:00
d36d8b6831 Bring in the fixes to make S3 lib work with google cloud storage. 2019-08-20 15:52:40 -05:00
94e77a33c5 MCOL-3441 Fix in-tree builds
A copy command was failing for in-tree builds.
2019-08-20 11:15:54 +01:00
7ddb755ee1 Merge pull request #829 from pleblanc1976/toolkit-fix
SM toolkit fix (mcol-3402)
2019-08-19 10:16:49 -05:00
95953624de Checkpointing some changes. 2019-08-16 10:16:52 -05:00
78eed09942 Remove mhash dep from cmake 2019-08-15 12:46:34 -05:00
b40b55d145 Made the SM cmakelist init/update submodules to be nice to buildbot. 2019-08-14 16:56:37 -05:00
b497d99ab3 Some SM cmakelists fixes. Looks like it all builds and installs to the right place. 2019-08-14 14:30:24 -05:00
1b1e956c63 Checkpointing changes for merging storagemanager -> CS. 2019-08-14 13:05:50 -05:00
fba0c1728c Relocating everything in the repo s.t. it can be merged into
the columnstore repo.
2019-08-14 11:07:24 -05:00