1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-07-29 08:21:15 +03:00
Commit Graph

481 Commits

Author SHA1 Message Date
37fd915a08 Serg`s patch for develop-6 revised for develop https://github.com/mariadb-corporation/mariadb-columnstore-engine/pull/2614 2022-11-09 22:41:38 +00:00
65252df4f6 C++20 fixes 2022-03-28 12:32:29 +00:00
3919c541ac New warnfixes (#2254)
* Fix clang warnings

* Remove vim tab guides

* initialize variables

* 'strncpy' output truncated before terminating nul copying as many bytes from a string as its length

* Fix ISO C++17 does not allow 'register' storage class specifier for outdated bison

* chars are unsigned on ARM, having  if (ival < 0) always false

* chars are unsigned by default on ARM and comparison with -1 if always true
2022-02-17 13:08:58 +03:00
04752ec546 clang format apply 2022-01-21 16:43:49 +00:00
12e3368ebf MCOL-4597: logic from postConfigure migrated to post-install to upgrade existing columnstore.cnf 2021-05-26 10:41:39 -05:00
846f7fb29b MCOL-4193: Delete unused OAM and applications, ProcMon, ProcMgr, and no longer build all tools for packages 2021-02-08 17:51:09 -06:00
90a167178a MCOL-3915 Update columnstore install and uninstall procedure 2020-05-22 23:13:09 +00:00
9e9f04eb79 Some minor cleanup in response to PR comment 2020-05-15 19:27:25 +00:00
32d3e3f6a3 MCOL-3991 MCS doesn't build mcsAdmin anymore.
Removed MDB network port check from postConfigure.
2020-05-14 14:07:13 +00:00
98abf95eae MCOL-3991 MCS is now single package and properly uninstalls 2020-05-12 13:36:24 +00:00
ea39ac7f43 MCOL-3915. Cleanup postConfigure. PostConfigure and Installer no longer editing config file 2020-05-12 13:35:25 +00:00
97b8e41ce3 MCOL-3915 Remove some configuration from postConfigure. Default Columnstore.xml to a standard single server config. 2020-05-12 13:33:37 +00:00
c60ec09fea MCOL-3915 Columnstore-post-install/postConfigure processed and columnstore started during installation 2020-05-12 13:33:37 +00:00
7b66bc7ead MCOL-3915 Run post-install, postConfigure, and start systemd units during package installation 2020-05-12 13:32:46 +00:00
75d1dbc592 MCOL-3494: initial commit of test program to verify storagemanager.cnf during postConfigure. 2020-04-30 16:59:56 -05:00
8edaffc433 Change includes for history.h 2020-04-21 11:46:01 -04:00
00828bf2df Merge branch 'develop' into mcol-3743 2020-04-13 12:49:28 -05:00
cfc2ae4762 Merge pull request #1107 from pleblanc1976/mcol-3887
MCOL-3887 - postconfig problem processing an upgrade
2020-04-06 15:59:22 -05:00
ab518fae8c MCOL-3743
Reformatted the usage stmt and help msg to fit on standard 80-col
terminals.
2020-03-17 14:36:23 -04:00
90d4b03fdb MCOL-3743: cmdline params to postConfig to configure SM
Made some changes.  It's working the way it should.
2020-03-17 13:55:08 -04:00
39c26f8c68 MCOL-3743: Add cmd-line config for S3 parameters to postConfig
Checkpointing the first cut.  It builds.  Not tested yet.
2020-03-16 17:53:19 -04:00
4e7e70b0f1 MCOL-3887 - postconfig problem processing an upgrade
mycnfUpgrade has been looking for our mysql config mods
in the /etc/my.cnf/ directory, which is wrong obviously.  Looks like
an oversight; all of the code around it is looking in the right place.
Changed it to look in the right place.
2020-03-16 13:42:44 -04:00
c073ae0ef9 MCOL-2022. Removed some test code. 2020-03-11 18:29:38 -04:00
efb97b1415 MCOL-2022: aws specific options can't use nvme dev names
Checkpointing a minimal attempt to disable the aws-specific
functionality for new installations.
2020-03-11 11:14:44 -04:00
f52ee922aa MCOL-3870 typo fix 2020-03-04 16:46:32 -06:00
ea478688d8 MCOL-3870 PostConfigure exit with mysqld running
"MariaDB Server is currently running on PID 1. Cannot run postConfigure whilst this is running"
Even though mysqld is not running
2020-03-04 15:20:36 -06:00
b72a9784ba Merge pull request #1014 from tntnatbry/MCOL-3675
MCOL-3675 and MCOL-3676 Detect in postConfigure if columnstore-post-install is run or not
2020-02-28 13:55:11 -05:00
ea3ced83af Merge pull request #1011 from pleblanc1976/minor-fixes
Minor fixes
2020-02-28 13:55:05 -05:00
5ff131e726 Merge pull request #1004 from pleblanc1976/runtime-disable-oam
Runtime disable oam
2020-02-28 13:54:39 -05:00
81017e2ec0 Merge pull request #993 from toddstoffel/spelling
Spelling
2020-01-03 09:42:11 -06:00
de4ab3e294 MCOL-3702 Fix replication config file changes
log_bin is missing from columnstore.cnf. This causes a search for the
master log file to fail and replication doesn't apply.

This patch adds a commented out log_bin which is enabled as needed.

It also fixes an error message.
2020-01-03 10:08:52 +00:00
1acec861e1 Fix Typos 2019-12-23 10:27:14 -08:00
7489d0bfd0 MCOL-3625 Rename packages
Rename packages to MariaDB-columnstore-engine, MariaDB-columnstore-libs
and MariaDB-columnstore-platform.

Also add the "columnstore-" prefix the the components so that MariaDB's
packaging system understands then and add a line to include them in
MariaDB's packaging.

In addition
* Fix S3 building for dist source build
* Fix Debian 10 dependency issue
* Fix git handling for dist builds
* Add support for MariaDB's RPM building
* Use MariaDB's PCRE and readline
* Removes a few dead files
* Fix Boost noncopyable includes
2019-12-04 11:04:39 +00:00
a8cd34f86d Add support for building from server 2019-12-03 21:21:28 +00:00
b3c22c8f9a MCOL-3626 Remove mcsmysql alias 2019-12-03 21:21:28 +00:00
95d0bceae7 MCOL-3559 Stop postConfigure if mysqld is running
It is possible that a user will already have a MariaDB Server running
before executing postConfigure. Exit early in this scenario.
2019-11-18 13:54:06 +00:00
a396b8cb80 Merge pull request #934 from LinuxJedi/MCOL-3613
MCOL-3613 Fix params in columnstore_installer
2019-11-14 19:32:36 +01:00
494ef36bca MCOL-3613 Fix params in columnstore_installer
Typo in arg count meant that -numBlocksPct and -totalUmMemory were not
processed correctly. Fixed the arg handling.
2019-11-14 15:02:56 +00:00
25d22381c6 MCOL-3608 Add module installer file copy
Adds back some stuff that was remove distributed that didn't work
properly for non-distributed. Also use /var/lib/columnstore/local
for module file as OAM intended.
2019-11-14 14:40:28 +00:00
8ab9ebb0f4 MCOL-3606 Make ColumnStore use generic paths
ColumnStore now uses standard bin/lib paths for pretty much everything.
Data path is now hard-coded to /var/lib/columnstore.

This patch also:

* Removes v1 decompression
* Removes a bunch of unneeded files
* Removes COLUMNSTORE_INSTALL_DIR / $INSTALLDIR
* Makes my.cnf.d work for all platforms (MCOL-3558)
* Changes configcpp to use recursive mutex (fixes possible config write deadlock)
* Fixes MCOL-3599 Fix regr functions, The library was installed in the wrong location
* Fixes a bunch of Ubuntu packaging issues
* Changes the binary names of several of the executables so as not to
clash with potential executables from other packages
2019-11-09 16:53:05 +00:00
1eaa83d852 Squash merge of the multithreaded PM join code.
Squashed commit of the following:

commit fe4cc375faf1588e30471062f78403e81229cd02
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Nov 1 13:38:11 2019 -0400

    Added some code comments to the new join code.

commit a7a82d093be4db3dfb44d33e4f514fd104b25f71
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Nov 1 13:17:47 2019 -0400

    Fixed an error down a path I think is unused.

commit 4e6c7c266a9aefd54c384ae2b466645770c81a5d
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Nov 1 13:12:12 2019 -0400

    std::atomic doesn't exist in C7, -> boost::atomic.

commit ed0996c3f4548fff0e19d43852d429ada1a72510
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Oct 16 12:47:32 2019 -0500

    Addition to the previous fix (join dependency projection).

commit 97bb806be9211e4688893460437f539c46f3796f
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Oct 15 15:22:09 2019 -0500

    Found and fixed a bad mem access, which may have been there for 8 years.

commit d8b0432d2abd70f28de5276daad758c494e4b04b
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Oct 15 14:04:48 2019 -0500

    Minor optimization in some code I happened to look at.

commit b6ec8204bf71670c7a8882464289e700aa5f7e33
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Oct 15 14:04:11 2019 -0500

    Fixed a compiler warning.

commit 0bf3e5218f71d92460ddc88090e3af77ecf28c35
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Oct 15 10:11:09 2019 -0500

    Undid part of the previous commit.

commit 5dfa1d23980e245c77c1644015b553aa4bcdf908
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Mon Oct 14 18:00:21 2019 -0500

    Proofread the diff vs base, added some comments, removed some debugging stuff.

commit 411fd955ebbae97ddab210a7b17fe5708538001d
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Oct 11 13:55:39 2019 -0500

    If a dev build (SKIP_OAM_INIT), made postConfigure exit before trying
    to start the system, because that won't work.

commit 634b1b8a7340b55fcaee045fd6d00b3e3a9269fa
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Mon Sep 30 14:55:45 2019 -0500

    Reduced crit section of BPP::addToJoiner a little.

commit 31f30c64dd95942f2c7a247cc81feaa5933c1a07
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 18 11:09:27 2019 -0500

    Checkpointing.  make the add joiner stuff free tmp mem quickly.

commit 9b7e788690546af7ddc4c921a0ab441ee9a8df02
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 18 10:38:57 2019 -0500

    Checkpoint.  Removed tmp hardcoding of bucket count.

commit fda4d8b7fb30d0431dc15e473042abb3d8121b19
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 18 10:20:09 2019 -0500

    Checkpoint.  Adjusted unproductive loop wait time.

commit 7b9a67df7d192f240e9e558e6e66c7aa9f1e8687
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 18 10:10:43 2019 -0500

    Checkpointing add'l optimizations.

    If we promote bpp::processorThreads / bucket count to a power of 2, we can
    use a bitmask instead of a mod operation to decide a bucket.

    Also, boosted utilization by not waiting for a bucket lock to become free.
    There are likely more gains to be had there with a smarter strategy.
    Maybe have each thread generate a random bucket access pattern to reduce
    chance of collision.  TBD.

commit abe7dab8661b5120f6ee268abc005dd66cd643e2
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Sep 17 16:15:51 2019 -0500

    Multithreaded PM hash table construction likely works here.

    A couple more fixes.
     - missed a mod after a hash in one place.
     - Made the PoolAllocator thread safe (small degree of performance hit
       there in threaded env).  May need to circle back to the table
       construction code to eliminate contention for the allocators instead.

commit ab308762fbd873dbf246a6d1574223087cd0d5f6
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Sep 17 12:14:14 2019 -0500

    Checkpointing.  Did some initial testing, fixed a couple things.

    Not done testing yet.

commit 3b161d74fa859edb8b5ba84bb905e586ac0586e6
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Sep 17 11:24:55 2019 -0500

    Checkpointing.  First cut of multithreaded PM join table building.

    Builds but is untested.

commit cb7e6e1c2761fc6c33b3b1c6b6cda488d7792bca
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Mon Sep 16 13:03:50 2019 -0500

    Increase the STLPoolAllocator window size to reduce destruction time.

commit b0ddaaae71a0a4959ad15c87579d85ed88e17e1f
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Sep 13 11:52:51 2019 -0500

    Fixed a bug preventing parallel table loading.  works now.

commit b87039604e312c1ddb88cdb226228b1c3addf018
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Thu Sep 12 22:04:15 2019 -0500

    Checkpointing some experimental changes.

     - Made the allocator type used by PM joins the STLPoolAllocator
     - Changed the default chunk size used by STLPoolAlloc based on a few test
        runs
     - Made BPP-JL interleave the PM join data by join # to take advantage
        of new locking env on PM.
     - While I was at it, fixed MCOL-1758.

commit fd4b09cc383d2b96959a8e5ca490c940bacb3d37
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Thu Sep 12 16:03:30 2019 -0500

    Speculative change.  Row estimator was stopping at 20 extents.

    Removed that limitation.

commit 7dcdd5b5455f9ac06121dd3cf1ba722150f3ee56
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Thu Sep 5 09:10:28 2019 -0500

    Inlined some hot simpleallocator fcns.

commit 6d84daceecc5499f6286cf3468c118b8b1d28d8f
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 4 17:02:29 2019 -0500

    Some optimizations to PM hash table creation.

    - made locks more granular.
    - reduced logic per iteration when adding elements.

commit b20bf54ed97c5a0a88d414a4dd844a0afc2e27f3
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 4 15:32:32 2019 -0500

    Reduced granularity of djLock in PrimProc.

commit 6273a8f3c4c62b87ef91c77a829033426e38e4d4
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 4 14:45:58 2019 -0500

    Added a timer to PM hash table construction

    signal USR1 will print cumulative wall time to stdout & reset the timer.
2019-11-01 17:34:33 -04:00
f93c3f5d9c Merge pull request #904 from LinuxJedi/plugin-load
MCOL-3551 Use generic MariaDB Server paths
2019-10-14 14:33:09 -05:00
ebb22a96a3 MCOL-3551 Use generic MariaDB Server paths
This branch enforces the use of generic MariaDB server paths for their
binaries and data rather than custom paths.

/usr/local/mariadb/columnstore is now only for columnstore with this
patch.

It should be noted that this removes the auto-mounting of external
MariaDB UM data storage for AWS.

This is also a fix for MCOL-3510 after buildbot changes are made.

Also... MCOL-3552 Use columnstore.cnf to load plugins

The ColumnStore plugins now load using a columnstore.cnf instead of a
SQL sequence to be more in-line with MariaDB's methods.
2019-10-13 09:34:37 +01:00
87204cfc4d If a dev build (SKIP_OAM_INIT), made postConfigure exit before trying
to start the system, because that won't work.
2019-10-11 15:43:21 -05:00
51129d6eae Merge pull request #882 from mariadb-corporation/revert-MCOL-2102
Revert "MCOL-2102 Set CrossEngineSupport Port number when setting MyS…
2019-10-07 15:01:26 +03:00
159317d2e3 MCOL-3534 Remove -n and -d from postConfigure msgs 2019-10-03 19:20:50 +00:00
c3dc25a4ad Revert "MCOL-2102 Set CrossEngineSupport Port number when setting MySQLPort via postConfigure -port"
This reverts commit 755efb3dfb.
2019-10-02 09:47:00 +01:00
8c207bf412 MCOL-3515 Remove distributed install
Non-distributed is now the only install method.
2019-09-24 12:02:10 +01:00
24e1c3ff2d Merge pull request #855 from mariadb-corporation/move-etc
Move config files
2019-09-09 15:54:30 -05:00
97bda78c3b Move config files
This patch:

* Moves config files from /usr/local/mariadb/columnstore/etc to
ENGINE_SYSCONFDIR/columnstore (ENGINE_SYSCONFDIR is /etc by default)
* Sets a define called MCSSYSCONFDIR whic contains the
ENGINE_SYSCONFDIR compile time setting
* Modifies scripts and code to use the new paths
* Removes a whole bunch of files we don't use
2019-09-09 14:13:56 +01:00