1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2026-01-06 08:21:10 +03:00
Commit Graph

4499 Commits

Author SHA1 Message Date
Gagan Goel
709290cc3d Port of commit: 1ff23d0cd70d576a0f4e512ce332cff348591d36
from server/columnstore_cache.

Commit message:

Fixed bug in free locks that caused rows in cache to not be properly flushed

Fixed by doing adding external_lock(F_UNLCK) in free_locks.
I also moved unlock of cache_handler to be after changing lock type of
cached tables to ensure that no one can use cached table while this is
happening (as cache table is locked with write lock).
I also fixed a wrong mutex order bug in ha_cache::flush_insert_cache()

Other things:
- Addded share::cached_rows to track inserted rows. This is only used
  for asserts to check the number of rows in the cache.
- Fixed wrong mysql_file_chsize() in case of repair
2020-08-18 18:01:40 -04:00
Gagan Goel
4afcba9520 Do not build the cache as a separate user-visible engine.
We are creating a new read-only system variable, columnstore_cache_inserts,
to enable/disable the cache. When this variable is set at server start up,
any table created with engine=columnstore will also create the corresponding
cache table in Aria engine for performing inserts.

It is important to note that a ColumnStore table created with this
option unset should not be queried when the server is restarted with
the option set, as this will most likely result in query failures.
2020-08-18 18:01:40 -04:00
Gagan Goel
86fb66365c 1. Set 1M as the threshold on the number of records to flush the cache.
2. Set 100k as the batch size when flushing records into ColumnStore, i.e.,
a flush of 1M records will be performed in 10 batches, each being 100k.

3. For INSERT ... SELECT on the cache, use the default insertion method of cpimport.
2020-08-18 18:01:40 -04:00
Gagan Goel
f5a8d228a0 Port of commit ba731bdc6a80e88d32e7440044b548c3e3edc591
from server/columnstore_cache

Commit message:

Fixed crashed bug on simple insert

Other things:
- Added test from columnstore team
- Fixed two reported bugs from columnstore team
- Call free_locks as part of start_trans() instead of get_status()
  to ensure that we have locks both for cached table and cache table
  before we try to free any.
- Store pointers to lock->get_status and lock->update_status for the
  cached table. Was needed by ha_tina in flush_insert_cache to make
  new insert rows visible for the SELECT that caused the flush
2020-08-18 18:01:40 -04:00
Gagan Goel
4ff4e9eb89 Re-enable the ColumnStore_Cache plugin. 2020-08-18 18:01:40 -04:00
Gagan Goel
cb2ddad308 Port of commit 0463e1f722d4d32526760c923e0092a380a9e634
from server/columnstore_cache.

Commit message:

Fixed bug in cache:

- The THR_LOCK org_lock must be stored in a shared structure so that all
  instances of a table can use it. Fixed by adding a ha_cache_share object
  that keeps track of this one.
- Fixed wrong test in get_status_and_flush_cache to detect in insert command
- Fixed in get_status_and_flush_cache that we always free the insert lock
  if we don't need it.
2020-08-18 18:01:40 -04:00
benthompson15
fd0ef267b5 Merge pull request #1368 from dhall-MariaDB/MCOL-4252
MCOL-4252 Fix up configuration
2020-08-18 14:37:53 -05:00
Gagan Goel
f1759e6560 Merge pull request #1367 from dhall-MariaDB/MCOL-3827
MCOL-3827 Optimize out sort on SubQuery in Select
2020-08-18 14:54:36 -04:00
Gagan Goel
59cae761d5 Merge pull request #1366 from dhall-MariaDB/MCOL-4247
MCOL-4247 TYpecast causes wrong virtual function
2020-08-18 14:30:35 -04:00
David.Hall
ada33d11c0 Merge pull request #1362 from benthompson15/MCOL-4155
MCOL-4155: rename the cnf file for enterprise builds.
2020-08-18 12:12:13 -05:00
David Hall
3673586267 MCOL-4252 Fix up configuration 2020-08-18 12:09:31 -05:00
David Hall
2880d0871f MCOL-4247 add break to case 2020-08-18 11:47:06 -05:00
Gagan Goel
d829470451 Merge pull request #1360 from dhall-MariaDB/MCOL-3814
MCOL-3814 add back lower to viewName assignment
2020-08-18 12:28:12 -04:00
Patrick LeBlanc
8e666afde8 Merge pull request #1365 from jmrojas2332/develop
MCOL-4227 Changes to mcs-loadbrm to work with multinode with s3
2020-08-18 10:51:56 -05:00
David Hall
7ba40a5544 MCOL-4247 TYpecast causes wrong virtual function 2020-08-17 17:34:58 -05:00
David Hall
e7b8abfdb9 MCOL-3827 Optimize out sort on SubQuery in Select 2020-08-17 14:40:22 -05:00
Jose Rojas
ceb44db95c Revert shebang change from docker 2020-08-14 16:10:54 -05:00
Jose Rojas
12fd673859 Merge branch 'develop' of https://github.com/mariadb-corporation/mariadb-columnstore-engine into develop 2020-08-14 16:08:18 -05:00
Jose Rojas
a2be47b4c7 Merge pull request #1363 from benthompson15/MCOL-4196
MCOL-4196: Runs 'dbbuilder 7' on all columnstore starts.
2020-08-13 10:33:12 -05:00
benthompson15
6eea5e072d MCOL-4196: Runs 'dbbuilder 7' on all columnstore starts. 2020-08-12 16:29:41 -05:00
benthompson15
30d3930f96 MCOL-4155: rename the cnf file for enterprise builds. 2020-08-12 15:21:32 -05:00
David Hall
478426c8bf MCOL-3814 add back lower to viewName assignment 2020-08-12 10:00:04 -05:00
Jose Rojas
70f40210d0 Merge pull request #1355 from drrtuy/docker_dispatcher
MCOL-4157 mcs-loadbrm.py doesn't start SM with systemd with a certain…
2020-08-05 09:43:38 -05:00
Jose Rojas
023defa8c6 Merge pull request #1358 from drrtuy/1.5.4-1
MCOL-4136 This patch makes an explicit dependency on generated errroi…
2020-08-05 09:23:42 -05:00
Roman Nozdrin
058db083e6 ExeMgr and PrimProc systemd units now prints a warning if no jemalloc found in ld cache 2020-08-05 13:01:46 +00:00
Roman Nozdrin
1c74f80d76 MCOL-4136 This patch makes an explicit dependency on generated errorids.h 2020-08-05 12:22:50 +00:00
Roman Nozdrin
6357ca1a72 Merge pull request #1353 from mariadb-corporation/ci-publish-pkgs-repodata
trivial repositories for pkgs
2020-08-04 12:06:53 +03:00
mariadb-RomanNavrotskiy
723433abc6 - trivial repositories for pkgs
- publish to event folder
- publish latest only on cron event
2020-08-04 07:57:26 +02:00
Gagan Goel
397945c2ea Merge pull request #1356 from mariadb-corporation/MCOL-4179
MCOL-4179 subquery returning bad data
2020-08-03 19:30:46 -04:00
David Hall
019ee27672 MCOL-4179 subquery returning bad data
in the one case, a typo caused rowgroup to be poorly formed.
2020-08-03 17:02:38 -05:00
Roman Nozdrin
ea4701caa7 MCOL-4157 mcs-loadbrm.py doesn't start SM with systemd with a certain argument
This feature is used to start multi-node MCS in containers
2020-08-03 17:45:21 +00:00
Jose Rojas
f764621615 Merge pull request #1352 from drrtuy/brm-4-s3-cluster
Introduced an artificial delay b/w SM and controllernode to allow SM …
2020-08-03 11:17:27 -05:00
Roman Nozdrin
4b0e8900d4 Introduced an artificial delay b/w SM and controllernode to allow SM to start
Introduced mcs-savebrm.py to avoid saving BRM on non-primary nodes
in multi-node S3 clusters

Refactored mcs-loadbrm.py to download BRM files from the primary in
in multi-node S3 clusters
2020-08-03 16:00:00 +00:00
Roman Nozdrin
15506e2bf1 Merge pull request #1350 from mariadb-corporation/ci-smoke-test-epel
ci: add epel for tests
2020-07-29 23:14:13 +03:00
Patrick LeBlanc
2db9a1d83b Merge pull request #1351 from pleblanc1976/update-version
Set version to 1.5.4-1
2020-07-29 11:48:41 -05:00
mariadb-RomanNavrotskiy
53253ae1db ci: add epel for tests 2020-07-29 18:47:18 +02:00
Patrick LeBlanc
e535c1d8d3 Set version to 1.5.4-1 2020-07-29 12:41:47 -04:00
Jose Rojas
c755f375dd Change for loadbrm s3 2020-07-27 17:41:24 -05:00
Roman Nozdrin
bca0d90d55 Merge pull request #1348 from drrtuy/increase_s3_timeout
Raised StopTimeout for mcs-workernode to allow SM to converge w/o tim…
2020-07-27 16:45:53 +03:00
Roman Nozdrin
5abb335595 Raised StopTimeout for mcs-workernode to allow SM to converge w/o timeout 2020-07-27 13:09:46 +00:00
Roman Nozdrin
3d61cfd8c7 Merge pull request #1347 from drrtuy/minor-upgrade
Remove x-columnstore.cnf b/c it breaks minor-upgrade on RPM-based dis…
2020-07-27 15:20:41 +03:00
Roman Nozdrin
26895ccdb3 Remove x-columnstore.cnf b/c it breaks minor-upgrade on RPM-based distros 2020-07-27 12:04:49 +00:00
Roman Nozdrin
5267dc381e Merge pull request #1345 from mariadb-corporation/docker-image-step
docker image
2020-07-24 18:23:01 +03:00
mariadb-RomanNavrotskiy
51e8d6de6a ci docker image steps 2020-07-24 13:52:01 +02:00
Gagan Goel
6fa984f883 Merge pull request #1344 from benthompson15/MCOL-4181-dev
MCOL-4181: Possible setting of ci->stats.fUser to NULL causing crash.
2020-07-21 15:11:35 -04:00
Roman Nozdrin
dc1852ad91 Merge pull request #1342 from mariadb-corporation/ci-publish-first
move publish pkgs step before regression
2020-07-21 17:03:25 +03:00
mariadb-RomanNavrotskiy
6519e7cd6a move publish pkgs before regression
set lower_case_table_names=1 for regression
2020-07-21 07:46:47 +02:00
benthompson15
219f67d162 MCOL-4181: Possible setting of ci->stats.fUser to NULL causing crash. 2020-07-20 15:18:58 -05:00
Roman Nozdrin
865988926e Merge pull request #1340 from jmrojas2332/develop
MCOL-4169 Prevent mcs-loadbrm from starting up twice
2020-07-13 21:11:31 +03:00
David.Hall
24f23c1d07 Merge pull request #1339 from drrtuy/MCOL-4152
MCOL-4152 Reset cond_stack with ha_mcs::reset() call
2020-07-13 10:49:43 -05:00