1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-07-29 08:21:15 +03:00
Commit Graph

239 Commits

Author SHA1 Message Date
1eaa83d852 Squash merge of the multithreaded PM join code.
Squashed commit of the following:

commit fe4cc375faf1588e30471062f78403e81229cd02
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Nov 1 13:38:11 2019 -0400

    Added some code comments to the new join code.

commit a7a82d093be4db3dfb44d33e4f514fd104b25f71
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Nov 1 13:17:47 2019 -0400

    Fixed an error down a path I think is unused.

commit 4e6c7c266a9aefd54c384ae2b466645770c81a5d
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Nov 1 13:12:12 2019 -0400

    std::atomic doesn't exist in C7, -> boost::atomic.

commit ed0996c3f4548fff0e19d43852d429ada1a72510
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Oct 16 12:47:32 2019 -0500

    Addition to the previous fix (join dependency projection).

commit 97bb806be9211e4688893460437f539c46f3796f
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Oct 15 15:22:09 2019 -0500

    Found and fixed a bad mem access, which may have been there for 8 years.

commit d8b0432d2abd70f28de5276daad758c494e4b04b
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Oct 15 14:04:48 2019 -0500

    Minor optimization in some code I happened to look at.

commit b6ec8204bf71670c7a8882464289e700aa5f7e33
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Oct 15 14:04:11 2019 -0500

    Fixed a compiler warning.

commit 0bf3e5218f71d92460ddc88090e3af77ecf28c35
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Oct 15 10:11:09 2019 -0500

    Undid part of the previous commit.

commit 5dfa1d23980e245c77c1644015b553aa4bcdf908
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Mon Oct 14 18:00:21 2019 -0500

    Proofread the diff vs base, added some comments, removed some debugging stuff.

commit 411fd955ebbae97ddab210a7b17fe5708538001d
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Oct 11 13:55:39 2019 -0500

    If a dev build (SKIP_OAM_INIT), made postConfigure exit before trying
    to start the system, because that won't work.

commit 634b1b8a7340b55fcaee045fd6d00b3e3a9269fa
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Mon Sep 30 14:55:45 2019 -0500

    Reduced crit section of BPP::addToJoiner a little.

commit 31f30c64dd95942f2c7a247cc81feaa5933c1a07
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 18 11:09:27 2019 -0500

    Checkpointing.  make the add joiner stuff free tmp mem quickly.

commit 9b7e788690546af7ddc4c921a0ab441ee9a8df02
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 18 10:38:57 2019 -0500

    Checkpoint.  Removed tmp hardcoding of bucket count.

commit fda4d8b7fb30d0431dc15e473042abb3d8121b19
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 18 10:20:09 2019 -0500

    Checkpoint.  Adjusted unproductive loop wait time.

commit 7b9a67df7d192f240e9e558e6e66c7aa9f1e8687
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 18 10:10:43 2019 -0500

    Checkpointing add'l optimizations.

    If we promote bpp::processorThreads / bucket count to a power of 2, we can
    use a bitmask instead of a mod operation to decide a bucket.

    Also, boosted utilization by not waiting for a bucket lock to become free.
    There are likely more gains to be had there with a smarter strategy.
    Maybe have each thread generate a random bucket access pattern to reduce
    chance of collision.  TBD.

commit abe7dab8661b5120f6ee268abc005dd66cd643e2
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Sep 17 16:15:51 2019 -0500

    Multithreaded PM hash table construction likely works here.

    A couple more fixes.
     - missed a mod after a hash in one place.
     - Made the PoolAllocator thread safe (small degree of performance hit
       there in threaded env).  May need to circle back to the table
       construction code to eliminate contention for the allocators instead.

commit ab308762fbd873dbf246a6d1574223087cd0d5f6
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Sep 17 12:14:14 2019 -0500

    Checkpointing.  Did some initial testing, fixed a couple things.

    Not done testing yet.

commit 3b161d74fa859edb8b5ba84bb905e586ac0586e6
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Sep 17 11:24:55 2019 -0500

    Checkpointing.  First cut of multithreaded PM join table building.

    Builds but is untested.

commit cb7e6e1c2761fc6c33b3b1c6b6cda488d7792bca
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Mon Sep 16 13:03:50 2019 -0500

    Increase the STLPoolAllocator window size to reduce destruction time.

commit b0ddaaae71a0a4959ad15c87579d85ed88e17e1f
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Sep 13 11:52:51 2019 -0500

    Fixed a bug preventing parallel table loading.  works now.

commit b87039604e312c1ddb88cdb226228b1c3addf018
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Thu Sep 12 22:04:15 2019 -0500

    Checkpointing some experimental changes.

     - Made the allocator type used by PM joins the STLPoolAllocator
     - Changed the default chunk size used by STLPoolAlloc based on a few test
        runs
     - Made BPP-JL interleave the PM join data by join # to take advantage
        of new locking env on PM.
     - While I was at it, fixed MCOL-1758.

commit fd4b09cc383d2b96959a8e5ca490c940bacb3d37
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Thu Sep 12 16:03:30 2019 -0500

    Speculative change.  Row estimator was stopping at 20 extents.

    Removed that limitation.

commit 7dcdd5b5455f9ac06121dd3cf1ba722150f3ee56
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Thu Sep 5 09:10:28 2019 -0500

    Inlined some hot simpleallocator fcns.

commit 6d84daceecc5499f6286cf3468c118b8b1d28d8f
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 4 17:02:29 2019 -0500

    Some optimizations to PM hash table creation.

    - made locks more granular.
    - reduced logic per iteration when adding elements.

commit b20bf54ed97c5a0a88d414a4dd844a0afc2e27f3
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 4 15:32:32 2019 -0500

    Reduced granularity of djLock in PrimProc.

commit 6273a8f3c4c62b87ef91c77a829033426e38e4d4
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 4 14:45:58 2019 -0500

    Added a timer to PM hash table construction

    signal USR1 will print cumulative wall time to stdout & reset the timer.
2019-11-01 17:34:33 -04:00
cbef44a0be MCOL-3503 add MODA aggregate function 2019-09-27 12:22:44 -05:00
70b3aa3159 Merge branch 'develop-1.2' into develop-merge-up-20190924-2 2019-09-24 14:17:57 +01:00
3fef0f21d3 Remove vpj files
They shouldn't be here
2019-09-05 17:38:03 +01:00
95fcc3dcbb MCOL-3423 Don't move decimal for LONG DOUBLE. Clear long double extra bits after copy, not before. 2019-08-20 09:50:43 -05:00
7af9771319 Merge branch 'develop' into develop-merge-up-20190729 2019-08-09 14:01:11 +01:00
d62b66ecf7 parse_item() in execplan code now always get an actual GWI structure
to avoid accedental crashes.

Add check for Conversion of Big IN Predicates Into Subqueries optimization
conditions.

Enabled derivedTableOptimization() for group by and derived handlers.

Disabled Conversion of Big IN Predicates Into Subqueries optimization.

Disabled most of optimizer_flags for now.

RowGroup + operator now correctly sets useStringTable flag that
instructs code to check StringStore instead of plain data buffer.
2019-08-01 14:29:55 -04:00
811909aa72 Merge branch 'develop-1.2' into develop-merge-up-20190729 2019-07-29 12:19:26 +01:00
c683b6bfc0 MCOL-3404 Revert experimental changes from MCOL-3343 2019-07-16 18:19:03 -05:00
5e4f1b9933 Merge branch 'develop' into MCOL-265 2019-06-10 13:58:03 +01:00
9022fa426d minor fixups 2019-05-04 02:14:40 -05:00
e078a416ec Fix build fail on buildbox 2019-04-29 06:00:53 -05:00
53afd89bb1 fully qualify identifiers 2019-04-29 02:07:56 -05:00
6bac032d56 break compile dependecy on ALARMManager.h 2019-04-29 01:28:55 -05:00
4b9d046c6e Fully resolve potentially ambiguous symbols by removing using namespace statements from headers which have a cascading effect. This causes potential behavior changes when switching to c++11 since symbols can be exported from std and boost while both have been imported into the global namespace. 2019-04-29 01:21:15 -05:00
1813add162 Merge pull request #743 from davidjmott/develop
Add -DSERVER_BUILD_DIR configure parameter to interrogate the server …
2019-04-26 14:02:20 +01:00
e65f80f493 delete visual c++ project files. cmake can generate these if needed 2019-04-25 23:35:03 -05:00
784bbe09d4 Merge branch 'develop-1.2' into develop-merge-up-20190425 2019-04-25 10:27:59 +01:00
e89d1ac3cf MCOL-265 Add support for TIMESTAMP data type 2019-04-23 00:00:09 -04:00
9846d6b595 MCOL-2091 align margin 2019-04-15 12:52:00 -05:00
853dc2a2c1 MCOL-2091 Don't use in-line initializers, a C++x11 feature. 2019-04-15 12:49:43 -05:00
f1b908abeb MCOL-2091 Special UDAF reset code for multi-distinct queries 2019-04-11 15:44:46 -05:00
5f76cee5f4 MCOL-2267 allow for long double in row::equals() 2019-04-02 11:36:56 -06:00
064d2ee9e4 Merge branch 'develop-1.2' into develop-merge-up-20190328 2019-03-28 15:09:21 +00:00
3f2c753947 MCOL-1822-c final checkin 2019-03-05 09:33:39 -06:00
cf056e42ac Merge branch 'develop-1.2' into MCOL-1822-c 2019-02-27 13:20:45 -06:00
c654e8621e MCOL-1822 interim checkin 2019-02-27 13:09:37 -06:00
a2aa4b8479 MCOL-1822 Intermediate checkin. DISTINCT not working. 2019-02-25 14:54:46 -06:00
a955b56f4d Merge branch 'develop-1.2' into develop-merge-up-20190218 2019-02-18 15:55:11 +00:00
c5b9ae11e5 MCOL-1822 add LONG DOUBLE support 2019-01-29 09:55:43 -06:00
6d65b13852 MCOL-901 Significanlty reduced memory consumption for group_concat().
RowGroup default constructor allocates memory too generously.

    Removed commented code from groupconcat.cpp.
2019-01-29 16:02:37 +03:00
d1ada75395 MCOL-270 Add support for MEDIUMINT data type 2018-12-30 19:13:16 -05:00
35a17a87c4 MCOL-1829 Subquery with limited order by could potentially return onordered set.
There were two code mistakes: Eq::operator() always returned true for
	any pair and Hasher::operator() always returned 0 as a key.
2018-12-26 10:39:32 +03:00
82bd9e29f5 Merge branch 'develop-1.1' into 1.1-merge-up-2018-12-20b 2018-12-20 20:55:00 +00:00
e4ee1095de Merge branch 'develop-1.1' into 1.1-merge-up-2018-12-20 2018-12-20 20:37:24 +00:00
7989ccefe0 MCOL-2018 Fix array bounds issue
StringStore NULL check includes a check for _CpNuLl_ in the
StringStore. This is a case should never happen but we keep it just in
case.

Unfortunately this check was skipping 4*8 bytes instead of just 4 bytes.
This is definitely bad behaviour but it could cause an out-of-bounds read
based crash.
2018-12-12 18:06:09 +00:00
9c1bc910a5 MCOL-1793, fix REGR_SLOPE calculations, change scale to DECIMAL_NOT_SPECIFIED (variable length) for most REGR_*** functions. 2018-11-20 16:29:17 -06:00
d930a1e322 MCOL-521 Some more fixes for multi-parm aggregates. Add regr slope 2018-09-25 16:31:10 -05:00
3fac7b1e19 MCOL-521 more multi-param work with constant NULL 2018-09-13 14:12:10 -05:00
580a3ec123 MCOL-1647 Fix TIME regressions
Fixes the following:

* Read past buffer end in intToDatetime / intToTime
* Allow intToTime to convert datetime
* Allow intToTime to convert shortened time values
* Allow stringToTime to convert datetime and int time values
* Fix saturation / bad values in intToTime and stringToTime
* Fix TIME return in STR_TO_DATE()
* Fix NULL return on type inequality for TIMEDIFF()
* Fix zero day calculation error in ADDTIME()/SUBTIME()
* Fix DATETIME to int calculation error in aggregate bit operations
* Make the new harderning flags optional with -DSECURITY_HARDENED_NEW
2018-08-17 07:55:51 +01:00
443a2867c4 MCOL-1385 Initial 10.3 support 2018-08-02 14:55:01 +01:00
bd641301da Merge pull request #504 from mariadb-corporation/1.1-merge-up-20180621
Merge develop-1.1 into develop
2018-07-04 22:52:04 +03:00
77b52a6a32 MCOL-1446 CS sorting direction aligned with the server`s. 2018-07-03 19:55:30 +03:00
8c90419852 Fix merge and coding style issues 2018-06-22 15:31:31 +01:00
ea70806e93 MCOL-1201 Add support for UDAF multiple parm constants 2018-06-05 14:10:04 -05:00
ec3a3846c3 MCOL-1201 manual rebase with develop. Obsoletes branch MCOL-1201 2018-06-05 14:06:58 -05:00
06e9772310 MCOL-1201 some fixes from testing 2018-06-05 13:54:17 -05:00
6fa7dded6f MCOL-1201 manual rebase with develop. Obsoletes branch MCOL-1201 2018-06-05 13:54:17 -05:00
1a582eed4a Merge branch 'develop-1.1' into 1.1-merge-up-20180509-a2 2018-05-09 09:20:55 +01:00
ac3e702a3e MCOL-1396 Allow StringStore to hold more than 2GB
StringStore originally worked by returning a 32bit pointer to a memory
location and storing the length with that pointer. This allowed 4GB to
be stored in 64KB blocks. With 1.1 we used the high bit to signify a
TEXT/BLOB string of > 64KB reducing the max capacity to 2GB but without
any bounds checking.

So, if you went over the 2GB mark the getter would think you are trying
to get a long string instead of a short one and come up empty. It would
then return NULL.

This patch uses 64bit memory points still retaining the high bit to
signify long strings. It also now stores the length with the string
rather than with the pointer to allow the full 64bits for pointers.

It also adds a bounds check for small strings.
2018-05-08 19:38:06 +01:00