1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-07-29 08:21:15 +03:00
Commit Graph

231 Commits

Author SHA1 Message Date
2ea73846b9 MCOL-4422 Remove mariadb.h and my_sys.h dependency from collation.h 2020-11-30 14:26:35 +04:00
3eb26c0d4a MCOL-4313 Introduced TSInt128 that is a storage class for int128
Removed uint128 from joblist/lbidlist.*

Another toString() method for wide-decimal that is EMPTY/NULL aware

Unified decimal processing in WF functions

Fixed a potential issue in EqualCompData::operator() for
    wide-decimal processing

Fixed some signedness warnings
2020-11-18 13:53:15 +00:00
129d5b5a0f MCOL-4174 Review/refactor frontend/connector code 2020-11-18 13:53:15 +00:00
844472d812 MCOL-4313 Very fragile but high speed approach with inline ASM
GCC compiler uses aligned versions of SIMD instructions expecting
aligned memory blocks that is hard to implement now
2020-11-18 13:52:20 +00:00
1588ebe439 MCOL-641 Clean up primitives code
Add int128_t support into ByteStream

Fixed UTs broken after collation patch
2020-11-18 13:52:19 +00:00
d3bc68b02f MCOL-641 Refactor initial extent elimination support.
This commit also adds support in TupleHashJoinStep::forwardCPData,
although we currently do not support wide decimals as join keys.

Row estimation to determine large-side of the join is also updated.
2020-11-18 13:52:19 +00:00
6aea838360 MCOL-641 Add support for functions (Part 2). 2020-11-18 13:51:55 +00:00
b5534eb847 MCOL-641 Refactored MultiplicationOverflowCheck but it still has flaws.
Introduced fDecimalOverflowCheck to enable/disable overflow check.

Add support into a FunctionColumn.

Low level scanning crashes on medium sized data sets.
2020-11-18 13:47:45 +00:00
b09f3088ca MCOL-641 Initial version of Math operations for wide decimal. 2020-11-18 13:47:44 +00:00
62d0c82d75 MCOL-641 1. Templatized convertValueNum() function.
2. Allocate int128_t buffers in batchprimitiveprocessor if
a query involves wide decimal columns.
2020-11-18 13:47:44 +00:00
824615a55b MCOL-641 Refactor empty value implementation in writeengine. 2020-11-18 13:47:44 +00:00
97ee1609b2 MCOL-641 Replaced NULL binary constants.
DataConvert::decimalToString, toString, writeIntPart, writeFractionalPart are not templates anymore.
2020-11-18 13:47:44 +00:00
c23ead2703 MCOL-641 This commit changes NULL and EMPTY values.
It also contains the refactored DataConvert::decimalToString().

Row::toString UT is finished.
2020-11-18 13:47:02 +00:00
de85e21c38 MCOL-641 This commit cleans up Row methods and adds couple UT for Row. 2020-11-18 13:47:02 +00:00
f73de30427 MCOL-641 This commit introduces GTest Suite into CS.
Binary NULL magic now consists of a series of BINARYEMPTYROW-s + BINARYNULL
in the end.

ByteStream now has hexbyte alias.

Added ColumnCommand::getEmptyRowValue to support 16 byte EMPTY values.
2020-11-18 13:47:01 +00:00
55afcd8890 MCOL-641 Basic extent elimination support for Decimal38. 2020-11-18 13:47:01 +00:00
0ff0472842 MCOL-641 sum() now works with DECIMAL(38) columns.
TupleAggregateStep class method and buildAggregateColumn() now properly set result data type.

doSum() now handles DECIMAL(38) in approprate manner.

Low-level null related methods for new binary-based datatypes now handles magic values for
binary-based DT.
2020-11-18 13:47:01 +00:00
98213c0094 MCOL-641 Addition now works for DECIMAL columns with precision > 18. 2020-11-18 13:47:01 +00:00
54c152d6c8 MCOL-641 This commit introduces templates for DataConvert and RowGroup methods. 2020-11-18 13:47:01 +00:00
32f6167067 MCOL-641 Work of Ivan Zuniga on basic read and write support for Binary16 2020-11-18 13:47:00 +00:00
98b94c0280 Remove deprecated boost::timer
Change the only one actual timer to std::chrono::steady_clock
2020-11-17 15:03:10 +03:00
b25fee320a Remove variable-length arrays (-Wvla) 2020-11-17 15:03:10 +03:00
ab44ef6ddb MCOL-4170 Refactor services/systemd units to finish their bootstrap ... 2020-11-09 12:01:16 +04:00
7e868bc588 add dependency for generated header files errorids.h messageids.h patch made by Ben. 2020-07-10 18:38:49 +00:00
eac7dab096 MCOL-4030: first commit of warning removals unneed const and missing virtual dtors. 2020-06-23 13:51:36 -05:00
f9078efbc6 MCOL-3536 Collation 2020-06-08 17:57:37 -05:00
236b92d706 MCOL-3536 Collation 2020-06-08 09:00:48 -05:00
d1e85c468c MCOL-3536 collation remove debug 1 thread 2020-06-05 10:22:31 -05:00
78ac310e42 MCOL-3536 Collation 2020-06-01 15:08:15 -05:00
06e50e0926 MCOL-3536 collation 2020-05-26 12:42:11 -05:00
11ba12f6ea MCOL-3536 collation 2020-05-19 16:22:44 -05:00
8479a87e46 Merge branch 'develop' into MCOL-3536 2020-05-18 16:22:01 -05:00
1f3d1e6fd6 MCOL-3536 collation 2020-05-14 16:02:49 -05:00
98abf95eae MCOL-3991 MCS is now single package and properly uninstalls 2020-05-12 13:36:24 +00:00
8595fa0226 MCOL-1734 CS now behaves similar to MDB in case of
NOT IN + correlated subquery if the subquery returns empty set.
CS now returns full outer record set.

gcc 8.2 complains about unused variable in ha_mcs_impl.cc
2020-01-07 17:23:17 +03:00
586391e1ca compilation failure
error: reference to 'mutex' is ambiguous
note: candidates are: 'class boost::mutex'
note:                 'class std::mutex'
2019-12-19 18:13:39 +01:00
d1199e03c3 Fix atomic usage
Using atomic without namespace broke the build on most platforms
2019-12-19 11:15:09 +02:00
0be00a3a73 Merge pull request #987 from LinuxJedi/MCOL-3677
Revert "MCOL-1734 This patch fixes the error for NOT IN + subquery wh…
2019-12-19 07:37:31 +02:00
3fb01d5f60 Revert "MCOL-1734 This patch fixes the error for NOT IN + subquery when outer query has NULLs in a key"
This reverts commit 5174c40bfe.
2019-12-18 15:09:47 +02:00
98d546c2c2 Found and fixed a race on a counter in the new HJ hash table
construction code in PrimProc.
2019-12-17 18:17:04 -05:00
5174c40bfe MCOL-1734 This patch fixes the error for NOT IN + subquery when outer query has NULLs in a key
column.
The if statement that decides whether to add the Row into a result set
or not has a logic error thus produces false positives for the case
described in MCOL-1734.
2019-12-06 20:31:23 +03:00
7489d0bfd0 MCOL-3625 Rename packages
Rename packages to MariaDB-columnstore-engine, MariaDB-columnstore-libs
and MariaDB-columnstore-platform.

Also add the "columnstore-" prefix the the components so that MariaDB's
packaging system understands then and add a line to include them in
MariaDB's packaging.

In addition
* Fix S3 building for dist source build
* Fix Debian 10 dependency issue
* Fix git handling for dist builds
* Add support for MariaDB's RPM building
* Use MariaDB's PCRE and readline
* Removes a few dead files
* Fix Boost noncopyable includes
2019-12-04 11:04:39 +00:00
1eaa83d852 Squash merge of the multithreaded PM join code.
Squashed commit of the following:

commit fe4cc375faf1588e30471062f78403e81229cd02
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Nov 1 13:38:11 2019 -0400

    Added some code comments to the new join code.

commit a7a82d093be4db3dfb44d33e4f514fd104b25f71
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Nov 1 13:17:47 2019 -0400

    Fixed an error down a path I think is unused.

commit 4e6c7c266a9aefd54c384ae2b466645770c81a5d
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Nov 1 13:12:12 2019 -0400

    std::atomic doesn't exist in C7, -> boost::atomic.

commit ed0996c3f4548fff0e19d43852d429ada1a72510
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Oct 16 12:47:32 2019 -0500

    Addition to the previous fix (join dependency projection).

commit 97bb806be9211e4688893460437f539c46f3796f
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Oct 15 15:22:09 2019 -0500

    Found and fixed a bad mem access, which may have been there for 8 years.

commit d8b0432d2abd70f28de5276daad758c494e4b04b
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Oct 15 14:04:48 2019 -0500

    Minor optimization in some code I happened to look at.

commit b6ec8204bf71670c7a8882464289e700aa5f7e33
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Oct 15 14:04:11 2019 -0500

    Fixed a compiler warning.

commit 0bf3e5218f71d92460ddc88090e3af77ecf28c35
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Oct 15 10:11:09 2019 -0500

    Undid part of the previous commit.

commit 5dfa1d23980e245c77c1644015b553aa4bcdf908
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Mon Oct 14 18:00:21 2019 -0500

    Proofread the diff vs base, added some comments, removed some debugging stuff.

commit 411fd955ebbae97ddab210a7b17fe5708538001d
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Oct 11 13:55:39 2019 -0500

    If a dev build (SKIP_OAM_INIT), made postConfigure exit before trying
    to start the system, because that won't work.

commit 634b1b8a7340b55fcaee045fd6d00b3e3a9269fa
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Mon Sep 30 14:55:45 2019 -0500

    Reduced crit section of BPP::addToJoiner a little.

commit 31f30c64dd95942f2c7a247cc81feaa5933c1a07
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 18 11:09:27 2019 -0500

    Checkpointing.  make the add joiner stuff free tmp mem quickly.

commit 9b7e788690546af7ddc4c921a0ab441ee9a8df02
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 18 10:38:57 2019 -0500

    Checkpoint.  Removed tmp hardcoding of bucket count.

commit fda4d8b7fb30d0431dc15e473042abb3d8121b19
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 18 10:20:09 2019 -0500

    Checkpoint.  Adjusted unproductive loop wait time.

commit 7b9a67df7d192f240e9e558e6e66c7aa9f1e8687
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 18 10:10:43 2019 -0500

    Checkpointing add'l optimizations.

    If we promote bpp::processorThreads / bucket count to a power of 2, we can
    use a bitmask instead of a mod operation to decide a bucket.

    Also, boosted utilization by not waiting for a bucket lock to become free.
    There are likely more gains to be had there with a smarter strategy.
    Maybe have each thread generate a random bucket access pattern to reduce
    chance of collision.  TBD.

commit abe7dab8661b5120f6ee268abc005dd66cd643e2
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Sep 17 16:15:51 2019 -0500

    Multithreaded PM hash table construction likely works here.

    A couple more fixes.
     - missed a mod after a hash in one place.
     - Made the PoolAllocator thread safe (small degree of performance hit
       there in threaded env).  May need to circle back to the table
       construction code to eliminate contention for the allocators instead.

commit ab308762fbd873dbf246a6d1574223087cd0d5f6
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Sep 17 12:14:14 2019 -0500

    Checkpointing.  Did some initial testing, fixed a couple things.

    Not done testing yet.

commit 3b161d74fa859edb8b5ba84bb905e586ac0586e6
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Tue Sep 17 11:24:55 2019 -0500

    Checkpointing.  First cut of multithreaded PM join table building.

    Builds but is untested.

commit cb7e6e1c2761fc6c33b3b1c6b6cda488d7792bca
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Mon Sep 16 13:03:50 2019 -0500

    Increase the STLPoolAllocator window size to reduce destruction time.

commit b0ddaaae71a0a4959ad15c87579d85ed88e17e1f
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Fri Sep 13 11:52:51 2019 -0500

    Fixed a bug preventing parallel table loading.  works now.

commit b87039604e312c1ddb88cdb226228b1c3addf018
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Thu Sep 12 22:04:15 2019 -0500

    Checkpointing some experimental changes.

     - Made the allocator type used by PM joins the STLPoolAllocator
     - Changed the default chunk size used by STLPoolAlloc based on a few test
        runs
     - Made BPP-JL interleave the PM join data by join # to take advantage
        of new locking env on PM.
     - While I was at it, fixed MCOL-1758.

commit fd4b09cc383d2b96959a8e5ca490c940bacb3d37
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Thu Sep 12 16:03:30 2019 -0500

    Speculative change.  Row estimator was stopping at 20 extents.

    Removed that limitation.

commit 7dcdd5b5455f9ac06121dd3cf1ba722150f3ee56
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Thu Sep 5 09:10:28 2019 -0500

    Inlined some hot simpleallocator fcns.

commit 6d84daceecc5499f6286cf3468c118b8b1d28d8f
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 4 17:02:29 2019 -0500

    Some optimizations to PM hash table creation.

    - made locks more granular.
    - reduced logic per iteration when adding elements.

commit b20bf54ed97c5a0a88d414a4dd844a0afc2e27f3
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 4 15:32:32 2019 -0500

    Reduced granularity of djLock in PrimProc.

commit 6273a8f3c4c62b87ef91c77a829033426e38e4d4
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date:   Wed Sep 4 14:45:58 2019 -0500

    Added a timer to PM hash table construction

    signal USR1 will print cumulative wall time to stdout & reset the timer.
2019-11-01 17:34:33 -04:00
3fef0f21d3 Remove vpj files
They shouldn't be here
2019-09-05 17:38:03 +01:00
6cdca1330b Merge pull request #808 from mariadb-corporation/develop-merge-up-20190729
Merge develop-1.2 into develop
2019-08-13 11:55:22 +03:00
a09a9d5d0f Mass substitution 'Corporaton' -> 'Corporation' 2019-08-07 14:43:25 -05:00
811909aa72 Merge branch 'develop-1.2' into develop-merge-up-20190729 2019-07-29 12:19:26 +01:00
5e4f1b9933 Merge branch 'develop' into MCOL-265 2019-06-10 13:58:03 +01:00
9390ee05fb Revert "MCOL-1559 Some string trailing blank stuff"
This reverts commit e5d76e142b.
2019-05-23 13:49:08 +01:00
020b211bb7 Merge branch 'develop-1.2' into develop-merge-up-20190514 2019-05-14 13:58:33 +01:00