1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-07-29 08:21:15 +03:00
Commit Graph

65 Commits

Author SHA1 Message Date
ab7dfaa25b Fix resource leak in DDLProc/DMLProc/PrimProc/WriteengineServer processes.
As part of the charset support, a call to MY_INIT() was added at the
initialization of the above processes. This call initializes the MySQL
thread environment required by the charset library. However, the
accompanying my_end() call required to terminate this thread environment
was not added at the termination of these process, hence leaking
resources. As a fix, we move the MY_INIT() calls to the Child()
functions of these services and also add the missing my_end() call.
2023-06-23 20:18:32 +00:00
2e1394149b MCOL-5464: Fixes of bugs from ASAN warnings, part one (#2792)
* Fixes of bugs from ASAN warnings, part one

* MQC as static library, with nifty counter for global map and mutex

* Switch clang to 16

* link messageqcpp to execplan
2023-04-04 02:33:23 +03:00
56f2346083 Remove windows ifdefs 2023-03-02 15:59:42 +00:00
96c40d5081 Add core dumps and stack trace (#2604)
* Add core dumps and stack trace
2022-11-22 03:29:34 +03:00
37fd915a08 Serg`s patch for develop-6 revised for develop https://github.com/mariadb-corporation/mariadb-columnstore-engine/pull/2614 2022-11-09 22:41:38 +00:00
b5d8e0324b extentmap iterator (#2540) 2022-08-29 13:30:03 +03:00
3919c541ac New warnfixes (#2254)
* Fix clang warnings

* Remove vim tab guides

* initialize variables

* 'strncpy' output truncated before terminating nul copying as many bytes from a string as its length

* Fix ISO C++17 does not allow 'register' storage class specifier for outdated bison

* chars are unsigned on ARM, having  if (ival < 0) always false

* chars are unsigned by default on ARM and comparison with -1 if always true
2022-02-17 13:08:58 +03:00
973e5024d8 MCOL-4957 Fix performance slowdown for processing TIMESTAMP columns.
Part 1:
 As part of MCOL-3776 to address synchronization issue while accessing
 the fTimeZone member of the Func class, mutex locks were added to the
 accessor and mutator methods. However, this slows down processing
 of TIMESTAMP columns in PrimProc significantly as all threads across
 all concurrently running queries would serialize on the mutex. This
 is because PrimProc only has a single global object for the functor
 class (class derived from Func in utils/funcexp/functor.h) for a given
 function name. To fix this problem:

   (1) We remove the fTimeZone as a member of the Func derived classes
   (hence removing the mutexes) and instead use the fOperationType
   member of the FunctionColumn class to propagate the timezone values
   down to the individual functor processing functions such as
   FunctionColumn::getStrVal(), FunctionColumn::getIntVal(), etc.

   (2) To achieve (1), a timezone member is added to the
   execplan::CalpontSystemCatalog::ColType class.

Part 2:
 Several functors in the Funcexp code call dataconvert::gmtSecToMySQLTime()
 and dataconvert::mySQLTimeToGmtSec() functions for conversion between seconds
 since unix epoch and broken-down representation. These functions in turn call
 the C library function localtime_r() which currently has a known bug of holding
 a global lock via a call to __tz_convert. This significantly reduces performance
 in multi-threaded applications where multiple threads concurrently call
 localtime_r(). More details on the bug:
   https://sourceware.org/bugzilla/show_bug.cgi?id=16145

 This bug in localtime_r() caused processing of the Functors in PrimProc to
 slowdown significantly since a query execution causes Functors code to be
 processed in a multi-threaded manner.

 As a fix, we remove the calls to localtime_r() from gmtSecToMySQLTime()
 and mySQLTimeToGmtSec() by performing the timezone-to-offset conversion
 (done in dataconvert::timeZoneToOffset()) during the execution plan
 creation in the plugin. Note that localtime_r() is only called when the
 time_zone system variable is set to "SYSTEM".

 This fix also required changing the timezone type from a std::string to
 a long across the system.
2022-02-14 14:12:27 -05:00
04752ec546 clang format apply 2022-01-21 16:43:49 +00:00
01f3ceb437 replace header guards with #pragma once 2022-01-21 15:24:58 +00:00
6c31f105ec [MCOL-4927] Fix DDLProc connection processing 2021-12-16 16:36:55 +03:00
5c5f103f98 MCOL-4839: Fix clang build (#2100)
* Fix clang build

* Extern C returned to plugin_instance

Co-authored-by: Leonid Fedorov <l.fedorov@mail.corp.ru>
2021-08-23 10:45:10 -05:00
9608533d92 MCOL-4734 Compilation failure: MariaDB-10.6 + ColumnStore-develop
mcsconfig.h and my_config.h have the following
pre-processor definitions:

1. Conflicting definitions coming from the standard cmake definitions:
- PACKAGE
- PACKAGE_BUGREPORT
- PACKAGE_NAME
- PACKAGE_STRING
- PACKAGE_TARNAME
- PACKAGE_VERSION
- VERSION

2. Conflicting definitions of other kinds:
- HAVE_STRTOLL - this is a dirt in MariaDB headers.
  Should be fixed in the server code. my_config.h erroneously
  performs "#define HAVE_STRTOLL" instead of "#define HAVE_STRTOLL 1".
  in some cases. The former is not CMake compatible style. The latter is.

3. Non-conflicting definitions:
  Otherwise, mcsconfig.h and my_config.h should be mutually compatible,
  because both are generated by cmake on the same host machine. So
  they should have exactly equal definitions like "HAVE_XXX", "SIZEOF_XXX", etc.

Observations:
- It's OK to include both mcsconfig.h and my_config.h providing that we
  suppress duplicate definition of the above conflicting types #1 and #2.
- There is no a need to suppress duplicate definitions mentioned in #3,
  as they are compatible!
- my_sys.h and m_ctype.h must always follow a CMake configuation header,
  either my_config.h or mcsconfig.h (or both).
  They must never be included without any preceeding configuration header.

This change make sure that we resolve conflicts by:
- either disallowing inclusion of mcsconfig.h and my_config.h
  at the same time
- or by hiding conflicting definitions #1 and #2
  (with their later restoring).
- also, by making sure that my_sys.h and m_ctype.h always follow
  a CMake configuration file.

Details:
- idb_mysql.h can now only be included only after my_config.h
  An attempt to use idb_mysql.h with mcsconfig.h instead of
  my_config.h is caught by the "#error" preprocessor directive.

- mariadb_my_sys.h can now be only included after mcsconfig.h.
  An attempt to use mariadb_my_sys.h without mcscofig.h
  (e.g. with my_config.h) is also caught by "#error".

- collation.h now can now be included in two ways.
  It now has the following effective structure:

    #if defined(PREFER_MY_CONFIG_H) && defined(MY_CONFIG_H)
    //  Remember current conflicting definitions on the preprocessor stack
    //  Undefine current conflicting definitions
    #endif
    #include "mcsconfig.h"
    #include "m_ctype.h"
    #if defined(PREFER_MY_CONFIG_H) && defined(MY_CONFIG_H)
    #    Restore conflicting definitions from the preprocessor stack
    #endif

  and can be included as follows:

  a. using only mcsconfig.h as a configuration header:

    // my_config.h must not be included so far
    #include "collation.h"

  b. using my_config.h as the first included configuration file:

    #define PREFER_MY_CONFIG_H // Force conflict resolution
    #include "my_config.h"     // can be included directly or indirectly
    ...
    #include "collation.h"

Other changes:

- Adding helper header files
     utils/common/mcsconfig_conflicting_defs_remember.h
     utils/common/mcsconfig_conflicting_defs_restore.h
     utils/common/mcsconfig_conflicting_defs_undef.h
  to perform conflict resolution easier.

- Removing `#include "collation.h"` from a number of files,
  as it's automatically included from rowgroup.h.

- Removing redundant `#include "utils_utf8.h"`.
  This change is not directly related to the problem being fixed,
  but it's nice to remove redundant directives for both collation.h
  and utils_utf8.h from all the files that do not really need them.
  (this change could probably have gone as a separate commit)

- Changing my_init() to MY_INIT(argv[0]) in the MCS services sources.
  After the fix of the complitation failure it appeared that ColumnStore
  services compiled with the debug build crash due to recent changes in
  safemalloc. The crash happened in strcmp() with `my_progname` as an argument
  (where my_progname is a mysys global variable). This problem should
  probably be fixed on the server side as well to avoid passing NULL.
  But, the majority of MariaDB executable programs also use MY_INIT(argv[0])
  rather than my_init(). So let's make MCS do like the other programs do.
2021-05-25 12:34:36 +04:00
846f7fb29b MCOL-4193: Delete unused OAM and applications, ProcMon, ProcMgr, and no longer build all tools for packages 2021-02-08 17:51:09 -06:00
2ea73846b9 MCOL-4422 Remove mariadb.h and my_sys.h dependency from collation.h 2020-11-30 14:26:35 +04:00
98b94c0280 Remove deprecated boost::timer
Change the only one actual timer to std::chrono::steady_clock
2020-11-17 15:03:10 +03:00
ab44ef6ddb MCOL-4170 Refactor services/systemd units to finish their bootstrap ... 2020-11-09 12:01:16 +04:00
ffd96d0315 A preparatory patch for MCOL-4170 Refactor services/systemd units to finish their bootstrap ...
Moving signal initialization code into separate functions.
2020-11-06 11:58:31 +04:00
7e868bc588 add dependency for generated header files errorids.h messageids.h patch made by Ben. 2020-07-10 18:38:49 +00:00
f9078efbc6 MCOL-3536 Collation 2020-06-08 17:57:37 -05:00
06e50e0926 MCOL-3536 collation 2020-05-26 12:42:11 -05:00
8479a87e46 Merge branch 'develop' into MCOL-3536 2020-05-18 16:22:01 -05:00
1f3d1e6fd6 MCOL-3536 collation 2020-05-14 16:02:49 -05:00
98abf95eae MCOL-3991 MCS is now single package and properly uninstalls 2020-05-12 13:36:24 +00:00
7489d0bfd0 MCOL-3625 Rename packages
Rename packages to MariaDB-columnstore-engine, MariaDB-columnstore-libs
and MariaDB-columnstore-platform.

Also add the "columnstore-" prefix the the components so that MariaDB's
packaging system understands then and add a line to include them in
MariaDB's packaging.

In addition
* Fix S3 building for dist source build
* Fix Debian 10 dependency issue
* Fix git handling for dist builds
* Add support for MariaDB's RPM building
* Use MariaDB's PCRE and readline
* Removes a few dead files
* Fix Boost noncopyable includes
2019-12-04 11:04:39 +00:00
3fef0f21d3 Remove vpj files
They shouldn't be here
2019-09-05 17:38:03 +01:00
5e4f1b9933 Merge branch 'develop' into MCOL-265 2019-06-10 13:58:03 +01:00
020b211bb7 Merge branch 'develop-1.2' into develop-merge-up-20190514 2019-05-14 13:58:33 +01:00
b2436502cb MCOL-537 Enabled -Wno-unused-result for OAM code.
Fixed pragmas that disables compilation checks.

    DDLProc now returns an error if it couldn't cwd.

    Use either auto_ptr or unique_ptr depending on GCC version.
2019-05-08 19:44:01 +03:00
7e2cb05624 MCOL-537 There are no CS-specific warnings building with gcc 8.2. 2019-05-07 16:00:05 +03:00
515b93cc3d remove faulty test code 2019-04-29 01:28:54 -05:00
4b9d046c6e Fully resolve potentially ambiguous symbols by removing using namespace statements from headers which have a cascading effect. This causes potential behavior changes when switching to c++11 since symbols can be exported from std and boost while both have been imported into the global namespace. 2019-04-29 01:21:15 -05:00
e65f80f493 delete visual c++ project files. cmake can generate these if needed 2019-04-25 23:35:03 -05:00
e89d1ac3cf MCOL-265 Add support for TIMESTAMP data type 2019-04-23 00:00:09 -04:00
101b36acb6 Merge branch 'develop' into MCOL-520 2018-10-08 14:23:48 -05:00
f1f13a09d1 Merge branch 'develop-1.1' into 1.1-merge-up-2018-10-05 2018-10-05 18:40:07 +01:00
9cfe199154 MCOL-520 2018-09-25 13:35:06 -05:00
4e6e5647ef MCOL-520 2018-09-21 10:40:05 -05:00
14d3a34c28 MCOL-1694 & MCOL-1505 Improved exception handling
This patch catches exceptions in DDLProc, DMLProc and ExeMgr which could
potentially happen during startup. Logging them instead of silently
ignoring them (or crashing in ExeMgr).
2018-09-07 11:43:54 +01:00
0d7c0f7ae4 Merge branch 'develop-1.1' into dev-merge-up-20180202 2018-02-02 14:53:36 +00:00
5d9f9c819b Merge branch 'develop-1.0' into 1.1-merge-up-20180118 2018-01-18 16:28:08 +00:00
c31c836352 MCOL-1085 Add crash dump to daemons
This patch adds an automated crash dump which logs in
/var/log/mariadb/columnstore/trace/ when one of the ColumnStore daemons
crashes.
2018-01-03 09:20:09 +00:00
01446d1e22 Reformat all code to coding standard 2017-10-26 17:18:17 +01:00
401e257df3 MCOL-879 Fix QueryStats linking issues
With 1.1 we have removed libdrizzle and used MariaDB's client library
instead for both CrossEngine and QueryStats. Unfortunately MariaDB 10.2
has two client libraries which have different structs with the same
name. When QueryStats was running inside the ColumnStore plugin this
symbol conflict was causing a crash.

The server's built-in client API has several different and several
missing functions so some additions to sm.cpp were made to fill the
gaps.

This patch does the following:

* Make sure that libmariadb is only linked to executables, not the
ColumnStore Plugin (to avoid symbol conflicts). Note that all
executables that link to CrossEngine and/or QueryStats need to link to
libmariadb to avoid missing symbol issues.
* Use the server's built-in client API for QueryStats when run in the
plugin
* Replace missing server built-in client API calls in sm.cpp (this is
for QueryStats and CrossEngine to keep the dynamic linker happy)
* Fixes issue where using 'localhost' as the MariaDB Server hostname
would fail in QueryStats.
2017-08-18 10:16:52 +01:00
3330495a2e MCOL-777 Cleanup source
Clean out autotools and some other things from the source tree.
2017-08-07 15:59:56 +01:00
bc2a4e7795 MCOL-523 Add UDAF and UDAnF SDK 2017-08-02 11:22:07 -05:00
87a679e6eb MCOL-513 use a single funcor per thread. ThreadPool was doing one at a time anyway, but in a convpluted way that made it easier to add more if wanted. But it was expensive. Cleanup and polish. 2017-02-17 09:46:42 -06:00
ffcfc41563 MCOL-507 Further ExeMgr performance improvements
This does the following:

* Switch resource manager to a singleton which reduces the amount of
times the XML data is scanned and objects allocated.
* Make the I_S tables use the FE implementation of the system catalog
* Make the I_S.columnstore_columns table use the RID list cache
* Make the extentmap pre-allocate a vector instead of many small allocs
2017-01-16 12:33:27 +00:00
3d91ac193c Merge branch 'develop' of git://github.com/mariadb-corporation/mariadb-columnstore-engine into MCOL-97 2016-09-22 18:50:59 -05:00
7d3cd26af9 MCOL-160 MCOL-262 CMake Build Fixes for out-of-tree builds and engine RPMS 2016-09-22 10:14:18 -04:00