1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-04-20 09:07:44 +03:00

42 Commits

Author SHA1 Message Date
Gagan Goel
ab7dfaa25b Fix resource leak in DDLProc/DMLProc/PrimProc/WriteengineServer processes.
As part of the charset support, a call to MY_INIT() was added at the
initialization of the above processes. This call initializes the MySQL
thread environment required by the charset library. However, the
accompanying my_end() call required to terminate this thread environment
was not added at the termination of these process, hence leaking
resources. As a fix, we move the MY_INIT() calls to the Child()
functions of these services and also add the missing my_end() call.
2023-06-23 20:18:32 +00:00
Leonid Fedorov
c2d0fa24da replace boost::shared_array<T> to std::shared_ptr<T[]> 2023-04-14 10:33:27 +00:00
Leonid Fedorov
56f2346083 Remove windows ifdefs 2023-03-02 15:59:42 +00:00
Leonid Fedorov
3919c541ac
New warnfixes (#2254)
* Fix clang warnings

* Remove vim tab guides

* initialize variables

* 'strncpy' output truncated before terminating nul copying as many bytes from a string as its length

* Fix ISO C++17 does not allow 'register' storage class specifier for outdated bison

* chars are unsigned on ARM, having  if (ival < 0) always false

* chars are unsigned by default on ARM and comparison with -1 if always true
2022-02-17 13:08:58 +03:00
Leonid Fedorov
04752ec546 clang format apply 2022-01-21 16:43:49 +00:00
Sergey Zefirov
9e0851e4cf MCOL-4766 ROLLBACK kept ranges changed inside rolled back transaction
Now ROLLBACK drops ranges to INVALID state which makes engine to rescan
blocks and discover correct ranges.
2021-07-07 18:16:56 +03:00
Leonid Fedorov
d08eff75ba Added _exit in Child dmlproc service 2021-05-28 10:09:35 +00:00
Roman Nozdrin
90397dfed0 MCOL-4675 DMLProc now automatically and gracefully shutdowns when a cluster state is set to
SS_SHUTDOWN_PENDING | SS_ROLLBACK
2021-05-27 11:07:32 +00:00
Alexander Barkov
9608533d92 MCOL-4734 Compilation failure: MariaDB-10.6 + ColumnStore-develop
mcsconfig.h and my_config.h have the following
pre-processor definitions:

1. Conflicting definitions coming from the standard cmake definitions:
- PACKAGE
- PACKAGE_BUGREPORT
- PACKAGE_NAME
- PACKAGE_STRING
- PACKAGE_TARNAME
- PACKAGE_VERSION
- VERSION

2. Conflicting definitions of other kinds:
- HAVE_STRTOLL - this is a dirt in MariaDB headers.
  Should be fixed in the server code. my_config.h erroneously
  performs "#define HAVE_STRTOLL" instead of "#define HAVE_STRTOLL 1".
  in some cases. The former is not CMake compatible style. The latter is.

3. Non-conflicting definitions:
  Otherwise, mcsconfig.h and my_config.h should be mutually compatible,
  because both are generated by cmake on the same host machine. So
  they should have exactly equal definitions like "HAVE_XXX", "SIZEOF_XXX", etc.

Observations:
- It's OK to include both mcsconfig.h and my_config.h providing that we
  suppress duplicate definition of the above conflicting types #1 and #2.
- There is no a need to suppress duplicate definitions mentioned in #3,
  as they are compatible!
- my_sys.h and m_ctype.h must always follow a CMake configuation header,
  either my_config.h or mcsconfig.h (or both).
  They must never be included without any preceeding configuration header.

This change make sure that we resolve conflicts by:
- either disallowing inclusion of mcsconfig.h and my_config.h
  at the same time
- or by hiding conflicting definitions #1 and #2
  (with their later restoring).
- also, by making sure that my_sys.h and m_ctype.h always follow
  a CMake configuration file.

Details:
- idb_mysql.h can now only be included only after my_config.h
  An attempt to use idb_mysql.h with mcsconfig.h instead of
  my_config.h is caught by the "#error" preprocessor directive.

- mariadb_my_sys.h can now be only included after mcsconfig.h.
  An attempt to use mariadb_my_sys.h without mcscofig.h
  (e.g. with my_config.h) is also caught by "#error".

- collation.h now can now be included in two ways.
  It now has the following effective structure:

    #if defined(PREFER_MY_CONFIG_H) && defined(MY_CONFIG_H)
    //  Remember current conflicting definitions on the preprocessor stack
    //  Undefine current conflicting definitions
    #endif
    #include "mcsconfig.h"
    #include "m_ctype.h"
    #if defined(PREFER_MY_CONFIG_H) && defined(MY_CONFIG_H)
    #    Restore conflicting definitions from the preprocessor stack
    #endif

  and can be included as follows:

  a. using only mcsconfig.h as a configuration header:

    // my_config.h must not be included so far
    #include "collation.h"

  b. using my_config.h as the first included configuration file:

    #define PREFER_MY_CONFIG_H // Force conflict resolution
    #include "my_config.h"     // can be included directly or indirectly
    ...
    #include "collation.h"

Other changes:

- Adding helper header files
     utils/common/mcsconfig_conflicting_defs_remember.h
     utils/common/mcsconfig_conflicting_defs_restore.h
     utils/common/mcsconfig_conflicting_defs_undef.h
  to perform conflict resolution easier.

- Removing `#include "collation.h"` from a number of files,
  as it's automatically included from rowgroup.h.

- Removing redundant `#include "utils_utf8.h"`.
  This change is not directly related to the problem being fixed,
  but it's nice to remove redundant directives for both collation.h
  and utils_utf8.h from all the files that do not really need them.
  (this change could probably have gone as a separate commit)

- Changing my_init() to MY_INIT(argv[0]) in the MCS services sources.
  After the fix of the complitation failure it appeared that ColumnStore
  services compiled with the debug build crash due to recent changes in
  safemalloc. The crash happened in strcmp() with `my_progname` as an argument
  (where my_progname is a mysys global variable). This problem should
  probably be fixed on the server side as well to avoid passing NULL.
  But, the majority of MariaDB executable programs also use MY_INIT(argv[0])
  rather than my_init(). So let's make MCS do like the other programs do.
2021-05-25 12:34:36 +04:00
benthompson15
846f7fb29b MCOL-4193: Delete unused OAM and applications, ProcMon, ProcMgr, and no longer build all tools for packages 2021-02-08 17:51:09 -06:00
Alexander Barkov
2ea73846b9 MCOL-4422 Remove mariadb.h and my_sys.h dependency from collation.h 2020-11-30 14:26:35 +04:00
Alexey Antipovsky
98b94c0280 Remove deprecated boost::timer
Change the only one actual timer to std::chrono::steady_clock
2020-11-17 15:03:10 +03:00
Alexander Barkov
ab44ef6ddb MCOL-4170 Refactor services/systemd units to finish their bootstrap ... 2020-11-09 12:01:16 +04:00
Alexander Barkov
ffd96d0315 A preparatory patch for MCOL-4170 Refactor services/systemd units to finish their bootstrap ...
Moving signal initialization code into separate functions.
2020-11-06 11:58:31 +04:00
David Hall
f9078efbc6 MCOL-3536 Collation 2020-06-08 17:57:37 -05:00
David Hall
06e50e0926 MCOL-3536 collation 2020-05-26 12:42:11 -05:00
David Hall
1f3d1e6fd6 MCOL-3536 collation 2020-05-14 16:02:49 -05:00
Andrew Hutchings
83a8924229 Fix namespace and class changes 2019-07-29 13:48:28 +01:00
Andrew Hutchings
811909aa72 Merge branch 'develop-1.2' into develop-merge-up-20190729 2019-07-29 12:19:26 +01:00
Andrew Hutchings
ea2ff9cd11 Merge branch 'develop-1.1' into develop-1.2-merge-up-20190517 2019-05-17 20:43:49 +01:00
Andrew Hutchings
020b211bb7 Merge branch 'develop-1.2' into develop-merge-up-20190514 2019-05-14 13:58:33 +01:00
David Hall
f2d8a33ee5 MCOL-3296 CTRL+C should not double remove from dequeue. 2019-05-09 12:42:18 -05:00
Roman Nozdrin
3c89a4bba4 MCOL-537 Preprocessor if blocks with pragmas now have else branch.
DMLProc exits on setupCwd failure.
2019-05-09 20:25:21 +03:00
David Mott
515b93cc3d remove faulty test code 2019-04-29 01:28:54 -05:00
David Mott
4b9d046c6e Fully resolve potentially ambiguous symbols by removing using namespace statements from headers which have a cascading effect. This causes potential behavior changes when switching to c++11 since symbols can be exported from std and boost while both have been imported into the global namespace. 2019-04-29 01:21:15 -05:00
david hill
101b36acb6
Merge branch 'develop' into MCOL-520 2018-10-08 14:23:48 -05:00
Andrew Hutchings
f1f13a09d1 Merge branch 'develop-1.1' into 1.1-merge-up-2018-10-05 2018-10-05 18:40:07 +01:00
Patrick LeBlanc
4215a47b5c MCOL-520. More untested changes.
Went down the list in shmkeys.h, added sane error msgs on perms
errors accessing rest of the shmsegs there.
Realized we probably do need 666 to the shmsegs.
Also deleted a little dead code that was copy-pasted everywhere.
2018-10-04 14:25:01 -05:00
David Hill
4e6e5647ef MCOL-520 2018-09-21 10:40:05 -05:00
Andrew Hutchings
14d3a34c28 MCOL-1694 & MCOL-1505 Improved exception handling
This patch catches exceptions in DDLProc, DMLProc and ExeMgr which could
potentially happen during startup. Logging them instead of silently
ignoring them (or crashing in ExeMgr).
2018-09-07 11:43:54 +01:00
david hill
0ea35564f0 MCOL-520 - remove sudo command 2018-04-12 10:49:09 -05:00
Andrew Hutchings
0d7c0f7ae4 Merge branch 'develop-1.1' into dev-merge-up-20180202 2018-02-02 14:53:36 +00:00
Andrew Hutchings
5d9f9c819b Merge branch 'develop-1.0' into 1.1-merge-up-20180118 2018-01-18 16:28:08 +00:00
David Hall
9749d825b5 MCOL-1165 use the threadpool's idle down feature 2018-01-16 13:26:44 -06:00
David Hall
504fcf0778 MCOL-1128 Allow exeMgrThreadPool to use unlimited threads. The number will be limited by the number of sessions allowed in server. 2018-01-12 16:29:01 -06:00
Andrew Hutchings
c31c836352 MCOL-1085 Add crash dump to daemons
This patch adds an automated crash dump which logs in
/var/log/mariadb/columnstore/trace/ when one of the ColumnStore daemons
crashes.
2018-01-03 09:20:09 +00:00
Andrew Hutchings
01446d1e22 Reformat all code to coding standard 2017-10-26 17:18:17 +01:00
David Hall
87a679e6eb MCOL-513 use a single funcor per thread. ThreadPool was doing one at a time anyway, but in a convpluted way that made it easier to add more if wanted. But it was expensive. Cleanup and polish. 2017-02-17 09:46:42 -06:00
Andrew Hutchings
ffcfc41563 MCOL-507 Further ExeMgr performance improvements
This does the following:

* Switch resource manager to a singleton which reduces the amount of
times the XML data is scanned and objects allocated.
* Make the I_S tables use the FE implementation of the system catalog
* Make the I_S.columnstore_columns table use the RID list cache
* Make the extentmap pre-allocate a vector instead of many small allocs
2017-01-16 12:33:27 +00:00
david hill
897d8d3f3d change snmp to alarm 2016-10-10 17:46:24 -05:00
David Hall
020c0ed3f5 MCOL-140 Add a mechanism to serialize transactions for a single table to prevent VSS clashes. Transactions for different tables will continue concurrently. 2016-07-28 09:25:49 -05:00
david hill
f6afc42dd0 the begginning 2016-01-06 14:08:59 -06:00