1
0
mirror of https://github.com/mariadb-corporation/mariadb-columnstore-engine.git synced 2025-08-01 06:46:55 +03:00
Commit Graph

8 Commits

Author SHA1 Message Date
1be82f859b Randomly start a new generation if the free memory is less than 30% 2022-08-23 17:59:52 +00:00
5f485f40ca MCOL-5153 This patch replaces MDB collation aware hash function with the (#2487)
exact functionality that does not use MDB hash function.
This patch also takes a bit from Robin Hood hash map implementation forgotten
that reduces hash function collision rate.
2022-08-04 16:22:11 -05:00
eabca67c8d MCOL-5153 This increases the size of the multiplier in the guarding
check in RowAggStorage::increaseSize() so that it doesn't throw w/o a
reason
2022-07-07 08:58:05 +00:00
7c808317dc clang format apply 2022-02-11 12:24:40 +00:00
2328f4ef2a [MCOL-4829] More accurate memory counting 2021-09-07 19:48:53 +03:00
bf1640be65 [MCOL-4829] Compression for the temp disk-based aggregation files 2021-09-02 19:31:38 +03:00
ef09342d47 MCOL-4839: Fix clang build (#2102)
* Fix clang build

* Extern C returned to plugin_instance

Co-authored-by: Leonid Fedorov <l.fedorov@mail.corp.ru>
2021-08-23 15:58:56 -05:00
475104e4d3 [MCOL-4709] Disk-based aggregation
* Introduce multigeneration aggregation

* Do not save unused part of RGDatas to disk
* Add IO error explanation (strerror)

* Reduce memory usage while aggregating
* introduce in-memory generations to better memory utilization

* Try to limit the qty of buckets at a low limit

* Refactor disk aggregation a bit
* pass calculated hash into RowAggregation
* try to keep some RGData with free space in memory

* do not dump more than half of rowgroups to disk if generations are
  allowed, instead start a new generation
* for each thread shift the first processed bucket at each iteration,
  so the generations start more evenly

* Unify temp data location

* Explicitly create temp subdirectories
  whether disk aggregation/join are enabled or not
2021-06-06 16:09:15 +03:00