This patch introduces INFORMATION_SCHEMA.COLUMNSTORE_PARTITIONS table to
query partitions of tables and columns. It has TABLE_SCHEMA, TABLE_NAME,
COLUMN_NAME and PARTITION_ID columns, all textual. PARTITION_ID encoding
is compatible with CALSHOWPARTITIONS and CALDROPPARTITIONS functions.
If two columns share partition, only one column will be listed in the
result set, but search for partitions of specific column will return
correct list of partitions for that column.
This patch adds flag to enable query accelerator by default and adds
three stored procedures to enable query accelerator, disable query
accelerator and execute just single query with query accelerator
enabled.
Here's how to use:
SET old_settings = queryacc.enable_queryacc();
SELECT ... FROM ...;
CALL queryacc.disable_queryacc(old_settings);
Example:
```sql
create table iftest(a int, b int) engine=columnstore;
insert into iftest values (1, null), (2, 1);
select if (b is null, 'b is null', 'b is not null') from iftest group by a;
```
Incorrect result:
```
ERROR 1815 (HY000): Internal error: MCS-2021: 'if (b is null, 'b is null', 'b is not null')' is not in GROUP BY clause. All non-aggregate columns in the SELECT and ORDER BY clause must be included in the GROUP BY clause.
```
Expected (as with InnoDB):
```
+----------------------------------------------+
| if (b is null, 'b is null', 'b is not null') |
+----------------------------------------------+
| b is null |
| b is not null |
+----------------------------------------------+
2 rows in set (0.000 sec)
```
* fix(cpimport): MCOL-6243 - cpimport fails at fields longer than 1M
This patch makes cpimport buffers resizeable, thus cpimport's
command-line buffer size argument is nothing but a hint now.
* No read past EOF
* MCOL-5983: Add COMPARE_BITMASK primitive filter for bitwise AND optimization
Add support for pushing down (col & mask) = mask patterns as primitive
filters instead of evaluating them in FE1 (Function Expression Group 1).
This optimization significantly improves query performance for Bloom filter
lookups commonly used in chemical/molecular databases where a signature
column stores precomputed bit patterns.
Changes:
- Add COMPARE_BITMASK (0x20) comparison operator to primitivemsg.h
- Implement scalar and SIMD handling for COMPARE_BITMASK in column.cpp
- Add pattern recognition for (col & mask) = mask in doSimpleFilter()
- Add cost-based filter reordering in walkTree() to prioritize cheap
bitmask filters before expensive LIKE filters
Performance results (21M rows, TEXT column with LIKE, BIGINT signature):
- Bitmask filter alone: 6s -> 0.23s (26x faster)
- LIKE alone: 8.2s (baseline, scans all rows)
- LIKE + bitmask combined: 8.2s -> 1.0s (8x faster)
The bitmask filter now executes as a primitive filter with SIMD
acceleration, filtering out non-matching rows before the expensive
dictionary lookup required for LIKE on TEXT columns.
* review comments addressed
The workernode process could crash if a signal (SIGINT, SIGTERM, SIGHUP)
was received during initialization. Signal handlers were registered
before the SlaveComm object was constructed, causing a null pointer
dereference when the handler tried to call comm->stop() or comm->reset().
The SlaveComm constructor can take significant time because it
initializes MasterSegmentTable which acquires RWLocks on shared memory.
If a signal arrived during this window, the process would crash.
Added null checks for the comm pointer in both stop() and reset()
signal handlers to prevent the crash.
ConstantFilter had its own fSimpleColumnListExtended member variable
that shadowed the base class Filter::fSimpleColumnListExtended.
When RBO called getSimpleColsExtended() to collect SimpleColumns for
rewriting, it used Filter::simpleColumnListExtended() which returned
the empty base class variable. Meanwhile, ConstantFilter::setSimpleColumnListExtended()
was populating its own shadowed variable.
This caused SimpleColumns inside ConstantFilter (used for IN predicates)
to not be updated during RBO parallel_ces rewriting, resulting in
incorrect query results where IN conditions were effectively ignored.
Fix: Remove the duplicate fSimpleColumnListExtended declaration from
ConstantFilter and use the inherited base class variable instead.
Also remove the redundant simpleColumnListExtended() method override.
Affected queries: TPC-H Q12, Q16, Q19 and any query using IN/NOT IN
conditions with RBO enabled.
Creating and dropping tables in a long-running Columnstore installation
were going to overflow used OIDs set, leading to semi-functional database.
Now DROP TABLE releases all OIDs back to reuse.
cloning but never freed, causing memory leaks detected by ASAN.
Refactored to use std::unique_ptr<ReturnedColumn> for RAII-style
memory management:
- rhs and lhs are now wrapped in unique_ptr upon extraction from stack
- For buildRowColumnFilter path: release() transfers ownership
- For ConstantFilter: lhs.release() transfers ownership to constructor
- rhs is automatically deleted after clone() via unique_ptr destructor
- Loop values wrapped in unique_ptr and auto-deleted after cloning
This follows the same RAII pattern already used in buildRowColumnFilter().
* fix(PP,THJS): MCOL-6083 fix THJS freezing in case of low memory
* fix TupleJoiner::copyForDiskJoin
* fix group_concat data deserialization
---------
Co-authored-by: Aleksei Antipovskii <alexey.antipovsky@gmail.com>
