in the query projection list.
With this patch, we enable SELECT/DERIVED handler execution for
queries involving 'get_user_var' function. Whereas for 'set_user_var'
function, the handlers are still disabled and the query execution
fallsback to the server, except for some edges case described in
the test file mcol-4410.test, in which case, an appropriate error
message is returned to the client.
Main theme of the patch is to fix joins processing in the plugin
code. We now use SELECT_LEX::top_join_list and process the nested
joins recursively, instead of SELECT_LEX::table_list struct which
we earlier used to build the join filters. The earlier approach
did not process certain nested join ON expressions, causing certain
queries to incorrectly error out such as that described in MCOL-4680.
In addition, some legacy code is also removed.
In this patch, we set the unioned type to a wide decimal, if any of the
numeric columns involved in the union operation have a precision > 18
(which is also possible for BIGINT/UBIGINT types) and <= 38.
We earlier leveraged the server functionality provided by
Item_in_subselect::create_in_to_exists_cond and
Item_in_subselect::inject_in_to_exists_cond
to create and inject the in-to-exists predicate into an IN
subquery's JOIN struct. With this patch, we leave the IN subquery's
JOIN unaltered and instead directly perform this predicate creation
and injection into ColumnStore's select execution plan.
This patch fixes:
- MCOL-4614 calShowPartitions() precision loss for huge narrow decimal
- MCOL-4615 GROUP_CONCAT() precision loss for huge narrow decimal
- MCOL-4660 Narow decimal to string conversion is inconsistent about zero integral
Changes:
- Implementing Row::getDecimalField()
- Removing double arithmetic from the code printing DECIMAL values
in TypeHandlerXDecimal::format64() and GroupConcator::outputRow().
Using Decimal::toString() instead.
- Rewriting Decimal::toStringTSInt64(). The old implementation
was wrong, too complex and slow (used unnecessary memmove, memcpy).
An additional cleanup:
- Removing the ENGINE=COLUMNSTORE clause from tests for MCOL-4532 and MCOL-4640
type_decimal.test is combinations-aware. It's run two times with
default_storage_engine=MyISAM and default_storage_engine=COLUMNSTORE.
So the CREATE TABLE statements should not specify the engine explicitly.
- Adding --disable_warnings in the old fixed test.
We needed to suppress warnings when the MyISAM combination is being run.
Previously the table was erroneously created with ENGINE=COLUMNSTORE
even with the MyISAM combination run. So warning were not generated.
Progress keep and test commit
Progress keep and test commit
Progress keep and test commit
Again, trying to pinpoint problematic part of a change
Revert "Again, trying to pinpoint problematic part of a change"
This reverts commit 71874e7c0d7e4eeed0c201b12d306b583c07b9e2.
Revert "Progress keep and test commit"
This reverts commit 63c7bc67ae55bdb81433ca58bbd239d6171a1031.
Revert "Progress keep and test commit"
This reverts commit 121c09febd78dacd37158caeab9ac70f65b493df.
Small steps - I walk minefield here
Propagating changes - now CPInfo in convertValArray
Progress keep commit
Restoring old functionality
Progress keep commit
Small steps to avoid/better locate old problem with the write engine.
Progress keeping commit
Thread the CPInfo up to convertValArray call in writeColumnRec
About to test changes - I should get no regression and no updates in
ranges either.
Testing out why I get a regression
Investigating source of regression
Debugging prints
Fix compile error
Debugging print - debug regression
I clearly see calls to writeColumnRec and prints there added to discern
between these.
Fix warning error
Possible culprit
Add forgotten default parameter for convertValArray
New logic to test
Max/min gets updated during value conversion
To test results of updates
Debug logs
Debug logs
An attempt to provide proper sequence index
Debug logs
An attempt to provide proper sequence index - now magic for resetting
Debug logs
Debug logs
Debug logs
Trying to perform correct updates
Trying to perform correct updates - seqNum woes fight
COMMIT after INSERT performs 'mark extent as invalid' operation - investigating
To test: cut setting of CPInfo upon commit from DML processor
It may be superfluous as write engine does that too
Debug logs
Debug logs
Better interface for CPMaxMin
Old interface forgot to set isBinaryColumn field
Possible fix for the problems
I forgot to reassign the value in cpinfoList
Debug logs
Computation of 'binary' column property
logs indicated that it was not set in getExtentCPMaxMin, and it was impossible to compute there so I had to move that into writeengine.
To test: code to allow cross-extent insertion
To test: removed another assertion for probable cause of errors
Debug logs
Dropped excessive logs
Better reset code
Again, trying to fix ordering
Fixing order of rowids for LBID computation
Debug logs
Remove update of second LBID in split insert
I have to validate incorrect behaviour for this test
Restoring the case where everything almost worked
Tracking changes in newly created extents
Progress keeping commit
Fixing build errors with recent server
An ability to get old values from blocks we update
Progress keeping commit
Adding analysis of old values to write engine code.
It is needed for updates and deletes.
Progress keeping commit
Moving max/min range update from convertValArray into separate function with simpler logic.
To test and debug - logic is there
Fix build errors
Update logic to debug
There is a suspicious write engine method updateColumnRecs which
receives a vector of column types but does not iterate over them
(otherwise it will be identical to updateColumnRec in logic).
Other than that, the updateColumnRec looks like the center of all
updates - deleteRow calls it, for example, dml processor also calls it.
Debug logs for insert bookkeeping regression
Set up operation type in externally-callable interface
Internal operations depend on the operation type and consistency is what matters there.
Debug logs
Fix for extent range update failure during update operation
Fix build error
Debug logs
Fix for update on deletion
I am not completely sure in it - to debug.
Debug log
writeColumnRec cannot set m_opType to UPDATE unconditionally
It is called from deleteRow
Better diagnostics
Debug logs
Fixed search condition
Debug logs
Debugging invalid LBID appearance
Debug logs - fixed condition
Fix problems with std::vector reallocation during growth
Fix growing std::vector data dangling access error
Still fixing indexing errors
Make in-range update to work
Correct sequence numbers
Debug logs
Debug logs
Remove range drop from DML part of write engine
A hack to test the culprit of range non-keeping
Tests - no results for now
MTR-style comments
Empty test results
To be filled with actual results.
Special database and result selects for all tests
Pleasing MTR with better folder name
Pleasing MTR - testing test result comparison
Pleasing MTR by disabling warnings
All test results
Cleaning up result files
Reset ranges before update
Remove comments from results - point of failure in MTR
Remove empty line from result - another MTR failure point
Probably fix for deletes
Possible fix for remaining failed delete test
Fix a bug in writeRows
It should not affect delete-with-range test case, yet it is a bug.
Debug logs
Debug logs
Tests reorganization and description
Support for unsigned integer for new tests
Fix type omission
Fix test failure due to warnings on clean installation
Support for bigint to test
Fix for failed signed bigint test
Set proper unsignedness flag
Removed that assignment during refactoring.
Tests for types with column width 1 and 2
Support for types in new tests
Remove trailing empty lines from results
Tests had failed because of extra empty lines.
Remove debug logs
Update README with info about new tests
Move tests for easier testing
Add task tag to tests
Fix invalid unsaigned range check
Fix for signed types
Fix regressions - progress keeping commit
Do not set invalid ranges into valid state
A possible fix for mcs81_self_join test
MCOL 2044 test database cleanup
Missing expected results
Delete extraneous assignment to m_opType
nullptr instead of NULL
Refactor extended CPInfo with TypeHandler
Better handling of ranges - safer types, less copy-paste
Fix logic error related to typo
Fix logic error related to typo
Trying to figure out why invalid ranges aren't displayed as NULL..NULL
Debug logs
Debug logs
Debug logs
Debug logs for worker node
Debug logs for worker node in extent map
Debugging virtual table fill operation
Debugging virtual table fill operation
Fix for invalid range computation
Remove debug logs
Change handling of invalid ranges
They are also set, but to invalid state.
Complete change
Fix typo
Remove unused code
"Fix" for tests - -1..0 instead of NULL..NULL for invalid unsigned ranges
Not a good change, yet I cannot do better for now.
MTR output requires tabs instead of spaces
Debug logs
Debug logs
Debug logs - fix build
Debug logs and logic error fix
Fix for clearly incorrect firstLBID in CPInfo being set - to test
Fix for system catalog operations suppot
Better interface to fix build errors
Delete tests we cannot satisfy due to extent rescan due to WHERE
Tests for wide decimals
Testing support for wide decimals
Fix for wide decimals tests
Fix for delete within range
Memory leak fix and, possible, double free fix
Dispatch on CalpontSystemCatalog::ColDataType is more robust
Add support for forgotten MEDINT type
Add forgottent BIGINT
empty() instead of size() > 0
Better layout
Remove confusing comment
Sensible names for special values of seqNum field
Tests for wide decimal support
Addressing concerns of drrtuy
Remove test we cannot satisfy
Final touches for PR
Remove unused result file
This patch is fixing the following bugs:
- MCOL-4609 TreeNode::getIntVal() does not round: implicit DECIMAL->INT cast is not MariaDB compatible
- MCOL-4610 TreeNode::getUintVal() looses precision for narrow decimal
- MCOL-4619 TreeNode::getUintVal() does not round: Implicit DECIMAL->UINT conversion is not like in InnoDB
- MCOL-4650 TreeNode::getIntVal() looses precision for narrow decimal
- MCOL-4651 SEC_TO_TIME(hugePositiveDecimal) returns a negative time
For a query of the form:
SELECT COUNT(c2) FROM (SELECT * FROM t1) q;
where t1 contains 10 columns c1, c2, ... , c10.
We currently create an intermediate RowGroup in ExeMgr with
a row of the form (1, c2_value1, 1, 1, 1, 1, 1, 1, 1, 1), i.e.
for all the columns of the subquery which are not referenced in
the outer query, we substitute a constant value, which is wasteful.
With this optimization, we are trimming the RowGroup to a row
of the form (1, c2_value1). This can have non-trivial query
execution time improvements if the subquery contains large number
of columns (such as a "select *" on a very wide table) and the outer
query is only referencing a subset of these columns with lower
index values from the subquery (as an example, c1 or c2 above).
That is, the current limitation of this optimization is we are not
removing those non-referenced subquery columns (c1 in the query above)
which are to the left of a referenced column.
After the cleanup work done for FETCH FIRST ... WITH TIES
SELECT_LEX members select_limit, explicit_limit and offset_limit are now moved
to SELECT_LEX::limit_params.
type is not LEFT/RIGHT.
In buildOuterJoin(), do not add ON expressions for WHERE
processing when the JOIN type is not LEFT/RIGHT.
Test cases to check correct processing of INNER JOIN ON expressions
with possible/impossible WHERE conditions are added for
1. One side of the LEFT JOIN being INNER JOIN.
2. One side of the LEFT JOIN being an INNER JOIN inside an INNER JOIN.
3. Both sides of the LEFT JOIN being an INNER JOIN.
An INSERT .. SELECT performed on a master node caused an infinite
loop on the slave node with columnstore_replication_slave=OFF.
The issue was ha_mcs_impl_select_next() was returning 0 if
the executing thread is a slave and returning early to avoid DML
execution on the slave. The calling code in select_handler::execute()
ran into an infinite loop due to this as it checked the return code
of the function as 0, and incorrectly thought there are more rows
to process.
The fix is to return HA_ERR_END_OF_FILE as the return value,
instead of 0. This is for the case of the query running on the
slave node with columnstore_replication_slave=OFF.
