It was a dead code. It was not even a part of the soures in CMakeList.txt.
The bit AND operator implemententation resides in func_bitwise.cpp
together with all other bit operators and functions.
This patch fixes:
- MCOL-4614 calShowPartitions() precision loss for huge narrow decimal
- MCOL-4615 GROUP_CONCAT() precision loss for huge narrow decimal
- MCOL-4660 Narow decimal to string conversion is inconsistent about zero integral
Changes:
- Implementing Row::getDecimalField()
- Removing double arithmetic from the code printing DECIMAL values
in TypeHandlerXDecimal::format64() and GroupConcator::outputRow().
Using Decimal::toString() instead.
- Rewriting Decimal::toStringTSInt64(). The old implementation
was wrong, too complex and slow (used unnecessary memmove, memcpy).
An additional cleanup:
- Removing the ENGINE=COLUMNSTORE clause from tests for MCOL-4532 and MCOL-4640
type_decimal.test is combinations-aware. It's run two times with
default_storage_engine=MyISAM and default_storage_engine=COLUMNSTORE.
So the CREATE TABLE statements should not specify the engine explicitly.
- Adding --disable_warnings in the old fixed test.
We needed to suppress warnings when the MyISAM combination is being run.
Previously the table was erroneously created with ENGINE=COLUMNSTORE
even with the MyISAM combination run. So warning were not generated.
Progress keep and test commit
Progress keep and test commit
Progress keep and test commit
Again, trying to pinpoint problematic part of a change
Revert "Again, trying to pinpoint problematic part of a change"
This reverts commit 71874e7c0d7e4eeed0c201b12d306b583c07b9e2.
Revert "Progress keep and test commit"
This reverts commit 63c7bc67ae55bdb81433ca58bbd239d6171a1031.
Revert "Progress keep and test commit"
This reverts commit 121c09febd78dacd37158caeab9ac70f65b493df.
Small steps - I walk minefield here
Propagating changes - now CPInfo in convertValArray
Progress keep commit
Restoring old functionality
Progress keep commit
Small steps to avoid/better locate old problem with the write engine.
Progress keeping commit
Thread the CPInfo up to convertValArray call in writeColumnRec
About to test changes - I should get no regression and no updates in
ranges either.
Testing out why I get a regression
Investigating source of regression
Debugging prints
Fix compile error
Debugging print - debug regression
I clearly see calls to writeColumnRec and prints there added to discern
between these.
Fix warning error
Possible culprit
Add forgotten default parameter for convertValArray
New logic to test
Max/min gets updated during value conversion
To test results of updates
Debug logs
Debug logs
An attempt to provide proper sequence index
Debug logs
An attempt to provide proper sequence index - now magic for resetting
Debug logs
Debug logs
Debug logs
Trying to perform correct updates
Trying to perform correct updates - seqNum woes fight
COMMIT after INSERT performs 'mark extent as invalid' operation - investigating
To test: cut setting of CPInfo upon commit from DML processor
It may be superfluous as write engine does that too
Debug logs
Debug logs
Better interface for CPMaxMin
Old interface forgot to set isBinaryColumn field
Possible fix for the problems
I forgot to reassign the value in cpinfoList
Debug logs
Computation of 'binary' column property
logs indicated that it was not set in getExtentCPMaxMin, and it was impossible to compute there so I had to move that into writeengine.
To test: code to allow cross-extent insertion
To test: removed another assertion for probable cause of errors
Debug logs
Dropped excessive logs
Better reset code
Again, trying to fix ordering
Fixing order of rowids for LBID computation
Debug logs
Remove update of second LBID in split insert
I have to validate incorrect behaviour for this test
Restoring the case where everything almost worked
Tracking changes in newly created extents
Progress keeping commit
Fixing build errors with recent server
An ability to get old values from blocks we update
Progress keeping commit
Adding analysis of old values to write engine code.
It is needed for updates and deletes.
Progress keeping commit
Moving max/min range update from convertValArray into separate function with simpler logic.
To test and debug - logic is there
Fix build errors
Update logic to debug
There is a suspicious write engine method updateColumnRecs which
receives a vector of column types but does not iterate over them
(otherwise it will be identical to updateColumnRec in logic).
Other than that, the updateColumnRec looks like the center of all
updates - deleteRow calls it, for example, dml processor also calls it.
Debug logs for insert bookkeeping regression
Set up operation type in externally-callable interface
Internal operations depend on the operation type and consistency is what matters there.
Debug logs
Fix for extent range update failure during update operation
Fix build error
Debug logs
Fix for update on deletion
I am not completely sure in it - to debug.
Debug log
writeColumnRec cannot set m_opType to UPDATE unconditionally
It is called from deleteRow
Better diagnostics
Debug logs
Fixed search condition
Debug logs
Debugging invalid LBID appearance
Debug logs - fixed condition
Fix problems with std::vector reallocation during growth
Fix growing std::vector data dangling access error
Still fixing indexing errors
Make in-range update to work
Correct sequence numbers
Debug logs
Debug logs
Remove range drop from DML part of write engine
A hack to test the culprit of range non-keeping
Tests - no results for now
MTR-style comments
Empty test results
To be filled with actual results.
Special database and result selects for all tests
Pleasing MTR with better folder name
Pleasing MTR - testing test result comparison
Pleasing MTR by disabling warnings
All test results
Cleaning up result files
Reset ranges before update
Remove comments from results - point of failure in MTR
Remove empty line from result - another MTR failure point
Probably fix for deletes
Possible fix for remaining failed delete test
Fix a bug in writeRows
It should not affect delete-with-range test case, yet it is a bug.
Debug logs
Debug logs
Tests reorganization and description
Support for unsigned integer for new tests
Fix type omission
Fix test failure due to warnings on clean installation
Support for bigint to test
Fix for failed signed bigint test
Set proper unsignedness flag
Removed that assignment during refactoring.
Tests for types with column width 1 and 2
Support for types in new tests
Remove trailing empty lines from results
Tests had failed because of extra empty lines.
Remove debug logs
Update README with info about new tests
Move tests for easier testing
Add task tag to tests
Fix invalid unsaigned range check
Fix for signed types
Fix regressions - progress keeping commit
Do not set invalid ranges into valid state
A possible fix for mcs81_self_join test
MCOL 2044 test database cleanup
Missing expected results
Delete extraneous assignment to m_opType
nullptr instead of NULL
Refactor extended CPInfo with TypeHandler
Better handling of ranges - safer types, less copy-paste
Fix logic error related to typo
Fix logic error related to typo
Trying to figure out why invalid ranges aren't displayed as NULL..NULL
Debug logs
Debug logs
Debug logs
Debug logs for worker node
Debug logs for worker node in extent map
Debugging virtual table fill operation
Debugging virtual table fill operation
Fix for invalid range computation
Remove debug logs
Change handling of invalid ranges
They are also set, but to invalid state.
Complete change
Fix typo
Remove unused code
"Fix" for tests - -1..0 instead of NULL..NULL for invalid unsigned ranges
Not a good change, yet I cannot do better for now.
MTR output requires tabs instead of spaces
Debug logs
Debug logs
Debug logs - fix build
Debug logs and logic error fix
Fix for clearly incorrect firstLBID in CPInfo being set - to test
Fix for system catalog operations suppot
Better interface to fix build errors
Delete tests we cannot satisfy due to extent rescan due to WHERE
Tests for wide decimals
Testing support for wide decimals
Fix for wide decimals tests
Fix for delete within range
Memory leak fix and, possible, double free fix
Dispatch on CalpontSystemCatalog::ColDataType is more robust
Add support for forgotten MEDINT type
Add forgottent BIGINT
empty() instead of size() > 0
Better layout
Remove confusing comment
Sensible names for special values of seqNum field
Tests for wide decimal support
Addressing concerns of drrtuy
Remove test we cannot satisfy
Final touches for PR
Remove unused result file
Bugs fixed:
- MCOL-4649 CAST(double AS UNSIGNED) returns 0
- MCOL-4631 CAST(double AS SIGNED) returns 0 or NULL
- MCOL-4647 SEC_TO_TIME(double_or_float) returns a wrong result
Problems:
- The code in Func_cast_unsigned::getUintVal() and
Func_cast_signed::getIntVal() did not properly check
the double value to fit inside a uint64_t/int64_t range.
So the corner cases:
- numeric_limits<uint64_t>::max()-2 for uint64_t
- numeric_limits<int64_t>::max() for int64_t
produced unexpected results.
The problem was in tests like this:
if (value > (double) numeric_limits<int64_t>::max())
A correct test would be:
if (value >= (double) numeric_limits<int64_t>::max())
- The code in Func_sec_to_time::getStrVal() searched for the decimal
dot character, assuming that the next character after the dot
was the leftmost fractional digit.
This assumption was wrong because huge double numbers use
scientific notation. So for example in "2.5e-40" the
digit "5" following the dot is NOT the leftmost fractional digit.
Also, the code in Func_sec_to_time::getStrVal() was slow
because of using non necessary to-string and from-string
data conversion.
Also, the code in Func_sec_to_time::getStrVal() evaluated
the argument two times: using getStrVal() then using getIntVal().
Solution:
- Adding new classes TDouble and TLongDouble.
- Adding a few function templates to reuse the code easier.
- Moving the conversion code inside TDouble and TLongDouble
methods toMCSSInt64Round() and toMCSUInt64Round().
- Reusing new classes and their methods in func_cast.cc and
func_sec_to_time.cc.
* This patch adds rebuildEM tool support to work with compressed files.
* This patch increases a version of the file header.
Note: Default version of the `rebuildEM` tool was using very old API,
those functions are not present currently. So `rebuildEM` will not work with
files created without compression, because we cannot deduce some info which are
needed to create column extent.
- MCOL-4618 FLOOR(-9999.0) returns a bad result
- MCOL-4653 CEIL(negativeNarrowDecimal) returns a wrong result
Main changes:
a. Moving ROUND, CEIL, FLOOR related code into a new simple
class template DecomposedDecimal, which is reused for
64 and 128 bit decimal.
b. Using DecomposedDecimal in TDecimal64 and TDecimal128
to implement ROUND, CEIL, FLOOR related methods.
c. Adding corresponding wrapper methods to the class Decimal.
d. Using new Decimal methods in Func_ceil and Func_floor.
Additional minor changed:
- Adding "explicit" to TSInt128 constructors to avoid
hidden data type conversion and erroneous choice between
64 vs 128 bit APIs when using Decimal.
Now one can call constructors in this self explanatory way:
- Decimal(TSInt128(some_int_value), scale, precision)
to create a wide decimal
- Decimal(TSInt64(some_int_value, scale, precision)
to create a narrow decimal
TODO:
Consider changing
Decimal(int64_t val, int8_t s, uint8_t p, const int128_t &val128 = 0)
to
Decimal(int64_t val, int8_t s, uint8_t p, const int128_t &val128)
(or even removing this constructor) to disallow compilation of:
Decimal(some_trivial_type_value, scale, precision)
This patch is fixing the following bugs:
- MCOL-4609 TreeNode::getIntVal() does not round: implicit DECIMAL->INT cast is not MariaDB compatible
- MCOL-4610 TreeNode::getUintVal() looses precision for narrow decimal
- MCOL-4619 TreeNode::getUintVal() does not round: Implicit DECIMAL->UINT conversion is not like in InnoDB
- MCOL-4650 TreeNode::getIntVal() looses precision for narrow decimal
- MCOL-4651 SEC_TO_TIME(hugePositiveDecimal) returns a negative time
The "SIGNED" part of the problem was previously fixed by MCOL-4640.
Fixing the "UNSIGNED" part.
- Adding TDecimal64::toUInt64Round() and Decimal::decimal64ToUInt64Round()
- Renaming Decimal::narrowRound() to decimal64ToSInt64Round(),
for a more self-descriptive name, and for symmetry with decimal64ToUInt64Round()
- Reusing TDecimal64::toSInt64Round() inside decimal64ToSInt64Round().
This change was forgotten in MCOL-4640 :(
- Removing the old code in Func_cast_unsigned::getUintVal with pow().
It caused precision loss, hence the bug. Adding a call for
Decimal::decimal64ToUInt64Round() instead.
- Adding tests for both SIGNED and UNSIGNED casts.
Additional change:
- Moving the wide-decimal-to-uint64_t rounding code from Func_cast_unsigned::getUintVal()
to TDecimal128::toUInt64Round() (with refactoring). Adding TDecimal::toUInt64Round()
for symmetry with TDecimal::toSInt64Round(). It will be easier to reuse the code
with way.
For a query of the form:
SELECT COUNT(c2) FROM (SELECT * FROM t1) q;
where t1 contains 10 columns c1, c2, ... , c10.
We currently create an intermediate RowGroup in ExeMgr with
a row of the form (1, c2_value1, 1, 1, 1, 1, 1, 1, 1, 1), i.e.
for all the columns of the subquery which are not referenced in
the outer query, we substitute a constant value, which is wasteful.
With this optimization, we are trimming the RowGroup to a row
of the form (1, c2_value1). This can have non-trivial query
execution time improvements if the subquery contains large number
of columns (such as a "select *" on a very wide table) and the outer
query is only referencing a subset of these columns with lower
index values from the subquery (as an example, c1 or c2 above).
That is, the current limitation of this optimization is we are not
removing those non-referenced subquery columns (c1 in the query above)
which are to the left of a referenced column.
After the cleanup work done for FETCH FIRST ... WITH TIES
SELECT_LEX members select_limit, explicit_limit and offset_limit are now moved
to SELECT_LEX::limit_params.
