Squashed commit of the following:
commit faaee9141a
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date: Wed Oct 3 13:03:02 2018 -0500
Commented a debugging printout.
commit e47e784c53
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date: Wed Oct 3 12:13:23 2018 -0500
Moved the SKIP_OAM_INIT check for cleanliness, added 'config.h' to
other places that need it.
commit 6626045535
Author: Patrick LeBlanc <patrick.leblanc@mariadb.com>
Date: Tue Oct 2 16:33:06 2018 -0500
WIP. First cut of excising OAM and root access from the dev process.
This passes most tests; need to make sure that the tests that fail
have nothing to do with this change.
DMLProc is doing something that results in a sudo password prompt.
Will obliterate that next.
This adds options which are user enabled to debug the LRU cache inside
ColumnStore. Specifically cache flushing.
It adds the following:
* PrimProc flush information when SIGUSR2 mode is enabled
* cpimport dictionary flush information when -d2 is used
* WriteEngineServer DML flush information to STDERR
Fixes the following:
* Generate error if calculateRowId fails
* No data written when first extent is completely full on a write, all
data going to second extent.
* 0 byte valArray malloc
* valArray free() on no malloc
* Column touched but no data written if all data going to second extent
* Wrong colWidth used on second extent calculateRowId
* Out of bounds memory write (crash) when no data for first extent
* Extent not committed if all data going to second extent
HWM for DML and API was being calculated using the first column in a
table instead of the smallest column. This shifts the calculation to the
correct column.
It turns out -c wasn't actually connected to anything and now with have
BLOB/TEXT it is pretty useful. If -c is set to < 1MB then 1MB is used,
otherwise it will use the selected buffer size.
When the API inserts data into ColumnStore which will roll over into a
new extent that data wasn't being put into the new extent and corruption
occured. This patch now tracks the additional data and inserts it into
the new extent. It also makes sure the LBIDs are stored so that they are
correctly committed.
cpimport would truncate UTF8 data half way through a character which
would cause problems for functions using that data. This patch
calculates the correct truncation point when inserting the data.
The signatures were getting destroyed whilst processing before being
used in the dictionary de-duplication cache making the cache full of
empty strings. This fix resets the signature after insertion for the
cache.
This fix also lets the signature cache be read when there is 1000
entries in it.
The rowID and therefore HWM for an insert was being calculated based on
the first column. If there are smaller columns in the table these will
insert in the middle of blocks instead of creating new blocks. This means
that we would no longer be crash safe and PrimProc gets very confused until
a cache flush.
We now use the smallest column to calculate the rowID and HWM increment
(as cpimport does).
