StringStore NULL check includes a check for _CpNuLl_ in the
StringStore. This is a case should never happen but we keep it just in
case.
Unfortunately this check was skipping 4*8 bytes instead of just 4 bytes.
This is definitely bad behaviour but it could cause an out-of-bounds read
based crash.
Fixes the following:
* Read past buffer end in intToDatetime / intToTime
* Allow intToTime to convert datetime
* Allow intToTime to convert shortened time values
* Allow stringToTime to convert datetime and int time values
* Fix saturation / bad values in intToTime and stringToTime
* Fix TIME return in STR_TO_DATE()
* Fix NULL return on type inequality for TIMEDIFF()
* Fix zero day calculation error in ADDTIME()/SUBTIME()
* Fix DATETIME to int calculation error in aggregate bit operations
* Make the new harderning flags optional with -DSECURITY_HARDENED_NEW
StringStore originally worked by returning a 32bit pointer to a memory
location and storing the length with that pointer. This allowed 4GB to
be stored in 64KB blocks. With 1.1 we used the high bit to signify a
TEXT/BLOB string of > 64KB reducing the max capacity to 2GB but without
any bounds checking.
So, if you went over the 2GB mark the getter would think you are trying
to get a long string instead of a short one and come up empty. It would
then return NULL.
This patch uses 64bit memory points still retaining the high bit to
signify long strings. It also now stores the length with the string
rather than with the pointer to allow the full 64bits for pointers.
It also adds a bounds check for small strings.
We should have been initalizing TEXT/BLOB 8 byte tokens for NULL,
instead we were initializing the entire length of the TEXT/BLOB which
can do a lot of damage.
StringStore as a vector of std::string had a performance regressions and
a rare crash.
This new version of StringStore restores the original StringStore with
the 64KB limitation and adds another vector to store strings that won't
fit into the small string storage.
The fix for MCOL-838 broke VARBINARY as it truncated on the first NUL on
StringStore deserialize. This fix uses append() to force a copy instead
whilst preserving length.
This fixes test012
* TEXT and BLOB now have separate identifiers internally
* TEXT columns are identified as such in system catalog
* cpimport only requires hex input for BLOB, not TEXT
