StringStore NULL check includes a check for _CpNuLl_ in the
StringStore. This is a case should never happen but we keep it just in
case.
Unfortunately this check was skipping 4*8 bytes instead of just 4 bytes.
This is definitely bad behaviour but it could cause an out-of-bounds read
based crash.
StringStore originally worked by returning a 32bit pointer to a memory
location and storing the length with that pointer. This allowed 4GB to
be stored in 64KB blocks. With 1.1 we used the high bit to signify a
TEXT/BLOB string of > 64KB reducing the max capacity to 2GB but without
any bounds checking.
So, if you went over the 2GB mark the getter would think you are trying
to get a long string instead of a short one and come up empty. It would
then return NULL.
This patch uses 64bit memory points still retaining the high bit to
signify long strings. It also now stores the length with the string
rather than with the pointer to allow the full 64bits for pointers.
It also adds a bounds check for small strings.
StringStore as a vector of std::string had a performance regressions and
a rare crash.
This new version of StringStore restores the original StringStore with
the 64KB limitation and adds another vector to store strings that won't
fit into the small string storage.
* TEXT and BLOB now have separate identifiers internally
* TEXT columns are identified as such in system catalog
* cpimport only requires hex input for BLOB, not TEXT
This patch adds enough support so that cross engines joins with blob
columns in the foreign engines will work. The modifications are as
follows:
* Add CrossEngine support for non-NULL-terminated (binary) data
* Add row data support for blobs (similar to varbinary)
* Add engine support for writing out blob data correctly to the storage
engine API
* Re-enable blob support in the engine plugin
It is possible to have a VARCHAR column that isn't NUL terminated, an
example of this is a union of two CHAR columns. So the length should
always act as a terminator when there is no NUL.
