DMLProcessor functor earlier did not check if the DBRM was in read-only mode.
This allowed DML statements to continue execution to the point where it locks
the table and then sends the statement down to the WriteEngineServer, which
ultimately returns back in an error state to DMLProc when it fails to perform
BRM updates due to DBRM in read-only mode. This caused a lingering table lock
in the system which could only be cleared on a system restart.
As a fix, we add a check in the DMLProcessor functor to detect if DBRM is in
read only mode, and if so, return back early in the execution of the DML
statement.
- occured -> occurred
- reponse -> response
- seperated -> separated
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
* Fix clang warnings
* Remove vim tab guides
* initialize variables
* 'strncpy' output truncated before terminating nul copying as many bytes from a string as its length
* Fix ISO C++17 does not allow 'register' storage class specifier for outdated bison
* chars are unsigned on ARM, having if (ival < 0) always false
* chars are unsigned by default on ARM and comparison with -1 if always true
1. Add a new system variable, columnstore_use_cpimport_for_cache_inserts,
that when set to ON, uses cpimport for the cache flush into ColumnStore.
This variable is set to OFF by default. By default, we perform batch inserts
for the cache flush.
2. Disable DMLProc logging of the SQL statement text for the cache
flush operation in case of batch inserts. Under certain heavy loads
involving INSERT statements, this logging becomes a bottleneck for
the cache flush, causing subsequent inserts into the cache table to hang.
1) Instead of making dbrm calls to writeVBEntry() per block,
we make these calls per batch. This can have non-trivial
reductions in the overhead of these calls if the batch size
is large.
2) In dmlproc, do not deserialize the whole insertpackage, which
consists of the complete record set per column, which would be
wasteful as we only need some metadata fields from insertpackage
here. This is only done for batch inserts at the moment, this
should also be applied to single inserts.
This patch catches exceptions in DDLProc, DMLProc and ExeMgr which could
potentially happen during startup. Logging them instead of silently
ignoring them (or crashing in ExeMgr).
This does the following:
* Switch resource manager to a singleton which reduces the amount of
times the XML data is scanned and objects allocated.
* Make the I_S tables use the FE implementation of the system catalog
* Make the I_S.columnstore_columns table use the RID list cache
* Make the extentmap pre-allocate a vector instead of many small allocs
