Merge 22.08.7 (#2678)

* fix C API includes

ColumnStore used to include server's mysql.h
but link all tools with libmariadb.so

There's no guarantee that this would work, even with workarounds
it had in dbcon/mysql/sm.cpp

Fix:
* tools (linked with libmariadb.so) *must* include libmariadb's mysql.h
* as a hack prevent service_thd_timezone.h from being loaded into tools,
  as it conflicts with libmariadb's mysql.h
* server plugin *must* include server's mysql.h
* also don't link every tool with libmariadb.so, link the helper library
  (liblibmysqlclient.so) that actually needs it, tools use this
  helper library, not libmariadb.so directly

* do *not* link ha_columnstore.so with libmariadb.so

this means some libraries have to be compiled twice -
for tools with libmariadb.so and for plugin, without.

* use system boost, if possible

boost 1.71.0 is what ubuntu focal has, so let's start with that version.
boost 1.77.0 is the first that supports c++20

* add dependency for generated header files errorids.h messageids.h

see 3edd51610

* bump the version

* MCOL-5322 This patch replaces boost::mutex with std::mutex b/c IMHO std::unique_lock::lock is
less troublesome comparing with the boost alternative

* MCOL-5310 This patch replaces move-assignment with copy-assignment to avoid memory corruption (#2661)

* Bump VERSION to 22.08.7-1

* MCOL-5306 Re-read the config (Columnstore.xml) file if it was updated.

The existing implementation of Config::makeConfig() factory method
was returning a possibly stale config to the caller, without checking
if the config file was updated since the last read. This bug triggered
a scenario as described in MCOL-5306 where after a failover in an MCS
cluster, the controllernode coordinates changed in the config file
after failover and the existing mariadbd process was still using the
old controllernode coordinates. This lead to failed network connection
between mariadbd and the new controllernode.

The change in this fix, however, is more generic and not just limited
to this above scenario.

* MCOL-5264 This patch replaces boost mutex locks with std analogs
boost::uniqie_lock dtor calls a fancy unlock logic that throws twice.
First if the mutex is 0 and second lock doesn't own the mutex.
The first condition failure causes unhandled exception for one of the clients
in DEC::writeToClient(). I was unable to find out why Linux can have a 0
mutex and replaced boost::mutex with std::mutex b/c stdlibc++ should
be more stable comparing with boost.

* MCOL-5311 Add timezone to jobList in subquerytransformer
TimeZone was uninitialized in this scenario and led to undefined behavior.

* patch_out_of_band
Some changes made to 10.6-enterprise make a build using the out-of-band method of compiling columnstore not work. Out-of band means the source for the engine is not in the storage subdir of server, but rather in a stand alone directory. This is used by developers for easier develop work. In the case of out-of-band, INSTALL_LAYOUT is false in CMakeLists.txt

* MCOL-5346 This patch forces TreeNode::getIntValue to use conversion for dict-based CHAR/VARCHAR and TEXT columns (#2657)

Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>

* MCOL-5263 Add support to ROLLBACK when PP were restarted.

DMLProc starts ROLLBACK when SELECT part of UPDATE fails b/c EM facility in PP were restarted.
Unfortunately this ROLLBACK stuck if EM/PP are not yet available.
DMLProc must have a t/o with re-try doing ROLLBACK.

* MCOL-3561 This patch updates Connector code after MDEV-29988

* This commit applies the code style format

Co-authored-by: Sergei Golubchik <serg@mariadb.com>
Co-authored-by: Roman Nozdrin <rnozdrin@mariadb.com>
Co-authored-by: David.Hall <david.hall@mariadb.com>
Co-authored-by: Gagan Goel <gagan.nith@gmail.com>
Co-authored-by: Denis Khalikov <dennis.khalikov@gmail.com>

VERSION

@@ -1,4 +1,4 @@
 COLUMNSTORE_VERSION_MAJOR=22
 COLUMNSTORE_VERSION_MINOR=08
-COLUMNSTORE_VERSION_PATCH=4
+COLUMNSTORE_VERSION_PATCH=7
 COLUMNSTORE_VERSION_RELEASE=1

dbcon/mysql/ha_mcs_execplan.cpp

@@ -3471,6 +3471,10 @@ ReturnedColumn* buildReturnedColumn(Item* item, gp_walk_info& gwi, bool& nonSupp
           break;
 
         default:
+          if (ref->ref_type() == Item_ref::DIRECT_REF)
+          {
+            return buildReturnedColumn(ref->real_item(), gwi, nonSupport);
+          }
           gwi.fatalParseError = true;
           gwi.parseErrorText = "Unknown REF item";
           break;
@@ -4053,15 +4057,14 @@ ReturnedColumn* buildFunctionColumn(Item_func* ifp, gp_walk_info& gwi, bool& non
 
         ReturnedColumn* rc = NULL;
 
-
         // Special treatment for json functions
         // All boolean arguments will be parsed as boolean string true(false)
         // E.g. the result of `SELECT JSON_ARRAY(true, false)` should be [true, false] instead of [1, 0]
-        bool mayHasBoolArg = ((funcName == "json_insert" || funcName == "json_replace" ||
+        bool mayHasBoolArg =
-                              funcName == "json_set" || funcName == "json_array_append" ||
+            ((funcName == "json_insert" || funcName == "json_replace" || funcName == "json_set" ||
-                              funcName == "json_array_insert") && i != 0 && i % 2 == 0) ||
+              funcName == "json_array_append" || funcName == "json_array_insert") &&
-                             (funcName == "json_array") ||
+             i != 0 && i % 2 == 0) ||
-                             (funcName == "json_object" && i % 2 == 1);
+            (funcName == "json_array") || (funcName == "json_object" && i % 2 == 1);
         bool isBoolType =
             (ifp->arguments()[i]->const_item() && ifp->arguments()[i]->type_handler()->is_bool_type());
 
@@ -6408,10 +6411,15 @@ void parse_item(Item* item, vector<Item_field*>& field_vec, bool& hasNonSupportI
 
     case Item::REF_ITEM:
     {
+      Item_ref* ref = (Item_ref*)item;
+      if (ref->ref_type() == Item_ref::DIRECT_REF)
+      {
+        parse_item(ref->real_item(), field_vec, hasNonSupportItem, parseInfo, gwi);
+        break;
+      }
       while (true)
       {
-        Item_ref* ref = (Item_ref*)item;
+        ref = (Item_ref*)item;
-
         if ((*(ref->ref))->type() == Item::SUM_FUNC_ITEM)
         {
           parseInfo |= AGG_BIT;

primitives/blockcache/stats.cpp

@@ -178,7 +178,7 @@ typedef map<uint32_t, TraceFileInfo> TraceFileMap_t;
 
 TraceFileMap_t traceFileMap;
 // map mutex
-boost::mutex traceFileMapMutex;
+std::mutex traceFileMapMutex;
 
 class StatMon
 {
@@ -197,7 +197,7 @@ class StatMon
   void operator()() const
   {
     // struct timespec ts = { 60 * 1, 0 };
-    boost::mutex::scoped_lock lk(traceFileMapMutex);
+    std::unique_lock lk(traceFileMapMutex);
     TraceFileMap_t::iterator iter;
     TraceFileMap_t::iterator end;
 
@@ -256,7 +256,7 @@ void Stats::touchedLBID(uint64_t lbid, pthread_t thdid, uint32_t session)
   if (session == 0)
     return;
 
-  boost::mutex::scoped_lock lk(traceFileMapMutex);
+  std::lock_guard lk(traceFileMapMutex);
   TraceFileMap_t::iterator iter = traceFileMap.find(session);
 
   if (iter == traceFileMap.end())
@@ -274,7 +274,7 @@ void Stats::markEvent(const uint64_t lbid, const pthread_t thdid, const uint32_t
   if (session == 0)
     return;
 
-  boost::mutex::scoped_lock lk(traceFileMapMutex);
+  std::lock_guard lk(traceFileMapMutex);
   TraceFileMap_t::iterator iter = traceFileMap.find(session);
 
   if (iter == traceFileMap.end())

primitives/primproc/sqlfrontsessionthread.cpp

@@ -124,8 +124,8 @@ namespace exemgr
 
 void SQLFrontSessionThread::setRMParms(const execplan::CalpontSelectExecutionPlan::RMParmVec& parms)
 {
-    for (execplan::CalpontSelectExecutionPlan::RMParmVec::const_iterator it = parms.begin();
+  for (execplan::CalpontSelectExecutionPlan::RMParmVec::const_iterator it = parms.begin(); it != parms.end();
-        it != parms.end(); ++it)
+       ++it)
   {
     switch (it->id)
     {
@@ -353,7 +353,9 @@ namespace exemgr
         // making the whole session wait.  It can take several seconds.
         std::unique_lock<std::mutex> scoped(jlMutex);
         destructing++;
-          std::thread bgdtor([jl, &jlMutex, &jlCleanupDone, &destructing] {
+        std::thread bgdtor(
+            [jl, &jlMutex, &jlCleanupDone, &destructing]
+            {
               std::unique_lock<std::mutex> scoped(jlMutex);
               const_cast<joblist::SJLP&>(jl).reset();  // this happens second; does real destruction
               if (--destructing == 0)
@@ -642,8 +644,7 @@ namespace exemgr
           bs >> qb;
 
           if (gDebug > 1 || (gDebug && !csep.isInternal()))
-              std::cout << "### For session id " << csep.sessionID() << ", got a command = " << qb
+            std::cout << "### For session id " << csep.sessionID() << ", got a command = " << qb << std::endl;
-                        << std::endl;
 
           if (qb == 0)
           {
@@ -667,8 +668,7 @@ namespace exemgr
             if (iter == tm.end())
             {
               if (gDebug > 1 || (gDebug && !csep.isInternal()))
-                  std::cout << "### For session id " << csep.sessionID() << ", returning end flag"
+                std::cout << "### For session id " << csep.sessionID() << ", returning end flag" << std::endl;
-                            << std::endl;
 
               bs.restart();
               bs << (messageqcpp::ByteStream::byte)1;
@@ -886,7 +886,9 @@ namespace exemgr
         int stmtID = csep.statementID();
         std::unique_lock<std::mutex> scoped(jlMutex);
         destructing++;
-          std::thread bgdtor([jl, &jlMutex, &jlCleanupDone, stmtID, &li, &destructing, &msgLog] {
+        std::thread bgdtor(
+            [jl, &jlMutex, &jlCleanupDone, stmtID, &li, &destructing, &msgLog]
+            {
               std::unique_lock<std::mutex> scoped(jlMutex);
               const_cast<joblist::SJLP&>(jl).reset();  // this happens second; does real destruction
               logging::Message::Args args;