Anything that links against joblist will spin up a threadpool upon
startup. This includes the tools setConfig/getConfig. It is possible on
a low core count machine or low CPU speed that the signal to the prune
thread to shutdown is sent before the thread has completed startup when
these quick-running tools are used.
This fix adds a mutex so that spin up and shutdown can't happen at the
same time as well as a stop watch in case we are shutting down when
either the thread is running or we haven't fully started.
Anything that links against joblist will spin up a threadpool upon
startup. This includes the tools setConfig/getConfig. It is possible on
a low core count machine or low CPU speed that the signal to the prune
thread to shutdown is sent before the thread has completed startup when
these quick-running tools are used.
This fix adds a mutex so that spin up and shutdown can't happen at the
same time as well as a stop watch in case we are shutting down when
either the thread is running or we haven't fully started.
When a thread has been idle for 10 minutes and we have too many threads
in the threadpool the thread will be pruned. This is done by the
thread's main function just returning. Unfortunately this does not free
up the memory, the thread either needs to be joined or detatched.
We cannot use detached threads since there are mutexes and conditional
variables between the main thread and the threadpool threads. If the
main thread finishes before the threadpool threads (as would happen in
cpimport) then crashes occur. The parent needs to wait on the child
threads which is the whole point in joining.
So this fix spawns a new thread which every minute will check the list
of threads to be joined due to timeout and join them.
We have had to use an adapted version of boost::thread_group so that we
can join a single thread based off its thread ID.
In addition with have modified PriorityThreadPool to use detached
threads since this does not need to signal the child threads at the end.
Ubuntu 18.04 uses GCC 7.3 which is a little stricter than before.
Fixes a few errors due to implicit includes that are no longer implicit
and a ton of warnings about the implied alignment of code in
utils/common/any.hpp
