The test is unstable because 'UPDATE t SET b = 100' latches a page and
waits for 'upd_cont' signal in lock_trx_handle_wait_enter sync point, then
purge requests RW_X_LATCH on the same page, and then 'SELECT * FROM t
WHERE a = 10 FOR UPDATE' requests RW_S_LATCH, waiting for RW_X_LATCH
requested by purge. 'UPDATE t SET b = 100' can't release page latch as
it waits for upd_cont signal, which must be emitted after 'SELECT * FROM
t WHERE a = 10 FOR UPDATE' acquired RW_S_LATCH. So we have a deadlock,
which is resolved by finishing the debug sync point wait by timeout, and
the 'UPDATE t SET b = 100' releases it's record locks rolling back the
transaction, and 'SELECT * FROM t WHERE a = 10 FOR UPDATE' is finished
successfully instead of finishing by lock wait timeout.
The fix is to forbid purging during the test by opening read view in a
separate connection before the first insert into the table.
Besides, 'lock_wait_end' syncpoint is not needed, as it enough to wait
the end of the SELECT execution to let the UPDATE to continue.
The issue is that trx_t::lock.was_chosen_as_deadlock_victim can be reset
before the transaction check it and set trx_t::error_state.
The fix is to reset trx_t::lock.was_chosen_as_deadlock_victim only in
trx_t::commit_in_memory(), which is invoked on full rollback. There is
also no need to have separate bit in
trx_t::lock.was_chosen_as_deadlock_victim to flag transaction it was
chosen as a victim of Galera conflict resolution, the same variable can be
used for both cases except debug build. For debug build we need to
distinguish deadlock and Galera's abort victims for debug checks. Also
there is no need to check for deadlock in lock_table_enqueue_waiting() for
Galera as the coresponding check presents in lock_wait().
Local variable "error_state" in lock_wait() was replaced with
trx->error_state, because before the replace
lock_sys_t::cancel<false>(trx, lock) and lock_sys.deadlock_check() could
change trx->error_state, which then could be overwritten with the local
"error_state" variable value.
The lock_wait_suspend_thread_enter DEBUG_SYNC point name is misleading,
because lock_wait_suspend_thread was eliminated in e71e613. It was renamed
to lock_wait_start.
Reviewed by: Marko Mäkelä, Jan Lindström.
