When replicating transactions from parallel slave replication
processing, Galera must respect the commit order of the parallel
slave replication. In the current implementation this is done by
calling `wait_for_prior_commit()` before the write set is
replicated and certified in before-prepare processing. This
however establishes a critical section which is held over
whole Galera replication step, and the commit rate will be
limited by Galera replication latency.
In order to allow concurrency in Galera replication step, the
critical section must be released at earliest point where Galera
can guarantee sequential consistency for replicated write sets.
This change passes a callback to release the critical section
by calling `wakeup_subsequent_commits()` to Galera library, which will
call the callback once the correct replication order can be established.
This functionality will be available from Galera 26.4.22 onwards.
Note that call to `wakeup_subsequent_commits()` at this stage is
safe from group commit point of view as Galera uses separate
`wait_for_commit` context to control commit ordering.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
This commit fixes a bug where Aria tables are used in
(master->slave1->slave2) and a backup is taken on slave2. In this case
it is possible that the replication position in the backup, stored in
mysql.gtid_slave_pos, will be wrong. This will lead to replication
errors if one is trying to use the backup as a new slave.
Analyze:
Replicated row events are committed with trans_commit_stmt() and
thd->transaction->all.ha_list != 0.
This means that backup_commit_lock is not taken for Aria tables,
which means the rows are committed and binary logged on the slave
under BLOCK_COMMIT which should not happen.
This issue does not occur on the master as thd->transaction->all.ha_list
is == 0 under AUTO_COMMIT, which sets 'is_real_trans' and 'rw_trans'
which in turn causes backup_commit_lock to be taken.
Fixed by checking in ha_check_and_coalesce_trx_read_only() if all handlers
supports rollback and if not, then wait for BLOCK_COMMIT also for
statement commit.
Issue:
Mariadb acquires additional MDL locks on UPDATE/INSERT/DELETE statements
on table with foreign keys. For example, table t1 references t2, an
UPDATE to t1 will MDL lock t2 in addition to t1.
A replica may deliver an ALTER t1 and UPDATE t2 concurrently for
applying. Then the UPDATE may acquire MDL lock for t1, followed by a
conflict when the ALTER attempts to MDL lock on t1. Causing a BF-BF
conflict.
Solution:
Additional keys for the referenced/foreign table needs to be added
to avoid potential MDL conflicts with concurrent update and DDLs.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
Problem was that initial GTID was set on wsrep_before_prepare
out-of-order. In practice GTID was set to same as previous
executed transaction GTID. In recovery valid GTID was found
from prepared transaction and this transaction is committed
leading to fact that same GTID was executed twice.
This is fixed by setting invalid GTID at wsrep_before_prepare
and later in wsrep_before_commit actual correct GTID is set
and this setting is done while we are in commit monitor i.e.
assigment is done in order of replication.
In recovery if prepared transaction is found we check its
GTID, if it is invalid transaction will be rolled back
and if it is valid it will be committed.
Initialize gtid seqno from recovered seqno when
bootstrapping a new cluster.
Added two test cases for both mariabackup and rsync SST methods
to show that GTIDs remain consistent on cluster and that
all expected rows are in the table.
Added tests for wsrep GTID recovery with binlog on and off.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
Problem was missing case from wsrep_handle_mdl_conflict. Test case
was trying to confirm that LOCK TABLE thread is not BF-aborted.
However as case was missing it was BF-aborted. Test case passed
because BF-aborting takes time and used wait condition might
see expected thread status before it was BF-aborted. Test naturally
failed if BF-aborting was done early enough.
Fix is to add missing case for SQLCOM_LOCK_TABLES to
wsrep_handle_mdl_conflict.
Note that using LOCK TABLE is still not recomended on cluster
because it could cause cluster hang.
This is a 10.5 specific commit that will then be overridden by
another one for 10.6+.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
Problem was missing case from wsrep_handle_mdl_conflict. Test case
was trying to confirm that LOCK TABLE thread is not BF-aborted.
However as case was missing it was BF-aborted. Test case passed
because BF-aborting takes time and used wait condition might
see expected thread status before it was BF-aborted. Test naturally
failed if BF-aborting was done early enough.
Fix is to add missing case for SQLCOM_LOCK_TABLES to
wsrep_handle_mdl_conflict.
Note that using LOCK TABLE is still not recomended on cluster
because it could cause cluster hang.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
* rpl.rpl_system_versioning_partitions updated for MDEV-32188
* innodb.row_size_error_log_warnings_3 changed error for MDEV-33658
(checks are done in a different order)
Fix a regression that caused assertion thd->is_error() after
sync wait failures. If wsrep_sync_wait() fails make sure a appropriate
error is set. Partially revert 75dd0246f8.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
Problem was incorrect handling of partitioned tables,
because db_type == DB_TYPE_PARTITION_DB
wsrep_should_replicate_ddl incorrectly marked
DDL as not replicatable. However, in partitioned
tables we should check implementing storage engine
from table->file->partition_ht() if available because
if partition handler is InnoDB all DDL should be allowed
even with wsrep_strict_ddl. For other storage engines
DDL should not be allowed and error should be issued.
This is 10.5 version of the fix.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
Issue:
Mariadb Galera cluster fails to replicate from
Mysql 5.7 when configured with MASTER_USE_GTID=no
option for CHANGE MASTER.
HOST: mysql, mysql 5.7.44 binlog_format=ROW
HOST: m1, mariadb 10.6 GALERA NODE replicating from HOST mysql, Using_Gtid: No (log file and position)
HOST: m2 mariadb 10.6 GALERA NODE
HOST: m3 mariadb 10.6 GALERA NODE
Error on m1:
2024-05-22 16:11:07 1 [ERROR] WSREP: Vote 0 (success) on 78cebda7-1876-11ef-896b-8a58fca50d36:2565 is inconsistent with group. Leaving cluster.
Error on m2 and m3:
2024-05-22 16:11:06 2 [ERROR] Error in Log_event::read_log_event(): 'Found invalid event in binary log', data_len: 42, event_type: -94
2024-05-22 16:11:06 2 [ERROR] WSREP: applier could not read binlog event, seqno: 2565, len: 482
It fails in Gtid_log_event::is_valid() check on
secondary node when sequence number sent from primary
is 0. On primary for applier or slave thread sequence
number is set to 0, when both thd->variables.gtid_seq_no
and thd->variables.wsrep_gtid_seq_no have value 0.
Solution:
Skip adding Gtid Event on primary for applier or
slave thread when both thd->variables.gtid_seq_no
and thd->variables.wsrep_gtid_seq_no have value 0.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
When client connections use threadpool, i.e. configuration has:
thread_handling = pool-of-threads it turned out that during wsrep
replication shutdown, not all client connections could be closed.
Reason was that some client threads has stmt_da in state DA_EOF,
and this state was earlier used to detect if client connection
was issuing SHUTDOWN command.
To fix this, the connection executing SHUTDOWN is now detected by
looking at the actual command being executed:
thd->get_command() == COM_SHUTDOWN
During replication shutdown, all other connections but the
SHUTDOWN executor, are terminated.
This commit has new mtr test galera.galera_threadpool, which
opens a number of threadpool client connections, and then
restarts the node to verify that connections in threadpool are
terminated during shutdown.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
DROP TABLE on child and UPDATE of parent table can cause an MDL BF-BF
conflict when applied concurrently.
DROP TABLE takes MDL locks on both child and its parent table, however
it only it did not add certification keys for the parent table.
This patch adds the following:
* Append certification keys corresponding to all parent tables
before DROP TABLE replication.
* Fix wsrep_append_fk_parent_table() so that it works when it is
given a table list containing temporary tables.
* Make sure function wsrep_append_fk_parent_table() is only called
for local transaction. That was not the case for ALTER TABLE.
* Add a test case that verifies that UPDATE parent depends on
preceeding DROP TABLE child.
* Adapt galera_ddl_fk_conflict test to work with DROP TABLE as well.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
