Fix old_mode flags conflict between OLD_MODE_NO_NULL_COLLATION_IDS
and OLD_MODE_LOCK_ALTER_TABLE_COPY.
Both flags used to be 1 << 6, now OLD_MODE_LOCK_ALTER_TABLE_COPY changed
to be 1 << 7
Fix some random test failures following MDEV-32168 push.
Don't blindly set $rpl_only_running_threads in many places. Instead explicit
stop only the IO or SQL thread, as appropriate. Setting it interfered with
rpl_end.inc in some cases. Rather than clearing it afterwards, better to
not set it at all when it is not needed, removing ambiguity in the test
about the state of the replication threads.
Don't fail the test if include/stop_slave_io.inc finds an error in the IO
thread after stop. Such errors can be simply because slave stop happened in
the middle of the IO thread's initial communication with the master.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
This patch provides a new mode FULL_NODUP to binlog_row_image system
variable. With FULL_NODUP mode, all columns are included in before
image, but only updated columns are included in after image for UPDATE.
While all columns are included in the after image for INSERT.
FULL_NODUP is for replacing FULL mode. It includes all data of
the before and after image as FULL mode, but it uses less storage
especially in the case that only a few columns are updated.
Note: It will binlog full before and after image for all modes if the
table has no primary key. FULL_NODUP follows the behavior.
Wait for the disconnect of the other connection to complete, before running
SHOW BINLOG EVENTS. Otherwise the DROP TEMPORARY TABLE that is binlogged
during disconnect may not have appeared yet depending on thread scheduling.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
Fix wrong change to rpl.rpl_shutdown_wait_slaves. After shutting down the
master, slaves may or may not succeed in reconnecting depending on the timing
on their reconnect relative to master restart. So don't assume all IO threads
will be running, just restart any slave that needs it.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
Fix sporadic test failure in rpl.rpl_ssl1. The test incorrectly did a STOP
SLAVE too early, which could race with the expected 'Access denied' error.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
Fix sporadic test failures in rpl.rpl_set_statement_default_master and
rpl.rpl_slave_load_tmpdir_not_exist. A race between START and STOP
SLAVE could leave an error condition that causes test failure after
MDEV-32168.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
Test rpl.show_status_stop_slave_race-7126 now fails sporadically
because it is expected to sometimes (but not always) leave an error
condition after slave stop. Fix by explicitly allowing the error
condition in this case.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
Fix a start/stop race that causes occasional test failure after more the more
strict error check of MDEV-32168.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
The reason was that Event e11 was re-executed before
"ALTER EVENT e11 DISABLE" had been executed.
Fixed by increasing re-schedule time
Other things:
- Removed double accounting of 'execution_count'. It was incremented in
top->mark_last_executed(thd) that was executed a few lines earlier.
The reason was that Event e11 was re-executed before
"ALTER EVENT e11 DISABLE" had been executed.
Fixed by increasing re-schedule time
Other things:
- Removed double accounting of 'execution_count'. It was incremented in
top->mark_last_executed(thd) that was executed a few lines earlier.
The MTR test rpl.rpl_semi_sync_slave_compressed_protocol scans the
log file to ensure there is no magic number error. It attempts to
only scan the log files of the current test; however, the variable
which controls this, , is initialized incorrectly,
and it thereby scans the entire log file, which includes output from
prior tests. This causes it to fail if a test which expects this
error runs previously on the same worker.
This patch fixes the assert_only_after so the test only scans
through its own log contents.
The test rpl.rpl_sql_thd_start_errno_cleared can lose a debug_sync
signal, as there is a RESET immediately following a SIGNAL. When the
signal is lost, the sql_thread is stuck in a WAIT_FOR clause until
it times out, resulting in long test times (albeit still
successful).
This patch extends the test to ensure the debug_sync signal was
received before issuing the RESET
This commit fixes several bugs in error handling around disk full when
writing the statement/transaction binlog caches:
1. If the error occurs during a non-transactional statement, the code
attempts to binlog the partially executed statement (as it cannot roll
back). The stmt_cache->error was still set from the disk full error. This
caused MYSQL_BIN_LOG::write_cache() to get an error while trying to read the
cache to copy it to the binlog. This was then wrongly interpreted as a disk
full error writing to the binlog file. As a result, a partial event group
containing just a GTID event (no query or commit) was binlogged. Fixed by
checking if an error is set in the statement cache, and if so binlog an
INCIDENT event instead of a corrupt event group, as for other errors.
2. For LOAD DATA LOCAL INFILE, if a disk full error occured while writing to
the statement cache, the code would attempt to abort and read-and-discard
any remaining data sent by the client. The discard code would however
continue trying to write data to the statement cache, and wrongly interpret
another disk full error as end-of-file from the client. This left the client
connection with extra data which corrupts the communication for the next
command, as well as again causing an corrupt/incomplete event to be
binlogged. Fixed by restoring the default read function before reading any
remaining data from the client connection.
Reviewed-by: Andrei Elkin <andrei.elkin@mariadb.com>
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
Compute binlog checksums (when enabled) already when writing events
into the statement or transaction caches, where before it was done
when the caches are copied to the real binlog file. This moves the
checksum computation outside of holding LOCK_log, improving
scalabitily.
At stmt/trx cache write time, the final end_log_pos values are not
known, so with this patch these will be set to 0. Events that are
written directly to the binlog file (not through stmt/trx cache) keep
the correct end_log_pos value. The GTID and COMMIT/XID events at the
start and end of event groups are written directly, so the zero
end_log_pos is only for events in the middle of event groups, which
do not negatively affect replication.
An option --binlog-legacy-event-pos, off by default, is provided to
disable this behavior to provide backwards compatibility with any
external applications that might rely on end_log_pos in events in the
middle of event groups.
Checksums cannot be pre-computed when binlog encryption is enabled, as
encryption relies on correct end_log_pos to provide part of the
nonce/IV.
Checksum pre-computation is also disabled for WSREP/Galera, as it uses
events differently in its write-sets and so on. Extending pre-computation of
checksums to Galera where it makes sense could be added in a future patch.
The current --binlog-checksum configuration is saved in
binlog_cache_data at transaction start and used to pre-compute
checksums in cache, if applicable. When the cache is later copied to
the binlog, a check is made if the saved value still matches the
configured global value; if so, the events are block-copied directly
into the binlog file. If --binlog-checksum was changed during the
transaction, events are re-written to the binlog file one-by-one and
the checksums recomputed/discarded as appropriate.
Reviewed-by: Monty <monty@mariadb.org>
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
Semisync ack (master side) receiver thread is made to report
details of faced errors.
In case of 'magic byte' error, a hexdump of the received packet
is always (level) NOTEd into the error log.
In other cases an exact server level error is print out
as a warning (as it may not be critical) under log_warnings > 2.
An MTR test added for the magic byte error. For others existing mtr
tests cover that, provided log_warnings > 2 is set.
If a replica is actively delaying a transaction when restarted (STOP
SLAVE/START SLAVE), when the sql thread is back up,
Seconds_Behind_Master will present as 0 until the configured
MASTER_DELAY has passed. That is, before the restart,
last_master_timestamp is updated to the timestamp of the delayed
event. Then after the restart, the negation of sql_thread_caught_up
is skipped because the timestamp of the event has already been used
for the last_master_timestamp, and their update is grouped together
in the same conditional block.
This patch fixes this by separating the negation of
sql_thread_caught_up out of the timestamp-dependent block, so it is
called any time an idle parallel slave queues an event to a worker.
Note that sql_thread_caught_up is still left in the check for internal
events, as SBM should remain idle in such case to not "magically" begin
incrementing.
Reviewed By:
============
Andrei Elkin <andrei.elkin@mariadb.com>
New Feature:
============
This patch extends the START SLAVE UNTIL command with options
SQL_BEFORE_GTIDS and SQL_AFTER_GTIDS to allow user control of
whether the replica stops before or after a provided GTID state. Its
syntax is:
START SLAVE UNTIL (SQL_BEFORE_GTIDS|SQL_AFTER_GTIDS)=”<gtid_list>”
When providing SQL_BEFORE_GTIDS=”<gtid_list>”, for each domain
specified in the gtid_list, the replica will execute transactions up
to the GTID found, and immediately stop processing events in that
domain (without executing the transaction of the specified GTID).
Once all domains have stopped, the replica will stop. Events
originating from domains that are not specified in the list are not
replicated.
START SLAVE UNTIL SQL_AFTER_GTIDS=”<gtid_list>” is an alias to the
default behavior of START SLAVE UNTIL master_gtid_pos=”<gtid_list>”.
That is, the replica will only execute transactions originating from
domain ids provided in the list, and will stop once all transactions
provided in the UNTIL list have all been executed.
Example:
=========
If a primary server has a binary log consisting of the following GTIDs:
0-1-1
1-1-1
0-1-2
1-1-2
0-1-3
1-1-3
If a fresh replica (i.e. one with an empty GTID position,
@@gtid_slave_pos='') is started with SQL_BEFORE_GTIDS, i.e.
START SLAVE UNTIL SQL_BEFORE_GTIDS=”1-1-2”
The resulting gtid_slave_pos of the replica will be “1-1-1”.
This is because the replica will execute only events from domain 1
until it sees the transaction with sequence number 2, and
immediately stop without executing it.
If the replica is started with SQL_AFTER_GTIDS, i.e.
START SLAVE UNTIL SQL_AFTER_GTIDS=”1-1-2”
then the resulting gtid_slave_pos of the replica will be “1-1-2”.
This is because it will only execute events from domain 1 until it
has executed the provided GTID.
Reviewed By:
============
Kristian Nielson <knielsen@knielsen-hq.org>
The assert's reason was in missed FL_DDL flagging of CREATE-or-REPLACE
Query event.
MDEV-27365 fixes covered only the non-pre-existing table execution branch so
did not see a possibility of implicit commit in
the middle of execution in a rollback branch when the being CREATEd
sequence table is actually replaced.
The pre-existing table branch cleared the DDL modification
flag so the query lost FL_DDL in binlog and its parallel execution
on slave may have ended up with the assert to indicate the query
is raced by a following in binlog order event.
Fixed with applying the MDEV-27365 pattern.
An mtr test is added to cover the rollback situation.
The description test [ pass ] with a generous number of mtr parallel
reties.
The MDEV-29693 conflict resolution is from Monty, as well as is
a bug fix where ANALYZE TABLE wrongly built histograms for
single-column PRIMARY KEY.
Also includes a fix for safe_malloc error reporting.
Other things:
- Copied main.log_slow from 10.4 to avoid mtr issue
Disabled test:
- spider/bugfix.mdev_27239 because we started to get
+Error 1429 Unable to connect to foreign data source: localhost
-Error 1158 Got an error reading communication packets
- main.delayed
- Bug#54332 Deadlock with two connections doing LOCK TABLE+INSERT DELAYED
This part is disabled for now as it fails randomly with different
warnings/errors (no corruption).
The crash inside my_vsnprintf_utf32() happened correctly,
because the caller methods:
Field_string::sql_rpl_type()
Field_varstring::sql_rpl_type()
mis-used the charset library and sent pure ASCII data to the
virtual function snprintf() of a utf32 CHARSET_INFO.
It was wrong to use Field::charset() in sql_rpl_type().
We're printing the metadata (the data type) here, not the column data.
The string contraining the data type of a CHAR/VARCHAR column
is a pure ASCII string.
Fixing to use res->charset() to print, like all virtual implementations
of sql_type() do.
Review was done by Andrei Elkin.
Thanks to Andrei for proposing MTR test improvents.
remove old deprecation helpers that were not used anywhere.
create new deprecation helpers and enforce their usage
this also removes inconsistencies in reporting deprecation:
sometimes it was ER_WARN_DEPRECATED_SYNTAX (1287),
sometimes ER_WARN_DEPRECATED_SYNTAX_NO_REPLACEMENT (1681),
sometimes a warning, sometimes a note.
it should always be
* ER_WARN_DEPRECATED_SYNTAX
* a warning (because it's something actionable, not purely informational)
In particular:
* @@debug
deprecated since 5.5.37
* sr_YU locale
deprecated since 10.0.11
* "engine_condition_pushdown" in the @@optimizer_switch
deprecated since 10.1.1
* @@date_format, @@datetime_format, @@time_format, @@max_tmp_tables
deprecated since 10.1.2
* @@wsrep_causal_reads
deprecated since 10.1.3
* "parser" in mroonga table comment
deprecated since 10.2.11
