Where io_setup fails, this is a serious issue, normally because
of the lack of fs.aio-max-nr configured in the kernel.
We adjust the error message to be a Warning, because like
"native AIO failed: falling back to innodb_use_native_aio=OFF",
its user actionable.
A default configuation of the server and indeed raising
innodb_write_io_threads and innodb_read_io_threads couldn't
exceed the default fs.aio-max-nr value. If a user is
constructing multipe instances of MariaDB that exceed this
value then they should be seeing the warning and taking action.
There are CI environments, as Otto points on on Launchpad,
that have insufficient fs.aio-max-nr configure to run mtr
in parallel. This however a genuine distro problem and
to resolve.
For us, and our developers, we'd rather see the warning
so we can fix CI and dev instances that are insufficiently
configured.
The io_setup man page as a very short but descript set of
causes for the io_setup failures. Its safer to refer to this,
now with a strerror description rather than a number.
The mtr suppressions are removed because a while ago
when the errors where moved out of InnoDB and recently
the server could never generate a warning of these forms.
This controls which linux implementation to use for
innodb_use_native_aio=ON.
innodb_linux_aio=auto is equivalent to innodb_linux_aio=io_uring when
it is available, and falling back to innodb_linux_aio=aio when not.
Debian packaging is no longer aio exclusive or uring, so
for those older Debian or Ubuntu releases, its a remove_uring directive.
For more recent releases, add mandatory liburing for consistent packaging.
WITH_LIBAIO is now an independent option from WITH_URING.
LINUX_NATIVE_AIO preprocessor constant is renamed to HAVE_LIBAIO,
analogous to existing HAVE_URING.
tpool::is_aio_supported(): A common feature check.
is_linux_native_aio_supported(): Remove. This had originally been added in
mysql/mysql-server@0da310b69d in 2012
to fix an issue where io_submit() on CentOS 5.5 would return EINVAL
for a /tmp/#sql*.ibd file associated with CREATE TEMPORARY TABLE.
But, starting with commit 2e814d4702 InnoDB
temporary tables will be written to innodb_temp_data_file_path.
The 2012 commit said that the error could occur on "old kernels".
Any GNU/Linux distribution that we currently support should be based
on a newer Linux kernel; for example, Red Hat Enterprise Linux 7
was released in 2014.
tpool::create_linux_aio(): Wraps the Linux implementations:
create_libaio() and create_liburing(), each defined in separate
compilation units (aio_linux.cc, aio_libaio.cc, aio_liburing.cc).
The CMake definitions are simplified using target_sources() and
target_compile_definitions(), all available since CMake 2.8.12.
With this change, there is no need to include ${CMAKE_SOURCE_DIR}/tpool
or add TPOOL_DEFINES flags anymore, target_link_libraries(lib tpool)
does all that.
This is joint work with Daniel Black and Vladislav Vaintroub.
The fixed limit of 1024 open files was preventing proper concurrency
testing in MTR. This commit removes the hardcoded value and adds a new
option to control the limit when running tests: --open-files-limit=X.
The default is still 1024, but it can now be changed when needed,
making it easier to test scenarios that require different number of
open file descriptors at the same time e.g: partition_notwin.test
Documentation is added to mtr's help file as well.
Signed-off-by: Vicențiu Ciorbaru <vicentiu@mariadb.org>
Added a test to verify mariadb-upgrade-service functionality on Windows.
The test runs mariadb-install-db.exe to create a Windows service, then
executes mariadb-upgrade-service.exe while the service is online
or offline.
There is no real cross-version upgrade in this test(not possible with
MTR), the actual goal is just to run smoke-test.
Last times mariadb-upgrade-service was broken (CONC-760, MDEV-30639)
there were problems with named pipe connections, which could be detected
and prevented by smoke-test alone.
During regular iteration the page cleaner does flush from flush list
with some flush target and then goes for generating free pages from LRU
tail. When asynchronous flush is triggered i.e. when 7/8 th of the LSN
margin is filled in the redo log, the flush target for flush list is
set to innodb_io_capacity_max. If it could flush all, the flush
bandwidth for LRU flush is currently set to zero. If the LRU tail has
dirty pages, page cleaner ends up freeing no pages in one iteration.
The scenario could repeat across multiple iterations till async flush
target is reached. During this time the DB system is starved of free
pages resulting in apparent stall and in some cases dict_sys latch
fatal error.
Fix: In page cleaner iteration, before LRU flush, ensure we provide
enough flush limit so that freeing pages is no blocked by dirty pages
in LRU tail. Log IO and flush state if double write flush wait is long.
Reviewed by: Marko Mäkelä
This is for preparing MariaDB for catalogs.
mtr tests should in the future use MARIADB_TOPDIR or MARIADB_DATADIR
instead of '@@datadir'. This is especially true for replication tests
that access binlog files.
MARIADB_TOPDIR is the top directory where binary log and engine log files
resides.
MARIADB_DATADIR is the directory where the database/schema directories
resides.
MARIADB_TOPDIR is not depending on catalogs.
When catalogs is used MARIADB_DATADIR will point to the directory of the
current catalog. If catalogs is not used then
MARIAD_DATADIR=MARIADB_TOPDIR.
the test failed almost always in release (but not in debug) builds with
--- galera_sst_mariabackup.result
+++ galera_sst_mariabackup.reject
@@ -5,7 +5,7 @@
connection node_1;
select @@innodb_undo_tablespaces;
@@innodb_undo_tablespaces
-0
+3
connection node_2;
select @@innodb_undo_tablespaces;
@@innodb_undo_tablespaces
and
[Warning] InnoDB: Cannot change innodb_undo_tablespaces=0 because previous shutdown was not with innodb_fast_shutdown=0
because mariadbd *before this test* wasn't using innodb_fast_shutdown=0
Fix the bootstrap to use innodb_fast_shutdown=0 (and the bootstrap
creates a starting point for any test that uses a .cnf file)
followup for cac0fc97cc
also, remove redundant include/have_innodb.inc
Some fixes related to commit f838b2d799 and
Rows_log_event::do_apply_event() and Update_rows_log_event::do_exec_row()
for system-versioned tables were provided by Nikita Malyavin.
This was required by test versioning.rpl,trx_id,row.
MDEV-33582 Add more warnings to be able to better diagnose network issues
- Disabled "Semisync ack receiver got hangup" warning
- One could get this warning from semisync if running
mtr --mysqld=log-warnings=3 rpl.rpl_semi_sync_shutdown_await_ack
- Fixed result file for engines/funcs/rpl_get_lock.test
Warnings are added to net_server.cc when
global_system_variables.log_warnings >= 4.
When the above condition holds then:
- All communication errors from net_serv.cc is also written to the
error log.
- In case of a of not being able to read or write a packet, a more
detailed error is given.
Other things:
- Added detection of slaves that has hangup to Ack_receiver::run()
- vio_close() is now first marking the socket closed before closing it.
The reason for this is to ensure that the connection that gets a read
error can check if the reason was that the socket was closed.
- Add a new state to vio to be able to detect if vio is acive, shutdown or
closed. This is used to detect if socket is closed by another thread.
- Testing of the new warnings is done in rpl_get_lock.test
- Suppress some of the new warnings in mtr to allow one to run some of
the tests with -mysqld=--log-warnings=4. All test in the 'rpl' suite
can now be run with this option.
- Ensure that global.log_warnings are restored at test end in a way
that allows one to use mtr --mysqld=--log-warnings=4.
Reviewed-by: <serg@mariadb.org>,<brandon.nesterenko@mariadb.com>
