Instead of trying to detect MinProtocol=TLSv1.2 in all possible
sections of files in all possible locations where openssl is configured
to find its configuration in all possible distros (and these locations
are mutually different in Debian, Ubuntu, and RHEL, btw) - just make
mtr to instruct openssl not to read any config files at all.
Do not use suppressions on Windows, the ASAN_OPTIONS can't be parsed
whenever full file names are used, because file names contain ':'
which is ASAN_OPTIONS delimeter.
Let MTR check for error existence after running a test and return it back to user.
Error reporting itset might be much better, but first of all we need to see that
something went wrong.
$glob_mysql_test_dir was the wrong directory for strace output as
it was for in-tree builds only so failed for:
* out of tree builds
* --parallel; and
* --mem
strace output wasn't saved.
strace-option never replaced existing arguments (so ammended
documentation).
strace-client didn't accept an argument as described.
Replaced specification of client with this with 'stracer' to be
consistent with --debugger option.
For consistency with debugger options, --client-strace was added to
execute the strace on the mysqltest.
Example: Running one test
$ ./mtr --strace --client-strace funcs_1.is_table_constraints
Logging: ./mtr --strace --client-strace funcs_1.is_table_constraints
vardir: /home/anel/mariadb/5.5/mysql-test/var
Checking leftover processes...
Removing old var directory...
- WARNING: Using the 'mysql-test/var' symlink
Creating var directory '/home/anel/mariadb/5.5/mysql-test/var'...
Checking supported features...
MariaDB Version 5.5.67-MariaDB-debug
Installing system database...
- SSL connections supported
- binaries are debug compiled
Collecting tests...
==============================================================================
TEST RESULT TIME (ms) or COMMENT
--------------------------------------------------------------------------
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
funcs_1.is_table_constraints [ pass ] 1270
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 1.270 of 3 seconds executing testcases
Completed: All 1 tests were successful
$ find -L . -name \*strace -ls
653 56 -rw-r--r-- 1 anel anel 57147 Nov 29 15:08 ./var/log/mysqltest.strace
646 1768 -rw-r--r-- 1 anel anel 1809855 Nov 29 15:08 ./var/log/mysqld.1.strace
Example: Running test in parallel
$ mysql-test/mtr --strace --client-strace --mem --parallel=3 main.select
Logging: /home/dan/software_projects/mariadb-server/mysql-test/mysql-test-run.pl --strace --client-strace --mem --parallel=3 main.select
vardir: /home/dan/software_projects/build-mariadb-10.3/mysql-test/var
Checking leftover processes...
Removing old var directory...
Creating var directory '/home/dan/software_projects/build-mariadb-10.3/mysql-test/var'...
- symlinking 'var' to '/dev/shm/var_auto_0v2E'
Checking supported features...
MariaDB Version 5.5.67-MariaDB
- SSL connections supported
Collecting tests...
Installing system database...
==============================================================================
TEST WORKER RESULT TIME (ms) or COMMENT
--------------------------------------------------------------------------
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
worker[3] - 'localhost:16040' was not free
worker[2] Using MTR_BUILD_THREAD 301, with reserved ports 16020..16039
worker[3] Using MTR_BUILD_THREAD 303, with reserved ports 16060..16079
main.select w1 [ pass ] 7310
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 7.310 of 11 seconds executing testcases
Completed: All 1 tests were successful.
$ find mysql-test/var/ -name \*strace -ls
5213766 1212 -rw-r--r-- 1 dan dan 1237817 May 20 16:47 mysql-test/var/1/log/mysqltest.strace
5214733 13016 -rw-r--r-- 1 dan dan 13328335 May 20 16:47 mysql-test/var/1/log/mysqld.1.strace
$ mysql-test/mtr --strace --client-strace --strace-option='-e' --strace-option='trace=openat' --mem --parallel=3 main.select
...
$ find mysql-test/var/ -name \*strace -ls
5220790 8 -rw-r--r-- 1 dan dan 6291 May 20 17:02 mysql-test/var/3/log/mysqltest.strace
5224140 308 -rw-r--r-- 1 dan dan 314356 May 20 17:02 mysql-test/var/3/log/mysqld.1.strace
$ more mysql-test/var/3/mysqltest.strace
1692 openat(AT_FDCWD, "/home/dan/software_projects/mariadb-server/libmysql/.libs/tls/x86_64/x86_64/libpthread.so.0", O_RDONLY|O_CLOEXEC) =
-1 ENOENT (No such file or directory)
1692 openat(AT_FDCWD, "/home/dan/software_projects/mariadb-server/libmysql/.libs/tls/x86_64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOE
NT (No such file or directory)
Closes#600
Historically, InnoDB split the redo log into at least 2 files.
MDEV-12061 allowed the minimum to be innodb_log_files_in_group=1,
but it kept the default at innodb_log_files_in_group=2.
Because performance seems to be slightly better with only one log file,
and because implementing an append-only variant of the log would require
a single file, let us define the default to be 1, and have
innodb_log_file_size=96M, to retain the same default total size.
This allows one to run the test suite even if any of the following
options are changed:
- character-set-server
- collation-server
- join-cache-level
- log-basename
- max-allowed-packet
- optimizer-switch
- query-cache-size and query-cache-type
- skip-name-resolve
- table-definition-cache
- table-open-cache
- Some innodb options
etc
Changes:
- Don't print out the value of system variables as one can't depend on
them to being constants.
- Don't set global variables to 'default' as the default may not
be the same as the test was started with if there was an additional
option file. Instead save original value and reset it at end of test.
- Test that depends on the latin1 character set should include
default_charset.inc or set the character set to latin1
- Test that depends on the original optimizer switch, should include
default_optimizer_switch.inc
- Test that depends on the value of a specific system variable should
set it in the test (like optimizer_use_condition_selectivity)
- Split subselect3.test into subselect3.test and subselect3.inc to
make it easier to set and reset system variables.
- Added .opt files for test that required specfic options that could
be changed by external configuration files.
- Fixed result files in rockdsb & tokudb that had not been updated for
a while.
- Note that some issues was also fixed in 10.2 and 10.4. I also fixed them
here to be able to continue with making 10.5 valgrind safe again
- Disable connection threads warnings when doing shutdown
The execution of mtr in the Windows environment fails due to
the fact that the new code from MDEV-18565 does not take into
account the need to add the ".exe" extension to the names of
executable files when searching for pre-requisites that are
needed to run SST scripts (especially when using mariabackup)
and when searching paths to some other Galera utilities.
This patch fixes this flaw.
Also adding paths to the PATH environment variable is now
done with the correct delimiter character.
The execution of mtr in the Windows environment fails due to
the fact that the new code from MDEV-18565 does not take into
account the need to add the ".exe" extension to the names of
executable files when searching for pre-requisites that are
needed to run SST scripts (especially when using mariabackup)
and when searching paths to some other Galera utilities.
This patch fixes this flaw.
Also adding paths to the PATH environment variable is now
done with the correct delimiter character.
The execution of mtr in the Windows environment fails due to
the fact that the new code from MDEV-18565 does not take into
account the need to add the ".exe" extension to the names of
executable files when searching for pre-requisites that are
needed to run SST scripts (especially when using mariabackup)
and when searching paths to some other Galera utilities.
This patch fixes this flaw.
Also adding paths to the PATH environment variable is now
done with the correct delimiter character.
The general reason why innodb redo log file is limited by 512G is that
log_block_convert_lsn_to_no() returns value limited by 1G. But there is no
need to have unique log block numbers in log group. The fix removes 512G
limit and limits log group size by
(uint32_t maximum value) * (minimum page size), which, in turns, can be
removed if fil_io() is no longer used for innodb redo log io.
Currently, running mtr with an incorrect (for example, new or
obsolete) version of wsrep_provider (for example, with the 26
version of libgalera_smm.so) leads to the failure of tests in
several suites with vague error diagnostics.
As for the galera_3nodes suite, the mtr also does not effectively
check all the prerequisites after merge with MDEV-18426 fixes.
For example, tests that using mariabackup do not check for presence
of ss and socat/nc. This is due to improper handling of relative
paths in mtr scripts.
In addition, some tests in different suites can be run without
setting the environment variables such as MTR_GALERA_TFMT, XBSTREAM,
and so on.
To eliminate all these issues, this patch makes the following changes:
1. Added auxiliary wsrep_mtr_check utility (which located in the
mysql-test/lib/My/SafeProcess subdirectory), which compares the
versions of the wsrep API that used by the server and by the wsrep
provider library, and it does this comparison safely, without
accessing the API if the versions do not match.
2. All checks related to the presence of mariabackup and utilities
that necessary for its operation transferred from the local directories
of different mtr suites (from the suite.pm files) to the main suite.pm
file. This not only reduces the amount of code and eliminates duplication
of identical code fragments, but also avoids problems due to the inability
of mtr to consider relative paths to include files when checking skip
combinations.
3. Setting the values of auxiliary environment variables that
are necessary for Galera, SST scripts and mariabackup (to work
properly) is moved to the main mysql-test-run.pl script, so as
not to duplicate this code in different suites, and to avoid
partial corrections of the same errors for different suites
(while other suites remain uncorrected).
4. Fixed duplication of the have_file_key_management.inc and
have_filekeymanagement.inc files between different suites,
these checks are also transferred to the top level.
5. Added garbd presence check and garbd path variable.
https://jira.mariadb.org/browse/MDEV-18565
Currently, running mtr with an incorrect (for example, new or
obsolete) version of wsrep_provider (for example, with the 26
version of libgalera_smm.so) leads to the failure of tests in
several suites with vague error diagnostics.
As for the galera_3nodes suite, the mtr also does not effectively
check all the prerequisites after merge with MDEV-18426 fixes.
For example, tests that using mariabackup do not check for presence
of ss and socat/nc. This is due to improper handling of relative
paths in mtr scripts.
In addition, some tests in different suites can be run without
setting the environment variables such as MTR_GALERA_TFMT, XBSTREAM,
and so on.
To eliminate all these issues, this patch makes the following changes:
1. Added auxiliary wsrep_mtr_check utility (which located in the
mysql-test/lib/My/SafeProcess subdirectory), which compares the
versions of the wsrep API that used by the server and by the wsrep
provider library, and it does this comparison safely, without
accessing the API if the versions do not match.
2. All checks related to the presence of mariabackup and utilities
that necessary for its operation transferred from the local directories
of different mtr suites (from the suite.pm files) to the main suite.pm
file. This not only reduces the amount of code and eliminates duplication
of identical code fragments, but also avoids problems due to the inability
of mtr to consider relative paths to include files when checking skip
combinations.
3. Setting the values of auxiliary environment variables that
are necessary for Galera, SST scripts and mariabackup (to work
properly) is moved to the main mysql-test-run.pl script, so as
not to duplicate this code in different suites, and to avoid
partial corrections of the same errors for different suites
(while other suites remain uncorrected).
4. Fixed duplication of the have_file_key_management.inc and
have_filekeymanagement.inc files between different suites,
these checks are also transferred to the top level.
5. Added garbd presence check and garbd path variable.
https://jira.mariadb.org/browse/MDEV-18565
