Problem was the parsing of test suite files for various tags and options.
This was done inefficiently, and include files were re-parsed for every
place they were included. This caused a delay of 20 seconds or so before
the first test started to run.
By parsing more efficiently and re-using first parse for subsequent
inclusion of the same file, time spent parsing is reduced to less than
1 second, and start appears instantaneous.
(With this patch, full ./mtr runs in 3 minutes on my laptop (release
build.)
mysql-test/suite/innodb_plugin/t/innodb_bug52663.test:
Test is fairly slow, so try to avoid getting stuck with it at the end
while other workers are idle.
Fixed lp:814237: Wrong mutex usage in Aria
storage/maria/ma_bitmap.c:
Added call to _ma_bitmap_mark_file_changed() when flushing bitmap. This fixes lp:814237
The bug happens when one uses MAX_ROWS=# with Aria & row_format=page and one insert more than # rows.
mysql-test/mysql-test-run.pl:
Ignore table is full error messages
mysql-test/suite/maria/r/max_length.result:
Test case for 'Table is full'
mysql-test/suite/maria/t/max_length.test:
Test case for 'Table is full'
storage/maria/ma_bitmap.c:
Ensure that we don't allocate bits outside of max_data_file_size.
Adjust max_data_file_size based on bitmap alignments.
Backport fix to adjust wrong first_bitmap_with_space.
storage/maria/ma_blockrec.c:
Calculate value of max_data_file_length
storage/maria/ma_blockrec.h:
Updated prototype for _ma_bitmap_init()
storage/maria/ma_check.c:
Give warnings if file sizes are above max file sizes.
Give more warnings in case of errors.
Have maria_chk write if table is recreated.
storage/maria/ma_create.c:
Better calculation of max_data_file_length and thus data pointer length.
Fixes some wrong pointer lengths when using MAX_ROWS=#
storage/maria/ma_open.c:
Removed duplicate assigment.
Use block size from file instead of global variable.
storage/maria/maria_chk.c:
Remove -1 from printed file length
storage/maria/maria_def.h:
Update struct st_maria_file_bitmap
There are 2 volatile condition constructions AND/OR constructions and fields(references) when first
good supported to be top elements of conditions because it is normal practice
(see copy_andor_structure for example) fields without any expression in the condition is really rare
and mostly useless case however it could lead to problems when optimiser changes/moves them unaware
of other variables referring to them. An easy solution of this problem is just to replace single field
in a condition with equivalent expression well supported by the server (<field> -> <field> != 0).
mysql-test/r/view.result:
New test added.
mysql-test/t/view.test:
New test added.
sql/sql_parse.cc:
<field> -> <field> != 0
sql/sql_yacc.yy:
<field> -> <field> != 0
- Fixed assert in transaction log handler when aria_check was run on block-record table that was much bigger than expected.
- Fixed warnings about wrong mutex order between bitmap and intern_lock
- Fixed error in bitmap that could cause two rows to use same block for a block record.
- Fixed wrong test that could cause error if last page for a bitmap was used by a blob.
- Fixed several bugs in pagecache for the case where pagecase had very few blocks and there was a lot of threads competing to get the blocks (very unlikely case).
mysql-test/suite/maria/r/maria-recovery3.result:
Updated results
sql/mysqld.cc:
Allow mi_check() to send information messages for log file
storage/maria/ma_bitmap.c:
Fixed problem with wrong mutex order when bitmap was the first page that was flushed out of page cache
- Fixed by introducing _ma_bitmap_mark_file_changed() that marks file changed without a bitmap lock.
- Fixed one case in _ma_change_bitmap_page() where we didn't mark the bitmap changed. This could cause to rows to reuse same block if this was the only change to the bitmap.
- Split _ma_bitmap_get_page_bits() in two parts to not take a bitmap lock when we already have it
- Fixed bug in _ma_bitmap_set_full_page_bits() that caused an error if last page for a bitmap was used by a blob
storage/maria/ma_check.c:
Better handling of wrong file length.
Fixed bug when we tried to write to transaction log when it was not opened (happened when block record file was bigger than expected)
storage/maria/ma_pagecache.c:
Fixed several bugs in pagecache for the case where pagecase had very few blocks and there was a lot of threads competing to get the blocks:
- In link_block() mark a block given to another thread with PCBLOCK_REASSIGNED to ensure that no other threads can start re-using the block
before the thread that requsted a block.
- In free_block(), don't reset status for a block that is in re-assign by link_block() (we don't want to loose the PCBLOCK_REASSIGNED flag).
- Added call to wait_for_flush() when we got a new block in find_block() to ensure that we don't use a block that is beeing flushed by another thread.
- Moved setting of hits_left and last_hit_time in find_block() to where we assign the block.
Code cleanup and making code uniform:
- Changed a lot of KEYCACHE_DBUG_PRINT to use DBUG_PRINT
- Streamlined all reporting of 'signal' and 'wait' between threads to be identical.
- Use thread name instead of thread number (for each match against --debug)
- Added more DBUG_ENTER, DBUG_PRINT and DBUG_ASSERT()
- Added more comments
storage/myisam/ha_myisam.cc:
Only print information about that we make a backup if we are really making a backup.
storage/myisam/mi_check.c:
Inform mysqld that we are creating a backup of the data file (for inclusion in error log).
mysql-test/r/join.result:
Test case for LP:798597
mysql-test/t/join.test:
Test case for LP:798597
sql/sql_select.cc:
In simplify_joins we reset table->maybe_null for outer join tables that can't ever be NULL.
This caused a conflict between the previously calculated items and the group_buffer against the fields
in the temporary table that are created as not null thanks to the optimization.
The fix is to correct the group by items to also be not_null so that they match the used fields and keys.
Added option to aria_read_log to crash recovery at certain points in the recovery process.
Fixed bug that caused future recovery attempts to fail if we got a crash/got killed during closing of tables at end of recovery process.
mysql-test/mysql-test-run.pl:
Don't abort if 'var' points to stale place; Just remove it.
mysql-test/suite/maria/r/maria.result:
Fixed wrong indentation
mysql-test/t/partition_error.test:
Fixed portability problem with partiton_error.test
storage/maria/ma_close.c:
More DBUG_PRINT info
storage/maria/ma_pagecache.c:
Copy flush_log_callback when writing to page cache. This fixes problem in recovery when switching from mode of file
storage/maria/ma_recovery.c:
Added option to aria_read_log to crash recovery at certain points in the recovery process.
storage/maria/ma_recovery.h:
Added option to aria_read_log to crash recovery at certain points in the recovery process.
storage/maria/maria_chk.c:
Align aria_chk -d output
Don't write warning Aria table '...' is usable but should be fixed if the table was before marked as crashed but now is ok
storage/maria/maria_read_log.c:
Added option to aria_read_log to crash recovery at certain points in the recovery process.
Noted that there was no memory leak, just a lot of used partitioned tables.
Fixed old bug: 'show status' now shows memory usage when compiled with safemalloc.
Added option --flush to mysqlcheck.c to run a 'flush tables' between each check to keep down memory usage.
Changed '--safemalloc' options to mysqld so that one can use --safemalloc and --skip-safemalloc.
Now skip-safemalloc is default (ie, we only do checking of memory overrun during free()) to speed up tests.
client/client_priv.h:
Added OPT_FLUSH_TABLES
client/mysqlcheck.c:
Added option --flush to mysqlcheck.c to run a 'flush tables' between each check to keep down memory usage.
mysql-test/mysql-test-run.pl:
Always run tests with --loose-skip-safemysqld for higher speed
sql/mysqld.cc:
Changed '--safemalloc' options so that one can use --safemalloc and --skip-safemalloc.
Now skip-safemalloc is default (ie, we only do checking of memory overrun during free()) to speed up tests
sql/sql_parse.cc:
Fixed old bug: 'show status' now shows memory usage when compiled with safemalloc.
storage/archive/archive_reader.c:
Changed all malloc() calls to use my_malloc()/my_free()
Added checks of malloc() calls.
storage/archive/ha_archive.cc:
Detect failure if init_archive_reader() and return errno. This fixed assert crash in my_seek().
Changed all malloc() calls to use my_malloc()/my_free()
Safety check that could cause core dump when doing create table with virtual column.
mysql-test/mysql-test-run.pl:
Show also warnings from thr_lock (which starts with just Warning, not Warning:)
mysql-test/r/lock.result:
Added test that showed not relevant warning when using table locks.
mysql-test/t/lock.test:
Added test that showed not relevant warning when using table locks.
mysys/thr_lock.c:
Fixed sorting of locks.
(Old sort code didn't handle case where TL_WRITE_CONCURRENT_INSERT must be sorted before TL_WRITE)
Added more information to check_locks warning output.
Fixed wrong testing of multiple different write locks for same table.
sql/item_cmpfunc.cc:
Safety check that could cause core dump when doing create table with virtual column.
mysql-test/suite/innodb/r/innodb_bug60049.result:
Updated results
mysql-test/suite/innodb/t/innodb_bug60049.test:
Force global.innodb_fast_shutdown to 0 as test require it
mysql-test/suite/innodb_plugin/t/innodb_bug60049.test:
Force global.innodb_fast_shutdown to 0 as test require it
storage/pbxt/src/pthread_xt.cc:
Disable call to setpriority()
The printing of include stack in the error case in mysqltest omitted the
bottom of the stack (the line number in original test case file), and instead
printed the top of the stack twice. Fix to print each element on the stack
once and only once.
aria_chk now returns 1 if one got any warnings during check and 2 if one got errors.
lp:728919 maria_chk should fail on all detected corruptions
mysql-test/lib/My/SafeProcess.pm:
Removed ^M lines from the file
storage/maria/ma_check.c:
Fixed indentation
storage/maria/maria_chk.c:
aria_chk now returns 1 if one got any warnings during check and 2 if one got errors.
storage/maria/unittest/ma_control_file-t.c:
Use unique sub directory for test results
storage/maria/unittest/ma_maria_log_cleanup.c:
Added help function create_tmpdir()
storage/maria/unittest/ma_pagecache_consist.c:
Use unique sub directory for test results
storage/maria/unittest/ma_pagecache_rwconsist.c:
Use unique sub directory for test results
storage/maria/unittest/ma_pagecache_rwconsist2.c:
Use unique sub directory for test results
storage/maria/unittest/ma_pagecache_single.c:
Use unique sub directory for test results
storage/maria/unittest/ma_test_all-t:
Use unique sub directory for test results
storage/maria/unittest/ma_test_loghandler-t.c:
Use unique sub directory for test results
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
Use unique sub directory for test results
remove delete of files that are already deleted by maria_log_remove()
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
Use unique sub directory for test results
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Use unique sub directory for test results
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Use unique sub directory for test results
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
Use unique sub directory for test results
remove delete of files that are already deleted by maria_log_remove()
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
Use unique sub directory for test results
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
Use unique sub directory for test results
remove delete of files that are already deleted by maria_log_remove()
storage/maria/unittest/ma_test_loghandler_purge-t.c:
Use unique sub directory for test results
support-files/compiler_warnings.supp:
Supress some warnings
unittest/unit.pl:
Run 4 unittests in parallel
mysql-test/r/create.result:
test of renaming
mysql-test/r/upgrade.result:
Now such behaviour prohibited to avoid problems.
mysql-test/t/create.test:
test of renaming
mysql-test/t/upgrade.test:
Now such behaviour prohibited to avoid problems.
sql/mysql_priv.h:
Function to test table name presence added.
sql/sql_rename.cc:
Rename fixed.
sql/sql_table.cc:
Function to test table name presence added.
Create fixed.
Problem was setting DEBUG_SYNC twice in a row too fast in the test case; this
could cause the second setting to override the first before the code had time
to react to the first, causing the signal to get lost.
Fixed by waiting for the code to receive the first signal before overwriting
it in the test case.
Fixed race condition in event that could cause hang when stopping event scheduler with SET GLOBAL event_scheduler=OFF
client/mysqltest.cc:
Write command to be executed to the log BEFORE executing the command.
This makes it easier to debug crashes as the log will contain the fatal command.
mysql-test/r/mysqltest.result:
Updated results (we now get more things logged)
sql/event_queue.cc:
Fixed race condition in event that could cause hang when stopping event scheduler with SET GLOBAL event_scheduler=OFF.
The reason was that a kill signal could be sent between last check of thd->killed and before thd->enter_cond() in which case the signal
would be missed and we would be stuck in Event_scheduler::stop() forever.
Fixed compiler warnings
client/readline.cc:
Fixed compiler warning
mysql-test/suite/innodb/t/innodb_bug60049.test:
This test failed when running with --mysqld=--loose-innodb-fast-shutdown=2 which we do on some machines
mysql-test/t/mysqldump.test:
Only run test if utf8 is used
sql/log.cc:
Fixed compiler warning
sql/mysql_priv.h:
Fixed compiler warnings
tests/mysql_client_test.c:
Don't abort test if ucs2 is not in use.
The reason for mtr warning is that collect_mysqld_features() starts mysqld with --datadir=/tmp and this
directory does not exist on Windows.
Fix : instead of passing --datadir=$opt_vardir/tmp in collect_mysqld_features() just use --datadir=.
mysqld does not need a correct directory, just an existing one, as it is started with --help ---verbose -skip-grant-tables.
mysql-test/suite/funcs_1/r/is_columns_is.result:
Updated test results
mysql-test/suite/pbxt/r/client_xml.result:
Updated test results
mysys/Makefile.am:
Added back my_port.c
storage/xtradb/buf/buf0buddy.c:
Removed not used variables
storage/xtradb/fil/fil0fil.c:
Added casts to fix compiler warnings
calc_daynr() function returns negative result
if malformed date with zero year and month is used.
Attempt to calculate week day on negative value
leads to crash. The fix is return NULL for
'W', 'a', 'w' specifiers if zero year and month is used.
Additional fix for calc_daynr():
--added assertion that result can not be negative
--return 0 if zero year and month is used
mysql-test/r/func_time.result:
test case
mysql-test/t/func_time.test:
test case
sql-common/my_time.c:
--added assertion that result can not be negative
--return 0 if zero year and month is used
sql/item_timefunc.cc:
eturn NULL for 'W', 'a', 'w' specifiers
if zero year and month is used.
mysql-test/t/loaddata.test:
test for bug; without fix, running the test with --valgrind would show the leak
and make the test fail.
sql/sql_load.cc:
* In READ_INFO class, 'need_end_io_cache' is true as long as init_io_cache() was called,
so if it's true, we need to call end_io_cache(), to free memory allocated
by init_io_cache(). No matter the value of 'error'. In the bug's scenario,
'error' was set to true in read_sep_field() because
'1' (read from file) isn't suitable to load into a geometric column. Because of
'error', end_io_cache() was not called.
Note: end_io_cache() calls my_b_flush_io_cache(), which will do nothing wrong given
that the file is opened for reads only; see the init_io_cache() call which uses
only those read-only types:
(get_it_from_net) ? READ_NET : (is_fifo ? READ_FIFO : READ_CACHE).
IF the cache were rather used to write to the file, my_b_flush_io_cache() may
write to it, and it may be questionable to write to the file
if 'error' is true. But here there's no problem.
* Now that 'need_end_io_cache' is checked even if 'error' is true, it needs
to be initialized in all cases.
* Bonus: move some variables to the initialization list.
Before sorting HAVING condition is split into two parts,
first part is a table related condition and the rest of is
HAVING part. Extraction of HAVING part does not take into account
the fact that some of conditions might be non-const but
have 'used_tables' == 0 (independent subqueries)
and because of that these conditions are cut off by
make_cond_for_table() function.
The fix is to use (table_map) 0 instead of used_tables in
third argument for make_cond_for_table() function.
It allows to extract elements which belong to sorted
table and in addition elements which are independend
subqueries.
mysql-test/r/having.result:
test case
mysql-test/t/having.test:
test case
sql/sql_select.cc:
The fix is to use (table_map) 0 instead of used_tables in
third argument for make_cond_for_table() function.
It allows to extract elements which belong to sorted
table and in addition elements which are independend
subqueries.
