tmp_table_size can now be set to 0 (to disable in memory internal temp tables)
Improved speed for internal Maria temp tables:
- Don't use packed keys, except with long text fields.
- Don't copy key all accessed pages during key search.
Some new benchmark tests to sql-bench (for group by)
BUILD/compile-pentium64-gcov:
Update script to use same pentium_config flags as other tests
BUILD/compile-pentium64-gprof:
Update script to use same pentium_config flags as other tests
include/my_sys.h:
Added count of my_sync calls
mysql-test/r/variables.result:
tmp_table_size can now be set to 0
sql-bench/test-select.sh:
Added some new test for GROUP BY on a not key field and group by with different order by
sql/mysqld.cc:
Added count of my_sync calls
tmp_table_size can now be set to 0 (to disable in memory internal temp tables)
sql/sql_select.cc:
If tmp_table_size is 0, don't use in memory temp tables (good for benchmarking MyISAM/Maria temp tables)
Don't pack keys for Maria tables; The 8K page size makes packed keys too slow for temp tables.
storage/maria/ma_key_recover.h:
Moved definition to maria_def.h
storage/maria/ma_page.c:
Moved code used to simplify comparing of identical Maria tables to own function (page_cleanup())
Fixed that one can read a page with a read lock.
storage/maria/ma_rkey.c:
For not exact key reads, cache the page where we found key (to speed up future read-next/read-prev calls)
storage/maria/ma_search.c:
Moved code to cache last key page to separate function.
Instead of copying pages, only get a link to the page. This notable speeds up key searches on bigger tables.
storage/maria/ma_write.c:
Added comment
storage/maria/maria_def.h:
Moved page_cleanup() to separate function.
* Implemented a second partial matching strategy via table scan.
This strategy is a fallback when there is no memory for rowid merging.
* Refactored the selection and creation of partial matching strategies,
so that the choice of strategy is encapsulated in a separate method
choose_partial_match_strategy().
* Refactored the representation of partial match strategies so that:
- each strategy is represented by a polymorphic class, and
- the base class for all partial match strategies contains common
execution code.
* Added an estimate of the memory needed for the rowid merge strategy,
and the system variable "rowid_merge_buff_size" to control the maximum
memory to be used by the rowid merge algorithm.
* Added two optimizer_switch system variables to control the choice of
partial match strategy:
"partial_match_rowid_merge", "partial_match_table_scan".
* Fixed multiple problems with deallocation of resources by the partial
match strategies.
sql/mysql_priv.h:
* Added two optimizer_switch system variables to control the choice of
partial match strategy:
"partial_match_rowid_merge", "partial_match_table_scan".
sql/mysqld.cc:
* Added two optimizer_switch system variables to control the choice of
partial match strategy:
"partial_match_rowid_merge", "partial_match_table_scan".
* Added a system variable "rowid_merge_buff_size" to control the maximum
memory to be used by the rowid merge algorithm.
sql/set_var.cc:
* Added a system variable "rowid_merge_buff_size" to control the maximum
memory to be used by the rowid merge algorithm.
sql/sql_class.h:
* Added a system variable "rowid_merge_buff_size" to control the maximum
memory to be used by the rowid merge algorithm.
support-files/build-tags:
Newer versions of BZR require the recursive flag in order to list all files.
This patch fixes some typos and poorly formulated sentences in
the output from mysqld --help --verbose.
Some of the problems described in the bug report are already
handled by the patch for Bug#49447, and are therefore not
included in this patch.
A client doing multiple mysql_library_init() and
mysql_library_end() calls over the lifetime of the process may
experience lost character set data, potentially even a
SIGSEGV.
This patch reinstates the reloading of character set data when
a mysql_library_init() is done after a mysql_library_end().
backporting of bug@30703 to 5.1.
The fixes are backed up with a regression test.
mysql-test/include/test_fieldsize.inc:
waiting to stop is to be actually exclusively for SQL thread.
mysql-test/suite/rpl/r/rpl_show_slave_running.result:
new results file is added.
mysql-test/suite/rpl/t/rpl_show_slave_running.test:
regression test for bug#30703 is added.
sql/mysqld.cc:
refining `show status like slave_running' handler to correspond to one of
`show slave status'.
sql/slave.cc:
A dbug-sync point is added to complement the regression test.
myisam-recover options changed from OFF to 'DEFAULT' to get less change of data loss when using MyISAM.
(The disadvantage is that changed MyISAM tables will be checked at access time; Use --myisam-recover=OFF for old behavior)
Don't call extra(HA_EXTRA_FORCE_REOPEN) in ALTER TABLE if table is locked as this will mark table as crashed!
Added assert to detect if we accidently would use MyISAM versioning in MySQL
include/my_base.h:
Mark NOT_USED as USED, as we now use this as a flag to not call extra()
mysql-test/mysql-test-run.pl:
Don't write all options when there is something wrong with the arguments
mysql-test/r/sp-destruct.result:
Add missing flush of mysql.proc (as the test copied live tables)
mysql-test/r/variables.result:
myisam-recover options changed to 'default'
mysql-test/r/view.result:
Don't show create time in result
mysql-test/suite/maria/t/maria-recovery2-master.opt:
Don't run test with myisam-recover (as this produces extra warnings during simulated death)
mysql-test/t/sp-destruct.test:
Add missing flush of mysql.proc (as the test copied live tables)
mysql-test/t/view.test:
Don't show create time in result
sql/lock.cc:
Added marker if table was deleted to argument list
sql/mysql_priv.h:
Added marker if table was deleted to argument list
sql/mysqld.cc:
myisam-recover options changed from OFF to 'DEFAULT' to get less change of data loss when using MyISAM
Allow one to specify OFF as argument to myisam-recover (was default before but one couldn't specify it)
sql/sql_base.cc:
Mark if table is going to be deleted
sql/sql_delete.cc:
Mark if table is going to be deleted
sql/sql_table.cc:
Mark if table is going to be deleted
Don't call extra(HA_EXTRA_FORCE_REOPEN) in ALTER TABLE if table is locked as this will mark table as crashed!
sql/table.cc:
Signal to handler if table is getting deleted as part of getting droped from table cache.
sql/table.h:
Added marker if table is going to be deleted.
storage/maria/ha_maria.cc:
Don't search for transaction handler if file is not transactional or outside of transaction
(Fixed possible core dump)
storage/maria/ma_blockrec.c:
Don't write changed information if table is going to be deleted.
storage/maria/ma_close.c:
Don't write changed information if table is going to be deleted.
storage/maria/ma_extra.c:
Mark tables that are deleted as crased, to ensure good behavior on restart if we suddenly crash.
storage/maria/ma_locking.c:
Cleanup
storage/maria/ma_recovery.c:
We need trnman to be inited during redo phase (to be able to open tables checked with maria_chk)
storage/maria/maria_def.h:
Added marker if table is going to be deleted.
storage/myisam/mi_close.c:
Don't write changed information if table is going to be deleted.
storage/myisam/mi_extra.c:
Mark tables that are deleted as crased, to ensure good behavior on restart if we suddenly crash.
storage/myisam/mi_open.c:
Added assert to detect if we accidently would use MyISAM versioning in MySQL
storage/myisam/myisamdef.h:
Added marker if table is going to be deleted.
Fix for Bug#31173: mysqlslap.exe crashes if called without any parameters
.bzrignore:
Fixed .bzrignore rules. Many were simply not ignoring what they were meant to.
client/mysqlslap.c:
Fixed bug for Bug#31173: mysqlslap.exe crashes if called without any parameters
The original patch could cause memory leaks and odd problems depending on how connection was made.
This code ensures that all mysql_options() are set for each mysql_real_connect().
(This patch by Monty)
mysys/my_thr_init.c:
Fixed multiply-initialized critical section on Windows, due to code incorrectly
checking the wrong field in an attempt to prevent multiple-initialization.
sql-common/client.c:
Don't use shared memory if it's not set (for example after failed mysql_real_connect).
Ensure that mysql_close() resets all resources so that it's safe to call it twice.
(Patch by monty, related to Bug#31173: mysqlslap.exe crashes if called without any parameters)
sql/CMakeLists.txt:
Added page fault counters for SHOW PROFILE on Windows.
sql/mysqld.cc:
Fixed attempt to set a NULL event. The code now only sets the event if appropriate (i.e. shared memory is being used)
sql/sql_profile.cc:
Added page fault counters for SHOW PROFILE on Windows.
sql/sql_profile.h:
Added page fault counters for SHOW PROFILE on Windows.
sql/udf_example.def:
Some cleanup functions were not exported from udf_example.dll, causing them to
never be executed, and as a result multiple-initialization of kernel objects
occurred and resources were not being freed correctly.
storage/maria/ma_close.c:
Condition variable share->key_del_cond was never being destroyed, while its
containing heap block was being freed in maria_close(), leaking kernel
resources.
The problem was that the dbug facility was being used after the
per-thread dbug state had already been finalized. The was present
in a few functions which invoked decrement_handler_count, which
in turn invokes my_thread_end on Windows. In my_thread_end, the
per-thread dbug state is finalized. Any use after the state is
finalized ends up creating a new state.
The solution is to process the exit of a function before the
decrement_handler_count function is called.
sql/mysqld.cc:
Process the function exit before decrement_handler_count is
called, as it can end the per-thread dbug state on Windows.
--default-character-set and --character-set-server such
that only the first will give a deprecation warning.
Apart from that, the two options should do the same.
WL#5182 is a follow-up to WL#5154, deprecating a few more options
and system variables.
client/client_priv.h:
The warning message has been changed to not include
a specific version number in the text.
client/mysql.cc:
--no-tee is deprecated
client/mysqldump.c:
--all is deprecated
-a now points to create-options
mysql-test/r/mysqlbinlog.result:
Warning text changed
mysql-test/suite/rpl/r/rpl_row_mysqlbinlog.result:
Warning text changed
sql/mysql_priv.h:
The warning message has been changed to not include
a specific version number in the text.
sql/mysqld.cc:
--use-symbolic-links is deprecated
-s now points to --symbolic-links
--warnings is deprecated
-W now points to --log-warnings
myisam_max_extra_sort_file_size is deprecated
record_buffer is deprecated
--log-update is deprecated
--sql-bin-update-same is deprecated
--skip-locking is deprecated
--skip-symlink is deprecated
--enable-locking is deprecated
--delay-key-write-for-all-tables is deprecated
Several items said to be deprecated in the 4.1 manual
have never been removed. This worklog adds deprecation
warnings when these items are used, and warns the user
that the items will be removed in MySQL 5.6.
A couple of previously deprecation decision have been
reversed (see single file comments)
client/client_priv.h:
Macro similar to the one in the server (mysql_priv.h)
for printing a deprecation warning message
client/mysql.cc:
no-auto-rehash will not be deprecated
skip-line-numbers will not be deprecated
skip-column-names will not be deprecated
no-pager is deprecated
set-variable is deprecated
no-named-commands is deprecated
client/mysqladmin.cc:
set-variable is deprecated
client/mysqlbinlog.cc:
position is deprecated
client/mysqldump.c:
first-slave is deprecated
no-set-names is deprecated
set-variable is deprecated
mysql-test/r/mysqlbinlog.result:
Adding the [Warning] to the test case, just to show that the
deprecation works.
The test case will be changed in Celosia to use --start-position.
mysys/my_getopt.c:
set-variable (include -O) is deprecated
scripts/mysqld_multi.sh:
Warning for mysqld_multi
sql/mysqld.cc:
default-collation is deprecated
log-bin-trust-routine-creators is deprecated
set-variable is deprecated
default-character-set is deprecated
safe-show-database is deprecated
sql/share/errmsg.txt:
Added version number for sql_log_update deprecation message.
It is well-known that due to concurrency issues, a slave can become
inconsistent when a transaction contains updates to both transaction and
non-transactional tables in statement and mixed modes.
In a nutshell, the current code-base tries to preserve causality among the
statements by writing non-transactional statements to the txn-cache which
is flushed upon commit. However, modifications done to non-transactional
tables on behalf of a transaction become immediately visible to other
connections but may not immediately get into the binary log and therefore
consistency may be broken.
In general, it is impossible to automatically detect causality/dependency
among statements by just analyzing the statements sent to the server. This
happen because dependency may be hidden in the application code and it is
necessary to know a priori all the statements processed in the context of
a transaction such as in a procedure. Moreover, even for the few cases that
we could automatically address in the server, the computation effort
required could make the approach infeasible.
So, in this patch we introduce the option
- "--binlog-direct-non-transactional-updates" that can be used to bypass
the current behavior in order to write directly to binary log statements
that change non-transactional tables.
mysql-test/extra/rpl_tests/rpl_mixing_engines.inc:
Backported this from Celosia to improve the test cases related to the NTM issue.
sql/log.cc:
Checks the --binlog-direct-non-transactional-updates before choosing
to either use the trxn-cache or not.
sql/mysqld.cc:
Introduces the option --binlog-direct-non-transactional-updates.
sql/set_var.cc:
Introduces the option --binlog-direct-non-transactional-updates.
sql/sql_class.h:
Introduces the option --binlog-direct-non-transactional-updates.
- Marked a couple of tests with --big
- Fixed xtradb/handler/ha_innodb.cc to call explain_filename()
storage/xtradb/handler/ha_innodb.cc:
Call explain_filename() to get proper names for partitioned tables
- Make index condition pushdown be controlled by an @@optimizer_switch flag,
not by @@engine_condition_pushdown
- Make MRR buffer size be controlled by @@mrr_buffer_size, not
by @@read_rnd_buffer_size
- Move parts of code to separate files
- Code cleanup
- Add --sorted_result to some SELECTs in tests.
When compressed myisam files are opened, they are always memory mapped
sometimes causing memory swapping problems.
When we mmap the myisam compressed tables of size greater than the memory
available, the kswapd0 process utilization is very high consuming 30-40% of
the cpu. This happens only with linux kernels older than 2.6.9
With newer linux kernels, we don't have this problem of high cpu consumption
and this option may not be required.
The option 'myisam_mmap_size' is added to limit the amount of memory used for
memory mapping of myisam files. This option is not dynamic.
The default value on 32 bit system is 4294967295 bytes and on 64 bit system it
is 18446744073709547520 bytes.
Note: Testcase only tests the option variable. The actual bug has be to
tested manually.
include/my_global.h:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
define SIZE_T_MAX
include/myisam.h:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
declare 'myisam_mmap_size' and 'myisam_mmap_used' variables and the mutex
THR_LOCK_myisam_mmap
myisam/mi_packrec.c:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
add 'myisam_mmap_size' option which limits the memory available to mmap of
myisam files
myisam/mi_static.c:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
declare 'myisam_mmap_size' and 'myisam_mmap_used' variables and the mutex
THR_LOCK_myisam_mmap
myisam/myisamdef.h:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
move MEMMAP_EXTRA_MARGIN to myisam.h so that it can be used in mysqld.cc
mysql-test/r/variables.result:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
Testcase for BUG#37408 to test the myisam_mmap_size option
mysql-test/t/variables.test:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
Testcase for BUG#37408 to test the myisam_mmap_size option
mysys/my_thr_init.c:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
intialize the lock THR_LOCK_myisam_mmap
sql/mysqld.cc:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
add the 'myisam_mmap_size' option
sql/set_var.cc:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
add the 'myisam_mmap_size' to the SHOW VARIABLES list
WL#2474 "Multi Range Read: Change the default MRR implementation to implement new MRR interface"
WL#2475 "Batched range read functions for MyISAM/InnoDb"
"Index condition pushdown for MyISAM/InnoDB"
Igor's fix from sp1r-igor@olga.mysql.com-20080330055902-07614:
There could be observed the following problems:
1. EXPLAIN did not mention pushdown conditions from on expressions in the
'extra' column. As a result if a query had no where conditions pushed
down to a table, but had on conditions pushed to this table the 'extra'
column in the EXPLAIN for the table missed 'using where'.
2. Conditions for ref access were not eliminated from on expressions
though such conditions were eliminated from the where condition.
As documented in the bug report, the double checked locking
pattern has inherent issues, and cannot guarantee correct
initialization.
This patch replaces the logic in init_available_charsets()
with the use of pthread_once(3). A wrapper function,
my_pthread_once(), is introduced and is used in lieu of direct
calls to init_available_charsets(). Related defines
MY_PTHREAD_ONCE_* are also introduced.
For the Windows platform, the implementation in lp:sysbench is
ported. For single-thread use, a simple define calls the
function and sets the pthread_once control variable.
Charset initialization is modified to use my_pthread_once().
include/my_no_pthread.h:
Dummy my_pthread_once() for single thread use.
include/my_pthread.h:
Declaration for new function my_pthread_once().
mysys/charset.c:
Logic in init_available_charsets() is simplified.
Using my_pthread_once() for all calls to this func.
mysys/my_winthread.c:
Windows implementation of my_pthread_once().
The help text for --init-slave=name:
"Command(s) that are executed when a slave connects to this master".
This text indicate that the --init-slave option is set on a master
server, and the master server passes the option's argument to slave
which connects to it. This is wrong. Actually the --init-slave option
just can be set on a slave server, and then the slave server executes
the argument each time the SQL thread starts.
Correct the help text for --init-slave option as following:
"Command(s) that are executed by a slave server each time the SQL thread starts."
sql/mysqld.cc:
Correct the help text for --init-slave option.
The help text for --init-slave=name:
"Command(s) that are executed when a slave connects to this master".
This text indicate that the --init-slave option is set on a master
server, and the master server passes the option's argument to slave
which connects to it. This is wrong. Actually the --init-slave option
just can be set on a slave server, and then the slave server executes
the argument each time the SQL thread starts.
Correct the help text for --init-slave option as following:
"Command(s) that are executed by a slave server each time the SQL thread starts."
sql/mysqld.cc:
Correct the help text for --init-slave option.
Fixed coredump in sql_plugin.cc:intern_plugin_lock() on mysqld start with PBXT
sql/mysqld.cc:
Fixed coredump in sql_plugin.cc:intern_plugin_lock() on mysqld start with PBXT
sql/share/errmsg.txt:
Row numbers are always positive
sql/sql_base.cc:
Fixed race condition in lock tables when killing insert_delayed thread.
This fixes Bug#47017 rpl_timezone fails on PB-2 with mismatch error
(Note that the patch only adds a continue; The rest is (required) indentation changes)
sql/sql_class.cc:
Fixed wrong output for high end machines in outfile_loaddata.
(Problem was that ER_TRUNCATED_WRONG_VALUE_FOR_FIELD expects ulong, not ulonglong)
sql/sql_insert.cc:
Ensure that if we get a lock problem with delayed_insert, the error is logged.
This patch fixes three bugs as follows. First, aborting the server while purging
binary logs might generate orphan files due to how the purge operation was
implemented:
(purge routine - sql/log.cc - MYSQL_BIN_LOG::purge_logs)
1 - register the files to be removed in a temporary buffer.
2 - update the log-bin.index.
3 - flush the log-bin.index.
4 - erase the files whose names where register in the temporary buffer
in step 1.
Thus a failure while executing step 4 would generate an orphan file. Second,
a similar issue might happen while creating a new binary as follows:
(create routine - sql/log.cc - MYSQL_BIN_LOG::open)
1 - open the new log-bin.
2 - update the log-bin.index.
Thus a failure while executing step 1 would generate an orphan file.
To fix these issues, we record the files to be purged or created before really
removing or adding them. So if a failure happens such records can be used to
automatically remove dangling files. The new steps might be outlined as follows:
(purge routine - sql/log.cc - MYSQL_BIN_LOG::purge_logs)
1 - register the files to be removed in the log-bin.~rec~ placed in
the data directory.
2 - update the log-bin.index.
3 - flush the log-bin.index.
4 - delete the log-bin.~rec~.
(create routine - sql/log.cc - MYSQL_BIN_LOG::open)
1 - register the file to be created in the log-bin.~rec~
placed in the data directory.
2 - open the new log-bin.
3 - update the log-bin.index.
4 - delete the log-bin.~rec~.
(recovery routine - sql/log.cc - MYSQL_BIN_LOG::open_index_file)
1 - open the log-bin.index.
2 - open the log-bin.~rec~.
3 - for each file in log-bin.~rec~.
3.1 Check if the file is in the log-bin.index and if so ignore it.
3.2 Otherwise, delete it.
The third issue can be described as follows. The purge operation was allowing
to remove a file in use thus leading to the loss of data and possible
inconsistencies between the master and slave. Roughly, the routine was only
taking into account the dump threads and so if a slave was not connect the
file might be delete even though it was in use.
implement Davi's review suggestions (post-push fixes)
include/violite.h:
Use official abbreviation for milliseconds (ms)
sql/mysqld.cc:
Fix formatting
Add error handling for the case of CreateEvent error
vio/vio.c:
Use official abbreviation for milliseconds(ms)
Remove superfluous memset
Fix formatting
vio/viosocket.c:
Use official abbreviation for milliseconds (ms)
Use size_t datatype instead of int in pipe_complete_io
- Moved some code from innodb_plugin to xtradb, to ensure that all tests runs
- Did changes in pbxt and maria storage engines becasue of changes in thd->query
- Reverted wrong code in sql_table.cc for how ROW_FORMAT is used.
This is a re-commit of Monty's merge to eliminate an extra commit from
MySQL-5.1.42 that was accidentally included in the merge.
This is a merge of the MySQL 5.1.41 clone-off (clone-5.1.41-build). In
case there are any extra changes done before final MySQL 5.1.41
release, these will need to be merged later before MariaDB 5.1.41
release.
Bug#31621: Windows server hanging during shutdown using named pipes
and idle connection
Problem: when idle pipe connection is forcefully closed with KILL
statement or when the server goes down, thread that is closing connection
would hang infinitely in CloseHandle(). The reason for the hang is that
named pipe operations are performed synchronously. In this mode all IOs
on pipe are serialized, that is CloseHandle() will not abort ReadFile()
in another thread, but wait for ReadFile() to complete.
The fix implements asynchrnous mode for named pipes, where operation of file
are not synchronized. Read/Write operation would fire an async IO and wait for
either IO completion or timeout.
Note, that with this patch timeouts are properly handled for named pipes.
Post-review: Win32 timeout code has been fixed for named pipes and shared
memory. We do not store pointer to NET in vio structure, only the read and
write timeouts.
include/violite.h:
Add pipe_overlapped to Vio structure for async IO for named pipes.
sql-common/client.c:
Use asynchronous pipe IO.
sql/mysqld.cc:
Use asynchronous pipe IO.
vio/vio.c:
-Refactor timeouts for win32 protocols: shared memory and named pipes.
Store read/write timeout in VIO structure, instead of storing pointer
to NET. New function vio_win32_timeout called indirectly via
vio_timeout changes these values.
vio/vio_priv.h:
Remove vio_ignore_timeout.
Add vio_win32_timeout to be used for named pipes and shared memory.
vio/viosocket.c:
Use async IO for named pipes.
After issuing IO, wait for either IO completion, pipe_close_event
or timeout.
Refactor timeouts for named pipe and shared memory.
sql/mysqld.cc:
Explicit type casting required by windows x64 compiler.
storage/xtradb/include/srv0srv.h:
Parameters should be declared as ulong.
storage/xtradb/srv/srv0srv.c:
Parameters should be declared as ulong.
This makes them suitable for tar archices right away and also are easier to copy
Don't disable federated storage engine by default.
Don't allow one to disable the Maria storage engine if it's used for temp tables
BUILD/SETUP.sh:
Compile by default MySQL clients with libmysqldclient.a (not .so)
This makes them suitable for tar archices right away and also are easier to copy
scripts/make_binary_distribution.sh:
Abort it clients are compiled with the shared libmysqlclient.so library
sql/mysqld.cc:
Don't call kill_mysql() if signal handler is not setup (causes a core dump).
This is only relevant when starting with --gdb.
sql/sql_plugin.cc:
Don't disable federated storage engine by default.
Don't allow one to disable the Maria storage engine if it's used for temp tables
