The original code comes, as far as I know, from Google (Mark Callaghan's team) with additional work from Percona, Ourdelta and Weldon Whipple.
This code provides the same functionallity, but with a lot of changes to make it faster and better fit the MariaDB infrastucture.
Added new status variables:
- Com_show_client_statistics, Com_show_index_statistics, Com_show_table_statistics, Com_show_user_statistics
- Access_denied_errors, Busy_time (clock time), Binlog_bytes_written, Cpu_time, Empty_queries, Rows_sent, Rows_read
Added new variable / startup option 'userstat' to control if user statistics should be enabled or not
Added my_getcputime(); Returns cpu time used by this thread.
New FLUSH commands:
- FLUSH SLOW QUERY LOG
- FLUSH TABLE_STATISTICS
- FLUSH INDEX_STATISTICS
- FLUSH USER_STATISTICS
- FLUSH CLIENT_STATISTICS
New SHOW commands:
- SHOW CLIENT_STATISTICS
- SHOW USER_STATISTICS
- SHOW TABLE_STATISTICS
- SHOW INDEX_STATISTICS
New Information schemas:
- CLIENT_STATISTICS
- USER_STATISTICS
- INDEX_STATISTICS
- TABLE_STATISTICS
Added support for all new flush commands to mysqladmin
Added handler::ha_... wrappers for all handler read calls to do statistics counting
- Changed all code to use new ha_... calls
- Count number of read rows, changed rows and rows read trough an index
Added counting of number of bytes sent to binary log (status variable Binlog_bytes_written)
Added counting of access denied errors (status variable Access_denied_erors)
Bugs fixed:
- Fixed bug in add_to_status() and add_diff_to_status() where longlong variables where threated as long
- CLOCK_GETTIME was not propely working on Linuxm
client/mysqladmin.cc:
Added support for all new flush commmands and some common combinations:
flush-slow-log
flush-table-statistics
flush-index-statistics
flush-user-statistics
flush-client-statistics
flush-all-status
flush-all-statistics
configure.in:
Added checking if clock_gettime needs the librt.
(Fixes Bug #37639 clock_gettime is never used/enabled in Linux/Unix)
include/my_sys.h:
Added my_getcputime()
include/mysql_com.h:
Added LIST_PROCESS_HOST_LEN & new REFRESH target defines
mysql-test/r/information_schema.result:
New information schema tables added
mysql-test/r/information_schema_all_engines.result:
New information schema tables added
mysql-test/r/information_schema_db.result:
New information schema tables added
mysql-test/r/log_slow.result:
Added testing that flosh slow query logs is accepted
mysql-test/r/status_user.result:
Basic testing of user, client, table and index statistics
mysql-test/t/log_slow.test:
Added testing that flosh slow query logs is accepted
mysql-test/t/status_user-master.opt:
Ensure that we get a fresh restart before running status_user.test
mysql-test/t/status_user.test:
Basic testing of user, client, table and index statistics
mysys/my_getsystime.c:
Added my_getcputime()
Returns cpu time used by this thread.
sql/authors.h:
Updated authors to have core and original MySQL developers first.
sql/event_data_objects.cc:
Updated call to mysql_reset_thd_for_next_command()
sql/event_db_repository.cc:
Changed to use new ha_... calls
sql/filesort.cc:
Changed to use new ha_... calls
sql/ha_partition.cc:
Changed to use new ha_... calls
Fixed comment syntax
sql/handler.cc:
Changed to use new ha_... calls
Reset table statistics
Added code to update global table and index status
Added counting of rows changed
sql/handler.h:
Added table and index statistics variables
Added function reset_statistics()
Added handler::ha_... wrappers for all handler read calls to do statistics counting
Protected all normal read calls to ensure we use the new calls in the server.
Made ha_partition a friend class so that partition code can call the old read functions
sql/item_subselect.cc:
Changed to use new ha_... calls
sql/lex.h:
Added keywords for new information schema tables and flush commands
sql/log.cc:
Added flush_slow_log()
Added counting of number of bytes sent to binary log
Removed not needed test of thd (It's used before, so it's safe to use)
Added THD object to MYSQL_BIN_LOG::write_cache() to simplify statistics counting
sql/log.h:
Added new parameter to write_cache()
Added flush_slow_log() functions.
sql/log_event.cc:
Updated call to mysql_reset_thd_for_next_command()
Changed to use new ha_... calls
sql/log_event_old.cc:
Updated call to mysql_reset_thd_for_next_command()
Changed to use new ha_... calls
sql/mysql_priv.h:
Updated call to mysql_reset_thd_for_next_command()
Added new statistics functions and variables needed by these.
sql/mysqld.cc:
Added new statistics variables and structures to handle these
Added new status variables:
- Com_show_client_statistics, Com_show_index_statistics, Com_show_table_statistics, Com_show_user_statistics
- Access_denied_errors, Busy_time (clock time), Binlog_bytes_written, Cpu_time, Empty_queries, Rows_set, Rows_read
Added new option 'userstat' to control if user statistics should be enabled or not
sql/opt_range.cc:
Changed to use new ha_... calls
sql/opt_range.h:
Changed to use new ha_... calls
sql/opt_sum.cc:
Changed to use new ha_... calls
sql/records.cc:
Changed to use new ha_... calls
sql/set_var.cc:
Added variable 'userstat'
sql/sp.cc:
Changed to use new ha_... calls
sql/sql_acl.cc:
Changed to use new ha_... calls
Added counting of access_denied_errors
sql/sql_base.cc:
Added call to statistics functions
sql/sql_class.cc:
Added usage of org_status_var, to store status variables at start of command
Added functions THD::update_stats(), THD::update_all_stats()
Fixed bug in add_to_status() and add_diff_to_status() where longlong variables where threated as long
sql/sql_class.h:
Added new status variables to status_var
Moved variables that was not ulong in status_var last.
Added variables to THD for storing temporary values during statistics counting
sql/sql_connect.cc:
Variables and functions to calculate user and client statistics
Added counting of access_denied_errors and lost_connections
sql/sql_cursor.cc:
Changed to use new ha_... calls
sql/sql_handler.cc:
Changed to use new ha_... calls
sql/sql_help.cc:
Changed to use new ha_... calls
sql/sql_insert.cc:
Changed to use new ha_... calls
sql/sql_lex.h:
Added SQLCOM_SHOW_USER_STATS, SQLCOM_SHOW_TABLE_STATS, SQLCOM_SHOW_INDEX_STATS, SQLCOM_SHOW_CLIENT_STATS
sql/sql_parse.cc:
Added handling of:
- SHOW CLIENT_STATISTICS
- SHOW USER_STATISTICS
- SHOW TABLE_STATISTICS
- SHOW INDEX_STATISTICS
Added handling of new FLUSH commands:
- FLUSH SLOW QUERY LOGS
- FLUSH TABLE_STATISTICS
- FLUSH INDEX_STATISTICS
- FLUSH USER_STATISTICS
- FLUSH CLIENT_STATISTICS
Added THD parameter to mysql_reset_thd_for_next_command()
Added initialization and calls to user statistics functions
Added increment of statistics variables empty_queries, rows_sent and access_denied_errors.
Added counting of cpu time per query
sql/sql_plugin.cc:
Changed to use new ha_... calls
sql/sql_prepare.cc:
Updated call to mysql_reset_thd_for_next_command()
sql/sql_select.cc:
Changed to use new ha_... calls
Indentation changes
sql/sql_servers.cc:
Changed to use new ha_... calls
sql/sql_show.cc:
Added counting of access denied errors
Added function for new information schema tables:
- CLIENT_STATISTICS
- USER_STATISTICS
- INDEX_STATISTICS
- TABLE_STATISTICS
Changed to use new ha_... calls
sql/sql_table.cc:
Changed to use new ha_... calls
sql/sql_udf.cc:
Changed to use new ha_... calls
sql/sql_update.cc:
Changed to use new ha_... calls
sql/sql_yacc.yy:
Add new show and flush commands
sql/structs.h:
Add name_length to KEY to avoid some strlen
Added cache_name to KEY for fast storage of keyvalue in cache
Added structs USER_STATS, TABLE_STATS, INDEX_STATS
Added function prototypes for statistics functions
sql/table.cc:
Store db+table+index name into keyinfo->cache_name
sql/table.h:
Added new information schema tables
sql/tztime.cc:
Changed to use new ha_... calls
Implemented the server infrastructure for the fix:
1. Added a function LEX_STRING *thd_query_string(THD) to return
a LEX_STRING structure instead of char *.
This is the function that must be called in innodb instead of
thd_query()
2. Did some encapsulation in THD : aggregated thd_query and
thd_query_length into a LEX_STRING and made accessor and mutator
methods for easy code updating.
3. Updated the server code to use the new methods where applicable.
Implemented the server infrastructure for the fix:
1. Added a function LEX_STRING *thd_query_string(THD) to return
a LEX_STRING structure instead of char *.
This is the function that must be called in innodb instead of
thd_query()
2. Did some encapsulation in THD : aggregated thd_query and
thd_query_length into a LEX_STRING and made accessor and mutator
methods for easy code updating.
3. Updated the server code to use the new methods where applicable.
Post-push fix.
Problem: After the original bugfix, if a statement is unsafe,
binlog_format=mixed, and engine is statement-only, a warning was
generated and the statement executed. However, it is a fundamental
principle of binlogging that binlog_format=mixed should guarantee
correct logging, no compromise. So correct behavior is to generate
an error and don't execute the statement.
Fix: Generate error instead of warning.
Since issue_unsafe_warnings can only generate one error message,
this allows us to simplify the code a bit too:
decide_logging_format does not have to save the error code for
issue_unsafe_warnings
mysql-test/suite/binlog/r/binlog_statement_insert_delayed.result:
updated result file
mysql-test/suite/binlog/r/binlog_stm_ps.result:
updated result file
mysql-test/suite/binlog/r/binlog_stm_unsafe_warning.result:
updated result file
mysql-test/suite/binlog/r/binlog_unsafe.result:
updated result file
mysql-test/suite/rpl/r/rpl_stm_found_rows.result:
updated result file
mysql-test/suite/rpl/r/rpl_stm_loadfile.result:
updated result file
mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result:
updated result file
mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors.test:
updated test:
- ER_BINLOG_UNSAFE_AND_STMT_ENGINE is now an error.
- added test for multiple types of unsafety
sql/share/errmsg.txt:
- Reformulated ER_BINLOG_UNSAFE_AND_STMT_ENGINE to reflect that it
is now an error, not a warning.
- Added "Reason for unsafeness" to ER_BINLOG_UNSAFE_STATEMENT and
ER_BINLOG_UNSAFE_AND_STMT_ENGINE.
sql/sql_class.cc:
In decide_logging_format:
- generate an error immediately in case 3, instead of scheduling a
warning to be generated later. also updated comments accordingly
- in case 7, there is only one unsafe warning error code now, so we
don't need to store it in binlog_unsafe_warning_flags
(see changes in sql_lex.h)
- fixed compilation warning in DBUG_PRINT
In issue_binlog_warning:
- moved array of error codes to sql_lex.h (so that they are
accessible also from decide_logging_format)
- simplified code after the first set of bits in
binlog_unsafe_warning_flags was removed
sql/sql_class.h:
- got rid of enum_binlog_stmt_warning. It's not needed anymore
since we only have one type of unsafe warning (one of them
turned into an error)
- updated comments accordingly
sql/sql_lex.cc:
added initialization of the array of error codes that has been
moved from THD::issue_unsafe_warnings to LEX.
sql/sql_lex.h:
Moved array of error codes from THD::issue_unsafe_warnings to LEX.
Post-push fix.
Problem: After the original bugfix, if a statement is unsafe,
binlog_format=mixed, and engine is statement-only, a warning was
generated and the statement executed. However, it is a fundamental
principle of binlogging that binlog_format=mixed should guarantee
correct logging, no compromise. So correct behavior is to generate
an error and don't execute the statement.
Fix: Generate error instead of warning.
Since issue_unsafe_warnings can only generate one error message,
this allows us to simplify the code a bit too:
decide_logging_format does not have to save the error code for
issue_unsafe_warnings
local storage for query cache).
We need more than one pointer in a thread to
represent the query cache and net->query_cache_query can not be used
any more (due to ABI compatibility issues and to different life
time of NET and THD).
This is a backport of the following patch from 6.0:
----------------------------------------------------------
revno: 2476.1157.2
committer: kostja@bodhi.(none)
timestamp: Sat 2007-06-16 13:29:24 +0400
include/mysql.h.pp:
Update the ABI (NET::query_cache_query is now unused).
include/mysql_com.h:
NET::query_cache_query is now unused.
sql/mysql_priv.h:
Update signatures of ex-functios now member functions.
sql/sql_cache.cc:
Introduce THD::query_cache_tls.
sql/sql_cache.h:
Introduce THD::query_cache_tls.
sql/sql_class.cc:
Introduce THD::query_cache_tls.
sql/sql_class.h:
Introduce THD::query_cache_tls.
sql/sql_parse.cc:
Use THD::query_cache_tls.
sql/sql_select.cc:
Use THD::query_cache_tls.
local storage for query cache).
We need more than one pointer in a thread to
represent the query cache and net->query_cache_query can not be used
any more (due to ABI compatibility issues and to different life
time of NET and THD).
This is a backport of the following patch from 6.0:
----------------------------------------------------------
revno: 2476.1157.2
committer: kostja@bodhi.(none)
timestamp: Sat 2007-06-16 13:29:24 +0400
case than in corr index".
Server was unable to find existing or explicitly created supporting
index for foreign key if corresponding statement clause used field
names in case different than one used in key specification and created
yet another supporting index.
In cases when name of constraint (and thus name of generated index)
was the same as name of existing/explicitly created index this led
to duplicate key name error.
The problem was that unlike all other code Key_part_spec::operator==()
compared field names in case sensitive fashion. As result routines
responsible for getting rid of redundant generated supporting indexes
for foreign key were not working properly for versions of field names
using different cases.
mysql-test/r/innodb_mysql.result:
Added test case for bug #39932 "create table fails if column for FK
is in different case than in corr index".
mysql-test/t/innodb_mysql.test:
Added test case for bug #39932 "create table fails if column for FK
is in different case than in corr index".
sql/sql_class.cc:
Make field name comparison case-insensitive like it is
in the rest of server.
case than in corr index".
Server was unable to find existing or explicitly created supporting
index for foreign key if corresponding statement clause used field
names in case different than one used in key specification and created
yet another supporting index.
In cases when name of constraint (and thus name of generated index)
was the same as name of existing/explicitly created index this led
to duplicate key name error.
The problem was that unlike all other code Key_part_spec::operator==()
compared field names in case sensitive fashion. As result routines
responsible for getting rid of redundant generated supporting indexes
for foreign key were not working properly for versions of field names
using different cases.
storing and restoring information about foreign keys in the .FRM files and
properly displaying it in SHOW CREATE TABLE output and I_S tables.
The idea of this patch is to change type of Key_part_spec::field_name and
Key::name to LEX_STRING in order to avoid extra strlen() calls during
semantic analysis and statement execution, particularly, in code to be
implemented on the 2nd milestone of WL#148.
Note that since we are not using LEX_STRING everywhere yet (e.g. in
Create_field and KEY) and we want to limit scope of our changes we
have to do strlen() in places where we create Key and Key_part_spec
instances from objects using plain (char*) for strings. These calls
will go away during the process of further (char*) -> LEX_STRING
refactoring.
We have introduced these changes in 6.0 and backported them to 5.5
tree to make people aware of these changes as early as possible and
to simplify merges with mysql-fk and mysql-6.1-fk trees.
No test case is needed since this patch does not introduce any
user visible changes.
sql/sql_class.cc:
Key_part_spec::field_name is now LEX_STRING. Adjusted code accordingly.
sql/sql_class.h:
Changed type of Key_part_spec::field_name and Key::name to LEX_STRING in
order to avoid extra strlen() calls in code responsible for semantic
analysis and statement execution (e.g. in future code responsible for
saving/restoring info about foreign keys).
sql/sql_lex.cc:
Moved null_lex_str from sql_yacc.yy to sql_lex.cc and added its
declaration to sql_lex.h to make it accessible in other SQL-layer
modules (e.g. sql_parse.cc).
sql/sql_lex.h:
Made null_lex_str accessible from outside of sql_lex.cc.
sql/sql_parse.cc:
Key_part_spec::field_name and Key::name are now LEX_STRING. Adjusted
code accordingly.
sql/sql_table.cc:
Adjusted code to accomodate change of type to LEX_STRING for
Key_part_spec::field_name and Key::name.
sql/sql_yacc.yy:
Now Key::name and Key_part_spec::field_name are LEX_STRINGs. Adjusted
grammar to be able properly initialize them. This should allow us to
save on some strlen() calls during later stages of statement execution.
storing and restoring information about foreign keys in the .FRM files and
properly displaying it in SHOW CREATE TABLE output and I_S tables.
The idea of this patch is to change type of Key_part_spec::field_name and
Key::name to LEX_STRING in order to avoid extra strlen() calls during
semantic analysis and statement execution, particularly, in code to be
implemented on the 2nd milestone of WL#148.
Note that since we are not using LEX_STRING everywhere yet (e.g. in
Create_field and KEY) and we want to limit scope of our changes we
have to do strlen() in places where we create Key and Key_part_spec
instances from objects using plain (char*) for strings. These calls
will go away during the process of further (char*) -> LEX_STRING
refactoring.
We have introduced these changes in 6.0 and backported them to 5.5
tree to make people aware of these changes as early as possible and
to simplify merges with mysql-fk and mysql-6.1-fk trees.
No test case is needed since this patch does not introduce any
user visible changes.
"have_profiling"
1) Renamed have_community_features server system variable to
have_profiling.
2) Removed eable-community-features configure option and
ENABLE_COMMUNITY_FEATURES macro.
3) Removed COMMUNITY_SERVER macro and replaced its usage by
ENABLED_PROFILING.
Only --enable-profiling is now needed to enable profiling.
It was the only existing "community feature", so there was
no need for both configure options.
Using --enable-community-features will give a warning message
since it no longer exists.
"have_profiling"
1) Renamed have_community_features server system variable to
have_profiling.
2) Removed eable-community-features configure option and
ENABLE_COMMUNITY_FEATURES macro.
3) Removed COMMUNITY_SERVER macro and replaced its usage by
ENABLED_PROFILING.
Only --enable-profiling is now needed to enable profiling.
It was the only existing "community feature", so there was
no need for both configure options.
Using --enable-community-features will give a warning message
since it no longer exists.
Backport from 6.0 to 5.1.
Only those sync points are included, which are used in debug_sync.test.
The Debug Sync Facility allows to place synchronization points
in the code:
open_tables(...)
DEBUG_SYNC(thd, "after_open_tables");
lock_tables(...)
When activated, a sync point can
- Send a signal and/or
- Wait for a signal
Nomenclature:
- signal: A value of a global variable that persists
until overwritten by a new signal. The global
variable can also be seen as a "signal post"
or "flag mast". Then the signal is what is
attached to the "signal post" or "flag mast".
- send a signal: Assign the value (the signal) to the global
variable ("set a flag") and broadcast a
global condition to wake those waiting for
a signal.
- wait for a signal: Loop over waiting for the global condition until
the global value matches the wait-for signal.
Please find more information in the top comment in debug_sync.cc
or in the worklog entry.
.bzrignore:
WL#4259 - Debug Sync Facility
Added the symbolic link libmysqld/debug_sync.cc.
CMakeLists.txt:
WL#4259 - Debug Sync Facility
Added definition for ENABLED_DEBUG_SYNC.
configure.in:
WL#4259 - Debug Sync Facility
Added definition for ENABLED_DEBUG_SYNC.
include/my_sys.h:
WL#4259 - Debug Sync Facility
Added definition for the DEBUG_SYNC_C macro.
libmysqld/CMakeLists.txt:
WL#4259 - Debug Sync Facility
Added sql/debug_sync.cc.
libmysqld/Makefile.am:
WL#4259 - Debug Sync Facility
Added sql/debug_sync.cc.
mysql-test/include/have_debug_sync.inc:
WL#4259 - Debug Sync Facility
New include file.
mysql-test/mysql-test-run.pl:
WL#4259 - Debug Sync Facility
Added option --debug_sync_timeout.
mysql-test/r/debug_sync.result:
WL#4259 - Debug Sync Facility
New test result.
mysql-test/r/have_debug_sync.require:
WL#4259 - Debug Sync Facility
New require file.
mysql-test/t/debug_sync.test:
WL#4259 - Debug Sync Facility
New test file.
mysys/my_static.c:
WL#4259 - Debug Sync Facility
Added definition for debug_sync_C_callback_ptr.
mysys/thr_lock.c:
WL#4259 - Debug Sync Facility
Added sync point "wait_for_lock".
sql/CMakeLists.txt:
WL#4259 - Debug Sync Facility
Added debug_sync.cc and debug_sync.h.
sql/Makefile.am:
WL#4259 - Debug Sync Facility
Added debug_sync.cc and debug_sync.h.
sql/debug_sync.cc:
WL#4259 - Debug Sync Facility
New source file.
sql/debug_sync.h:
WL#4259 - Debug Sync Facility
New header file.
sql/mysqld.cc:
WL#4259 - Debug Sync Facility
Added opt_debug_sync_timeout.
Added calls to debug_sync_init() and debug_sync_end().
Fixed a purecov comment (unrelated).
sql/set_var.cc:
WL#4259 - Debug Sync Facility
Added server variable "debug_sync".
sql/set_var.h:
WL#4259 - Debug Sync Facility
Added declaration for server variable "debug_sync".
sql/share/errmsg.txt:
WL#4259 - Debug Sync Facility
Added error messages ER_DEBUG_SYNC_TIMEOUT and ER_DEBUG_SYNC_HIT_LIMIT.
sql/sql_base.cc:
WL#4259 - Debug Sync Facility
Added sync points "after_flush_unlock" and "before_lock_tables_takes_lock".
sql/sql_class.cc:
WL#4259 - Debug Sync Facility
Added initialization for debug_sync_control to THD::THD.
Added calls to debug_sync_init_thread() and debug_sync_end_thread().
sql/sql_class.h:
WL#4259 - Debug Sync Facility
Added element debug_sync_control to THD.
storage/myisam/myisamchk.c:
Fixed a typo in an error message string (unrelated).
Backport from 6.0 to 5.1.
Only those sync points are included, which are used in debug_sync.test.
The Debug Sync Facility allows to place synchronization points
in the code:
open_tables(...)
DEBUG_SYNC(thd, "after_open_tables");
lock_tables(...)
When activated, a sync point can
- Send a signal and/or
- Wait for a signal
Nomenclature:
- signal: A value of a global variable that persists
until overwritten by a new signal. The global
variable can also be seen as a "signal post"
or "flag mast". Then the signal is what is
attached to the "signal post" or "flag mast".
- send a signal: Assign the value (the signal) to the global
variable ("set a flag") and broadcast a
global condition to wake those waiting for
a signal.
- wait for a signal: Loop over waiting for the global condition until
the global value matches the wait-for signal.
Please find more information in the top comment in debug_sync.cc
or in the worklog entry.
NOTE: Backporting the patch to next-mr.
WL#4828 Augment DBUG_ENTER/DBUG_EXIT to crash MySQL in different functions
-------
The assessment of the replication code in the presence of faults is extremely
import to increase reliability. In particular, one needs to know if servers
will either correctly recovery or print out appropriate error messages thus
avoiding unexpected problems in a production environment.
In order to accomplish this, the current patch refactories the debug macros
already provided in the source code and introduces three new macros that
allows to inject faults, specifically crashes, while entering or exiting a
function or method. For instance, to crash a server while returning from
the init_slave function (see module sql/slave.cc), one needs to do what
follows:
1 - Modify the source replacing DBUG_RETURN by DBUG_CRASH_RETURN;
DBUG_CRASH_RETURN(0);
2 - Use the debug variable to activate dbug instructions:
SET SESSION debug="+d,init_slave_crash_return";
The new macros are briefly described below:
DBUG_CRASH_ENTER (function) is equivalent to DBUG_ENTER which registers the
beginning of a function but in addition to it allows for crashing the server
while entering the function if the appropriate dbug instruction is activate.
In this case, the dbug instruction should be "+d,function_crash_enter".
DBUG_CRASH_RETURN (value) is equivalent to DBUG_RETURN which notifies the
end of a function but in addition to it allows for crashing the server
while returning from the function if the appropriate dbug instruction is
activate. In this case, the dbug instruction should be
"+d,function_crash_return". Note that "function" should be the same string
used by either the DBUG_ENTER or DBUG_CRASH_ENTER.
DBUG_CRASH_VOID_RETURN (value) is equivalent to DBUG_VOID_RETURN which
notifies the end of a function but in addition to it allows for crashing
the server while returning from the function if the appropriate dbug
instruction is activate. In this case, the dbug instruction should be
"+d,function_crash_return". Note that "function" should be the same string
used by either the DBUG_ENTER or DBUG_CRASH_ENTER.
To inject other faults, for instance, wrong return values, one should rely
on the macros already available. The current patch also removes a set of
macros that were either not being used or were redundant as other macros
could be used to provide the same feature. In the future, we also consider
dynamic instrumentation of the code.
BUG#45747 DBUG_CRASH_* is not setting the strict option
---------
When combining DBUG_CRASH_* with "--debug=d:t:i:A,file" the server crashes
due to a call to the abort function in the DBUG_CRASH_* macro althought the
appropriate keyword has not been set.
NOTE: Backporting the patch to next-mr.
WL#4828 Augment DBUG_ENTER/DBUG_EXIT to crash MySQL in different functions
-------
The assessment of the replication code in the presence of faults is extremely
import to increase reliability. In particular, one needs to know if servers
will either correctly recovery or print out appropriate error messages thus
avoiding unexpected problems in a production environment.
In order to accomplish this, the current patch refactories the debug macros
already provided in the source code and introduces three new macros that
allows to inject faults, specifically crashes, while entering or exiting a
function or method. For instance, to crash a server while returning from
the init_slave function (see module sql/slave.cc), one needs to do what
follows:
1 - Modify the source replacing DBUG_RETURN by DBUG_CRASH_RETURN;
DBUG_CRASH_RETURN(0);
2 - Use the debug variable to activate dbug instructions:
SET SESSION debug="+d,init_slave_crash_return";
The new macros are briefly described below:
DBUG_CRASH_ENTER (function) is equivalent to DBUG_ENTER which registers the
beginning of a function but in addition to it allows for crashing the server
while entering the function if the appropriate dbug instruction is activate.
In this case, the dbug instruction should be "+d,function_crash_enter".
DBUG_CRASH_RETURN (value) is equivalent to DBUG_RETURN which notifies the
end of a function but in addition to it allows for crashing the server
while returning from the function if the appropriate dbug instruction is
activate. In this case, the dbug instruction should be
"+d,function_crash_return". Note that "function" should be the same string
used by either the DBUG_ENTER or DBUG_CRASH_ENTER.
DBUG_CRASH_VOID_RETURN (value) is equivalent to DBUG_VOID_RETURN which
notifies the end of a function but in addition to it allows for crashing
the server while returning from the function if the appropriate dbug
instruction is activate. In this case, the dbug instruction should be
"+d,function_crash_return". Note that "function" should be the same string
used by either the DBUG_ENTER or DBUG_CRASH_ENTER.
To inject other faults, for instance, wrong return values, one should rely
on the macros already available. The current patch also removes a set of
macros that were either not being used or were redundant as other macros
could be used to provide the same feature. In the future, we also consider
dynamic instrumentation of the code.
BUG#45747 DBUG_CRASH_* is not setting the strict option
---------
When combining DBUG_CRASH_* with "--debug=d:t:i:A,file" the server crashes
due to a call to the abort function in the DBUG_CRASH_* macro althought the
appropriate keyword has not been set.
binlog-db-db / binlog-ignore-db
InnoDB will return an error if statement based replication is used
along with transaction isolation level READ-COMMITTED (or weaker),
even if the statement in question is filtered out according to the
binlog-do-db rules set. In this case, an error should not be printed.
This patch addresses this issue by extending the existing check in
external_lock to take into account the filter rules before deciding to
print an error. Furthermore, it also changes decide_logging_format to
take into consideration whether the statement is filtered out from
binlog before decision is made.
sql/sql_base.cc:
Changed the check on decide_logging_format to take into account
whether statement is filtered or not in SBR.
sql/sql_class.cc:
Added the thd_binlog_filter_ok to INNODB_COMPATIBILITY_HOOKS set.
storage/innobase/handler/ha_innodb.cc:
Extended check in external_lock to take into consideration the
filtering when deciding to throw an error.
storage/innobase/handler/ha_innodb.h:
Added declaration of new hook.
storage/innodb_plugin/handler/ha_innodb.cc:
Extended check in external_lock to take into consideration the
filtering when deciding to throw an error.
storage/innodb_plugin/handler/ha_innodb.h:
Added declaration of new hook.
binlog-db-db / binlog-ignore-db
InnoDB will return an error if statement based replication is used
along with transaction isolation level READ-COMMITTED (or weaker),
even if the statement in question is filtered out according to the
binlog-do-db rules set. In this case, an error should not be printed.
This patch addresses this issue by extending the existing check in
external_lock to take into account the filter rules before deciding to
print an error. Furthermore, it also changes decide_logging_format to
take into consideration whether the statement is filtered out from
binlog before decision is made.
partial backport of bug43138 fix
mysql-test/r/warnings.result:
test result
mysql-test/t/warnings.test:
test case
sql/sql_class.cc:
partial backport of bug43138 fix
sql/sql_class.h:
partial backport of bug43138 fix
sql/sql_table.cc:
partial backport of bug43138 fix
* Finished Monty and Jani's merge
* Some InnoDB tests still fail (because it's old xtradb code run against
newer testsuite). They are expected to go after mergning with the latest
xtradb.
Added (rewritten) patch from Percona to get extended statistics in slow.log:
- Added handling of 'set' variables to set_var.cc. Changed sql_mode to use this
- Added extra logging to slow log of 'Thread_id, Schema, Query Cache hit, Rows sent and Rows examined'
- Added optional logging to slow log, through log_slow_verbosity, of query plan statistics
- Added new user variables log_slow_rate_limit, log_slow_verbosity, log_slow_filter
- Added log-slow-file as synonym for 'slow-log-file', as most slow-log variables starts with 'log-slow'
- Added log-slow-time as synonym for long-query-time
Some trivial MyISAM optimizations:
- In prepare for drop, flush key blocks
- Don't call mi_lock_database if my_disable_locking is used
KNOWN_BUGS.txt:
Updated file to reflect MariaDB and not the Maria storage engine
README:
Updated file to reflect MariaDB
mysql-test/r/log_slow.result:
Test new options for slow query log
mysql-test/r/variables.result:
Updated result (old version cut of things at 79 characters)
mysql-test/t/log_slow.test:
Test new options for slow query log
sql/Makefile.am:
Added log_slow.h
sql/event_data_objects.cc:
Removed not needed test for enable_slow_log (is done when the flag is tested elsewhere)
sql/events.cc:
Use the general make_set() function instead of 'symbolic_mode_representation'
sql/filesort.cc:
Added status for used query plans
sql/log.cc:
Reset counters if no query_length (from Percona's patch; Not sure if needed, but can do no harm)
Added extra logging to slow log of 'Thread_id, Schema, Query Cache hit, Rows sent and Rows examined'
Added optional logging to slow log, through log_slow_verbosity, of query plan statistics
Fixed wrong test of error condition
sql/log_slow.h:
Defines and variables for log_slow_verbosity and log_slow_filter
sql/mysql_priv.h:
Include log_slow.h
sql/mysqld.cc:
Added new user variables log_slow_rate_limit, log_slow_verbosity, log_slow_filter
Added log-slow-file as synonym for 'slow-log-file', as most slow-log variables starts with 'log-slow'
Added log-slow-time as synonym for long-query-time
Added note that one should use log-slow-filter instead of log-slow-admin-statements
Updated comment from 'slow_query_log_file'
sql/set_var.cc:
Added long_slow_time as synonym for long_query_time
Added new user variables log_slow_rate_limit, log_slow_verbosity, log_slow_filter
dded handling of 'set' variables to set_var.cc. Changed sql_mode to use this
sql/set_var.h:
- Added handling of 'set' variables. Changed sql_mode to use this
sql/slave.cc:
Use global filter also for slaves
sql/sp_head.cc:
Simplify saving of general_slow_log state
Use the general make_set() function instead of 'symbolic_mode_representation'
sql/sql_cache.cc:
Added status for used query plans
sql/sql_class.cc:
Remember/restore query_plan_flags over complex statements
sql/sql_class.h:
Added variables to handle extended slow log statistics
sql/sql_parse.cc:
Added status for used query plans
Added test for filtering slow_query_log
sql/sql_select.cc:
Added status for used query plans
sql/sql_show.cc:
Use the general make_set() function instead of 'symbolic_mode_representation'
sql/strfunc.cc:
Report first error (not last) if something is wrong in a set
Removed compiler warning
storage/myisam/mi_extra.c:
In prepare for drop, flush key blocks (speed optimization)
storage/myisam/mi_locking.c:
Don't call mi_lock_database if my_disable_locking is used (speed optimization)
Removed hard-coded error messages. All messages are now in
errmsg.txt
Also renamed enumeration value BINLOG_STMT_UNSAFE_FUNCTION to
BINLOG_STMT_UNSAFE_SYSTEM_FUNCTION to make the naming consistent
with BINLOG_STMT_UNSAFE_SYSTEM_VARIABLE.
sql/item_create.cc:
Renamed BINLOG_STMT_UNSAFE_FUNCTION to
BINLOG_STMT_UNSAFE_SYSTEM_FUNCTION
sql/share/errmsg.txt:
Moved hard-coded strings from THD::issue_unsafe_warnings() to
errmsg.txt.
sql/sql_class.cc:
- Moved error messages to errmsg.txt.
- Updated comment above THD::issue_unsafe_warnings().
sql/sql_lex.h:
Renamed BINLOG_STMT_UNSAFE_FUNCTION to
BINLOG_STMT_UNSAFE_SYSTEM_FUNCTION
sql/sql_yacc.yy:
Renamed BINLOG_STMT_UNSAFE_FUNCTION to
BINLOG_STMT_UNSAFE_SYSTEM_FUNCTION
Removed hard-coded error messages. All messages are now in
errmsg.txt
Also renamed enumeration value BINLOG_STMT_UNSAFE_FUNCTION to
BINLOG_STMT_UNSAFE_SYSTEM_FUNCTION to make the naming consistent
with BINLOG_STMT_UNSAFE_SYSTEM_VARIABLE.
when used with --tab
1) New syntax: added CHARACTER SET clause to the
SELECT ... INTO OUTFILE (to complement the same clause in
LOAD DATA INFILE).
mysqldump is updated to use this in --tab mode.
2) ESCAPED BY/ENCLOSED BY field parameters are documented as
accepting CHAR argument, however SELECT .. INTO OUTFILE
silently ignored rests of multisymbol arguments.
For the symmetrical behavior with LOAD DATA INFILE the
server has been modified to fail with the same error:
ERROR 42000: Field separator argument is not what is
expected; check the manual
3) Current LOAD DATA INFILE recognizes field/line separators
"as is" without converting from client charset to data
file charset. So, it is supposed, that input file of
LOAD DATA INFILE consists of data in one charset and
separators in other charset. For the compatibility with
that [buggy] behaviour SELECT INTO OUTFILE implementation
has been saved "as is" too, but the new warning message
has been added:
Non-ASCII separator arguments are not fully supported
This message warns on field/line separators that contain
non-ASCII symbols.
client/mysqldump.c:
mysqldump has been updated to call SELECT ... INTO OUTFILE
statement with a charset from the --default-charset command
line parameter.
mysql-test/r/mysqldump.result:
Added test case for bug #30946.
mysql-test/r/outfile_loaddata.result:
Added test case for bug #30946.
mysql-test/t/mysqldump.test:
Added test case for bug #30946.
mysql-test/t/outfile_loaddata.test:
Added test case for bug #30946.
sql/field.cc:
String conversion code has been moved from check_string_copy_error()
to convert_to_printable() for reuse.
sql/share/errmsg.txt:
New WARN_NON_ASCII_SEPARATOR_NOT_IMPLEMENTED message has been added.
sql/sql_class.cc:
The select_export::prepare() method has been modified to:
1) raise the ER_WRONG_FIELD_TERMINATORS error on multisymbol
ENCLOSED BY/ESCAPED BY field arguments like LOAD DATA INFILE;
2) warn with a new WARN_NON_ASCII_SEPARATOR_NOT_IMPLEMENTED
message on non-ASCII field or line separators.
The select_export::send_data() merhod has been modified to
convert item data to output charset (see new SELECT INTO OUTFILE
syntax). By default the BINARY charset is used for backward
compatibility.
sql/sql_class.h:
The select_export::write_cs field added to keep output
charset.
sql/sql_load.cc:
mysql_load has been modified to warn on non-ASCII field or
line separators with a new WARN_NON_ASCII_SEPARATOR_NOT_IMPLEMENTED
message.
sql/sql_string.cc:
New global function has been added: convert_to_printable()
(common code has been moved from check_string_copy_error()).
sql/sql_string.h:
New String::is_ascii() method and new global convert_to_printable()
function have been added.
sql/sql_yacc.yy:
New syntax: added CHARACTER SET clause to the
SELECT ... INTO OUTFILE (to complement the same clause in
LOAD DATA INFILE). By default the BINARY charset is used for
backward compatibility.
