mysqldump generates view defitions in two stages:
- dump CREATE TABLE statements for the temporary tables. For each view a
temporary table, that has the same structure as the view is created.
- dump DROP TABLE statements for the temporary tables and CREATE VIEW
statements for the view.
This approach is required because views can have dependencies on each other
(a view can use other views). So, they should be created in the particular
order. mysqldump however is not smart enough, so in order to resolve
dependencies it creates temporary tables first of all.
The problem was that mysqldump might have generated incorrect dump for the
temporary table when a view has non-ASCII column name. That happened when
default-character-set is not utf8.
The fix is to:
1. Switch character_set_client for the mysqldump's connection to binary
before issuing SHOW FIELDS statement in order to avoid conversion.
2. Dump switch character_set_client statements to UTF8 and back for
CREATE TABLE statement that is issued to create temporary table.
Bug#25422 (Hang with log tables)
Bug 17876 (Truncating mysql.slow_log in a SP after using cursor locks the
thread)
Bug 23044 (Warnings on flush of a log table)
Bug 29129 (Resetting general_log while the GLOBAL READ LOCK is set causes
a deadlock)
Prior to this fix, the server would hang when performing concurrent
ALTER TABLE or TRUNCATE TABLE statements against the LOG TABLES,
which are mysql.general_log and mysql.slow_log.
The root cause traces to the following code:
in sql_base.cc, open_table()
if (table->in_use != thd)
{
/* wait_for_condition will unlock LOCK_open for us */
wait_for_condition(thd, &LOCK_open, &COND_refresh);
}
The problem with this code is that the current implementation of the
LOGGER creates 'fake' THD objects, like
- Log_to_csv_event_handler::general_log_thd
- Log_to_csv_event_handler::slow_log_thd
which are not associated to a real thread running in the server,
so that waiting for these non-existing threads to release table locks
cause the dead lock.
In general, the design of Log_to_csv_event_handler does not fit into the
general architecture of the server, so that the concept of general_log_thd
and slow_log_thd has to be abandoned:
- this implementation does not work with table locking
- it will not work with commands like SHOW PROCESSLIST
- having the log tables always opened does not integrate well with DDL
operations / FLUSH TABLES / SET GLOBAL READ_ONLY
With this patch, the fundamental design of the LOGGER has been changed to:
- always open and close a log table when writing a log
- remove totally the usage of fake THD objects
- clarify how locking of log tables is implemented in general.
See WL#3984 for details related to the new locking design.
Additional changes (misc bugs exposed and fixed):
1)
mysqldump which would ignore some tables in dump_all_tables_in_db(),
but forget to ignore the same in dump_all_views_in_db().
2)
mysqldump would also issue an empty "LOCK TABLE" command when all the tables
to lock are to be ignored (numrows == 0), instead of not issuing the query.
3)
Internal errors handlers could intercept errors but not warnings
(see sql_error.cc).
4)
Implementing a nested call to open tables, for the performance schema tables,
exposed an existing bug in remove_table_from_cache(), which would perform:
in_use->some_tables_deleted=1;
against another thread, without any consideration about thread locking.
This call inside remove_table_from_cache() was not required anyway,
since calling mysql_lock_abort() takes care of aborting -- cleanly -- threads
that might hold a lock on a table.
This line (in_use->some_tables_deleted=1) has been removed.
sql/handler.cc:
Moved logic for system / log tables in the SQL layer.
sql/handler.h:
Moved logic for system / log tables in the SQL layer.
sql/lock.cc:
Revised locking of log tables
sql/log.cc:
Major cleanup: changed how log tables are locked / written to.
sql/log.h:
Major cleanup: changed how log tables are locked / written to.
sql/mysql_priv.h:
performance schema helpers
sql/slave.cc:
open_ltable() lock flags
sql/sp.cc:
open_ltable() lock flags
sql/sql_acl.cc:
open_ltable() lock flags
sql/sql_class.h:
performance schema helpers
sql/sql_delete.cc:
log tables cleanup in TRUNCATE
sql/sql_error.cc:
Internal handlers can also intercept warnings
sql/sql_insert.cc:
open_ltable() lock flags
sql/sql_parse.cc:
performance schema helpers
sql/sql_plugin.cc:
open_ltable() lock flags
sql/sql_rename.cc:
log tables cleanup in RENAME
sql/sql_servers.cc:
open_ltable() lock flags
sql/sql_show.cc:
Move INFORMATION_SCHEMA_NAME to table.cc
sql/sql_table.cc:
log tables cleanup (admin operations, ALTER TABLE)
sql/sql_udf.cc:
open_ltable() lock flags
sql/table.cc:
Implemented TABLE_CATEGORY.
sql/share/errmsg.txt:
Changed the wording and name of ER_CANT_READ_LOCK_LOG_TABLE
sql/table.h:
Implemented TABLE_CATEGORY.
storage/csv/ha_tina.cc:
Moved logic for system / log tables in the SQL layer.
storage/csv/ha_tina.h:
Moved logic for system / log tables in the SQL layer.
storage/myisam/ha_myisam.cc:
Moved logic for system / log tables in the SQL layer.
storage/myisam/ha_myisam.h:
Moved logic for system / log tables in the SQL layer.
client/mysqldump.c:
Don't lock tables in the ignore list.
Don't issue empty LOCK TABLES queries.
sql/sql_base.cc:
log tables cleanup
performance schema helpers
mysql-test/r/ps.result:
Adjust test results
mysql-test/r/show_check.result:
Adjust test results
mysql-test/r/status.result:
Adjust test results
mysql-test/t/log_state.test:
Added tests for Bug#29129
mysql-test/t/ps.test:
Make the test output deterministic
mysql-test/t/show_check.test:
Make the test output deterministic
mysql-test/r/log_state.result:
Changed the default location of the log output to LOG_FILE,
for backward compatibility with MySQL 5.0
---
Adjust test results
mysql-test/r/log_tables.result:
cleanup for -ps-protocol
mysql-test/t/log_tables.test:
cleanup for -ps-protocol
sql/set_var.cc:
Changed the default location of the log output to LOG_FILE,
for backward compatibility with MySQL 5.0
---
log tables cleanup
Bug#25422 (Hang with log tables)
Bug 17876 (Truncating mysql.slow_log in a SP after using cursor locks the
thread)
Bug 23044 (Warnings on flush of a log table)
Bug 29129 (Resetting general_log while the GLOBAL READ LOCK is set causes
a deadlock)
Prior to this fix, the server would hang when performing concurrent
ALTER TABLE or TRUNCATE TABLE statements against the LOG TABLES,
which are mysql.general_log and mysql.slow_log.
The root cause traces to the following code:
in sql_base.cc, open_table()
if (table->in_use != thd)
{
/* wait_for_condition will unlock LOCK_open for us */
wait_for_condition(thd, &LOCK_open, &COND_refresh);
}
The problem with this code is that the current implementation of the
LOGGER creates 'fake' THD objects, like
- Log_to_csv_event_handler::general_log_thd
- Log_to_csv_event_handler::slow_log_thd
which are not associated to a real thread running in the server,
so that waiting for these non-existing threads to release table locks
cause the dead lock.
In general, the design of Log_to_csv_event_handler does not fit into the
general architecture of the server, so that the concept of general_log_thd
and slow_log_thd has to be abandoned:
- this implementation does not work with table locking
- it will not work with commands like SHOW PROCESSLIST
- having the log tables always opened does not integrate well with DDL
operations / FLUSH TABLES / SET GLOBAL READ_ONLY
With this patch, the fundamental design of the LOGGER has been changed to:
- always open and close a log table when writing a log
- remove totally the usage of fake THD objects
- clarify how locking of log tables is implemented in general.
See WL#3984 for details related to the new locking design.
Additional changes (misc bugs exposed and fixed):
1)
mysqldump which would ignore some tables in dump_all_tables_in_db(),
but forget to ignore the same in dump_all_views_in_db().
2)
mysqldump would also issue an empty "LOCK TABLE" command when all the tables
to lock are to be ignored (numrows == 0), instead of not issuing the query.
3)
Internal errors handlers could intercept errors but not warnings
(see sql_error.cc).
4)
Implementing a nested call to open tables, for the performance schema tables,
exposed an existing bug in remove_table_from_cache(), which would perform:
in_use->some_tables_deleted=1;
against another thread, without any consideration about thread locking.
This call inside remove_table_from_cache() was not required anyway,
since calling mysql_lock_abort() takes care of aborting -- cleanly -- threads
that might hold a lock on a table.
This line (in_use->some_tables_deleted=1) has been removed.
binary SHOW CREATE TABLE or SELECT FROM I_S.
The problem is that mysqldump generates incorrect dump for a table
with non-ASCII column name if the mysqldump's character set is
ASCII.
The fix is to:
1. Switch character_set_client for the mysqldump's connection
to binary before issuing SHOW CREATE TABLE statement in order
to avoid conversion.
2. Dump switch character_set_client statements to UTF8 and back
for CREATE TABLE statement.
client/mysqldump.c:
1. Switch character_set_client for the mysqldump's connection
to binary before issuing SHOW CREATE TABLE statement in order
to avoid conversion.
2. Dump switch character_set_client statements to UTF8 and back
for CREATE TABLE statement.
mysql-test/r/mysqldump-max.result:
Update result file.
mysql-test/r/mysqldump.result:
Update result file.
mysql-test/r/openssl_1.result:
Update result file.
mysql-test/r/show_check.result:
Update result file.
mysql-test/t/show_check.test:
Test case:
- create a table with non-ASCII column name;
- dump the database by mysqldump using ASCII character set;
- drop the database;
- load the dump;
- check that the table has been re-created properly.
binary SHOW CREATE TABLE or SELECT FROM I_S.
The problem is that mysqldump generates incorrect dump for a table
with non-ASCII column name if the mysqldump's character set is
ASCII.
The fix is to:
1. Switch character_set_client for the mysqldump's connection
to binary before issuing SHOW CREATE TABLE statement in order
to avoid conversion.
2. Dump switch character_set_client statements to UTF8 and back
for CREATE TABLE statement.
After dumping triggers mysqldump copied
the value of the OLD_SQL_MODE variable to the SQL_MODE
variable. If the --compact option of the mysqldump was
not set the OLD_SQL_MODE variable had the value
of the uninitialized SQL_MODE variable. So
usually the NO_AUTO_VALUE_ON_ZERO option of the
SQL_MODE variable was discarded.
This fix is for non-"--compact" mode of the mysqldump,
because mysqldump --compact never set SQL_MODE to the
value of NO_AUTO_VALUE_ON_ZERO.
The dump_triggers_for_table function has been modified
to restore previous value of the SQL_MODE variable after
dumping triggers using the SAVE_SQL_MODE temporary
variable.
client/mysqldump.c:
Fixed bug #29788.
The dump_triggers_for_table function has been modified
to restore previous value of the SQL_MODE variable after
dumping triggers using the SAVE_SQL_MODE temporary
variable.
mysql-test/r/mysqldump.result:
Updated test case for bug #29788.
mysql-test/t/mysqldump.test:
Updated test case for bug #29788.
After dumping triggers mysqldump copied
the value of the OLD_SQL_MODE variable to the SQL_MODE
variable. If the --compact option of the mysqldump was
not set the OLD_SQL_MODE variable had the value
of the uninitialized SQL_MODE variable. So
usually the NO_AUTO_VALUE_ON_ZERO option of the
SQL_MODE variable was discarded.
This fix is for non-"--compact" mode of the mysqldump,
because mysqldump --compact never set SQL_MODE to the
value of NO_AUTO_VALUE_ON_ZERO.
The dump_triggers_for_table function has been modified
to restore previous value of the SQL_MODE variable after
dumping triggers using the SAVE_SQL_MODE temporary
variable.
into gleb.loc:/home/uchum/work/bk/5.1-opt
client/mysqldump.c:
Auto merged
include/my_base.h:
Auto merged
mysql-test/include/mix1.inc:
Auto merged
mysql-test/r/innodb_mysql.result:
Auto merged
mysql-test/t/disabled.def:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/sql_update.cc:
Auto merged
storage/myisam/ha_myisam.cc:
Auto merged
into gleb.loc:/home/uchum/work/bk/5.1-opt
client/mysqldump.c:
Auto merged
mysql-test/t/mysqldump.test:
Auto merged
mysql-test/r/mysqldump.result:
Merge with 5.0-opt.
For each view the mysqldump utility creates a temporary table
with the same name and the same columns as the view
in order to satisfy views that depend on this view.
After the creation of all tables, mysqldump drops all
temporary tables and creates actual views.
However, --skip-add-drop-table and --compact flags disable
DROP TABLE statements for those temporary tables. Thus, it was
impossible to create the views because of existence of the
temporary tables with the same names.
client/mysqldump.c:
Fixed bug #28524.
The mysqldump utility has been modified to unconditionally drop
temporary tables before the creation of views.
mysql-test/t/mysqldump.test:
Updated test case for bug #28524 and updated result of previous tests.
mysql-test/r/mysqldump.result:
Updated test case for bug #28524 and updated result of previous tests.
For each view the mysqldump utility creates a temporary table
with the same name and the same columns as the view
in order to satisfy views that depend on this view.
After the creation of all tables, mysqldump drops all
temporary tables and creates actual views.
However, --skip-add-drop-table and --compact flags disable
DROP TABLE statements for those temporary tables. Thus, it was
impossible to create the views because of existence of the
temporary tables with the same names.
into bodhi.(none):/opt/local/work/mysql-5.1-runtime
client/mysqldump.c:
Auto merged
mysql-test/r/show_check.result:
Auto merged
mysql-test/t/show_check.test:
Auto merged
sql/ha_ndbcluster.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/sql_cache.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/share/errmsg.txt:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_show.cc:
Auto merged
mysql-test/r/events_bugs.result:
Manual merge.
SHOW CREATE TABLE or SELECT FROM I_S.
This is the last patch for this bug, which depends on the big
CS patch and was pending.
The problem was that SHOW CREATE statements returned original
queries in the binary character set. That could cause the query
to be unreadable.
The fix is to use original character_set_client when sending
the original query to the client. In order to preserve the query
in mysqldump, 'binary' character set results should be set when
issuing SHOW CREATE statement. If either source or destination
character set is 'binary' , no conversion is performed.
The idea is that since the source character set is no longer
'binary', we fix the destination character set to still produce
valid dumps.
client/mysqldump.c:
Switch character_set_results of mysqldump-connection before
calling SHOW CREATE statements for the objects.
mysql-test/r/show_check.result:
Result file.
mysql-test/t/show_check.test:
Add test case for the part of BUG#10491.
sql/events.cc:
Send original query in the original character set.
sql/sp_head.cc:
Send original query in the original character set.
sql/sql_show.cc:
Send original query in the original character set.
SHOW CREATE TABLE or SELECT FROM I_S.
This is the last patch for this bug, which depends on the big
CS patch and was pending.
The problem was that SHOW CREATE statements returned original
queries in the binary character set. That could cause the query
to be unreadable.
The fix is to use original character_set_client when sending
the original query to the client. In order to preserve the query
in mysqldump, 'binary' character set results should be set when
issuing SHOW CREATE statement. If either source or destination
character set is 'binary' , no conversion is performed.
The idea is that since the source character set is no longer
'binary', we fix the destination character set to still produce
valid dumps.
into maint1.mysql.com:/data/localhome/tsmith/bk/maint/51
client/mysqldump.c:
Auto merged
mysql-test/lib/mtr_cases.pl:
Auto merged
mysql-test/lib/mtr_report.pl:
Auto merged
mysql-test/r/mysqldump.result:
Auto merged
mysql-test/suite/binlog/r/binlog_multi_engine.result:
Auto merged
mysql-test/suite/binlog/t/binlog_multi_engine.test:
Auto merged
mysql-test/suite/ndb/r/ndb_read_multi_range.result:
Auto merged
mysql-test/suite/ndb/r/ndb_sp.result:
Auto merged
mysql-test/suite/ndb/r/ndb_trigger.result:
Auto merged
mysql-test/suite/ndb/r/ps_7ndb.result:
Auto merged
mysql-test/suite/ndb/t/loaddata_autocom_ndb.test:
Auto merged
mysql-test/suite/ndb/t/ndb_alter_table.test:
Auto merged
mysql-test/suite/ndb/t/ndb_alter_table2.test:
Auto merged
mysql-test/suite/ndb/t/ndb_alter_table3.test:
Auto merged
mysql-test/suite/ndb/t/ndb_autodiscover.test:
Auto merged
mysql-test/suite/ndb/t/ndb_autodiscover2.test:
Auto merged
mysql-test/suite/ndb/t/ndb_autodiscover3.test:
Auto merged
mysql-test/suite/ndb/t/ndb_basic.test:
Auto merged
mysql-test/suite/ndb/t/ndb_binlog_ddl_multi.test:
Auto merged
mysql-test/suite/ndb/t/ndb_binlog_log_bin.test:
Auto merged
mysql-test/suite/ndb/t/ndb_binlog_multi.test:
Auto merged
mysql-test/suite/ndb/t/ndb_bitfield.test:
Auto merged
mysql-test/suite/ndb/t/ndb_blob.test:
Auto merged
mysql-test/suite/ndb/t/ndb_blob_partition.test:
Auto merged
mysql-test/suite/ndb/t/ndb_cache.test:
Auto merged
mysql-test/suite/ndb/t/ndb_cache2.test:
Auto merged
mysql-test/suite/ndb/t/ndb_cache_multi.test:
Auto merged
mysql-test/suite/ndb/t/ndb_cache_multi2.test:
Auto merged
mysql-test/suite/ndb/t/ndb_charset.test:
Auto merged
mysql-test/suite/ndb/t/ndb_condition_pushdown.test:
Auto merged
mysql-test/suite/ndb/t/ndb_config.test:
Auto merged
mysql-test/suite/ndb/t/ndb_config2.test:
Auto merged
mysql-test/suite/ndb/t/ndb_cursor.test:
Auto merged
mysql-test/suite/ndb/t/ndb_database.test:
Auto merged
mysql-test/suite/ndb/t/ndb_dd_alter.test:
Auto merged
mysql-test/suite/ndb/t/ndb_dd_backuprestore.test:
Auto merged
mysql-test/suite/ndb/t/ndb_dd_basic.test:
Auto merged
mysql-test/suite/ndb/t/ndb_dd_ddl.test:
Auto merged
mysql-test/suite/ndb/t/ndb_dd_disk2memory.test:
Auto merged
mysql-test/suite/ndb/t/ndb_dd_dump.test:
Auto merged
mysql-test/suite/ndb/t/ndb_dd_sql_features.test:
Auto merged
mysql-test/suite/ndb/t/ndb_gis.test:
Auto merged
mysql-test/suite/ndb/t/ndb_index.test:
Auto merged
mysql-test/suite/ndb/t/ndb_index_ordered.test:
Auto merged
mysql-test/suite/ndb/t/ndb_index_unique.test:
Auto merged
mysql-test/suite/ndb/t/ndb_insert.test:
Auto merged
mysql-test/suite/ndb/t/ndb_limit.test:
Auto merged
mysql-test/suite/ndb/t/ndb_loaddatalocal.test:
Auto merged
mysql-test/suite/ndb/t/ndb_lock.test:
Auto merged
mysql-test/suite/ndb/t/ndb_minmax.test:
Auto merged
mysql-test/suite/ndb/t/ndb_multi.test:
Auto merged
mysql-test/suite/ndb/t/ndb_partition_error.test:
Auto merged
mysql-test/suite/ndb/t/ndb_partition_key.test:
Auto merged
mysql-test/suite/ndb/t/ndb_partition_list.test:
Auto merged
mysql-test/suite/ndb/t/ndb_partition_range.test:
Auto merged
mysql-test/suite/ndb/t/ndb_read_multi_range.test:
Auto merged
mysql-test/suite/ndb/t/ndb_rename.test:
Auto merged
mysql-test/suite/ndb/t/ndb_replace.test:
Auto merged
mysql-test/suite/ndb/t/ndb_restore.test:
Auto merged
mysql-test/suite/ndb/t/ndb_restore_partition.test:
Auto merged
mysql-test/suite/ndb/t/ndb_restore_print.test:
Auto merged
mysql-test/suite/ndb/t/ndb_row_format.test:
Auto merged
mysql-test/suite/ndb/t/ndb_single_user.test:
Auto merged
mysql-test/suite/ndb/t/ndb_sp.test:
Auto merged
mysql-test/suite/ndb/t/ndb_subquery.test:
Auto merged
mysql-test/suite/ndb/t/ndb_temporary.test:
Auto merged
mysql-test/suite/ndb/t/ndb_transaction.test:
Auto merged
mysql-test/suite/ndb/t/ndb_trigger.test:
Auto merged
mysql-test/suite/ndb/t/ndb_truncate.test:
Auto merged
mysql-test/suite/ndb/t/ndb_types.test:
Auto merged
mysql-test/suite/ndb/t/ndb_update.test:
Auto merged
mysql-test/suite/ndb/t/ndb_view.test:
Auto merged
mysql-test/suite/ndb/t/ndbapi.test:
Auto merged
mysql-test/suite/ndb/t/ps_7ndb.test:
Auto merged
mysql-test/suite/ndb/t/strict_autoinc_5ndb.test:
Auto merged
mysql-test/suite/rpl/r/rpl_events.result:
Auto merged
mysql-test/suite/rpl/r/rpl_replicate_do.result:
Auto merged
mysql-test/suite/rpl/r/rpl_row_UUID.result:
Auto merged
mysql-test/suite/rpl/r/rpl_sp.result:
Auto merged
mysql-test/suite/rpl/r/rpl_switch_stm_row_mixed.result:
Auto merged
mysql-test/suite/rpl/r/rpl_trigger.result:
Auto merged
mysql-test/suite/rpl/r/rpl_view.result:
Auto merged
mysql-test/suite/rpl_ndb/r/rpl_ndb_UUID.result:
Auto merged
mysql-test/suite/rpl_ndb/r/rpl_ndb_dd_advance.result:
Auto merged
mysql-test/suite/rpl_ndb/r/rpl_ndb_stm_innodb.result:
Auto merged
mysql-test/suite/rpl_ndb/t/rpl_ndb_commit_afterflush.test:
Auto merged
mysql-test/suite/rpl_ndb/t/rpl_ndb_dd_advance.test:
Auto merged
mysql-test/suite/rpl_ndb/t/rpl_ndb_innodb_trans.test:
Auto merged
mysql-test/suite/rpl_ndb/t/rpl_ndb_stm_innodb-master.opt:
Auto merged
mysql-test/suite/rpl_ndb/t/rpl_ndb_stm_innodb.test:
Auto merged
mysql-test/t/sp.test:
Auto merged
sql/Makefile.am:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/sql_class.h:
Auto merged
mysql-test/t/disabled.def:
manual merge
(Will need to follow up with moving a few test cases around post-merge)
1. Fix ddl_i18n_koi8r, ddl_i18n_utf8: explicitly specify character-sets
directory for mysqldump;
2. Fix crash in mysqldump if collation is not found;
3. Use proper way to compare character set names.
client/mysqldump.c:
Prevent crash if collation does not exist.
mysql-test/t/ddl_i18n_koi8r.test:
Explicitly specify character-sets-directory for mysqldump.
mysql-test/t/ddl_i18n_utf8.test:
Explicitly specify character-sets-directory for mysqldump.
sql/sql_lex.cc:
Use my_charset_same() function to check if the charset names are equal.
1. Fix ddl_i18n_koi8r, ddl_i18n_utf8: explicitly specify character-sets
directory for mysqldump;
2. Fix crash in mysqldump if collation is not found;
3. Use proper way to compare character set names.
- BUG#11986: Stored routines and triggers can fail if the code
has a non-ascii symbol
- BUG#16291: mysqldump corrupts string-constants with non-ascii-chars
- BUG#19443: INFORMATION_SCHEMA does not support charsets properly
- BUG#21249: Character set of SP-var can be ignored
- BUG#25212: Character set of string constant is ignored (stored routines)
- BUG#25221: Character set of string constant is ignored (triggers)
There were a few general problems that caused these bugs:
1. Character set information of the original (definition) query for views,
triggers, stored routines and events was lost.
2. mysqldump output query in client character set, which can be
inappropriate to encode definition-query.
3. INFORMATION_SCHEMA used strings with mixed encodings to display object
definition;
1. No query-definition-character set.
In order to compile query into execution code, some extra data (such as
environment variables or the database character set) is used. The problem
here was that this context was not preserved. So, on the next load it can
differ from the original one, thus the result will be different.
The context contains the following data:
- client character set;
- connection collation (character set and collation);
- collation of the owner database;
The fix is to store this context and use it each time we parse (compile)
and execute the object (stored routine, trigger, ...).
2. Wrong mysqldump-output.
The original query can contain several encodings (by means of character set
introducers). The problem here was that we tried to convert original query
to the mysqldump-client character set.
Moreover, we stored queries in different character sets for different
objects (views, for one, used UTF8, triggers used original character set).
The solution is
- to store definition queries in the original character set;
- to change SHOW CREATE statement to output definition query in the
binary character set (i.e. without any conversion);
- introduce SHOW CREATE TRIGGER statement;
- to dump special statements to switch the context to the original one
before dumping and restore it afterwards.
Note, in order to preserve the database collation at the creation time,
additional ALTER DATABASE might be used (to temporary switch the database
collation back to the original value). In this case, ALTER DATABASE
privilege will be required. This is a backward-incompatible change.
3. INFORMATION_SCHEMA showed non-UTF8 strings
The fix is to generate UTF8-query during the parsing, store it in the object
and show it in the INFORMATION_SCHEMA.
Basically, the idea is to create a copy of the original query convert it to
UTF8. Character set introducers are removed and all text literals are
converted to UTF8.
This UTF8 query is intended to provide user-readable output. It must not be
used to recreate the object. Specialized SHOW CREATE statements should be
used for this.
The reason for this limitation is the following: the original query can
contain symbols from several character sets (by means of character set
introducers).
Example:
- original query:
CREATE VIEW v1 AS SELECT _cp1251 'Hello' AS c1;
- UTF8 query (for INFORMATION_SCHEMA):
CREATE VIEW v1 AS SELECT 'Hello' AS c1;
client/mysqldump.c:
Set original character set and collation before dumping definition query.
include/my_sys.h:
Move out-parameter to the end of list.
mysql-test/lib/mtr_report.pl:
Ignore server-warnings during the test case.
mysql-test/r/create.result:
Update result file.
mysql-test/r/ctype_cp932_binlog_stm.result:
Update result file.
mysql-test/r/events.result:
Update result file.
mysql-test/r/events_bugs.result:
Update result file.
mysql-test/r/events_grant.result:
Update result file.
mysql-test/r/func_in.result:
Update result file.
mysql-test/r/gis.result:
Update result file.
mysql-test/r/grant.result:
Update result file.
mysql-test/r/information_schema.result:
Update result file.
mysql-test/r/information_schema_db.result:
Update result file.
mysql-test/r/lowercase_view.result:
Update result file.
mysql-test/r/mysqldump.result:
Update result file.
mysql-test/r/ndb_sp.result:
Update result file.
mysql-test/r/ps.result:
Update result file.
mysql-test/r/rpl_replicate_do.result:
Update result file.
mysql-test/r/rpl_sp.result:
Update result file.
mysql-test/r/rpl_trigger.result:
Update result file.
mysql-test/r/rpl_view.result:
Update result file.
mysql-test/r/show_check.result:
Update result file.
mysql-test/r/skip_grants.result:
Update result file.
mysql-test/r/sp-destruct.result:
Update result file.
mysql-test/r/sp-error.result:
Update result file.
mysql-test/r/sp-security.result:
Update result file.
mysql-test/r/sp.result:
Update result file.
mysql-test/r/sql_mode.result:
Update result file.
mysql-test/r/system_mysql_db.result:
Update result file.
mysql-test/r/temp_table.result:
Update result file.
mysql-test/r/trigger-compat.result:
Update result file.
mysql-test/r/trigger-grant.result:
Update result file.
mysql-test/r/trigger.result:
Update result file.
mysql-test/r/view.result:
Update result file.
mysql-test/r/view_grant.result:
Update result file.
mysql-test/t/events.test:
Update test case (new columns added).
mysql-test/t/information_schema.test:
Update test case (new columns added).
mysql-test/t/show_check.test:
Test case for SHOW CREATE TRIGGER in prepared statements and
stored routines.
mysql-test/t/sp-destruct.test:
Update test case (new columns added).
mysql-test/t/sp.test:
Update test case (new columns added).
mysql-test/t/view.test:
Update test.
mysys/charset.c:
Move out-parameter to the end of list.
scripts/mysql_system_tables.sql:
Add new columns to mysql.proc and mysql.event.
scripts/mysql_system_tables_fix.sql:
Add new columns to mysql.proc and mysql.event.
sql/event_data_objects.cc:
Support new attributes for events.
sql/event_data_objects.h:
Support new attributes for events.
sql/event_db_repository.cc:
Support new attributes for events.
sql/event_db_repository.h:
Support new attributes for events.
sql/events.cc:
Add new columns to SHOW CREATE event resultset.
sql/mysql_priv.h:
1. Introduce Object_creation_ctx;
2. Introduce SHOW CREATE TRIGGER;
3. Introduce auxilary functions.
sql/sp.cc:
Add support for new store routines attributes.
sql/sp_head.cc:
Add support for new store routines attributes.
sql/sp_head.h:
Add support for new store routines attributes.
sql/sql_lex.cc:
Generate UTF8-body on parsing/lexing.
sql/sql_lex.h:
1. Generate UTF8-body on parsing/lexing.
2. Introduce SHOW CREATE TRIGGER.
sql/sql_parse.cc:
Introduce SHOW CREATE TRIGGER.
sql/sql_partition.cc:
Update parse_sql().
sql/sql_prepare.cc:
Update parse_sql().
sql/sql_show.cc:
Support new attributes for views
sql/sql_trigger.cc:
Support new attributes for views
sql/sql_trigger.h:
Support new attributes for views
sql/sql_view.cc:
Support new attributes for views
sql/sql_yacc.yy:
1. Add SHOW CREATE TRIGGER statement.
2. Generate UTF8-body for views, stored routines, triggers and events.
sql/table.cc:
Introduce Object_creation_ctx.
sql/table.h:
Introduce Object_creation_ctx.
sql/share/errmsg.txt:
Add new errors.
mysql-test/include/ddl_i18n.check_events.inc:
Aux file for test suite.
mysql-test/include/ddl_i18n.check_sp.inc:
Aux file for test suite.
mysql-test/include/ddl_i18n.check_triggers.inc:
Aux file for test suite.
mysql-test/include/ddl_i18n.check_views.inc:
Aux file for test suite.
mysql-test/include/have_cp1251.inc:
Aux file for test suite.
mysql-test/include/have_cp866.inc:
Aux file for test suite.
mysql-test/include/have_koi8r.inc:
Aux file for test suite.
mysql-test/include/have_utf8.inc:
Aux file for test suite.
mysql-test/r/ddl_i18n_koi8r.result:
Result file.
mysql-test/r/ddl_i18n_utf8.result:
Result file.
mysql-test/r/have_cp1251.require:
Aux file for test suite.
mysql-test/r/have_cp866.require:
Aux file for test suite.
mysql-test/r/have_koi8r.require:
Aux file for test suite.
mysql-test/r/have_utf8.require:
Aux file for test suite.
mysql-test/t/ddl_i18n_koi8r.test:
Complete koi8r test case for the CS patch.
mysql-test/t/ddl_i18n_utf8.test:
Complete utf8 test case for the CS patch.
- BUG#11986: Stored routines and triggers can fail if the code
has a non-ascii symbol
- BUG#16291: mysqldump corrupts string-constants with non-ascii-chars
- BUG#19443: INFORMATION_SCHEMA does not support charsets properly
- BUG#21249: Character set of SP-var can be ignored
- BUG#25212: Character set of string constant is ignored (stored routines)
- BUG#25221: Character set of string constant is ignored (triggers)
There were a few general problems that caused these bugs:
1. Character set information of the original (definition) query for views,
triggers, stored routines and events was lost.
2. mysqldump output query in client character set, which can be
inappropriate to encode definition-query.
3. INFORMATION_SCHEMA used strings with mixed encodings to display object
definition;
1. No query-definition-character set.
In order to compile query into execution code, some extra data (such as
environment variables or the database character set) is used. The problem
here was that this context was not preserved. So, on the next load it can
differ from the original one, thus the result will be different.
The context contains the following data:
- client character set;
- connection collation (character set and collation);
- collation of the owner database;
The fix is to store this context and use it each time we parse (compile)
and execute the object (stored routine, trigger, ...).
2. Wrong mysqldump-output.
The original query can contain several encodings (by means of character set
introducers). The problem here was that we tried to convert original query
to the mysqldump-client character set.
Moreover, we stored queries in different character sets for different
objects (views, for one, used UTF8, triggers used original character set).
The solution is
- to store definition queries in the original character set;
- to change SHOW CREATE statement to output definition query in the
binary character set (i.e. without any conversion);
- introduce SHOW CREATE TRIGGER statement;
- to dump special statements to switch the context to the original one
before dumping and restore it afterwards.
Note, in order to preserve the database collation at the creation time,
additional ALTER DATABASE might be used (to temporary switch the database
collation back to the original value). In this case, ALTER DATABASE
privilege will be required. This is a backward-incompatible change.
3. INFORMATION_SCHEMA showed non-UTF8 strings
The fix is to generate UTF8-query during the parsing, store it in the object
and show it in the INFORMATION_SCHEMA.
Basically, the idea is to create a copy of the original query convert it to
UTF8. Character set introducers are removed and all text literals are
converted to UTF8.
This UTF8 query is intended to provide user-readable output. It must not be
used to recreate the object. Specialized SHOW CREATE statements should be
used for this.
The reason for this limitation is the following: the original query can
contain symbols from several character sets (by means of character set
introducers).
Example:
- original query:
CREATE VIEW v1 AS SELECT _cp1251 'Hello' AS c1;
- UTF8 query (for INFORMATION_SCHEMA):
CREATE VIEW v1 AS SELECT 'Hello' AS c1;
- Move the check of too long path to 'get_one_option'
client/mysqldump.c:
Move the check of too long path to 'get_one_option'
mysql-test/r/mysqldump.result:
Update result file after changing error message
into quadxeon.mysql.com:/benchmarks/ext3/TOSAVE/tsmith/bk/maint/jun05/51
client/mysqldump.c:
Auto merged
mysql-test/mysql-test-run.pl:
Auto merged
mysql-test/r/innodb_mysql.result:
Auto merged
sql/field.cc:
Auto merged
sql/field.h:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
sql/mysqld.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
into quadxeon.mysql.com:/benchmarks/ext3/TOSAVE/tsmith/bk/maint/jun05/50
client/mysqldump.c:
Auto merged
mysql-test/mysql-test-run.pl:
Auto merged
sql/field.cc:
Auto merged
sql/field.h:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
sql/mysqld.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
into quadxeon.mysql.com:/benchmarks/ext3/TOSAVE/tsmith/bk/maint/jun05/51
client/mysqldump.c:
Auto merged
include/m_string.h:
Auto merged
include/mysql_com.h:
Auto merged
include/violite.h:
Auto merged
libmysql/libmysql.c:
Auto merged
mysql-test/include/mix1.inc:
Auto merged
mysql-test/mysql-test-run.pl:
Auto merged
mysql-test/r/innodb_mysql.result:
Auto merged
mysql-test/r/ps.result:
Auto merged
mysql-test/t/mysqltest.test:
Auto merged
mysql-test/t/ps.test:
Auto merged
mysys/my_init.c:
Auto merged
server-tools/instance-manager/mysql_connection.cc:
Auto merged
sql/field.cc:
Auto merged
sql/item.cc:
Auto merged
sql/item.h:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
sql/log_event.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/net_serv.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/sql_connect.cc:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_prepare.cc:
Auto merged
sql/sql_repl.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_show.cc:
Auto merged
sql-common/client.c:
Auto merged
sql/sql_yacc.yy:
Auto merged
strings/ctype-mb.c:
Auto merged
strings/ctype-ucs2.c:
Auto merged
strings/strtod.c:
Auto merged
vio/vio_priv.h:
Auto merged
vio/viosocket.c:
Auto merged
client/mysqltest.c:
SCCS merged
include/my_global.h:
SCCS merged
sql/field.h:
SCCS merged
sql/sql_table.cc:
Manual merge
strings/ctype-bin.c:
Manual merge
into bk-internal.mysql.com:/data0/bk/mysql-5.1-opt
client/mysqldump.c:
Auto merged
sql/field.cc:
Auto merged
sql/field.h:
Auto merged
sql/field_conv.cc:
Auto merged
sql/handler.h:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_func.h:
Auto merged
sql/item_subselect.cc:
Auto merged
sql/item_sum.cc:
Auto merged
sql/item_timefunc.cc:
Auto merged
sql/my_decimal.h:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/sql_view.cc:
Auto merged
sql/table.cc:
Auto merged
sql/table.h:
Auto merged
strings/decimal.c:
Auto merged
sql/sql_update.cc:
Manual merge
into bk-internal.mysql.com:/data0/bk/mysql-5.1-opt
BitKeeper/deleted/.del-ps_6bdb.result:
Auto merged
client/mysqldump.c:
Auto merged
mysql-test/r/mysqldump.result:
Auto merged
mysql-test/r/ps_2myisam.result:
Auto merged
mysql-test/r/ps_3innodb.result:
Auto merged
mysql-test/r/ps_4heap.result:
Auto merged
mysql-test/r/ps_5merge.result:
Auto merged
mysql-test/r/ps_7ndb.result:
Auto merged
sql/field.h:
Auto merged
sql/item_func.cc:
Auto merged
sql/my_decimal.h:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_table.cc:
Auto merged
into bk-internal.mysql.com:/data0/bk/mysql-5.0-opt
client/mysqldump.c:
Auto merged
mysql-test/r/mysqldump.result:
Auto merged
mysql-test/r/ps_2myisam.result:
Auto merged
mysql-test/r/ps_3innodb.result:
Auto merged
mysql-test/r/ps_4heap.result:
Auto merged
mysql-test/r/ps_5merge.result:
Auto merged
mysql-test/r/ps_6bdb.result:
Auto merged
mysql-test/r/ps_7ndb.result:
Auto merged
sql/field.h:
Auto merged
sql/item_func.cc:
Auto merged
sql/my_decimal.h:
Auto merged
sql/sql_base.cc:
Auto merged
into gleb.loc:/home/uchum/work/bk/mysql-5.1-opt
client/mysqldump.c:
Auto merged
mysql-test/r/join_outer.result:
Auto merged
mysql-test/r/kill.result:
Auto merged
mysql-test/r/view.result:
Auto merged
mysql-test/t/join_outer.test:
Auto merged
mysql-test/t/kill.test:
Auto merged
sql/field.h:
Auto merged
sql/item_func.cc:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sql_base.cc:
Auto merged
mysql-test/r/mysqldump.result:
Merge with 5.0-opt
mysql-test/t/mysqldump.test:
Merge with 5.0-opt
mysql-test/t/view.test:
Merge with 5.0-opt
sql/sql_select.cc:
Merge with 5.0-opt
fixed by using flush logs, dumping, then doing PURGE MASTER LOGS TO 'binfile', instead
of deleting the log files at the beginning.
NOTE: previously the delete-master-logs would reset the log names back to
filename.00001. Now the trailing number doesn't get reset. This may need to be
documented.
client/mysqldump.c:
changed the code the --delete-master-logs option is used from this:
take locks
delete bin logs
do dump (if this is a lock-based dump)
release locks
do dump (if this is a consistent-read-dump)
to this:
take locks
flush logs
remember the name of the new log
do dump (if this is a lock-based dump)
release locks
do dump (if this is a consistent-read-dump)
fflush output sql file if specified, to ensure the backup is commited to disk
--- yes, dump succeeded ---
do PURGE MASTER LOGS TO up to the new log
fixed by using flush logs, dumping, then doing PURGE MASTER LOGS TO 'binfile', instead
of deleting the log files at the beginning.
NOTE: previously the delete-master-logs would reset the log names back to
filename.00001. Now the trailing number doesn't get reset. This may need to be
documented.
sometimes `mysqldump --hex-blob' overruned output buffer by '\0' byte.
The dump_table() function has been fixed to reserve 1 byte more for the
last '\0' byte of dumped string.
client/mysqldump.c:
Fixed bug #28522.
The dump_table() function has been fixed to reserve 1 byte more for the
last '\0' byte of dumped string.
mysql-test/t/mysqldump.test:
Updated test case for bug #28522.
mysql-test/r/mysqldump.result:
Updated test case for bug #28522.
