row_build_index_entry(): In innodb_file_format=Barracuda
(ROW_FORMAT=DYNAMIC or ROW_FORMAT=COMPRESSED), a secondary index on a
full column can refer to a field that is stored off-page in the
clustered index record. Take that into account.
rb:692 approved by Jimmy Yang
If the expression for a derived table contained a clause LIMIT 0
SELECT from such derived table incorrectly returned a non-empty set.
Fixed by ensuring JOIN::do_send_rows to be updated after the call
of st_select_lex_unit::set_limit that sets the value of
JOIN::unit->select_limit_cnt.
Due to this bug in the function generate_derived_keys_for_table some
key definitions to access materialized derived tables or materialized
views were constructed with invalid info for their key parts.
This could make the server crash when it optimized queries using
materialized derived tables or materialized views.
Before this fix, the test performance_schema.relaylog would fail
with sporadic failures related to statistics on update_cond.
The reason for these failures is that thread scheduling makes
impossible to predict if instrumented conditions will be used on not.
The fix is to relax the test case, to not collect statistics about:
- wait/synch/cond/sql/MYSQL_BIN_LOG::update_cond
- wait/synch/cond/sql/MYSQL_RELAY_LOG::update_cond
- The crash was because a NOT NULL table column inside the subquery was considered NULLable
because the code thought it was on the inner side of an outer join nest.
- Fixed by making correct distinction between tables inside outer join nests and inside semi-join nests.
This crashing bug could manifest itself at execution of join queries
over materialized derived tables with IN subquery predicates in the
where clause. If for such a query the optimizer chose to use duplicate
weed-out with duplicates in a materialized derived table and chose to
employ join cache the the execution could cause a crash of the server.
It happened because the JOIN_CACHE::init method assumed that the value
of TABLE::file::ref is set at the moment when the method was called
for the employed join cache. It's true for regular tables, but it's
not true for materialized derived tables that are filled now at the
first access to them, i.e. after the JOIN_CACHE::init has done its job.
To fix this problem for any ROWID field of materialized derived table
the procedure that copies fields from record buffers into the employed
join buffer first checks whether the value of TABLE::file::ref has
been set for the table, and if it's not so the procedure sets this value.
- Update test results
- Fix a problem with PS:
= convert_subq_to_sj() should not save where to prep_where or on_expr to prep_on_expr.
= After an unmerged subquery predicate has been pulled, it should call fix_after_pullout() for
outer_refs.
Analysis:
The failed assert ensured that the choice of subquery strategy
is performed only for queries with at least one table. If there
is a LIMIT 0 clause all tables are removed, and the subquery is
neither optimized, nor executed during actual optimization. However,
if the query is EXPLAIN-ed, the EXPLAIN execution path doesn't remove
the query tables if there is a LIMIT 0 clause. As a result, the
subquery optimization code is called, which violates the ASSERT
condition.
Solution:
Transform the assert into a condition, and if the outer query
has no tables assume that there will be at most one subquery
execution.
There is potentially a better solution by reengineering the
EXPLAIN/optimize code, so that subquery optimization is not
done if not needed. Such a solution would be a lot bigger and
more complex than a bug fix.
DB_COL_APPEARS_TWICE_IN_INDEX: Remove. This condition is already
checked and reported by MySQL before passing the index definition to
the storage engine.
row_create_index_for_mysql(): Remove the redundant check for
DB_COL_APPEARS_TWICE_IN_INDEX. When enforcing the column prefix index
limit, invoke dict_mem_index_free(index) to plug the memory leak. In
the loop, use index->n_def instead of dict_index_get_n_fields(index),
because the latter would be 0 for indexes that have not been copied to
the data dictionary cache.
innodb-use-sys-malloc.test:
Add test cases for attempting to trigger the error checks in
row_create_index_for_mysql(). Before MySQL 5.5 and WL#5743, the leak
is only reproducible if ha_innobase::max_supported_key_part_length()
returned a higher limit than the one used in
row_create_index_for_mysql().
In MySQL 5.5 and later, the leak is reproducible with
innodb_large_prefix=true.
rb:688 approved by Jimmy Yang
Split status variable Rows_read to Rows_read and Rows_tmp_read so that one can see how much real data is read.
Same was done with with Handler_update and Handler_write.
Fixed bug in MEMORY tables where some variables was counted twice.
Added new internal handler call 'ha_close()' to have one place to gather statistics.
Fixed bug where thd->open_options was set to wrong value when doing admin_recreate_table()
mysql-test/r/status.result:
Updated test results and added new tests
mysql-test/r/status_user.result:
Udated test results
mysql-test/t/status.test:
Added new test for temporary table status variables
sql/ha_partition.cc:
Changed to call ha_close() instead of close()
sql/handler.cc:
Added internal_tmp_table variable for easy checking of temporary tables.
Added new internal handler call 'ha_close()' to have one place to gather statistics.
Gather statistics for internal temporary tables.
sql/handler.h:
Added handler variables internal_tmp_table, rows_tmp_read.
Split function update_index_statistics() to two.
Added ha_update_tmp_row() for faster tmp table handling with more statistics.
sql/item_sum.cc:
ha_write_row() -> ha_write_tmp_row()
sql/multi_range_read.cc:
close() -> ha_close()
sql/mysqld.cc:
New status variables: Rows_tmp_read, Handler_tmp_update and Handler_tmp_write
sql/opt_range.cc:
close() -> ha_close()
sql/sql_base.cc:
close() -> ha_close()
sql/sql_class.cc:
Added handling of rows_tmp_read
sql/sql_class.h:
Added new satistics variables.
rows_read++ -> update_rows_read() to be able to correctly count reads to internal temp tables.
Added handler::ha_update_tmp_row()
sql/sql_connect.cc:
Added comment
sql/sql_expression_cache.cc:
ha_write_row() -> ha_write_tmp_row()
sql/sql_select.cc:
close() -> ha_close()
ha_update_row() -> ha_update_tmp_row()
sql/sql_show.cc:
ha_write_row() -> ha_write_tmp_row()
sql/sql_table.cc:
Fixed bug where thd->open_options was set to wrong value when doing admin_recreate_table()
sql/sql_union.cc:
ha_write_row() -> ha_write_tmp_row()
sql/sql_update.cc:
ha_write_row() -> ha_write_tmp_row()
sql/table.cc:
close() -> ha_close()
storage/heap/ha_heap.cc:
Removed double counting of statistic variables.
close -> ha_close() to get tmp table statistics.
storage/maria/ha_maria.cc:
close -> ha_close() to get tmp table statistics.
The following were missing in the patch for mwl106:
- KEY_PART_INFO::fieldnr were not set for generated keys to access
tmp tables storing the rows of materialized derived tables/views
- TABLE_SHARE::column_bitmap_size was not set for tmp tables storing
the rows of materialized derived tables/views.
These could cause crashes or memory overwrite.
If a view/derived table is non-mergeable then the definition of the tmp table
to store the rows for it is created at the prepare stage. In this case if the
view definition uses outer joins and a view column belongs to an inner table
of one of them then the column should be considered as nullable independently
on nullability of the underlying column. If the underlying column happens to be
defined as non-nullable then the function create_tmp_field_from_item rather
than the function create_tmp_field_from_field should be employed to create
the definition of the interesting column in the tmp table.
- Fixed assert in transaction log handler when aria_check was run on block-record table that was much bigger than expected.
- Fixed warnings about wrong mutex order between bitmap and intern_lock
- Fixed error in bitmap that could cause two rows to use same block for a block record.
- Fixed wrong test that could cause error if last page for a bitmap was used by a blob.
- Fixed several bugs in pagecache for the case where pagecase had very few blocks and there was a lot of threads competing to get the blocks (very unlikely case).
mysql-test/suite/maria/r/maria-recovery3.result:
Updated results
sql/mysqld.cc:
Allow mi_check() to send information messages for log file
storage/maria/ma_bitmap.c:
Fixed problem with wrong mutex order when bitmap was the first page that was flushed out of page cache
- Fixed by introducing _ma_bitmap_mark_file_changed() that marks file changed without a bitmap lock.
- Fixed one case in _ma_change_bitmap_page() where we didn't mark the bitmap changed. This could cause to rows to reuse same block if this was the only change to the bitmap.
- Split _ma_bitmap_get_page_bits() in two parts to not take a bitmap lock when we already have it
- Fixed bug in _ma_bitmap_set_full_page_bits() that caused an error if last page for a bitmap was used by a blob
storage/maria/ma_check.c:
Better handling of wrong file length.
Fixed bug when we tried to write to transaction log when it was not opened (happened when block record file was bigger than expected)
storage/maria/ma_pagecache.c:
Fixed several bugs in pagecache for the case where pagecase had very few blocks and there was a lot of threads competing to get the blocks:
- In link_block() mark a block given to another thread with PCBLOCK_REASSIGNED to ensure that no other threads can start re-using the block
before the thread that requsted a block.
- In free_block(), don't reset status for a block that is in re-assign by link_block() (we don't want to loose the PCBLOCK_REASSIGNED flag).
- Added call to wait_for_flush() when we got a new block in find_block() to ensure that we don't use a block that is beeing flushed by another thread.
- Moved setting of hits_left and last_hit_time in find_block() to where we assign the block.
Code cleanup and making code uniform:
- Changed a lot of KEYCACHE_DBUG_PRINT to use DBUG_PRINT
- Streamlined all reporting of 'signal' and 'wait' between threads to be identical.
- Use thread name instead of thread number (for each match against --debug)
- Added more DBUG_ENTER, DBUG_PRINT and DBUG_ASSERT()
- Added more comments
storage/myisam/ha_myisam.cc:
Only print information about that we make a backup if we are really making a backup.
storage/myisam/mi_check.c:
Inform mysqld that we are creating a backup of the data file (for inclusion in error log).
mysql-test/r/join.result:
Test case for LP:798597
mysql-test/t/join.test:
Test case for LP:798597
sql/sql_select.cc:
In simplify_joins we reset table->maybe_null for outer join tables that can't ever be NULL.
This caused a conflict between the previously calculated items and the group_buffer against the fields
in the temporary table that are created as not null thanks to the optimization.
The fix is to correct the group by items to also be not_null so that they match the used fields and keys.
The function simple_pred did not take into account that a multiple equality
could include ref items (more exactly items of the class Item_direct_view_ref).
It caused crashes for queries over derived tables or views if the
min/max optimization could be applied to these queries.
BOGUS "THE TABLE MYSQL.PROC IS MISSING,..."
There was a race condition between loading a stored routine
(function/procedure/trigger) specified by fully qualified name
SCHEMA_NAME.PROC_NAME and dropping the stored routine database.
The problem was that there is a window for race condition when one server
thread tries to load a stored routine being executed and the other thread
tries to drop the stored routine schema.
This condition race window exists in implementation of function
mysql_change_db() called by db_load_routine() during loading of stored
routine to cache. Function mysql_change_db() calls check_db_dir_existence()
that might failed because specified database was dropped during concurrent
execution of DROP SCHEMA statement. db_load_routine() calls mysql_change_db()
with flag 'force_switch' set to 'true' value so when referenced db is not found
then my_error() is not called and function mysql_change_db() returns ok.
This shadows information about schema opening error in db_load_routine().
Then db_load_routine() makes attempt to parse stored routine that is failed.
This makes to return error to sp_cache_routines_and_add_tables_aux() but since
during error generation a call to my_error wasn't made and hence
THD::main_da wasn't set we set the generic "mysql.proc table corrupt" error
when running sp_cache_routines_and_add_tables_aux().
The fix is to install an error handler inside db_load_routine() for
the mysql_op_change_db() call, and check later if the ER_BAD_DB_ERROR
was caught.
sql/sql_db.cc:
Added synchronization point "before_db_dir_check" to emulate a race condition during
processing of CALL/DROP SCHEMA.
Fix a failure of the re-enabled innodb-index.test in the embedded server.
Apparently, the embedded server does not default to ENGINE=InnoDB when
copying an InnoDB table by CREATE TABLE t2 SELECT * FROM t1;
The patch for bugs 717577 and 724942 has missed to make adjustments for the
call item_equal->add_const(const_item, orig_field_item) in the function
check_simple_equality that builds multiple equality for a field and a constant.
As a result, when this field happens to be a view field and the corresponding
Item_field object F is wrapped in an Item_direct_view_ref object R the object
F is placed in the multiple equality instead of the object R.
A substitution of an equal item for F potentially can cause very serious
problems and in some cases can lead to crashes of the server.
OLD VALUE OF INPUT PARAMETER.
The user-visible problem was that CASE-control-flow function
(not CASE-statement) misbehaved in stored routines under some
circumstances. The problem resulted in a crash or wrong data
returned. The error happened when expressions in CASE-function
were not of the same character set.
A CASE-function should return values of the same character set
for all branches. Internally, that means a new Item-instance
for the CONVERT(... USING <some charset>)-function is added
to the item tree when needed. The problem was that such changes
were not properly recorded using THD::change_item_tree(),
thus dangling pointers remain in the item tree after
THD::rollback_item_tree_changes(), which lead to undefined
behavior (i.e. crash / wrong data) for subsequent executions of
the stored routine.
This bug was introduced by a patch for Bug 11753363
(44793 - CHARACTER SETS: CASE CLAUSE, UCS2 OR UTF32, FAILURE).
The fixed function is Item_func_case::fix_length_and_dec().
New CONVERT-items are added in agg_item_set_converter(),
which calls THD::change_item_tree().
The problem was that an intermediate array was passed
to agg_item_set_converter(). Thus, THD::change_item_tree() there
was called on intermediate objects.
Note: those intermediate objects are allocated on THD's
memory root, so it's Ok to put them into "changed item lists".
The fix is to track changes on the correct objects.
- Added regression test with queries over the WORLD database.
- Discovered and fixed several bugs in the related cost calculation
functionality both in the semijoin and non-semijon subquery code.
- Added DBUG printing of the cost variables used to decide between
IN-EXISTS and MATERIALIZATION.
TO POSITION FIRST CAN CAUSE DATA TO BE CORRUPTED".
ALTER TABLE MODIFY/CHANGE ... FIRST did nothing except renaming
columns if new version of the table had exactly the same
structure as the old one (i.e. as result of such statement, names
of columns changed their order as specified but data in columns
didn't). The same thing happened for ALTER TABLE DROP COLUMN/ADD
COLUMN statements which were supposed to produce new version of
table with exactly the same structure as the old version of table.
I.e. in the latter case the result was the same as if old column
was renamed instead of being dropped and new column with default
as value being created.
Both these problems were caused by the fact that ALTER TABLE
implementation incorrectly interpreted both these situations as
simple renaming of columns and assumed that in-place ALTER TABLE
algorithm could have been used for them.
This patch fixes this problem by ensuring that in cases when some
column is moved to the first position or some column is dropped
the default ALTER TABLE algorithm involving table copying is
always used. This is achieved by detecting such situations in
mysql_prepare_alter_table() and setting Alter_info::change_level
to ALTER_TABLE_DATA_CHANGED for them.
mysql-test/r/alter_table.result:
Added test for bug #12652385 - "61493: REORDERING COLUMNS TO
POSITION FIRST CAN CAUSE DATA TO BE CORRUPTED".
mysql-test/t/alter_table.test:
Added test for bug #12652385 - "61493: REORDERING COLUMNS TO
POSITION FIRST CAN CAUSE DATA TO BE CORRUPTED".
sql/sql_table.cc:
Changed mysql_prepare_alter_table() to detect situations in
which we some column moved to the first position or some column
is dropped and ensure that such ALTER TABLE statements won't
be carried out using in-place algorithm. The latter could have
happened before this patch if new version of table had the same
structure as the old one (except the column names).
UPDATED TWICE
For multi update it is not allowed to update a column
of a table if that table is accessed through multiple aliases
and either
1) the updated column is used as partitioning key
2) the updated column is part of the primary key
and the primary key is clustered
This check is done in unsafe_key_update().
The bug was that for case 2), it was checked whether
updated_column_number == table_share->primary_key
However, the primary_key variable is the index number of the
primary key, not a column number.
Prior to this bugfix, the first column was wrongly believed to be
the primary key. The columns covered by an index is found in
table->key_info[idx_number]->key_part. The bugfix is to check if
any of the columns in the keyparts of the primary key are
updated.
The user-visible effect is that for storage engines with
clustered primary key (e.g. InnoDB but not MyISAM) queries
like
"UPDATE t1 AS A JOIN t2 AS B SET A.primkey=..."
will now error with
"ERROR HY000: Primary key/partition key update is not allowed
since the table is updated both as 'A' and 'B'."
instead of
"ERROR 1032 (HY000): Can't find record in 't1_tb'"
even if primkey is not the first column in the table. This
was the intended behavior of bugfix 11764529.
mysql-test/r/multi_update.result:
Add test for bug#11882110
mysql-test/r/multi_update_innodb.result:
Add test for bug#11882110
mysql-test/t/multi_update.test:
Add test for bug#11882110
mysql-test/t/multi_update_innodb.test:
Add test for bug#11882110
sql/sql_update.cc:
unsafe_key_update() wrongly checked if the primary key index
number was the same as updated column number. Now it is checked
whether any of the columns making up the primary key is updated.
sql/table.h:
Fix comment on TABLE_SHARE::primary_key. Incorrect comment
was introduced by an earlier merge conflict (as per dlenev)
