Issue:
------
VALUES doesn't have a type() function and is considered a
Item_field.
Solution for 5.7:
-----------------
Add a new type() function for Item_values_insert.
On 8.0 and trunk it was fixed by Mithun's Bug#19601973.
Solution for 5.6:
-----------------
Additionally Bug#17458914 is backported.
This will address the problem of using VALUES() in
INSERT ... ON DUPLICATE KEY UPDATE. Create a field object
only if it is in the UPDATE clause, else return a NULL
item.
This will also address the problems mentioned in
Bug#14789787 and Bug#16756402.
Solution for 5.5:
-----------------
As mentioned above Bug#17458914 is backported.
Additionally Bug#14786324 is also backported.
When VALUES() is detected outside its meaningful place,
it should be treated as NULL and is thus replaced with a
Field_null object, with the same name as the original
field.
Fields with type NULL are generally not handled well inside
the server (e.g Innodb will not accept them and it is
impossible to create them in regular tables). So create a
new const NULL item instead.
Shared variables of Delayed_insert may be updated without mutex protection
when delayed insert thread gets an error.
Re-acquire mutex earlier, so that shared variables are protected.
INCORRECT ERROR.
Analysis
========
INSERT with DUPLICATE KEY UPDATE and REPLACE on a table
where foreign key constraint is defined fails with an
incorrect 'duplicate entry' error rather than foreign
key constraint violation error.
As part of the bug fix for BUG#22037930, a new flag
'HA_CHECK_FK_ERROR' was added while checking for non fatal
errors to manage FK errors based on the 'IGNORE' flag. For
INSERT with DUPLICATE KEY UPDATE and REPLACE queries, the
foreign key constraint violation error was marked as non-fatal,
even though IGNORE was not set. Hence it continued with the
duplicate key processing resulting in an incorrect error.
Fix:
===
Foreign key violation errors are treated as non fatal only when
the IGNORE is not set in the above mentioned queries. Hence reports
the appropriate foreign key violation error.
1. the same message text for INSERT and INSERT IGNORE
2. no new warnings in UPDATE IGNORE yet (big change for 5.5)
and replace a commonly used expression with a
named constant
CONSTRAINT.
Analysis
=======
INSERT and UPDATE operations using the IGNORE keyword which
causes FOREIGN KEY constraint violations reports an error
despite using the IGNORE keyword.
Foreign key violation errors were not ignored and reported
as errors instead of warnings even when IGNORE was set.
Fix
===
Added code to ignore the foreign key violation errors and
report them as warnings when the IGNORE keyword is used.
Problem was that created table was not marked as used (not set query_id) and so opening tables for stored function pick it up (as opened place holder for it) and used changing TABLE internals.
CONVERT_CHARSET_PARTITION_CONSTANT:
SQL/SQL_PARTITION..CC:202
Issue:
-----
This problem happens under the following conditions:
1) A table partitioned with a character column as the key.
2) The expressions specified in the partition definition
requires a charset conversion. This can happen when the
server's default collation is different from the
expression's collation.
3) INSERT DELAYED is used to insert data into the table.
SOLUTION:
---------
While creating the delayed_insert object, initialize it
with the relevant select_lex.
send_result_set_metadata
Analysis
--------
Cursor inside trigger accessing NEW/OLD row leads server exit.
The reason for the bug was that implementation of function
create_tmp_table() was not considering Item::TRIGGER_FIELD_ITEM
as possible alternative for type of class being instantiated.
This was resulting in a mismatch between a number of columns
in result list and temp table definition. This mismatch leads
to the failure of assertion
DBUG_ASSERT(send_result_set_metadata.elements == item_list.elements)
in the method Materialized_cursor::send_result_set_metadata
in debug mode.
Fix:
---
Added code to consider Item::TRIGGER_FIELD_ITEM as valid
type while creating fields.
* Wait for aborted thd (victim) to release MDL locks
* Skip aborting an already aborted thd
* Defer setting OK status in case of CTAS
* Minor cosmetic changes
* Added a test case
when restoring auto-inc value in INSERT ... ON DUPLICATE KEY UPDATE, take into account that
1. it may be changed in the UPDATE clause (old code did that)
2. it may be changed in the INSERT clause and then cause a dup key (old code missed that)
bzr merge -r4264 maria/5.5
Text conflict in sql/mysqld.cc
Text conflict in storage/xtradb/btr/btr0cur.c
Text conflict in storage/xtradb/buf/buf0buf.c
Text conflict in storage/xtradb/buf/buf0lru.c
Text conflict in storage/xtradb/handler/ha_innodb.cc
5 conflicts encountered.
~40% bugfixed(*) applied
~40$ bugfixed reverted (incorrect or we're not buggy)
~20% bugfixed applied, despite us being not buggy
(*) only changes in the server code, e.g. not cmakefiles
SHOW PROCESSLIST, SHOW BINLOGS
Problem: A deadlock was occurring when 4 threads were
involved in acquiring locks in the following way
Thread 1: Dump thread ( Slave is reconnecting, so on
Master, a new dump thread is trying kill
zombie dump threads. It acquired thread's
LOCK_thd_data and it is about to acquire
mysys_var->current_mutex ( which LOCK_log)
Thread 2: Application thread is executing show binlogs and
acquired LOCK_log and it is about to acquire
LOCK_index.
Thread 3: Application thread is executing Purge binary logs
and acquired LOCK_index and it is about to
acquire LOCK_thread_count.
Thread 4: Application thread is executing show processlist
and acquired LOCK_thread_count and it is
about to acquire zombie dump thread's
LOCK_thd_data.
Deadlock Cycle:
Thread 1 -> Thread 2 -> Thread 3-> Thread 4 ->Thread 1
The same above deadlock was observed even when thread 4 is
executing 'SELECT * FROM information_schema.processlist' command and
acquired LOCK_thread_count and it is about to acquire zombie
dump thread's LOCK_thd_data.
Analysis:
There are four locks involved in the deadlock. LOCK_log,
LOCK_thread_count, LOCK_index and LOCK_thd_data.
LOCK_log, LOCK_thread_count, LOCK_index are global mutexes
where as LOCK_thd_data is local to a thread.
We can divide these four locks in two groups.
Group 1 consists of LOCK_log and LOCK_index and the order
should be LOCK_log followed by LOCK_index.
Group 2 consists of other two mutexes
LOCK_thread_count, LOCK_thd_data and the order should
be LOCK_thread_count followed by LOCK_thd_data.
Unfortunately, there is no specific predefined lock order defined
to follow in the MySQL system when it comes to locks across these
two groups. In the above problematic example,
there is no problem in the way we are acquiring the locks
if you see each thread individually.
But If you combine all 4 threads, they end up in a deadlock.
Fix:
Since everything seems to be fine in the way threads are taking locks,
In this patch We are changing the duration of the locks in Thread 4
to break the deadlock. i.e., before the patch, Thread 4
('show processlist' command) mysqld_list_processes()
function acquires LOCK_thread_count for the complete duration
of the function and it also acquires/releases
each thread's LOCK_thd_data.
LOCK_thread_count is used to protect addition and
deletion of threads in global threads list. While show
process list is looping through all the existing threads,
it will be a problem if a thread is exited but there is no problem
if a new thread is added to the system. Hence a new mutex is
introduced "LOCK_thd_remove" which will protect deletion
of a thread from global threads list. All threads which are
getting exited should acquire LOCK_thd_remove
followed by LOCK_thread_count. (It should take LOCK_thread_count
also because other places of the code still thinks that exit thread
is protected with LOCK_thread_count. In this fix, we are changing
only 'show process list' query logic )
(Eg: unlink_thd logic will be protected with
LOCK_thd_remove).
Logic of mysqld_list_processes(or file_schema_processlist)
will now be protected with 'LOCK_thd_remove' instead of
'LOCK_thread_count'.
Now the new locking order after this patch is:
LOCK_thd_remove -> LOCK_thd_data -> LOCK_log ->
LOCK_index -> LOCK_thread_count
SHOW PROCESSLIST, SHOW BINLOGS
Problem: A deadlock was occurring when 4 threads were
involved in acquiring locks in the following way
Thread 1: Dump thread ( Slave is reconnecting, so on
Master, a new dump thread is trying kill
zombie dump threads. It acquired thread's
LOCK_thd_data and it is about to acquire
mysys_var->current_mutex ( which LOCK_log)
Thread 2: Application thread is executing show binlogs and
acquired LOCK_log and it is about to acquire
LOCK_index.
Thread 3: Application thread is executing Purge binary logs
and acquired LOCK_index and it is about to
acquire LOCK_thread_count.
Thread 4: Application thread is executing show processlist
and acquired LOCK_thread_count and it is
about to acquire zombie dump thread's
LOCK_thd_data.
Deadlock Cycle:
Thread 1 -> Thread 2 -> Thread 3-> Thread 4 ->Thread 1
The same above deadlock was observed even when thread 4 is
executing 'SELECT * FROM information_schema.processlist' command and
acquired LOCK_thread_count and it is about to acquire zombie
dump thread's LOCK_thd_data.
Analysis:
There are four locks involved in the deadlock. LOCK_log,
LOCK_thread_count, LOCK_index and LOCK_thd_data.
LOCK_log, LOCK_thread_count, LOCK_index are global mutexes
where as LOCK_thd_data is local to a thread.
We can divide these four locks in two groups.
Group 1 consists of LOCK_log and LOCK_index and the order
should be LOCK_log followed by LOCK_index.
Group 2 consists of other two mutexes
LOCK_thread_count, LOCK_thd_data and the order should
be LOCK_thread_count followed by LOCK_thd_data.
Unfortunately, there is no specific predefined lock order defined
to follow in the MySQL system when it comes to locks across these
two groups. In the above problematic example,
there is no problem in the way we are acquiring the locks
if you see each thread individually.
But If you combine all 4 threads, they end up in a deadlock.
Fix:
Since everything seems to be fine in the way threads are taking locks,
In this patch We are changing the duration of the locks in Thread 4
to break the deadlock. i.e., before the patch, Thread 4
('show processlist' command) mysqld_list_processes()
function acquires LOCK_thread_count for the complete duration
of the function and it also acquires/releases
each thread's LOCK_thd_data.
LOCK_thread_count is used to protect addition and
deletion of threads in global threads list. While show
process list is looping through all the existing threads,
it will be a problem if a thread is exited but there is no problem
if a new thread is added to the system. Hence a new mutex is
introduced "LOCK_thd_remove" which will protect deletion
of a thread from global threads list. All threads which are
getting exited should acquire LOCK_thd_remove
followed by LOCK_thread_count. (It should take LOCK_thread_count
also because other places of the code still thinks that exit thread
is protected with LOCK_thread_count. In this fix, we are changing
only 'show process list' query logic )
(Eg: unlink_thd logic will be protected with
LOCK_thd_remove).
Logic of mysqld_list_processes(or file_schema_processlist)
will now be protected with 'LOCK_thd_remove' instead of
'LOCK_thread_count'.
Now the new locking order after this patch is:
LOCK_thd_remove -> LOCK_thd_data -> LOCK_log ->
LOCK_index -> LOCK_thread_count
Added variable "OLD_MODE" that can be used to turn off the new behavior
mysql-test/r/insert.result:
Added test case
mysql-test/r/mysqld--help.result:
Added old_mode
mysql-test/suite/sys_vars/r/old_mode_basic.result:
Added testing of new variable
mysql-test/suite/sys_vars/t/old_mode_basic.test:
Added testing of new variable
mysql-test/t/insert.test:
Added test case
sql/sql_class.h:
Added bit flags for OLD_MODE
sql/sql_insert.cc:
Disable duplicate key warnings for INSERT IGNORE of OLD_MODE NO_DUP_KEY_WARNINGS_WITH_IGNORE is used
sql/sql_show.cc:
Don't show progress reporting on SHOW PROCESSLIST if OLD_MODE NO_PROGRESS_INFO is used
sql/sys_vars.cc:
Added OLD_MODE
Main fix was to not cache derivied tables as they may be temporary tables that are deleted before the next query.
This was a bit tricky as Item_field::fix_fields depended on cached_tables to be set to resolve some columns.
mysql-test/r/sp-bugs.result:
Added test case
mysql-test/t/sp-bugs.test:
Added test case
sql/item.cc:
Fixed fix_outer_field to handle case where found field did not have in cached_table
Idea is that if cached_table is not avaliable, use from_field->table->pos_in_table_list instead
sql/records.cc:
Also accept INTERNAL_TMP_TABLE for memmap
sql/sql_base.cc:
More DBUG_PRINT
Fixed that setup_natural_join_row_types() is not run twice.
Original code modified context->first_name_resolution_table also for second executions.
This was wrong as this could give wrong results if some joins had been optimized away between calls.
sql/sql_derived.cc:
Mark derived tables as internal temporary tables (INTERNAL_TMP_TABLE), not as NON_TRANSACTIONAL_TMP_TABLE.
This is more correct as the tables are not visible by the end user.
sql/sql_insert.cc:
Reset pos_in_table_list before calling fix_fields.
One of the consequences of the change of not caching all generated tables in Item_ident is that
pos_in_table_list needs to be correct in calls to fix_fields.
sql/sql_lex.cc:
More DBUG_PRINT
sql/sql_parse.cc:
Don't cache derivied tables as they may be temporary tables that are deleted before the next query
sql/sql_select.cc:
Reset table_vector. This was required as some code checked the vector to see if temporary tables had already been created.
sql/table.cc:
Mark tables with field translations as cacheable (as these will not disapper between stmt executions.
bzr merge lp:maria/5.5 -rtag:mariadb-5.5.31
Text conflict in cmake/cpack_rpm.cmake
Text conflict in debian/dist/Debian/control
Text conflict in debian/dist/Ubuntu/control
Text conflict in sql/CMakeLists.txt
Conflict adding file sql/db.opt. Moved existing file to sql/db.opt.moved.
Conflict adding file sql/db.opt.moved. Moved existing file to sql/db.opt.moved.moved.
Text conflict in sql/mysqld.cc
Text conflict in support-files/mysql.spec.sh
8 conflicts encountered.
Problem:
Insert with 'on duplicate key update' on a view,
crashes the server.
Analysis:
During an insert on to a view, we do the following:
For insert fields and values -
1. Resolve insert values.
2. Resolve insert fields.
3. Check if the fields and values are all from a
single table of a view in case of INSERT VALUES.
Do not check the same in case of INSERT SELECT,
as the values can be read from different table than
that of the view.
For the update fields (if DUP UPDATE is used)
1. Create a name resolution context with 'table_list' only.
2. Resolve update fields in this context.
3. Check if update fields and values are from the same
table as the insert fields.
4. Get the next name resolution context. Concatinate this
with the previous one.
5. Resolve update values in this context as we can refer
to other tables in the values clause.
Note that at step 3(of update fields), we check for
'used_tables map' of update values, without resolving them
first. Hence the crash.
Fix:
At step 3, do not pass the update values to check if its a
single table view update, as update values can refer other table.
Code has been re-organized to function like check_insert_fields.
sql/sql_insert.cc:
Do not pass update_values as they are not resolved yet.
Problem:
Insert with 'on duplicate key update' on a view,
crashes the server.
Analysis:
During an insert on to a view, we do the following:
For insert fields and values -
1. Resolve insert values.
2. Resolve insert fields.
3. Check if the fields and values are all from a
single table of a view in case of INSERT VALUES.
Do not check the same in case of INSERT SELECT,
as the values can be read from different table than
that of the view.
For the update fields (if DUP UPDATE is used)
1. Create a name resolution context with 'table_list' only.
2. Resolve update fields in this context.
3. Check if update fields and values are from the same
table as the insert fields.
4. Get the next name resolution context. Concatinate this
with the previous one.
5. Resolve update values in this context as we can refer
to other tables in the values clause.
Note that at step 3(of update fields), we check for
'used_tables map' of update values, without resolving them
first. Hence the crash.
Fix:
At step 3, do not pass the update values to check if its a
single table view update, as update values can refer other table.
Code has been re-organized to function like check_insert_fields.
