Apply the patch from Patryk Pomykalski:
- create_internal_tmp_table_from_heap() will now return information whether
the last row that we tried to write was a duplicate row.
(mysql-5.6 also has this change)
Includes 5.6 changesets for:
*****
Fix for BUG#13489996 valgrind:conditional jump or move depends on uninitialised values-field_blob.
blob_ptr_size was not initialized properly: remove this variable.
*****
Bug#14021323 CRASH IN FIELD::SET_NULL WHEN INSERTING ROWS TO NEW TABLE
*****
MDEV-4643 MariaDB crashes consistently when trying a SELECT on VIEW with a UNION and an additional JOIN in SELECT
open derived temp tables *before* trying QUICK_SELECT for them,
handler::multi_range_read_info() needs an open table.
- Make query plan be re-saved after the first join execution
(saving it after JOIN::cleanup is too late because EXPLAIN output
is currently produced before that)
- Handle QPF allocation/deallocation for edge cases, like unsuccessful
BINLOG command.
- Work around the problem with UNION's direct subselects not being visible.
- Update test results ("Using temporary; Using filesort" are now always printed
last in the Extra column)
- This cset gets rid of memory leaks/crashes. Some result mismatches still remain.
- Update view.result (old EXPLAIN didn't match the execution)
- Put in a stub code to work around the
SELECT ... UNION SELECT ... ORDER BY (subuqery) problem
- Handle another specific case where there the JOIN
never had a query plan, but had multiple join->cleanup(full=true) calls
- The idea that there can only be MAX_TABLES subuqeries/unions was
wrong. Switch QPF_query to using a Dynamic_array.
= make Dynamic_array template support size growth. its underlying
DYNAMIC_ARRAY supports it. (this part will need more polishing)
- Handle statements inside SPs:
= regular statements
= SET command, which does not have its own statement.
- Handle execution of subquery from range optimizer:
allocate subquery QPFs on the same MEM_ROOT as the whole
query plan was allocated.
This requires that subselect's footprints are saved before it is deleted.
Attempt to save select's QPF exposes one to a variety of edge cases:
- the select may be a UNION's "fake select" which has no valid id
- optimization may fail in the middle (but subsequent JOIN::optimize() calls
will succeed, despite the fact that there never was a query plan)
- Introduce "Query Plan Footprints" (abbrev. QPFs)
QPF is a part of query plan that is
1. sufficient to produce EXPLAIN output,
2. can be used to produce EXPLAIN output even after its subquery/union
was executed and deleted
3. is cheap to save so that we can always save query plans
- This patch doesn't fully address #2, we make/save strings for
a number of EXPLAIN's columns. This will be fixed.
-Added test and extra code to ensure we don't leave keyread on for a handler table.
-Create on disk temporary files always with long data pointers if SQL_SMALL_RESULT is not used. This ensures that we can handle temporary files bigger than 4G.
mysql-test/include/default_mysqld.cnf:
Run test suite with smaller aria keybuffer size
mysql-test/suite/maria/maria3.result:
Run test suite with smaller aria keybuffer size
mysql-test/suite/sys_vars/r/aria_pagecache_buffer_size_basic.result:
Run test suite with smaller aria keybuffer size
sql/handler.cc:
Disable key read (extra safety if something went wrong)
sql/multi_range_read.cc:
Ensure we have don't leave keyread on for secondary_file
sql/opt_range.cc:
Simplify code with mark_columns_used_by_index_no_reset()
Ensure that read_keys_and_merge() disableds keyread if it enables it
sql/opt_subselect.cc:
Remove not anymore used argument for create_internal_tmp_table()
sql/sql_derived.cc:
Remove not anymore used argument for create_internal_tmp_table()
sql/sql_select.cc:
Use 'enable_keyread()' instead of calling HA_EXTRA_RESET. (Makes debugging easier)
Create on disk temporary files always with long data pointers if SQL_SMALL_RESULT is not used. This ensures that we can handle temporary files bigger than 4G.
Remove not anymore used argument for create_internal_tmp_table()
More DBUG
sql/sql_select.h:
Remove not anymore used argument for create_internal_tmp_table()
WITH COMPOSITE KEY COLUMNS
Problem:-
While running a SELECT query with several AGGR(DISTINCT) function
and these are referring to different field of same composite key,
Returned incorrect value.
Analysis:-
In a table, where we have composite key like (a,b,c)
and when we give a query like
select COUNT(DISTINCT b), SUM(DISTINCT a) from ....
here, we first make a list of items in Aggr(distinct) function
(which is a, b), where order of item doesn't matter.
and then we see, whether we have a composite key where the prefix
of index columns matches the items of the aggregation function.
(in this case we have a,b,c).
if yes, so we can use loose index scan and we need not perform
duplicate removal to distinct in our aggregate function.
In our table, we traverse column marked with <-- and get the result as
(a,b,c) count(distinct b) sum(distinct a)
treated as count b treated as sum(a)
(1,1,2)<-- 1 1
(1,2,2)<-- 1++=2 1+1=2
(1,2,3)
(2,1,2)<-- 2++=3 1+1+2=4
(2,2,2)<-- 3++=4 1+1+2+2=6
(2,2,3)
result will be 4,6, but it should be (2,3)
As in this case, our assumption is incorrect. If we have
query like
select count(distinct a,b), sum(distinct a,b)from ..
then we can use loose index scan
Solution:-
In our query, when we have more then one aggr(distinct) function
then they should refer to same fields like
select count(distinct a,b), sum(distinct a,b) from ..
-->we can use loose scan index as both aggr(distinct) refer to same fields a,b.
If they are referring to different field like
select count(distinct a), sum(distinct b) from ..
-->will not use loose scan index as both aggr(distinct) refer to different fields.
This could happen when using Aria for internal temporary files (default case) and using DISTINCT.
_ma_scan_restore_block_record() didn't work correctly if there was rows inserted, updated or deleted on the handler
between calls to _ma_scan_remember_block_record() and _ma_scan_restore_block_record().
The effect was that some DISTINCT queries that used remove_dup_with_compare() could fail.
.bzrignore:
Ignore sql_yacc.hh
mysql-test/suite/maria/r/distinct.result:
Test case for MDEV-4280
mysql-test/suite/maria/t/distinct.test:
Test case for MDEV-4280
mysql-test/t/mysql.test:
Fixed test suite (we could get error -1 in some cases)
sql/sql_select.cc:
Break loop if restart_rnd_next() gives an error
storage/maria/ha_maria.cc:
scan_restore_pos() can return disk fault error.
storage/maria/ma_blockrec.c:
_ma_scan_remember_block_record() did incorrectly update scan.dir instead of scan_save.dir .
_ma_scan_restore_block_record() didn't work correctly if there was rows inserted,updated or deleted on the handler
between calls to _ma_scan_remember_block_record() and _ma_scan_restore_block_record().
Fixed by adding counters for row changes and reading the current scan page if changes had been made.
storage/maria/ma_blockrec.h:
scan_restore_pos() can return disk fault error.
storage/maria/ma_delete.c:
Increment row_changes
storage/maria/ma_scan.c:
scan_restore_pos() can return disk fault error.
storage/maria/ma_update.c:
Increment row_changes
storage/maria/ma_write.c:
Increment row_changes
storage/maria/maria_def.h:
scan_restore_pos() can return disk fault error.
STRING CONVERSION FUNCTIONS
Problem:
While executing the prepared statement, user variable is
set to memory which would be freed at the end of
execution.
If the statement is executed again, valgrind throws
error when accessing this pointer.
Analysis:
1. First time when Item_func_set_user_var::check is called,
memory is allocated for "value" to store the result.
(In the call to copy_if_not_alloced).
2. While sending the result, Item_func_set_user_var::check
is called again. But, this time, its called with
"use_result_field" set to true.
As a result, we call result_field->val_str(&value).
3. Here memory allocated for "value" gets freed. And "value"
gets set to "result_field", with "str_length" being that of
result_field's.
4. In the call to JOIN::cleanup, result_field's memory gets
freed as this is allocated in a chunk as part of the
temporary table which is needed to execute the query.
5. Next time, when execute of the same statement is called,
"value" will be set to memory which is already freed.
Valgrind error occurs as "str_length" is positive
(set at Step 3)
Note that user variables list is stored as part of the Lex object
in set_var_list. Hence the persistance across executions.
Solution:
Patch for Bug#11764371 fixed in mysql-5.6+ fixes this problem
as well.So backporting the same.
In the solution for Bug#11764371, we create another object of
user_var and repoint it to temp_table's field. As a result while
deleting the alloced buffer in Step 3, since the cloned object
does not own the buffer, deletion will not happen.
So at step 5 when we execute the statement second time, the
original object will be used and since deletion did not happen
valgrind will not complain about dangling pointer.
sql/item_func.h:
Add constructors.
sql/sql_select.cc:
Change user variable assignment functions to read from fields after
tables have been unlocked.
- Call tmp_having->update_used_tables() *before* we have call JOIN::cleanup().
Making the call after join::cleanup() is not allowed, because subquery
predicate items walk parent join's JOIN_TAB structures. Which can be
invalidated by JOIN::cleanup().
When iterating over a list of conditions using List_iterator
the function remove_eq_conds should skip all predicates that
replace a condition from the list. Otherwise it can come to
an infinite recursion.
<non-nullable datatime field> IS NULL in outer joins with
that in inner joins.
Previously such condition was transformed into the condition
<non-nullable datatime field> = 0 unless the field belonged
to an inner table of an outer join. In this case the predicate
was interpreted as for any other field.
Now if the field in the predicate <non-nullable datatime field> IS NULL
belongs to an inner table of an outer join the predicate is
transformed into the disjunction
<non-nullable datatime field> = 0 OR <non-nullable datatime field> IS NULL.
This is fully compatible with the semantics of such predicates in 5.5.
- When restore_prev_nj_state() is called for the table that is
the last remaining child of a nested join, do not leave that
nested join's bit in join->cur_embedding_map.
This bug was the result of incompleteness of the patch for bug mdev-4177.
When an OR condition is simplified to a single conjunct it is merged
into the embedding AND condition. Multiple equalities are also merged,
and any field item involved in those equality should acquire a pointer
to a the multiple equality formed by this merge.
The function make_join_statistics checks whether eq_ref access uses only
constant expressions, and, if this is the case the function performs
constant row substitution. The code of this check must take into account
hidden components of extended secondary keys.
