When a versioned table stores its history in partitions, an implicit
query (no FOR SYSTEM_TIME) needs only current rows, which all live in
the current partition. vers_setup_conds() no longer adds a row_end
condition for such a table, and prune_partitions() restricts
read_partitions to the current partition instead.
read_partitions is reset to all partitions every statement in
open_table(), so the restriction must be applied on each execution.
prune_partitions() runs on each execution, also for merged derived
tables and views, where vers_setup_conds() is not called again. Both
functions guard with !vers_conditions.was_set(). In prune_partitions()
!is_set() would not do: set_all() sets type (flips is_set()) but not
orig_type (was_set()), so is_set() would be true on prepared-statement /
stored-procedure re-executions and skip the restriction, leaving history
partitions readable - rows matched then grow on each execute/call.
A query whose time comes from @@system_versioning_asof must still read
history, so the restriction is also skipped when from_sysvar is set.
init_from_sysvar() sets this new flag rather than orig_type, so
was_set(), print() and the write lock check still see only an explicit
FOR SYSTEM_TIME. Like orig_type, the flag persists to prepared-statement
re-execution.
EXPLAIN no longer shows "Using where" for such a query: correctness now
comes from partition selection rather than a row_end filter.
With sql_safe_updates, DELETE without WHERE on such a table is now
rejected even with LIMIT, as for a non-versioned table. With an index
on row_end, UPDATE or DELETE without WHERE (UPDATE also without LIMIT)
is now rejected too: it was accepted as a range scan on row_end. A
versioned table without SYSTEM_TIME partitions keeps the old behavior
(MDEV-41393).
read_partitions is reset to all partitions (or the PARTITION() set) once
per statement in open_table() (set_partition_bitmaps()), and
prune_partitions() only intersects it with lock_partitions and the pruned
set instead of rebuilding it.
mark_all_partitions_as_used() is dropped.
Pure refactoring, results unchanged. It lets a restriction set on
read_partitions before pruning survive it - needed when read_partitions
is pre-restricted, e.g. a versioned table whose history is partitioned
reads only the current partition.
prune_partitions() built the used-partition set directly in
read_partitions: it cleared it, let find_used_partitions*() fill it, and
in the index-merge case used it as scratch too - both buffer and result.
The patch adds PART_PRUNE_PARAM::parts_bitmap and builds the return
into it instead. read_partitions is produced from it only at the end:
copied when the condition was analyzed (IMPOSSIBLE included,
parts_bitmap left empty), or set to all locked partitions via
mark_all_partitions_as_used() when nothing could be pruned.
Pure refactoring: read_partitions ends up identical on every path. It
decouples the pruning scratch from read_partitions, so a later commit can
switch the final bitmap_copy() to bitmap_intersect() - letting pruning
narrow a pre-set read_partitions (e.g. a versioned table selecting only
its current partition) instead of overwriting it.
When an uncommitted transaction inserts rows into a table and
another statement locks rows in the same table (SELECT ... FOR UPDATE)
while computing a MIN or MAX, then:
1. In a Debug build, the server aborts on an assertion
2. In a Release build, the server returns wrong results
These errors occur because, while reading a group of rows for computing
a MAX, the transaction timeout error was swallowed.
Under the scenario described above and captured in the new test at this
commit, QUICK_GROUP_MIN_MAX_SELECT::next_max() emits a lock timeout error
during QUICK_GROUP_MIN_MAX_SELECT::get_next() but the error was suppressed
if we computed a MIN.
The InnoDB storage engine has an unwritten convention that after it has
returned a fatal error (which is any error except HA_ERR_END_OF_FILE
or HA_ERR_KEY_NOT_FOUND), then the SQL layer should not try to make
any further reads. This is because InnoDB might have rolled back
the current transaction already. So in the case of an error, return
immediately from QUICK_GROUP_MIN_MAX_SELECT::get_next().
An incorrectly backported test from 11.x revealed an UBSAN error in 10.11, so
fix that problem by preventing a division-by-zero from happening.
Remove the other incorrectly backported tests and relabel the retained test
in terms of the current ticket.
When the optimizer_max_sel_arg_weight is set to 1, a nested query
crashed while tracing.
SEL_ARG object has a field named 'field', that is not set when the
type is other than KEY_RANGE. But, the field was accessed to store
its name, and weight to the trace. This resulted in a crash due to NULL
pointer.
Added a check to access field if the type is KEY_RANGE, and if not, just
trace the type.
Loose Index Scan optimization can only be used when the index it is
uses covers all columns used by the query. So, it would not be
attempted for indexes that are not present in table->covering_keys.
Before commit ed0a723566 (pushed to 11.0), Clustered PK would be
present in table->covering_keys if primary key columns are covering.
However, that commit gave Clustered PKs special treatment: they are
no longer present in table->covering_keys. This caused Loose Scan not
to be considered for Clustered PKs.
Fixed this by making the check in get_best_group_min_max() to allow
Loose Scan if all fields used by the query are parts of the Clustered PK.
Also made changes:
- The logic in JOIN_TAB::save_explain_data() assumed that Loose Scan
could only be used for index-only scans. This is not the case for
clustered PKs.
- One query in group_min_max_innodb started to use Loose Scan. It did so
in 10.11, too.
(Variant 2)
SEL_TREE* tree_or(SEL_TREE *X, SEL_TREE *Y) tries to conserve memory by
reusing object *X for the return value when possible.
MDEV-34620 has added logic to disable construction of index_merge plans
for N-way ORs with large N. That logic interfered with object reuse logic:
for the parameters of:
X = SEL_TREE{ trees=[key1_treeA, key2_treeB]}
Y = SEL_TREE{ trees=[key1_treeC]}
we would decide to reuse object X.
For key1, we would produce key_or(key1_treeA, key1_treeC)
but key2_treeB would be just left there.
Then, we would construct a range scan from key2_treeB.
Fixed by moving the "disable building index_merge plans" logic into a
location where it would not interfere with object reuse logic.
The "FETCH FIRST n ROWS WITH TIES" was not enforced when the SELECT used
"using index for group-by".
This was caused by an optimization which removed the ORDER BY clause
when the GROUP BY clause prescribed a compatible ordering.
Other GROUP BY strategies used workarounds to still handle WITH TIES,
see comment to using_with_ties_and_group_min_max() in this patch for
details. QUICK_GROUP_MIN_MAX_SELECT didn't have a workaround.
Fix this by disabling removal of ORDER BY when
QUICK_GROUP_MIN_MAX_SELECT is used.
- Removed duplicate words, like "the the" and "to to"
- Removed duplicate lines (one double sort line found in mysql.cc)
- Fixed some typos found while searching for duplicate words.
Command used to find duplicate words:
egrep -rI "\s([a-zA-Z]+)\s+\1\s" | grep -v param
Thanks to Artjoms Rimdjonoks for the command and pointing out the
spelling errors.
Description:
The function get_best_ror_intersect is responsible for selecting
the optimal combination of ROR scans that minimize cost while
improving selectivity. It iteratively adds scans to a selected
set (S), ensuring that each addition results in improved selectivity.
If selectivity improves, the function then evaluates whether
the cost is minimized.
The comment contained some inaccuracies:
- Incorrect Selectivity Condition:
A missing parentheses caused the condition to be misinterpreted,
leading to incorrect logic. The function intends to check whether
adding a scan improves selectivity before including it in the set.
- Loop Condition Issue:
The condition for continuing the loop did not properly reduce R in
the comment which meant it was an infinite loop.
Fix:
The comment of the function is fixed addressing the issues.
The set should include the scan before comparing the
selectivity with the initial set and thus the selectivity condition
in the comment is fixed by properly enclosing the expression in
parentheses to reflect the intended logic.
Ensured that R is properly reduced in each iteration to maintain
correctness.
Change-Id: Ie197af8211a5ef05a5118a33b8b543d354475780
If a table had a primary_key index, non_primary_key index,
and an extended index with primary key in desc order,
then the query with where clause predicates containing these
index fields was picking an index_merge plan which was causing
a wrong result.
The plan shouldn't have contained index_merge in the first place,
because the rows produced by one index have rows ordered by rowid
in ascending order, and the rows coming from the other index
were in a different order.
The solution is to not produce index_merge plan in such scenarios.
is_key_scan_ror() would now return false, for non_primary_key indexes
if any primary key part with a reverse sort is present in them.
handler::clone() call did not work with read only tables like S3.
It gave a wrong error message (out of memory instead of a permission
error) and aborted the query.
The issue was that the clone call had a wrong parameter to ha_open().
This now fixed. I also changed the clone call to provide the correct
error message if things fails.
This patch fixes an 'out of memory' error when using the S3 engine
for queries that could use multiple indexes together to find the matching
rows, like the following:
SELECT * FROM t1 WHERE key1 = 99 OR key2 = 2
Disallow range optimization for BETWEEN when casting one of the arguments
from STRING to a numeric type would be required to construct a range for
the query.
Adds a new method on Item_func_between called can_optimize_range_const
which allows range optimization when the types of the arguments to BETWEEN
would permit it.
Use reclength because rec_buff_length is the actual reclength with
padding, whose use could cause ASAN unknown-crash, presumably caused
by memory violation.
Loose index scan currently only supports ASC key. When searching for
the next MAX value, the search starts from the rightmost range and
moves towards the left. If the key has "moved" as the result of a
successful ha_index_read_map() call when handling a previous range, we
check if the key is to the left of the current range, and if so, we
can skip to the next range.
The existing check on whether the loop has iterated at least once is
not sufficient, as an unsuccessful ha_index_read_map() often (always?)
does not "move" the key.
If a query has many OR-ed constructs which can use multiple indexes
(key1=1 AND key2=10) OR
(key1=2 AND key2=20) OR
(key1=3 AND key2=30) OR
...
The range optimizer would construct and then discard a lot of potential
index_merge plans. This process
1. is CPU-intensive
2. can hit the @@optimizer_max_sel_args limitation after which all
potential range or index_merge plans are discarded.
The fix is to apply a heuristic: if there is an OR clause with more than
MAX_OR_ELEMENTS_FOR_INDEX_MERGE=100 branches (hard-coded constant),
disallow construction of index_merge plans for the OR branches.
normalize_cond() translated `WHERE col` into `WHERE col<>0`
But the opetator "not equal to 0" does not necessarily exists
for all data types.
For example, the query:
SELECT * FROM t1 WHERE inet6col;
was translated to:
SELECT * FROM t1 WHERE inet6col<>0;
which further failed with this error:
ERROR : Illegal parameter data types inet6 and bigint for operation '<>'
This patch changes the translation from `col<>0` to `col IS TRUE`.
So now
SELECT * FROM t1 WHERE inet6col;
gets translated to:
SELECT * FROM t1 WHERE inet6col IS TRUE;
Details:
1. Implementing methods:
- Field_longstr::val_bool()
- Field_string::val_bool()
- Item::val_int_from_val_str()
If the input contains bad data,
these methods raise a better error message:
Truncated incorrect BOOLEAN value
Before the change, the error was:
Truncated incorrect DOUBLE value
2. Fixing normalize_cond() to generate Item_func_istrue/Item_func_isfalse
instances instead of Item_func_ne/Item_func_eq
3. Making Item_func_truth sargable, so it uses the range optimizer.
Implementing the following methods:
- get_mm_tree(), get_mm_leaf(), add_key_fields() in Item_func_truth.
- get_func_mm_tree(), for all Item_func_truth descendants.
4. Implementing the method negated_item() for all Item_func_truth
descendants, so the negated item has a chance to be sargable:
For example,
WHERE NOT col IS NOT FALSE -- this notation is not sargable
is now translated to:
WHERE col IS FALSE -- this notation is sargable
Make Item_func_eq of the following forms sargable by updating the relevant range
analysis methods:
1. substr(col, 1, n) = str
2. str = substr(col, 1, n)
3. left(col, n) = str
4. str = left(col, n)
where col is a indexed column and str is a const and inexpensive item
of length n.
We do this by factoring out Item_func_like::get_mm_leaf() and apply it
to a string obtained from escaping str and then appending a wildcard
"%" to it.
The addition of the two Functype enums, LEFT_FUNC and SUBSTR_FUNC,
requires changes in the spider group by handler to continue handling
LEFT and SUBSTR correctly.
Co-authored-by: Yuchen Pei <ycp@mariadb.com>
Co-authored-by: Sergei Petrunia <sergey@mariadb.com>
Partial commit of the greater MDEV-34348 scope.
MDEV-34348: MariaDB is violating clang-16 -Wcast-function-type-strict
The functions queue_compare, qsort2_cmp, and qsort_cmp2
all had similar interfaces, and were used interchangable
and unsafely cast to one another.
This patch consolidates the functions all into the
qsort_cmp2 interface.
Reviewed By:
============
Marko Mäkelä <marko.makela@mariadb.com>
