database/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2026-10-23 18:49:29 +03:00
Code Activity

MDEV-40620: Allow a nested join as a FULL JOIN operand

Browse Source 
Phase 2 rejected two shapes.  A FULL JOIN whose right operand was a
nested join raised ER_FULL_JOIN_BASE_TABLES_ONLY, and a FULL JOIN on the
inner side of an enclosing LEFT or RIGHT JOIN raised
ER_FULL_JOIN_NOT_ALLOWED_IN_OUTER_JOIN.  Both shapes are allowed now and
neither error is raised any more.

A FULL JOIN emits the rows of one side that never matched with the other
side null complemented.  An enclosing join's condition has to see those
rows complete, which it cannot do while the operand is only a set of
tables spread through the join order.  An operand that is a nested join
is therefore computed into a temporary table before the enclosing join
runs, so one JOIN_TAB stands for the whole operand and carries the FULL
JOIN's own marks.

The chosen plan has to keep such an operand on an unbroken span of the
join order, and that span becomes a run of JOIN_TABs of its own, the way
a materialized semi join already is.  Nothing outside the operand is
positioned while the operand is computed, so a table inside it can carry
no condition that names an outside table, an equality class that spans
the boundary is split at it, and the outer join scope of a join outside
it stops there.  A nest goes on naming a table that table elimination
removed, and an eliminated table is a const table, so the test for
entering the run leaves the eliminated tables out.  The test for
entering a materialized semi join run reads the same way.

The operand keeps a copy of its own ON expression, since simplifying the
join tree merges an inner join's ON into the enclosing outer join's ON,
which is not equivalent for a FULL JOIN because it changes which rows of
the right side never match.  An operand can itself be built from more
than one join, so an ON expression can sit a level or two below the top
of it, and every such expression is recorded on the operand as a whole,
which is the nest that reads them back.  A predicate over the operand is
checked again at the entry standing for it, once the null complementing
has happened.  A FULL JOIN's left side WHERE predicates are deferred to
the table that completes its right side even when that side is a nest.
The enclosing ON contributes no access path for the operand.

Reading a row back restores what a nest table cannot hold on its own,
whether the row was null complemented for a column declared NOT NULL,
and the status that says whether the table holds a usable row at all.

The JOIN_TAB standing for a run covers every table of the nest, the
eliminated ones included.  The plan output therefore skips only a
JOIN_TAB that is a table of the query, since a JOIN_TAB standing for a
run is never eliminated.

The FULL JOIN code moves out of sql_select.cc, sql_base.cc and table.cc
into opt_full_join.cc and opt_full_join.h.  JOIN::full_joins holds a
Full_join_info, declared in opt_full_join_info.h, for each FULL JOIN
that simplify_joins did not rewrite.  It holds the two sides, the WHERE
conjuncts held back for the FULL JOIN, the tab that reads its right
side, the duplicate filter of the right rows that matched, and the next
FULL JOIN whose null complement pass runs from the same tab.  It
replaces fj_dups, fj_first_target and fj_next_target on JOIN_TAB and
fj_left_cond on TABLE_LIST.  The list is built for each JOIN, so a
prepared statement starts every execution with nothing held.

opt_join_runs.cc and opt_join_runs.h hold the runs.
get_best_combination records every run of the chosen join order in a
Join_run_layout before it writes any JOIN_TAB, and the layout gives the
exact number of top level JOIN_TABs.  make_join_select walks the plan
with Join_tab_walk, which keeps the tables a run sets aside on entry
until the walk reaches the run's placeholder.  A FULL JOIN run can be
inside a materialized semi join, so the walk keeps the runs on a stack.

make_join_select decides again which conditions wait for a FULL JOIN's
right partner, on each condition about to be attached to a table.
Distributing the WHERE builds conditions for one table out of
disjunctions that span both sides, and a HAVING predicate moved into
the WHERE can name the left side alone, so the choice made while the
WHERE is rewritten misses both.  That earlier choice stays, since it
keeps the left side out of ref and range analysis.  Whether a condition
reads a table after the partner is decided from the join order
positions of the tables it reads.

An enclosing join's ON supplies no key value for a table on a FULL
JOIN's left side, because the null complement pass is complete only
once every left row has been read.  A key value that reads only tables
of the same left side is still allowed.  An ORDER BY over a FULL JOIN
that simplify_joins kept sorts through a temporary table, because the
null complement pass emits right rows after the scan of the first
table.  EXPLAIN gathers the tables of each run at its own depth, and an
operand nest inside a materialized semi join reports under that semi
join's id.

Sections 22 to 28 cover the new shapes.  The section covering more than
one matching right side row becomes Section 28, which adds a nest operand
to the same case, and the two sections that came with the fixes before
this one are renumbered to follow it.  Sections 35 to 42 cover an ON
expression below the top of an operand, an enclosing ON that keeps the
left side NULLs, operand nests that table elimination shrinks, a
condition offered to a table inside a run, a prepared WHERE that folds
away on a later execution, a FULL JOIN run inside a materialized semi
join, and a left side condition over a table before the partner.
Sections 8, 18 and 27 gain left side predicates that appear only once
the WHERE is rewritten.  The helper that gives every column an index of
its own comes with them, ahead of the axis that turns it on.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Dave Gosselin
2026-08-17 11:41:56 -04:00
parent a877d79674
commit 4561ccad91
27 changed files with 14049 additions and 2705 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
libmysqld/CMakeLists.txt
View File

@@ -124,6 +124,8 @@ SET(SQL_EMBEDDED_SOURCES emb_qcache.cc libmysqld.c lib_sql.cc
../sql/multi_range_read.cc
../sql/opt_index_cond_pushdown.cc
../sql/opt_subselect.cc
../sql/opt_full_join.cc
../sql/opt_join_runs.cc
../sql/create_options.cc
../sql/rpl_utility.cc
../sql/rpl_utility_server.cc

43
mysql-test/include/full_join_add_indexes.inc Normal file
View File

@@ -0,0 +1,43 @@
# Give every column of every base table in the current database a
# non-unique index on itself, unless the column already leads an index.
# Does nothing outside the allidx combination of full_join_index.inc,
# which is the file that declares the axis and must be sourced once by
# any test that sources this one.
#
# Source this after every group of CREATE TABLE and INSERT statements.
# The work is decided by what the current database holds at that moment,
# so a table created later in a section needs its own source after it.
# Sourcing it again over tables that are already covered adds nothing,
# which is what lets a section place it wherever it is convenient.
#
# A non-unique index is the only kind that can be added without knowing
# the data. A primary key or a unique index would fail on the duplicates
# and the NULLs that many of these sections put in a join column on
# purpose, so those stay written out in the sections that want them.
#
# What this does not reach: a temporary table, which the information
# schema does not report, and a vector column, which is excluded because
# an index on one is a vector index rather than the ordinary index this
# axis is about.
#
# Identifiers are not quoted below because a backtick would end the query
# MTR is reading. Every table and column name in the FULL JOIN tests is
# a plain identifier, so none needs quoting.
if ($MTR_COMBINATION_ALLIDX)
{
--disable_query_log
--disable_result_log
let $fj_idx_query= select concat('alter table ', c.table_name, ' add index fj_idx_', c.column_name, ' (', c.column_name, ')') from information_schema.columns c join information_schema.tables t on t.table_schema = c.table_schema and t.table_name = c.table_name where c.table_schema = database() and t.table_type = 'BASE TABLE' and c.data_type <> 'vector' and not exists (select 1 from information_schema.statistics s where s.table_schema = c.table_schema and s.table_name = c.table_name and s.column_name = c.column_name and s.seq_in_index = 1) order by c.table_name, c.ordinal_position limit 1;
let $fj_idx_stmt= `$fj_idx_query`;
while ($fj_idx_stmt != '')
{
eval $fj_idx_stmt;
let $fj_idx_stmt= `$fj_idx_query`;
}
--enable_result_log
--enable_query_log
}

5973
mysql-test/main/full_join.result
View File

File diff suppressed because it is too large Load Diff

4166
mysql-test/main/full_join.test
View File

File diff suppressed because it is too large Load Diff

54
mysql-test/main/natural_full_join_grants.result
View File

@@ -17,13 +17,19 @@ use mysqltest_nfj;
create table t1 (a int, b int);
create table t2 (a int, c int);
create table t3 (a int, d int);
create table t4 (a int, e int);
create table t5 (a int, f int);
insert into t1 values (1,10),(2,20);
insert into t2 values (2,200),(3,300);
insert into t3 values (2,2000),(9,9000);
insert into t4 values (2,20000),(4,40000);
insert into t5 values (2,200000),(5,500000);
create user nfj_user@localhost;
grant select (a,b) on mysqltest_nfj.t1 to nfj_user@localhost;
grant select (c) on mysqltest_nfj.t2 to nfj_user@localhost;
grant select (d) on mysqltest_nfj.t3 to nfj_user@localhost;
grant select (e) on mysqltest_nfj.t4 to nfj_user@localhost;
grant select (f) on mysqltest_nfj.t5 to nfj_user@localhost;
connect nfj_con, localhost, nfj_user,, mysqltest_nfj;
# nfj_user lacks SELECT on t2.a, so the coalesced a is denied in
# the SELECT list and in WHERE.
@@ -58,13 +64,44 @@ a
2
3
9
# The chained join above puts the nested join on the left of the
# outer NATURAL FULL JOIN. The two below put it on the right and
# then on both sides, so the walk has to descend into an operand
# from either direction.
# Nested join on the right; t4.a is not granted.
select a from t1 natural full join (t2 natural full join t4);
ERROR 42000: SELECT command denied to user 'nfj_user'@'localhost' for column 'a' in table 't4'
connection default;
grant select (a) on mysqltest_nfj.t4 to nfj_user@localhost;
connection nfj_con;
# With t4.a granted the read works.
select a from t1 natural full join (t2 natural full join t4);
a
1
2
3
4
# Nested joins on both sides; t5.a is not granted.
select a from (t1 natural full join t2)
natural full join (t4 natural full join t5);
ERROR 42000: SELECT command denied to user 'nfj_user'@'localhost' for column 'a' in table 't5'
connection default;
grant select (a) on mysqltest_nfj.t5 to nfj_user@localhost;
connection nfj_con;
# With t5.a granted the read works.
select a from (t1 natural full join t2)
natural full join (t4 natural full join t5);
a
1
2
3
4
5
connection default;
disconnect nfj_con;
# The same check covers views, and across view kinds. A view on the
# right side of a FULL JOIN is rejected as a non-base table, so the
# executable shape is a view on the left. Reading the coalesced
# column then requires the column grant on the view, however the view
# is materialized.
# The same check covers views, and across view kinds. Reading the
# coalesced column requires the column grant on the view, however the
# view is materialized.
#
# v_merge mergeable view
# v_tmp temptable (materialized) view
@@ -76,9 +113,12 @@ create algorithm=temptable view v_tmp as select a, b from t1;
create view v_expr as select a + 0 as a, b from t1;
create view v_over as select a, b from v_merge;
create view v_union as select a, b from t1 union select a, b from t1;
# A view on the right side is rejected (run as the table owner).
# A view on the right side runs (run as the table owner).
select a from t2 natural full join v_merge;
ERROR HY000: FULL JOIN is only supported with base tables on the right side; 'v_merge' is not a base table
a
1
2
3
create user nfj_vuser@localhost;
grant select (a,c) on mysqltest_nfj.t2 to nfj_vuser@localhost;
grant select (b) on mysqltest_nfj.v_merge to nfj_vuser@localhost;

49
mysql-test/main/natural_full_join_grants.test
View File

@@ -21,13 +21,19 @@ use mysqltest_nfj;
create table t1 (a int, b int);
create table t2 (a int, c int);
create table t3 (a int, d int);
create table t4 (a int, e int);
create table t5 (a int, f int);
insert into t1 values (1,10),(2,20);
insert into t2 values (2,200),(3,300);
insert into t3 values (2,2000),(9,9000);
insert into t4 values (2,20000),(4,40000);
insert into t5 values (2,200000),(5,500000);
create user nfj_user@localhost;
grant select (a,b) on mysqltest_nfj.t1 to nfj_user@localhost;
grant select (c) on mysqltest_nfj.t2 to nfj_user@localhost;
grant select (d) on mysqltest_nfj.t3 to nfj_user@localhost;
grant select (e) on mysqltest_nfj.t4 to nfj_user@localhost;
grant select (f) on mysqltest_nfj.t5 to nfj_user@localhost;
connect (nfj_con, localhost, nfj_user,, mysqltest_nfj);
@@ -62,14 +68,43 @@ connection nfj_con;
--sorted_result
select a from (t1 natural full join t2) natural full join t3;
--echo # The chained join above puts the nested join on the left of the
--echo # outer NATURAL FULL JOIN. The two below put it on the right and
--echo # then on both sides, so the walk has to descend into an operand
--echo # from either direction.
--echo # Nested join on the right; t4.a is not granted.
--error ER_COLUMNACCESS_DENIED_ERROR
select a from t1 natural full join (t2 natural full join t4);
connection default;
grant select (a) on mysqltest_nfj.t4 to nfj_user@localhost;
connection nfj_con;
--echo # With t4.a granted the read works.
--sorted_result
select a from t1 natural full join (t2 natural full join t4);
--echo # Nested joins on both sides; t5.a is not granted.
--error ER_COLUMNACCESS_DENIED_ERROR
select a from (t1 natural full join t2)
natural full join (t4 natural full join t5);
connection default;
grant select (a) on mysqltest_nfj.t5 to nfj_user@localhost;
connection nfj_con;
--echo # With t5.a granted the read works.
--sorted_result
select a from (t1 natural full join t2)
natural full join (t4 natural full join t5);
connection default;
disconnect nfj_con;
--echo # The same check covers views, and across view kinds. A view on the
--echo # right side of a FULL JOIN is rejected as a non-base table, so the
--echo # executable shape is a view on the left. Reading the coalesced
--echo # column then requires the column grant on the view, however the view
--echo # is materialized.
--echo # The same check covers views, and across view kinds. Reading the
--echo # coalesced column requires the column grant on the view, however the
--echo # view is materialized.
--echo #
--echo # v_merge mergeable view
--echo # v_tmp temptable (materialized) view
@@ -82,8 +117,8 @@ create view v_expr as select a + 0 as a, b from t1;
create view v_over as select a, b from v_merge;
create view v_union as select a, b from t1 union select a, b from t1;
--echo # A view on the right side is rejected (run as the table owner).
--error ER_FULL_JOIN_BASE_TABLES_ONLY
--echo # A view on the right side runs (run as the table owner).
--sorted_result
select a from t2 natural full join v_merge;
create user nfj_vuser@localhost;

16
mysql-test/main/table_elim.result
View File

@@ -1129,8 +1129,20 @@ id select_type table type possible_keys key key_len ref rows filtered Extra
Warnings:
Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` full join `test`.`t2` on(`test`.`t2`.`a` = `test`.`t1`.`a` and `test`.`t1`.`a` is not null) where 1
explain extended select t1.a from t1 full join (t2 full join t3 on t2.b=t3.b) on t2.a=t1.a and t3.a=t1.a;
ERROR HY000: FULL JOIN is only supported with base tables on the right side; '(nest_last_join)' is not a base table
id select_type table type possible_keys key key_len ref rows filtered Extra
1 SIMPLE <derived> ALL NULL NULL NULL NULL 0 0.00
1 SIMPLE t1 ALL NULL NULL NULL NULL 4 100.00 Using where
1 MATERIALIZED t2 ALL NULL NULL NULL NULL 2 100.00
1 MATERIALIZED t3 ALL NULL NULL NULL NULL 2 100.00 Using where
Warnings:
Note 1003 select `test`.`t1`.`a` AS `a` from (`test`.`t2` full join `test`.`t3` on(`test`.`t2`.`b` = `test`.`t3`.`b`)) full join `test`.`t1` on(`test`.`t3`.`a` = `test`.`t2`.`a` and `test`.`t1`.`a` = `test`.`t2`.`a`) where 1
explain extended select t1.a from t1 full join (t2 left join t3 on t2.b=t3.b) on t2.a=t1.a and t3.a=t1.a;
ERROR HY000: FULL JOIN is only supported with base tables on the right side; '(nest_last_join)' is not a base table
id select_type table type possible_keys key key_len ref rows filtered Extra
1 SIMPLE t1 ALL NULL NULL NULL NULL 4 100.00
1 SIMPLE <derived> ALL NULL NULL NULL NULL 0 0.00 Using where
1 MATERIALIZED t2 ALL PRIMARY NULL NULL NULL 2 50.00
1 MATERIALIZED t3 ALL NULL NULL NULL NULL 2 100.00 Using where
Warnings:
Note 1003 select `test`.`t1`.`a` AS `a` from `test`.`t1` full join (`test`.`t2` left join `test`.`t3` on(`test`.`t3`.`b` = `test`.`t2`.`b`)) on(`test`.`t2`.`a` = `test`.`t1`.`a` and `test`.`t3`.`a` = `test`.`t2`.`a`) where 1
drop table t1, t2, t3;
# End of 12.3 tests

2
mysql-test/main/table_elim.test
View File

@@ -859,9 +859,7 @@ create table t3 (a int primary key, b int)
--echo # These will not be eliminated because contains a FULL OUTER JOIN.
explain extended select t1.a from t1 full join t2 on t2.a=t1.a;
--error ER_FULL_JOIN_BASE_TABLES_ONLY
explain extended select t1.a from t1 full join (t2 full join t3 on t2.b=t3.b) on t2.a=t1.a and t3.a=t1.a;
--error ER_FULL_JOIN_BASE_TABLES_ONLY
explain extended select t1.a from t1 full join (t2 left join t3 on t2.b=t3.b) on t2.a=t1.a and t3.a=t1.a;
drop table t1, t2, t3;

1
sql/CMakeLists.txt
View File

@@ -172,6 +172,7 @@ SET (SQL_SOURCE
opt_histogram_json.cc
opt_index_cond_pushdown.cc opt_subselect.cc
opt_table_elimination.cc sql_expression_cache.cc
opt_full_join.cc opt_join_runs.cc
gcalc_slicescan.cc gcalc_tools.cc
my_apc.cc mf_iocache_encr.cc item_jsonfunc.cc
my_json_writer.cc json_schema.cc json_schema_helper.cc

1
sql/field.h
View File

@@ -6094,6 +6094,7 @@ public:
Copy_field() = default;
~Copy_field() = default;
void set(Field *to,Field *from,bool save); // Field to field
void set_restoring_null_row(Field *to, Field *from);
void set(uchar *to,Field *from); // Field to string
void (*do_copy)(const Copy_field *);
void (*do_copy2)(const Copy_field *); // Used to handle null values

47
sql/field_conv.cc
View File

@@ -303,6 +303,33 @@ static void do_copy_not_null(const Copy_field *copy)
}
/*
Copy: (NULL-able field) -> (not NULL-able field of a row that may be
NULL as a whole)
A field declared NOT NULL holds no null bit, so the only way for it to
read as NULL is TABLE::null_row, which says the whole row is a null
complemented one. A NULL reaching such a field therefore says the row
it came from was null complemented, and restoring that state is what
makes the field read as NULL again. This is the inverse of
do_outer_field_null, which recorded the state in the first place.
*/
static void do_copy_null_to_outer_field(const Copy_field *copy)
{
if (*copy->from_null_ptr & copy->from_bit)
{
copy->to_field->table->null_row= 1;
copy->to_field->reset();
}
else
{
copy->to_field->table->null_row= 0;
(copy->do_copy2)(copy);
}
}
/* Copy: (non-NULLable field) -> (NULLable field) */
static void do_copy_maybe_null(const Copy_field *copy)
{
@@ -777,6 +804,26 @@ void Copy_field::set(Field *to,Field *from,bool save)
}
/*
Same as set(), except that a NULL arriving at a destination declared
NOT NULL is kept rather than turned into a zero with a truncation
warning. It is kept as TABLE::null_row on the destination's table,
the only place such a field can carry a NULL.
This is for reading a row back out of a temporary table into the fields
it was collected from. A truncation warning would be wrong there, since
no value is being narrowed, and losing the NULL would turn a null
complemented row into a row of zeroes.
*/
void Copy_field::set_restoring_null_row(Field *to, Field *from)
{
set(to, from, FALSE);
if (from->maybe_null() && from->null_ptr && !to->real_maybe_null())
do_copy= do_copy_null_to_outer_field;
}
Field::Copy_func *Field_timestamp::get_copy_func(const Field *from) const
{
Field::Copy_func *copy= Field_temporal::get_copy_func(from);

16
sql/item_cmpfunc.cc
View File

@@ -7741,6 +7741,22 @@ Item* Item_equal::get_first(JOIN_TAB *context, Item *field_item)
}
}
}
else if ((emb_nest= embedding_materialized_nest(field)) &&
emb_nest->nested_join->materialized_full_join)
{
/*
It's a field of a FULL JOIN operand nest that is computed into a
temporary table. The tables outside the nest have no current row
while the nest is computed, so the substitute has to be a constant
or a field of the same nest.
*/
while ((item= it++))
{
if (item->const_item() ||
embedding_materialized_nest(it.get_curr_field()) == emb_nest)
return (item != field_item) ? item : NULL;
}
}
else
{
/*

2892
sql/opt_full_join.cc Normal file
View File

File diff suppressed because it is too large Load Diff

104
sql/opt_full_join.h Normal file
View File

@@ -0,0 +1,104 @@
/*
Copyright (c) 2026, MariaDB
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; version 2 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1335 USA */
/*
FULL JOIN rewrite, planning, and execution support. Declarations for
the functions in opt_full_join.cc that are called from outside it.
*/
COND *lift_full_join_left_where(JOIN *join, COND *conds);
bool full_join_left_side_allows(TABLE_LIST *tl,
table_map used_tables);
class Join_run_layout;
void record_full_join_runs(JOIN *join, Join_run_layout *layout);
bool
setup_full_join_materialization(JOIN_TAB *tab);
void attach_full_join_nest_conds(THD *thd, JOIN *join);
bool hold_full_join_left_cond(THD *thd, JOIN *join, table_map prefix,
COND *cond);
COND *restore_full_join_left_conds(JOIN *join, COND *conds);
bool attach_full_join_left_conds(THD *thd, JOIN *join);
COND *lift_full_join_nest_where(THD *thd, JOIN *join, COND *cond);
bool attach_full_join_nest_where(THD *thd, JOIN *join);
JOIN_TAB *fj_deferral_target(JOIN *join, JOIN_TAB *dest,
JOIN_TAB *first_inner_tab);
bool
setup_full_join_materialization_scan(JOIN_TAB *tab);
bool is_in_full_join_scope(TABLE_LIST *tl);
table_map usable_not_null_tables(COND *conds, TABLE_LIST *fj_operand);
COND *rewrite_full_outer_joins(JOIN *join,
COND *conds,
bool in_sj,
TABLE_LIST *fj_operand,
TABLE_LIST **right_table,
List_iterator<TABLE_LIST> *li,
table_map *used_tables,
table_map *not_null_tables);
void clear_full_join_nest_conds(List<TABLE_LIST> *join_list);
void record_full_join_nest_cond(THD *thd, TABLE_LIST *nest,
Item *on_expr);
bool
compute_full_join_nest_tables(JOIN *join, SELECT_LEX *lex);
table_map
restrict_to_unplaced_fj_tables(JOIN *join, uint idx, table_map pool);
bool alloc_full_join_duplicate_filters(JOIN *join, JOIN_TAB *start_tab,
uint count);
void free_full_join_duplicate_filters(JOIN *join);
void
reset_fj_duplicate_filters(JOIN_TAB *join_tab);
enum_nested_loop_state
run_fj_null_complement_passes(JOIN *join, JOIN_TAB *join_tab);
void mark_table_list_as_null_row(TABLE_LIST *tl);
enum_nested_loop_state
evaluate_fj_null_complement_row(JOIN *join, JOIN_TAB *join_tab,
COND *select_cond);
bool remember_full_join_right_rowid(JOIN_TAB *join_tab, THD *thd);
bool record_full_join_right_match(JOIN *join, JOIN_TAB *join_tab,
COND *select_cond);
int read_record_func_for_full_join_nest(READ_RECORD *info);
bool is_in_full_join_left_side(TABLE_LIST *tl);
int coalesce_natural_full_join(THD *thd,
List<Natural_join_column> *left_tab_col,
List<Natural_join_column> *right_tab_col);

89
sql/opt_full_join_info.h Normal file
View File

@@ -0,0 +1,89 @@
/*
Copyright (c) 2026, MariaDB
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; version 2 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1335 USA */
#ifndef OPT_FULL_JOIN_INFO_INCLUDED
#define OPT_FULL_JOIN_INFO_INCLUDED
#include "sql_alloc.h"
class JOIN;
struct TABLE_LIST;
class Item;
typedef class Item COND;
typedef struct st_join_table JOIN_TAB;
class full_join_duplicate_filter;
/*
A FULL JOIN that simplify_joins did not rewrite. It runs as a LEFT
JOIN of its left side over its right side, followed by a pass over the
right side that emits the rows no left row matched. A side is a join
nest when the query parenthesised it and a single table otherwise, and
each side is the other's foj_partner.
The sides are set when the JOIN is optimized, and the held conditions
while it is optimized. The members after them are set by
alloc_full_join_duplicate_filters at the start of each execution, and
dups is released by free_full_join_duplicate_filters at its end.
*/
class Full_join_info : public Sql_alloc
{
public:
/* The side marked JOIN_TYPE_FULL | JOIN_TYPE_LEFT. */
TABLE_LIST *left;
/* The side marked JOIN_TYPE_FULL | JOIN_TYPE_RIGHT. */
TABLE_LIST *right;
/*
The WHERE conjuncts that reference the left side, lifted out of the
WHERE before access selection so they neither prune nor drive access
on the left side. make_join_select attaches them to the right tab
under the found-match guard.
*/
COND *left_cond;
/*
When the chosen join order computes the right side into a temporary
table, the WHERE conjuncts that reference nothing but that side,
taken out of the WHERE once the join order is known. Left in the
WHERE they would be placed on a table inside the nest and decide
which rows the nest holds. make_join_select attaches them to the tab
that stands for the whole nest instead.
*/
COND *nest_where;
/*
The tab whose tab_list is right. It records the rowid of each right
row that matched, and the null complement pass scans it again.
*/
JOIN_TAB *rescan_tab;
/* The rowids of the right rows that matched. */
full_join_duplicate_filter *dups;
/*
The next FULL JOIN whose null complement pass runs when the scan of
the same tab ends, see JOIN_TAB::fj_passes.
*/
Full_join_info *next_pass;
Full_join_info(TABLE_LIST *left_arg, TABLE_LIST *right_arg)
: left(left_arg), right(right_arg), left_cond(NULL), nest_where(NULL),
rescan_tab(NULL), dups(NULL), next_pass(NULL) {}
table_map left_map() const;
table_map right_map() const;
JOIN_TAB *right_tab(JOIN *join) const;
JOIN_TAB *find_left_most_tab(JOIN *join) const;
};
#endif

290
sql/opt_join_runs.cc Normal file
View File

@@ -0,0 +1,290 @@
/*
Copyright (c) 2026, MariaDB
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; version 2 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1335 USA */
/**
@file
@brief
Where the runs of a picked join order begin and end, and the opening
and finishing of their JOIN_TAB arrays.
*/
#include "mariadb.h"
#include "sql_priv.h"
#include "sql_select.h"
#include "opt_full_join.h"
#include "opt_join_runs.h"
/* Used to build a pointer to a permanently null Item * for on_expr_ref. */
static Item * const null_ptr= NULL;
/*
The materialized semi join whose run begins at the given position of
the picked join order, otherwise NULL.
*/
static SJ_MATERIALIZATION_INFO *sjm_run_at(JOIN *join, uint pos)
{
POSITION *cur_pos= &join->best_positions[pos];
if (cur_pos->sj_strategy != SJ_OPT_MATERIALIZE &&
cur_pos->sj_strategy != SJ_OPT_MATERIALIZE_SCAN)
return NULL;
return cur_pos->table->emb_sj_nest->sj_mat_info;
}
/*
Whether run a is opened before run b, in the order described at
Join_run_layout.
*/
static bool opens_before(const Join_run *a, const Join_run *b)
{
if (a->first != b->first)
return a->first < b->first;
if (a->count != b->count)
return a->count > b->count;
return a->kind == JOIN_TAB_RANGE_SJM && b->kind != JOIN_TAB_RANGE_SJM;
}
/*
Record the runs of the picked join order. The semi join strategies
have to be final.
RETURN
false OK
true Out of memory
*/
bool Join_run_layout::init(JOIN *join)
{
const Join_run *top= NULL;
runs= NULL;
run_count= 0;
max_runs= 0;
/*
The FULL JOIN operand nests are strictly nested or disjoint and each
run of one holds at least two tables, so there are fewer of them than
there are tables. At most one semi join run begins at a position.
*/
if (join->full_join_nest_tables)
max_runs+= join->table_count;
for (uint i= 0; i < join->table_count; i++)
{
if (sjm_run_at(join, i))
max_runs++;
}
if (max_runs && !(runs= join->thd->alloc<Join_run>(max_runs)))
return true;
if (join->full_join_nest_tables)
record_full_join_runs(join, this);
for (uint i= 0; i < join->table_count; i++)
{
SJ_MATERIALIZATION_INFO *sjm= sjm_run_at(join, i);
if (sjm)
add(JOIN_TAB_RANGE_SJM, i, sjm->tables, NULL, sjm);
}
/*
A run that begins at a position the previous top level run covers is
held by it. Every other run takes one entry of the top level array
for all the positions it covers.
*/
top_level_count= join->table_count;
for (const Join_run *run= begin(); run != end(); run++)
{
if (top && run->first <= top->last())
continue;
top= run;
top_level_count-= run->count - 1;
}
return false;
}
/*
Add a run, keeping the runs in the order they are opened. Runs that
opens_before() does not order keep the order they were added in.
*/
void Join_run_layout::add(Join_tab_range_type kind, uint first, uint count,
TABLE_LIST *nest, SJ_MATERIALIZATION_INFO *sjm)
{
Join_run run= { kind, first, count, nest, sjm };
uint i;
DBUG_ASSERT(run_count < max_runs);
for (i= run_count; i && opens_before(&run, &runs[i - 1]); i--)
runs[i]= runs[i - 1];
runs[i]= run;
run_count++;
}
/*
Open a run at entry j of the array being filled in. j becomes the
run's placeholder and is then pointed at the first entry of the run's
own array, so the tables to follow and any run opened among them go
into it.
The array has an entry for each position the run covers, which is an
upper bound because a run inside it takes one entry for two positions
or more. The array is narrowed to the entries used when the run
finishes.
A semi join placeholder reads the temporary table holding the semi
join's result, and takes its row count from the materialization. A
FULL JOIN placeholder is recorded on its nest.
RETURN
false OK
true Out of memory
*/
bool Join_run_stack::open(THD *thd, JOIN *join, const Join_run *run,
JOIN_TAB *&j)
{
JOIN_TAB *children;
DBUG_ASSERT(!depth || run->last() <= last[depth - 1]);
if (!(children= thd->alloc<JOIN_TAB>(run->count)))
return true;
bzero((void*) j, sizeof(JOIN_TAB));
j->join= join;
j->table= NULL;
j->ref.key= -1;
j->on_expr_ref= (Item**) &null_ptr;
j->bush_root_tab= innermost();
j->bush_children.start= children;
j->bush_children.end= children + run->count;
j->bush_children.kind= run->kind;
j->bush_children.nest= run->nest;
if (run->kind == JOIN_TAB_RANGE_SJM)
{
SJ_MATERIALIZATION_INFO *sjm= run->sjm;
/* The unique index is always in 'possible keys' in EXPLAIN */
j->keys= key_map(1);
j->records_read= (sjm->is_sj_scan ? sjm->rows : 1.0);
j->records_init= j->records_out= j->records_read;
j->records= (ha_rows) j->records_read;
j->cond_selectivity= 1.0;
j->join_read_time= 0.0; /* Not saved currently */
j->join_loops= 0.0;
join->join_tab_ranges.push_back(&j->bush_children, thd->mem_root);
}
else
{
run->nest->nested_join->materialized_full_join= TRUE;
run->nest->nested_join->materialized_full_join_tab= j;
if (join->join_tab_ranges.push_back(&j->bush_children, thd->mem_root))
return true;
}
DBUG_ASSERT(depth < MAX_TABLES);
placeholder[depth]= j;
last[depth]= run->last();
depth++;
j= children;
return false;
}
/*
Finish the runs whose last position is pos, innermost first, once j,
the entry for pos, is filled in. j is the last entry of the innermost
of them, and the placeholder of each finished run is the last entry of
the run that holds it.
RETURN
The entry the next position follows, which is the placeholder of the
outermost run finished here, or j when no run ends at pos.
*/
JOIN_TAB *Join_run_stack::finish_ended(uint pos, JOIN_TAB *j)
{
while (depth && last[depth - 1] == pos)
{
JOIN_TAB *root= placeholder[--depth];
j->last_leaf_in_bush= TRUE;
root->bush_children.end= j + 1;
j= root;
}
return j;
}
Join_tab_walk::Join_tab_walk(JOIN *join_arg)
: join(join_arg), cur(first_depth_first_tab(join_arg)), depth(0)
{}
void Join_tab_walk::next()
{
if (ends_set_aside())
depth--;
cur= next_depth_first_tab(join, cur);
}
/*
The placeholder of the outermost run of the given kind that holds the
current JOIN_TAB, or NULL when no such run holds it.
*/
JOIN_TAB *Join_tab_walk::outermost_run(Join_tab_range_type kind) const
{
JOIN_TAB *found= NULL;
for (JOIN_TAB *root= cur->bush_root_tab; root; root= root->bush_root_tab)
{
if (root->bush_children.kind == kind)
found= root;
}
return found;
}
/*
Set map aside until the walk reaches run, the placeholder of a run
that holds the current JOIN_TAB.
*/
void Join_tab_walk::set_aside(JOIN_TAB *run, table_map map)
{
DBUG_ASSERT(run);
#ifndef DBUG_OFF
if (depth)
{
JOIN_TAB *root= run->bush_root_tab;
while (root && root != placeholder[depth - 1])
root= root->bush_root_tab;
DBUG_ASSERT(root);
}
#endif
DBUG_ASSERT(depth < MAX_TABLES);
placeholder[depth]= run;
set_aside_map[depth]= map;
depth++;
}

142
sql/opt_join_runs.h Normal file
View File

@@ -0,0 +1,142 @@
/*
Copyright (c) 2026, MariaDB
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; version 2 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1335 USA */
#ifndef OPT_JOIN_RUNS_INCLUDED
#define OPT_JOIN_RUNS_INCLUDED
/*
A run is a stretch of consecutive positions of the picked join order
whose tables get a JOIN_TAB array of their own. One entry of the
enclosing array, the placeholder, stands for all of them. A
materialized semi join gets a run, and so does a FULL JOIN operand nest
of two tables or more. Two runs are disjoint or one holds the other.
*/
struct Join_run
{
/* JOIN_TAB_RANGE_SJM or JOIN_TAB_RANGE_FULL_JOIN. */
Join_tab_range_type kind;
/* Index into JOIN::best_positions of the run's first table. */
uint first;
/* How many positions the run covers. */
uint count;
/* The FULL JOIN operand nest, for a run of kind JOIN_TAB_RANGE_FULL_JOIN. */
TABLE_LIST *nest;
/* The semi join's materialization, for a run of kind JOIN_TAB_RANGE_SJM. */
SJ_MATERIALIZATION_INFO *sjm;
uint last() const { return first + count - 1; }
};
/*
The runs of the picked join order, in the order their JOIN_TAB arrays
are opened. That is by first position, and among the runs that begin
at the same position, by the number of positions covered, most first.
A run that holds another has to be opened first, because the inner
run's placeholder is an entry of the outer run's array.
A semi join run is opened before a FULL JOIN run that begins at the
same position and covers as many positions, which places the FULL JOIN
run inside the semi join. FULL JOIN runs covering the same positions
are opened in the order of the join tree, outer nest first.
*/
class Join_run_layout
{
Join_run *runs;
uint run_count;
uint max_runs;
/* Entries of the top level JOIN_TAB array, one per run or table. */
uint top_level_count;
public:
bool init(JOIN *join);
void add(Join_tab_range_type kind, uint first, uint count,
TABLE_LIST *nest, SJ_MATERIALIZATION_INFO *sjm);
const Join_run *begin() const { return runs; }
const Join_run *end() const { return runs + run_count; }
uint top_level_entries() const { return top_level_count; }
};
/*
The runs opened and not yet finished while the JOIN_TAB arrays are
filled in, outermost first.
*/
class Join_run_stack
{
JOIN_TAB *placeholder[MAX_TABLES];
uint last[MAX_TABLES];
uint depth;
public:
Join_run_stack(): depth(0) {}
/* The placeholder of the innermost open run, NULL outside every run. */
JOIN_TAB *innermost() const
{
return depth ? placeholder[depth - 1] : NULL;
}
bool is_empty() const { return !depth; }
bool open(THD *thd, JOIN *join, const Join_run *run, JOIN_TAB *&j);
JOIN_TAB *finish_ended(uint pos, JOIN_TAB *j);
};
/*
A walk over the JOIN_TABs of a plan in the order of
first_depth_first_tab and next_depth_first_tab, which reaches the
tables of a run before the placeholder that stands for the run.
A caller entering a run can set a table map aside. The map is
available again at the run's placeholder and is dropped when the walk
moves past it. A run that sets a map aside is inside every run whose
map is still set aside, so the maps form a stack with the innermost
run on top.
*/
class Join_tab_walk
{
JOIN *join;
JOIN_TAB *cur;
/* The placeholders of the runs that set a map aside, outermost first. */
JOIN_TAB *placeholder[MAX_TABLES];
table_map set_aside_map[MAX_TABLES];
uint depth;
public:
explicit Join_tab_walk(JOIN *join_arg);
JOIN_TAB *tab() const { return cur; }
void next();
JOIN_TAB *outermost_run(Join_tab_range_type kind) const;
void set_aside(JOIN_TAB *run, table_map map);
/*
True when the current JOIN_TAB is the placeholder of the innermost
run that set a map aside.
*/
bool ends_set_aside() const
{
return depth && placeholder[depth - 1] == cur;
}
/* The map the current JOIN_TAB's run set aside, 0 when it set none. */
table_map tables_set_aside() const
{
return ends_set_aside() ? set_aside_map[depth - 1] : 0;
}
};
#endif

128
sql/opt_subselect.cc
View File

@@ -4330,39 +4330,6 @@ void fix_semijoin_strategies_for_picked_join_order(JOIN *join)
}
/*
Return the number of tables at the top-level of the JOIN
SYNOPSIS
get_number_of_tables_at_top_level()
join The join with the picked join order
DESCRIPTION
The number of tables in the JOIN currently include all the inner tables of the
mergeable semi-joins. The function would make sure that we only count the semi-join
nest and not the inner tables of teh semi-join nest.
*/
uint get_number_of_tables_at_top_level(JOIN *join)
{
uint j= 0, tables= 0;
while(j < join->table_count)
{
POSITION *cur_pos= &join->best_positions[j];
tables++;
if (cur_pos->sj_strategy == SJ_OPT_MATERIALIZE ||
cur_pos->sj_strategy == SJ_OPT_MATERIALIZE_SCAN)
{
SJ_MATERIALIZATION_INFO *sjm= cur_pos->table->emb_sj_nest->sj_mat_info;
j= j + sjm->tables;
}
else
j++;
}
return tables;
}
/*
Setup semi-join materialization strategy for one semi-join nest
@@ -4387,7 +4354,12 @@ uint get_number_of_tables_at_top_level(JOIN *join)
bool setup_sj_materialization_part1(JOIN_TAB *sjm_tab)
{
DBUG_ASSERT(sjm_tab->is_sjm_nest());
JOIN_TAB *tab= sjm_tab->bush_children.start;
/*
The first entry of the run can stand for a run of its own, which
reads a temporary table rather than a table of the query, so the nest
is reached from the first entry that does read a table of the query.
*/
JOIN_TAB *tab= enter_bushes(sjm_tab->bush_children.start);
TABLE_LIST *emb_sj_nest= tab->table->pos_in_table_list->embedding;
SJ_MATERIALIZATION_INFO *sjm;
THD *thd;
@@ -4453,6 +4425,34 @@ bool setup_sj_materialization_part1(JOIN_TAB *sjm_tab)
DBUG_RETURN(FALSE);
}
/*
Remove the injected semi-join IN-equalities from the conditions of every
JOIN_TAB of a run and of every run below it.
A JOIN_TAB that stands for a run of its own reads a temporary table
rather than a table of the query, and the tables of that run are
semi-join inner tables as well, so their conditions can hold an injected
IN-equality too. Counting JOIN_TABs of the run gives the wrong bound
because such a JOIN_TAB stands for more than one table.
*/
static bool remove_sj_conds_in_run(THD *thd, JOIN_TAB *start, JOIN_TAB *end)
{
for (JOIN_TAB *tab= start; tab < end; tab++)
{
if (tab->has_bush_children() &&
remove_sj_conds_in_run(thd, tab->bush_children.start,
tab->bush_children.end))
return TRUE;
if (remove_sj_conds(thd, &tab->select_cond) ||
(tab->select && remove_sj_conds(thd, &tab->select->cond)))
return TRUE;
}
return FALSE;
}
/**
@retval
FALSE ok
@@ -4463,7 +4463,12 @@ bool setup_sj_materialization_part2(JOIN_TAB *sjm_tab)
{
DBUG_ENTER("setup_sj_materialization_part2");
DBUG_ASSERT(sjm_tab->is_sjm_nest());
JOIN_TAB *tab= sjm_tab->bush_children.start;
/*
The first entry of the run can stand for a run of its own, which
reads a temporary table rather than a table of the query, so the nest
is reached from the first entry that does read a table of the query.
*/
JOIN_TAB *tab= enter_bushes(sjm_tab->bush_children.start);
TABLE_LIST *emb_sj_nest= tab->table->pos_in_table_list->embedding;
/* Walk out of outer join nests until we reach the semi-join nest we're in */
while (!emb_sj_nest->sj_mat_info)
@@ -4543,12 +4548,9 @@ bool setup_sj_materialization_part2(JOIN_TAB *sjm_tab)
sj-inner tables which are not available after the materialization
has been finished.
*/
for (i= 0; i < sjm->tables; i++)
{
if (remove_sj_conds(thd, &tab[i].select_cond) ||
(tab[i].select && remove_sj_conds(thd, &tab[i].select->cond)))
DBUG_RETURN(TRUE);
}
if (remove_sj_conds_in_run(thd, sjm_tab->bush_children.start,
sjm_tab->bush_children.end))
DBUG_RETURN(TRUE);
if (!(sjm->in_equality= create_subq_in_equalities(thd, sjm,
emb_sj_nest->sj_subq_pred)))
DBUG_RETURN(TRUE); /* purecov: inspected */
@@ -5584,6 +5586,15 @@ void destroy_sj_tmp_tables(JOIN *join)
}
join->sj_tmp_tables.empty();
join->sjm_info_list.empty();
Full_join_mat_info *mat;
List_iterator<Full_join_mat_info> it2(join->full_join_mat_list);
while ((mat= it2++))
{
delete [] mat->copy_field;
mat->copy_field= NULL;
}
join->full_join_mat_list.empty();
}
@@ -5616,6 +5627,13 @@ int clear_sj_tmp_tables(JOIN *join)
{
sjm->materialized= FALSE;
}
Full_join_mat_info *mat;
List_iterator<Full_join_mat_info> it3(join->full_join_mat_list);
while ((mat= it3++))
{
mat->materialized= FALSE;
}
return 0;
}
@@ -6004,7 +6022,8 @@ enum_nested_loop_state join_tab_execution_startup(JOIN_TAB *tab)
{
/* It's a merged SJM nest */
enum_nested_loop_state rc;
SJ_MATERIALIZATION_INFO *sjm= tab->bush_children.start->emb_sj_nest->sj_mat_info;
SJ_MATERIALIZATION_INFO *sjm=
enter_bushes(tab->bush_children.start)->emb_sj_nest->sj_mat_info;
if (!sjm->materialized)
{
@@ -6026,6 +6045,31 @@ enum_nested_loop_state join_tab_execution_startup(JOIN_TAB *tab)
sjm->materialized= TRUE;
}
}
else if (tab->is_full_join_nest())
{
Full_join_mat_info *mat= tab->bush_children.mat;
if (!mat->materialized)
{
enum_nested_loop_state rc;
JOIN *join= tab->join;
JOIN_TAB *join_tab= tab->bush_children.start;
JOIN_TAB *save_return_tab= join->return_tab;
/*
Compute the nest on its own. The first call runs the join over
the nest's tables, the second signals that no rows are left, which
a join strategy that buffers rows needs in order to flush them.
*/
if ((rc= sub_select(join, join_tab, FALSE)) < 0 ||
(rc= sub_select(join, join_tab, TRUE)) < 0)
{
join->return_tab= save_return_tab;
DBUG_RETURN(rc);
}
join->return_tab= save_return_tab;
mat->materialized= TRUE;
}
}
DBUG_RETURN(NESTED_LOOP_OK);
}

1
sql/opt_subselect.h
View File

@@ -340,7 +340,6 @@ void fix_semijoin_strategies_for_picked_join_order(JOIN *join);
bool setup_sj_materialization_part1(JOIN_TAB *sjm_tab);
bool setup_sj_materialization_part2(JOIN_TAB *sjm_tab);
uint get_number_of_tables_at_top_level(JOIN *join);
/*

11
sql/opt_trace.cc
View File

@@ -592,13 +592,22 @@ void get_table_name_for_trace(const JOIN_TAB *tab, String *out)
}
else if (tab->is_sjm_nest())
{
JOIN_TAB *ctab= tab->bush_children.start;
JOIN_TAB *ctab= enter_bushes(tab->bush_children.start);
size_t len= my_snprintf(table_name_buffer,
sizeof(table_name_buffer)-1,
"<subquery%d>",
ctab->emb_sj_nest->sj_subq_pred->get_identifier());
out->copy(table_name_buffer, len, &my_charset_bin);
}
else if (tab->is_full_join_nest())
{
/*
Named as EXPLAIN names it. The temporary table the nest is computed
into does not exist yet when the chosen join order is traced, so
there is nothing else to read a name from.
*/
out->set(STRING_WITH_LEN("<derived>"), &my_charset_bin);
}
else
{
TABLE_LIST *real_table= tab->table->pos_in_table_list;

125
sql/sql_base.cc
View File

@@ -41,6 +41,7 @@
#include "sql_partition.h" // ALTER_PARTITION_PARAM_TYPE
#include "log_event.h" // Query_log_event
#include "sql_select.h"
#include "opt_full_join.h"
#include "sp_head.h"
#include "sp.h"
#include "sp_cache.h"
@@ -7872,130 +7873,6 @@ err:
}
static inline bool is_coalesce_item(Item *item)
{
return item->type() == Item::FUNC_ITEM &&
((Item_func*) item)->functype() == Item_func::COALESCE_FUNC;
}
/*
Append 'item' to 'list', flattening it when it is itself a COALESCE so
COALESCE(a, b) contributes its arguments a, b rather than a nested
COALESCE. 'item' is only read, never mutated--it may be shared by an
already-built equi-join condition (see natural_join_eq_operand).
@return TRUE on out-of-memory, FALSE on success.
*/
static bool append_coalesce_or_item(THD *thd, List<Item> *list, Item *item)
{
if (is_coalesce_item(item))
{
Item_func *fc= (Item_func*) item;
for (Item **a= fc->arguments(), **end= a + fc->argument_count();
a < end; a++)
{
if (list->push_back(*a, thd->mem_root))
return TRUE;
}
return FALSE;
}
return list->push_back(item, thd->mem_root);
}
/*
Build COALESCE(first, second). When either operand is already a
COALESCE, return a single flattened COALESCE over all of the operands
instead of nesting one COALESCE inside another, so that a chain such as
(t1 natural full join t2) natural full join t3
yields COALESCE(t1.x, t2.x, t3.x) rather than
COALESCE(COALESCE(t1.x, t2.x), t3.x). Neither operand is mutated, since
an operand may be shared by an equi-join condition built earlier.
@return the COALESCE item, or NULL on out-of-memory.
*/
static Item_func_coalesce *coalesce_items(THD *thd, Item *first, Item *second)
{
if (!is_coalesce_item(first) && !is_coalesce_item(second))
return new (thd->mem_root) Item_func_coalesce(thd, first, second);
List<Item> list;
if (append_coalesce_or_item(thd, &list, first) ||
append_coalesce_or_item(thd, &list, second))
return NULL;
return new (thd->mem_root) Item_func_coalesce(thd, list);
}
/*
For some pair of tables (t1, t2) such that
t1 NATURAL FULL JOIN t2
generate a set of output columns
COALESCE(t1.x_1, t2.y_1), ..., COALESCE(t1.x_n, t2.y_n)
such that NULL results won't appear in the NATURAL FULL JOIN.
@param thd the current thread
@param left_tab_col common columns originating in t1
@param right_tab_col common columns originating in t2
@return TRUE on out-of-memory, FALSE on success.
*/
static int coalesce_natural_full_join(THD *thd,
List<Natural_join_column> *left_tab_col,
List<Natural_join_column> *right_tab_col)
{
/*
It's a NATURAL JOIN so the number of columns from the left table better
match the number from the right table.
*/
DBUG_ASSERT(left_tab_col->elements == right_tab_col->elements);
/*
Walk the left table and right table columns in lock-step, creating a
new COALESCE() over each pair of columns. The calling function relies
on the state of left_join_columns, so set the COALESCE() item instance
on members of that list.
*/
List_iterator<Natural_join_column> left(*left_tab_col);
List_iterator<Natural_join_column> right(*right_tab_col);
Natural_join_column *left_col= nullptr;
Natural_join_column *right_col= nullptr;
while ((left_col= left++) && (right_col= right++))
{
/*
When an operand is itself a NATURAL FULL JOIN its common column is the
COALESCE built for that join. coalesce_items() flattens such operands
so a chain such as
(t1 natural full join t2) natural full join t3
yields
COALESCE(t1.x, t2.x, t3.x)
rather than a nested COALESCE(COALESCE(t1.x, t2.x), t3.x).
TODO (future improvement):
Create a new function 'coalesce_items()'. If first item is of
type Item_func_coalesce() then just add the item to the arg list
otherwise call Item_func_coalesce(thd, a, b);
*/
Item *left_field= left_col->get_item();
Item *right_field= right_col->get_item();
Item_func_coalesce *coal= coalesce_items(thd, left_field, right_field);
if (!coal)
return TRUE; // out of memory
// Makes the field `COALESCE(left, right) AS left`.
coal->set_name(thd, left_field->name);
// Save the result into the set of left_join_columns.
left_col->natural_full_join_field= coal;
}
return FALSE;
}
/*
Materialize and store the row type of NATURAL/USING join.

13
sql/sql_explain.cc
View File

@@ -1119,6 +1119,19 @@ int Explain_basic_join::print_explain(Explain_query *query,
FALSE /*using filesort*/))
return 1;
}
/*
A run held by one of these tables is reported after them, the way a
run held by a table of the join's own plan is reported after that
plan. A run can hold a further run, so this reaches every depth.
*/
for (uint i=0; i< n_join_tabs; i++)
{
Explain_basic_join *nest= join_tabs[i]->sjm_nest;
if (nest && nest->print_explain(query, output, explain_flags, is_analyze))
return 1;
}
return 0;
}

2378
sql/sql_select.cc
View File

File diff suppressed because it is too large Load Diff

122
sql/sql_select.h
View File

@@ -469,6 +469,19 @@ typedef enum_nested_loop_state
Next_select_func setup_end_select_func(JOIN *join);
int rr_sequential(READ_RECORD *info);
int read_record_func_for_rr_and_unpack(READ_RECORD *info);
TABLE_LIST *embedding_materialized_nest(Field *field);
JOIN_TAB *first_depth_first_tab(JOIN* join);
JOIN_TAB *next_depth_first_tab(JOIN* join, JOIN_TAB* tab);
void add_cond_and_fix(THD *thd, Item **e1, Item *e2);
COND *add_found_match_trig_cond(THD *thd, JOIN_TAB *tab, COND *cond,
JOIN_TAB *root_tab);
COND *simplify_nested_join(JOIN *join, TABLE_LIST *table,
COND *conds, bool in_sj,
TABLE_LIST *fj_operand,
bool fj_left_operand,
table_map *used_tables,
table_map *not_null_tables);
bool join_transform_enabled(THD *thd, ulonglong switch_flag);
Item *remove_pushed_top_conjuncts(THD *thd, Item *cond);
Item *and_new_conditions_to_optimized_cond(THD *thd, Item *cond,
COND_EQUAL **cond_eq,
@@ -487,7 +500,7 @@ class AGGR_OP;
class Filesort;
struct SplM_plan_info;
class SplM_opt_info;
class full_join_duplicate_filter;
class Full_join_info;
/*
What a contiguous run of JOIN_TABs holds. A join's plan is one run of
@@ -538,22 +551,53 @@ enum Join_tab_range_type
};
/*
What a FULL JOIN operand nest needs in order to have its rows computed
into a temporary table and read back as one table of the enclosing
join.
*/
class Full_join_mat_info: public Sql_alloc
{
public:
TMP_TABLE_PARAM table_param;
/* One Item_field per column of the nest's tables that is read. */
List<Item> table_cols;
TABLE *table;
/*
Copies each column of a row read back from the table into the field
of the nest table it came from, one per entry of table_cols.
*/
Copy_field *copy_field;
/* True once the nest's rows have been written for this execution. */
bool materialized;
};
class JOIN_TAB_RANGE: public Sql_alloc
{
public:
JOIN_TAB *start;
JOIN_TAB *end;
Join_tab_range_type kind;
/*
The join nest whose tables the run holds, for a run of kind
JOIN_TAB_RANGE_FULL_JOIN. A materialized semi join run reaches its
nest through the semi join predicate carried by its tables instead,
and leaves this NULL.
*/
TABLE_LIST *nest;
/* The temporary table the nest's rows go into, and how to fill it. */
Full_join_mat_info *mat;
};
typedef struct st_join_table {
/*
Non-NULL only on the right side of a FULL JOIN. Tracks right-side
rowids matched during the LEFT JOIN pass so the null-complement
rescan can skip them.
The FULL JOIN this tab is the rescan_tab of, set for one execution by
alloc_full_join_duplicate_filters. NULL on every other tab.
*/
full_join_duplicate_filter *fj_dups;
Full_join_info *full_join;
/*
True when sending null-complemented rows to the result sink when
@@ -562,30 +606,17 @@ typedef struct st_join_table {
bool writing_null_complements{false};
/*
Linked list of FULL JOIN right side JOIN_TABs whose left-side
null-complement generation ends at this tab. At the end of
sub_select(this, 0), each target's null-complement rescan runs so
the rescan participates in the enclosing nested loop and
cross-products with any outer scope tables. This will be NULL on
tabs that are not the left-most JOIN_TAB of any FULL JOIN's left
side. Ordered inside-out for chained FULL JOINs so that an inner
FJ's rescan runs before an outer FJ's rescan and can update the
outer fj_dups filter through the normal forward chain.
To visit this linked list, the first element is on fj_first_target
and subsequent elements are on fj_next_target (see function
alloc_full_join_duplicate_filters).
The FULL JOINs whose null complement pass runs at the end of
sub_select(this, 0), chained through Full_join_info::next_pass. This
tab is the left-most tab of each one's left side, or the FULL JOIN's
own rescan_tab when no left-most tab is found. Running the pass there
makes it take part in the enclosing nested loop and pair its rows with
the current rows of any outer scope tables. Ordered inside out for
chained FULL JOINs, so that an inner FULL JOIN's pass runs before an
outer one's and can update the outer duplicate filter through the
normal forward chain.
*/
struct st_join_table *fj_first_target;
/*
Next pointer for the fj_first_target list. Not NULL only on FULL
JOIN right side JOIN_TABs that share a left-most JOIN_TAB with
another FULL JOIN right side JOIN_TAB (like in the case of chained
FULL JOINs). Put another way, one FULL JOIN's left-most JOIN_TAB
may be another's right JOIN_TAB.
*/
struct st_join_table *fj_next_target;
Full_join_info *fj_passes;
TABLE *table; /**< pointer to table cursor */
TABLE_LIST *tab_list; /**< pointer to query table, e.g. `t1` */
@@ -1075,6 +1106,17 @@ typedef struct st_join_table {
bool is_full_join_nest() const
{ return bush_children.kind == JOIN_TAB_RANGE_FULL_JOIN; }
/*
The tables of the nest whose rows a FULL JOIN run produces. The run
stands for all of them wherever the join order asks which tables an
entry covers.
*/
table_map full_join_nest_map() const
{
DBUG_ASSERT(is_full_join_nest());
return bush_children.nest->nested_join->used_tables;
}
/*
If this join_tab reads a non-merged semi-join (also called jtbm), return
the select's number. Otherwise, return 0.
@@ -1770,6 +1812,22 @@ public:
*/
table_map full_join_nest_tables;
/*
Sets of tables that must each occupy an unbroken span of the join
order. All of full_join_nest_tables is one such set, which is what
keeps a FULL JOIN's two operands beside each other, and every join
nest that is an operand of a FULL JOIN is another, which is what
keeps such a nest from being interleaved with the tables of the
operand it is joined to. The sets come from the join tree, so any
two of them are either disjoint or one contains the other, and a
join order satisfying all of them therefore exists.
*/
table_map *full_join_groups;
uint full_join_group_count;
/* The FULL JOINs that simplify_joins did not rewrite. */
List<Full_join_info> full_joins;
ha_rows send_records,found_records, accepted_rows;
/*
@@ -2077,6 +2135,12 @@ public:
List<TABLE> sj_tmp_tables;
/* SJM nests that are executed with SJ-Materialization strategy */
List<SJ_MATERIALIZATION_INFO> sjm_info_list;
/*
The FULL JOIN operand nests whose rows are computed into a temporary
table. The tables themselves go on sj_tmp_tables, which is where the
executor frees the temporary tables it made for the join's internals.
*/
List<Full_join_mat_info> full_join_mat_list;
/** Exec time only: TRUE <=> current group has been sent */
bool group_sent;
@@ -2301,7 +2365,7 @@ public:
bool transform_in_predicates_into_in_subq(THD *thd);
bool optimize_upper_rownum_func();
void calc_allowed_top_level_tables(SELECT_LEX *lex);
bool calc_allowed_top_level_tables(SELECT_LEX *lex);
table_map get_allowed_nj_tables(uint idx);
bool propagate_dependencies(JOIN_TAB *stat);
void update_key_dependencies();

16
sql/sql_view.cc
View File

@@ -41,8 +41,6 @@
const LEX_CSTRING view_type= { STRING_WITH_LEN("VIEW") };
extern bool check_full_join_base_tables(List<TABLE_LIST> *);
static int mysql_register_view(THD *thd, DDL_LOG_STATE *ddl_log_state,
TABLE_LIST *view, enum_view_create_mode mode,
char *backup_file_name);
@@ -573,20 +571,6 @@ bool mysql_create_view(THD *thd, TABLE_LIST *views,
goto err;
}
/*
Reject VIEW definitions that put a FULL JOIN on the right side of a
LEFT or RIGHT JOIN. CREATE VIEW only prepares the body (doesn't optimize),
so the same check that runs in JOIN::optimize_inner doesn't run here.
*/
for (sl= select_lex; sl; sl= sl->next_select())
{
if (check_full_join_base_tables(&sl->top_join_list))
{
res= TRUE;
goto err;
}
}
/* view list (list of view fields names) */
if (lex->view_list.elements)
{

38
sql/table.cc
View File

@@ -7282,44 +7282,6 @@ TABLE *TABLE_LIST::get_real_join_table()
}
/*
Return true when this view/derived table contains a FULL JOIN.
*/
static bool join_list_contains_full_join(List<TABLE_LIST> *join_list)
{
List_iterator<TABLE_LIST> it(*join_list);
TABLE_LIST *tbl;
while ((tbl= it++))
{
if (tbl->outer_join & JOIN_TYPE_FULL)
return true;
if (tbl->nested_join &&
join_list_contains_full_join(&tbl->nested_join->join_list))
return true;
}
return false;
}
bool TABLE_LIST::contains_full_join() const
{
List<TABLE_LIST> *join_list= nullptr;
if (view)
join_list= &view->first_select_lex()->top_join_list;
else if (derived)
join_list= &derived->first_select()->top_join_list;
else if (nested_join)
join_list= &nested_join->join_list;
else
return false;
return join_list_contains_full_join(join_list);
}
Natural_join_column::Natural_join_column(Field_translator *field_param,
TABLE_LIST *tab)
{

35
sql/table.h
View File

@@ -3280,6 +3280,7 @@ struct TABLE_LIST
return view != NULL ? view_name : table_name;
}
bool is_active_sjm();
bool is_materialized_full_join();
bool is_sjm_scan_table();
bool is_jtbm() { return MY_TEST(jtbm_subselect != NULL); }
bool is_pure_alias() const;
@@ -3329,15 +3330,6 @@ struct TABLE_LIST
SELECT ... FROM *this FULL OUTER JOIN foj_partner ...
*/
TABLE_LIST *foj_partner{nullptr};
/*
For the right side of a surviving FULL JOIN, the WHERE conjuncts that
reference the left side, lifted out of the WHERE before access
selection so they neither prune nor drive access on the left side.
make_join_select reattaches them here under the found-match guard.
Reset and recomputed each optimization. NULL otherwise.
*/
COND *fj_left_cond{nullptr};
private:
bool prep_check_option(THD *thd, uint8 check_opt_type);
bool prep_where(THD *thd, Item **conds, bool no_where_clause);
@@ -3527,7 +3519,30 @@ typedef struct st_nested_join
it. Only a NESTED_JOIN_OPERAND nest is ever rebalanced.
*/
bool rebalanced;
/*
/*
True when this nest is an operand of a FULL JOIN whose tables the
chosen join order gathers into a run computed into a temporary table.
Decided again for every optimization, since it follows from the join
order and not from the query.
*/
bool materialized_full_join;
/*
The placeholder JOIN_TAB that stands for this nest's run when
materialized_full_join is set. NULL otherwise.
*/
struct st_join_table *materialized_full_join_tab;
/*
The ON expressions of the inner joins inside this nest, which is only
collected when the nest is an operand of a FULL JOIN. Simplifying the
join tree moves such an expression into the enclosing FULL JOIN's ON,
where it is checked once that join has positioned the whole nest. A
nest computed into a temporary table is computed before that, so it
needs its own join condition here to not produce the cross product of
its tables. Collected again for every optimization, since the
expressions are restored from prep_on_expr each time.
*/
Item *fj_inner_cond;
/*
Bitmap of tables within this nested join (including those embedded within
its children), including tables removed by table elimination.
*/