SELECT COUNT(*), SUM(b) FROM t ran serially. The workers could pre-aggregate a
GROUP BY, but aggregation over the whole table was refused, because the shape
the grouped path merges through is not there: with no group the server
materialises nothing, and aggregates in end_send_group() straight off the last
join tab -- no aggregation table, no group key, nothing for a partial to be
merged into.
It turns out neither end needs new machinery, and this commit is mostly the
finding of that.
The manager already merges these. end_send_group() reaches Item_sum::add(), and
add() honours a value left by direct_add() for every aggregate the gate accepts
-- COUNT adds a count instead of incrementing, AVG takes a partial sum together
with the number of rows behind it. So the drain primes the aggregates and hands
the row to the plan's own terminal here exactly as it does through end_update()
for a GROUP BY, and the drain needed no case of its own.
The workers accumulate into the grouped machinery with a single group in it.
The layout gives this shape a group of one constant column, and from there it
is the grouped path: a worker keeps one partial in a keyed container, merges
into it through end_update(), and ships one row. Beyond costing a column per
shipped row -- of which there is one per worker -- that is also what keeps
Item_sum_avg::create_tmp_field() packing the count beside the sum, which it does
only for a container that has a group. An average built any other way would have
arrived without its count.
So what is actually new is the gate. The aggregate tests -- the five that can be
merged, no DISTINCT, one argument, an argument a worker can evaluate -- are the
same for both shapes and move out to pwt_aggregates_can_be_merged(). What is
left is the shape question, and which shape to ask about is the query's own
grouping rather than the plan's: a query that names a GROUP BY stays the grouped
question whatever the optimizer then did with it, so a plan with no group key to
merge through is a refusal of that query and not an invitation to read it as
aggregating over everything, which would answer something else. Only a query
that never grouped reaches the second question.
Refused there: a select-list item reading a column outside an aggregate. Its
value is whichever row the terminal happened to hold, and a worker holds a
different one.
An empty result needed nothing. No rows means no groups means no partials
shipped, and end_send_group() still sends its one row at end of records:
COUNT(*) is 0, SUM is NULL, one row and not one per worker. The new test checks
each of those rather than assuming them.
main.parallel_query_aggregate_whole is that test: serial against parallel for
COUNT, COUNT(col), SUM, MIN, MAX and AVG over integer, decimal and double
columns; the empty case; that exactly one row is sent; and that a DISTINCT
aggregate, STD, GROUP_CONCAT and a bare column outside an aggregate are still
refused and run serially. main.parallel_query_aggregate gains the fall-back
case, where the container turns out to have no group index: the constant group
reaches the same fall-back, ships plain rows, and the manager aggregates them a
row at a time.
Known and accepted: adding doubles is not associative, so SUM and AVG over DOUBLE
or FLOAT can differ from the serial answer, and between runs, in the last
significant bit -- which worker summed which chunk is scheduling. Decimal and
integer aggregates are exact and unaffected. The test rounds well clear of the
last bit rather than leaving floats uncovered.
Under a forced-worker sweep of the main suite (--parallel-worker-threads=4)
three more tests differ than before, all of them EXPLAIN text where a query of
this shape now says ALL_parallel or range_parallel: main.costs,
main.func_group_innodb and main.innodb_full_scan. No value changes in any of
them.
This commit was prepared with Claude Code (Opus 5), which established that
Item_sum::add() already honours a directly added partial -- the fact the whole
change rests on -- and found that a container without a group would have
dropped an average's count.
A worker that pre-aggregates kept two temporary tables of the same shape: the
grouping table its aggregates accumulated into, and the shipping container the
manager reads. At end of records flush_groups() walked the first and wrote every
row into the second -- a memcpy and a keyed insert per group -- so that the
manager could read rows it already had, in a table it could already have been
given. The tables hold the same rows because a worker produces one row per group
and ships one row per group; there was never anything for the second table to do
but hold a copy.
So there is one table now. setup_worker_preagg() builds the shipping container
itself, from the worker's own item list, and the aggregates accumulate straight
into it; flush_groups() is gone, and so is the per-group copy and the second
table's memory. What the manager reads is the table end_update() wrote.
The batch transport is disabled, which is what makes that possible to say
plainly. It carries record images through a buffer rather than handing a table
over, so everything that goes through it is copied twice, and a worker with a
finished table still has to stream it. pwt_create_transport() now builds the
temporary-table transport unconditionally and the debug flag that selected the
other one is gone. The classes are kept, compiled out, in case the streaming
shape is wanted again.
Two things the copy had been hiding had to be fixed before the table could be
handed over.
A rebuild on disk has to be the cross-thread kind. A container is instantiated
cross-thread, because the manager reads and frees what the worker wrote, and
the transport's own spill path passes that through. But a pre-aggregating
worker's table fills while end_update() is writing it, and end_update() rebuilt
it with cross_thread=false: Aria then ties the new handle's allocations to the
worker's thread, and the manager frees them after that thread is gone. The
decision now travels on TMP_TABLE_PARAM, set where the container is created and
read where the rebuild happens. Without it the new test fails with the
allocation crossing threads, named:
16416 bytes freed by T@3, allocated by T@71 at maria/ma_bitmap.c:242, ...
sql/sql_select.cc:23926, sql/sql_select.cc:26598
And the memory that rebuild allocates has to change hands. The transport
already moves the charge from the worker's THD to the manager's for a spill it
performs itself, but a spill inside end_update() never passes through the
transport. pwt_end_send() now brackets put_record() and hands anything the call
left outstanding to the sink: working memory a call allocates and frees nets to
nothing there, and the rebuild is what does not.
main.parallel_query_spill covers the new path: twenty thousand groups against a
64K heap, asserting the query ran in the workers, that a row per group arrived,
that the container really was rebuilt on disk, and that the answer matches the
serial one group for group. main.parallel_query_transport loses the halves that
exercised the batch transport, and with them its dependency on a debug build.
Measured against this tree with the same forced-worker sweep of the main suite
(--parallel-worker-threads=4): the same fifty tests fail before and after, the
same fifty by name.
This commit was prepared with Claude Code (Opus 5), which found that the two
tables held the same rows, and identified both the non-cross-thread rebuild and
the unaccounted memory before either could reach a test.
The we found a bug, we need to close the container's index before handing it
to the manager
A pre-aggregating worker leaves its container's index open: AGGR_OP::put_record()
opens it on its first call and nothing since has closed it. When the copy into a
separate shipping container went away, closing it moved to the worker's tidy-up
at the end of execute_and_handoff() -- which is after sink->flush(), and flush()
is what tells the transport this producer is finished.
The manager may claim the container the moment it is marked done. So a manager
that got there before the worker reached its tidy-up called ha_rnd_init() on a
table whose index was still open, and that asserts:
mariadbd: sql/handler.h:3770: int handler::ha_rnd_init(bool):
Assertion `inited==NONE || (inited==RND && scan)' failed.
handler::ha_rnd_init
pwt_tmp_table_source::next_row
pwt_manager::drain_and_send
Query: SELECT col1, COUNT(col2) FROM t1 GROUP BY col1 ORDER BY NULL
Which thread wins is scheduling, so this showed up as main.parallel_query_aggregate
failing about one run in eight with the whole parallel set running at
--parallel=18, and passing every time on its own.
Closing the index is moved to before the flush, where the hand-off is. The
tidy-up still closes it, for the paths that reach the end without handing
anything over; ha_index_or_rnd_end() on a handler that is already closed does
nothing.
pwt_tmp_table_sink::flush() now asserts what the ordering is for: that nothing
this producer had open on the table is still open when the container becomes the
consumer's to read. That turns a race the manager notices only when it wins into
a condition the producer checks every time -- with the close put back after the
flush, main.parallel_query_aggregate fails on the assertion on every run rather
than on one in eight.
This commit was prepared with Claude Code (Opus 5). The crash was found by
running the parallel test set repeatedly under load rather than once; the
producer-side assertion was added so the next mistake of this shape does not
need luck to be seen.
ANALYZE FORMAT=JSON says how many rows each worker read, which answers how the
chunks divided the work but not why a worker finished when it did. Two figures
the engine already keeps answer that, once they are kept per worker rather than
summed: how long each worker was busy inside the engine, and how much of that
was waiting for pages to come off disk.
"r_rows_per_worker": [181967, 93080, 159138, 165815],
"r_engine_time_per_worker_ms": [69.577, 74.895, 72.507, 69.902],
"r_pages_read_time_per_worker_ms": [0.276, 0.229, 0.285, 0.283],
"r_chunks": 18,
"r_chunks_resplit": 1,
The pair is what makes a straggler readable. Rows without time is a worker that
was handed more to do; time without rows is one that was blocked. Above, four
workers within seven per cent of each other on time despite a two-to-one spread
in rows says the spread is the pull queue's doing, not the disk's. The read
times sum to the pages_read_time_ms already reported in r_engine_stats, so the
array is a decomposition of that total rather than a second measurement of it.
Reading is the wait worth attributing this way: a parallel scan takes no row
locks, because the gate declines the scan when the read is a locking one, so
there is no lock wait to divide. The figures are undercounts wherever the
pre-fetcher did its job -- a page somebody else fetched costs its reader
nothing here -- so a low read time is not proof that nobody waited.
Two things had to be fixed before the numbers existed rather than merely being
hidden.
The page read time was being collected per worker all along and thrown away at
the merge: each worker resets its handler stats when it opens its tables and
snapshots them into tab_hstats before closing them, and quiesce_workers() added
those into the manager's handler without keeping the split. It now keeps it.
Engine time was always zero for a worker, for two reasons. It is folded into
the handler stats from the handler's Exec_time_tracker by close_thread_table(),
which a worker's tables never go through, so snapshot_table_stats() now does
that fold itself. Under that, the worker's handlers had no tracker at all to
fold: set_time_tracker() is called by the optimizer on the plan's tables, and a
worker's copies are opened after the plan exists, so nothing was timing them.
Each worker now owns one Exec_time_tracker per table, installed as the table is
opened.
Both arrays are omitted when the engine recorded no time, so a scan that never
touched the disk does not carry an array of zeroes; this is why the read-time
array appears in some plans and not others. Timings and a per-worker split are
both non-deterministic, so analyze-format.inc masks them, and the three .result
files that show a parallel scan are re-recorded.
This commit was prepared with Claude Code, which found that the read time was
already being collected and discarded, and traced the always-zero engine time
to the missing tracker on the worker's handlers.
Added, r_peak_to_average_ratio for worker engine time.
A benchmark of "select grp, sum(i) from perf group by grp" over a million
narrow rows showed the pages the engine touched jumping fivefold as the worker
count crossed a threshold, while the time barely moved: one more worker turned
19 chunks into 10127.
Exec_ctx::split() was the cause. It is reached when a worker runs dry and the
chunk it picked up was flagged to be divided, and it divides by asking
partition() for split_level 1 -- every child of the sub-tree's root. One level
down from a chunk that spans a whole sub-tree is one range per page below it,
so a chunk became hundreds or thousands of chunks where a handful would have
done. Each of those then costs a descent from the root when a worker takes it,
which is what the page counter was showing.
partition() is left alone; only the number of boundaries used changes. The
ranges it returns are contiguous intervals that share endpoints, so a run of
them merges into one by taking the first's start and the last's end, and every
m'th boundary is kept for a target of about one piece per worker.
The same benchmark, before and after, at the worker counts around the step:
workers chunks resplit pages chunks resplit pages
13 1403 2 20988 41 2 15540
14 282 1 16508 29 1 15496
15 15415 15 76984 240 15 16284
The re-split counts are unchanged, which is the point: the same chunks are
flagged and the same decisions are taken, only each division is now the size it
was meant to be. What was a fivefold rise in pages accessed is five per cent.
Those counts are the other half of this commit, because none of it was visible
before. ANALYZE FORMAT=JSON now reports, beside r_rows_per_worker on the table
that was divided:
"r_rows_per_worker": [...],
"r_chunks": 41,
"r_chunks_resplit": 2,
r_rows_per_worker on its own cannot answer this. It is the cut and the order
the pull queue handed chunks out, together, so an uneven row split says nothing
about whether the index was cut badly or the chunks merely landed unevenly.
r_chunks is the cut by itself: every chunk the engine produced, the ones a
re-split made included, and how many were divided rather than scanned.
The counts come from the coordinator through a new handler method,
parallel_get_chunk_stats(), which defaults to reporting nothing so that an
engine that does not divide scans adds no output. They are read once the
workers have been joined, from the handler parallel_init_coordinator() ran on
-- the manager's, not a worker's copy -- and only for the driving table, which
is the only one divided.
Both are masked by include/analyze-format.inc: a chunk is only re-split when a
worker runs dry and asks, so how often that happens, and therefore how many
chunks exist at the end, depends on which worker finished first.
What this does not address: create_contexts() still flips from flagging the
tail of the chunk list to flagging all of it the moment the chunk count stops
exceeding the worker count, which is the step visible above between 14 workers
and 15. It is a discontinuity where a slope belongs, and it is now cheap rather
than expensive, but it is still there.
This commit was prepared with Claude Code: the re-split was identified from the
chunk counts once they existed, and the before-and-after table above is its
measurement of the same test with the coalescing removed and restored.
(Slightly) tidied up by Rex.
Then...
Three fixes from review of the coalescing commit, none a wrong answer, each a
number coming out other than intended.
Round the merge stride down, not up. m was ceil(size/target), which makes
ceil(size/m) groups, and for a size just over the target that collapses to half
of it: 16 boundaries at target 15 merged into 8 pieces. Floor overshoots
instead, bounded by twice the target -- and too many small pieces costs less
than too few large ones, since the whole reason a chunk is being split is that
a worker had nothing to do.
Aim for twice the worker count, not once. The pieces a re-split produces are
never re-split again -- create_context() flags none of them -- so if the rows
inside the chunk are themselves skewed, having more pieces than workers is the
only slack left for evening that out. Twice keeps the piece count, and with it
the per-piece descent from the root, small, while leaving a second helping for
whoever finishes early.
The benchmark from the coalescing commit, re-measured across the same worker
counts:
workers chunks resplit pages
13 69 2 15652
14 45 1 15560
15 484 15 17260
16 511 15 17368
The 15-worker case reconciles exactly -- about 1027 leaf ranges per flagged
chunk, target 30, stride 34, so ~31 pieces each; 15 x 31 plus the 15 initial
chunks is the 484 -- and the headroom costs six per cent more page accesses
than the single-worker-count target did, against the fivefold the shattering
cost before coalescing existed. The step between 14 and 15 workers, where
create_contexts() goes from flagging the tail of the chunk list to flagging all
of it, is now a slope a reader can see the shape of rather than a cliff.
Make every statistic the parallel scan reports per-execution. quiesce_workers()
accumulated r_rows and its neighbours across executions of the same plan -- a
correlated subquery, a routine loop -- the way the serial path does, while
r_rows_per_worker and the chunk counts were overwritten each time, so ANALYZE
for a re-executed plan mixed totals with last-execution figures in one block.
The split and the chunk counts only describe one execution, and totals beside
them answer a different question about a different number of rows, so the
tracker now starts from zero each time the workers are reaped: everything
ANALYZE prints for a parallel scan describes the same, latest, run. This is a
deliberate departure from the serial convention, and r_scans is reset with the
rest so the figures stay consistent with each other.
This commit was prepared with Claude Code, from findings of its own review of
the coalescing commit: the half-target collapse and the mixed accumulate-and-
overwrite semantics were found by re-deriving what the arithmetic does at the
boundaries rather than re-running it.
TPC-H Q1 now runs in the workers. Two things were stopping it, and both are
here.
AVG
---
An average does not compose out of averages, so what a worker has for a group
is two numbers: the sum of its rows and how many there were. Nothing had to be
invented to carry them. Item_sum_avg::create_tmp_field() already packs a sum
and a count into one field, because that is how the serial grouped plan
accumulates an average, and a worker's grouped container was already
maintaining both.
What was missing was the merge. The manager primes its own aggregate with a
worker's partial through direct_add(), and Item_sum_sum::direct_add() carries
only the sum -- Item_sum_avg::count was left untouched, so every partial
counted as one row. direct_add() now takes the count alongside the sum, and
reset_field()/update_field() add it to the field's count instead of the 1 an
ordinary row contributes, in the same shape Item_sum_sum's already had.
Nothing changes for a serial plan: direct_added is false there, and the new
branches are not taken.
An aggregate inside an expression
---------------------------------
SUM(x)+1 was refused, and so was SUM(x)/COUNT(x) -- the obvious way to write an
average by hand. A select list item that is not itself an aggregate is
evaluated once per group from whatever row the terminal is holding, so a field
in it that GROUP BY does not name takes an arbitrary value; that is worth
refusing. But the check walked into the aggregates as well, found x under
SUM(x), and refused a query that was never in doubt.
It now steps over Item_sum subtrees, using with_sum_func() to answer an
aggregate-free subtree wholesale with the walk that was there before, and
decomposing only the node kinds that can hold an aggregate. Anything else
carrying one is refused rather than guessed at: being wrong in the permissive
direction here is wrong results. SUM(x)+x is still correctly refused, because
the bare x is walked and is not a group column.
The shipped row's shape
-----------------------
With more than one AVG the second onward came back wrong, which was not
arithmetic. Item_sum_avg is the one aggregate whose create_tmp_field() differs
between a keyed and an unkeyed temporary table -- it packs the count beside the
sum only for the keyed form -- and the two ends of the transport were built
differently:
recv, group_container make_container(..., group) 16-byte {sum, count}
exec.result (shipped) make_container(...) plain 8-byte double
flush_groups() copies a group container record into the shipping container by
reclength, so it wrote past a narrower record and the manager read back a wider
one, drifting by one AVG's worth of bytes at a time. For SUM, COUNT, MIN and
MAX the two widths are identical, which is why this sat there unnoticed. The
shipping container is now built with the same key; a worker ships one row per
group, so there is nothing for the key to collapse.
main.parallel_query_aggregate gains an AVG section -- serial against parallel
over a DECIMAL average, and three AVGs at once, which is the case that exposed
the shape bug -- and AVG leaves its "what is not accepted" list. STDDEV,
BIT_XOR and COUNT(DISTINCT) stay there.
This commit was prepared with Claude Code: it bisected TPC-H Q1 down to the two
refusals, found the container shape mismatch from the pattern that only the
last AVG was wrong, and fixed a null Item passed to VDec that the aggregate
test caught as a crash.
Not for the shipping tree. Every hunk is bracketed START PROTOTYPE / END
PROTOTYPE so it can be lifted out again in one pass.
Behind debug_dbug='+d,pwt_scan_only' the workers do nothing but read their
chunk of the driving table and ship the rows. There are no worker JOIN_TABs, no
cloned conditions or select list, and no join in a worker. This thread runs the
plan it would have run serially, and the only difference is that the driving
table's JOIN_TAB reads its rows from the transport rather than from the
handler: do_select() stops diverting, start_scan_only() displaces
read_first_record, and the executor above it is untouched.
Why bother. The gate admits a few per cent of the queries in the test suite,
because almost every refusal it makes is about evaluating cloned Items in
another thread rather than about dividing a scan. Take the evaluation away and
nearly all of them go, so the chunked scan and the transport -- the two layers
underneath -- can be run over the whole suite instead of over that sliver.
How to run it. The scan-only flag on its own does nothing; the workers still
have to be asked for:
./mtr --parallel=16 --force --max-test-fail=0 --ignore-parallel-diff \
--mysqld=--parallel-worker-threads=4 \
--mysqld=--debug-dbug=+d,pwt_scan_only \
--suite=main
--ignore-parallel-diff is what makes the run readable: without it a third of
the failures are nothing but the _parallel suffix EXPLAIN adds to the access
type. It does not cover FORMAT=JSON, where the value is quoted, so a handful of
json tests still differ on access_type alone. Never combine it with --record.
For one test, or from inside a test, the flag can be set on a live server:
SET @sd=@@global.debug_dbug;
SET GLOBAL debug_dbug='+d,pwt_scan_only';
...
SET GLOBAL debug_dbug=@sd;
Reading the results. main/parallel_query_* fail by construction -- they measure
that the workers ran the join, and here they do not -- as do the environmental
five this tree always fails. What is worth reading is anything with a row
difference. As it stands the whole main suite gives 1358 passes and 39
failures, of which none is a wrong answer: 10 are trace, counter or JSON
EXPLAIN content, 7 are the parallel_query tests, 5 environmental, 2 EXPLAIN
only, 2 an unordered select whose rows arrive in another order, and one an
unordered LIMIT 2 that picks a different pair. No crashes.
What it cannot tell you, which is the more important half. Everything a worker
does with an Item is gone, and that is where the defects have actually been: a
worker THD that did not carry the session's time zone, a condition left only in
a Filesort, a reader that wanted a SQL_SELECT the worker copy did not have, a
materialized subquery re-opened per worker. None of those are reachable here;
most cannot even exist in this shape. Passing in this mode says nothing about
the path that ships. Its value is as a lower layer's test, and as a bisection
tool: a query that answers wrongly in both modes is wrong in the scan or the
transport, and one that answers wrongly only in the full mode is wrong in the
worker join.
Two things the mode needs that the full path gets for free. The displaced
reader is put back by finalize_parallel_workers(), because a JOIN_TAB outlives
one execution -- a prepared statement, a correlated subquery, a routine loop
all run the same plan again -- while the manager does not. And a tab whose
condition was partly pushed into the index is given the whole condition back
for the length of the scan: the pushed half lives in the manager handler's
pushed_idx_cond, which is no longer the handler producing the rows, so it would
otherwise be applied nowhere.
This commit was prepared with Claude Code: it wrote the mode, and found three
defects in it by running the suite -- a flush skipped because the scan loop
ends on HA_ERR_END_OF_FILE, the reader left installed across a re-execution,
and the index-condition pushdown above, which is the same trap the full path
hit from the opposite direction.
A plain ORDER BY was refused, so most sorted queries could not run in the
workers at all. The refusal was right at the time: make_aggr_tables_info()
sorts the driving table's own read for a plan that needs no temporary table,
and the workers take that read over, so the sort lost the scan it was attached
to. This shape has no aggregation table to move it to either -- the terminal
after the driving tab sends straight to the client.
The manager does it instead, over the rows it drains. A sort is the one
post-join step indifferent to the order its input arrives in, which is what
lets a plan that ends in one be handed out in chunks.
It sorts a container of the transport's own layout, not the base table, so the
plan's Filesort cannot be reused: its order names the manager's base-table
fields and the rows are in the container. pwt_row_layout remembers which
shipped column each ORDER element sorts on and builds the equivalent order over
a container's own fields, the same way the group key is rebuilt for the
pre-aggregation containers. The drain collects instead of sending, filesort()
runs, and each row read back takes the path a drained row would have taken:
copy_back_row() into the manager's base-table records, then out. The container
is rebuilt on disk if it fills, which needs none of the cross-thread accounting
a worker's does, this one being written, sorted and read in one thread.
pwt_manager_sort_order() decides whether a plan's sort is ours, and the gate and
the setup ask the same function. Every ORDER BY element has to name a column the
container holds -- a field of a scanned table that the query reads, and so ships.
An expression has no column there, and a sort that returns row ids, unpacks into
other fields or stops early is doing something for the plan beyond ordering; both
are left serial. A plan with an aggregation table is a different shape whose sort
AGGR_OP::end_send() already performs.
pwt_table_conds() had to change with it and the order of its three sources is
load-bearing. add_sorting_to_table() hands tab->select to the Filesort and nulls
tab->select_cond, so a sorted tab keeps its condition only in the filesort -- but
push_index_cond() ran long before and left that SQL_SELECT holding just the
remainder. pre_idx_push_select_cond therefore still wins; the filesort is asked
only when there was no pushdown. Taking them the other way round silently drops
the pushed half, which main.innodb_icp catches as rows the serial plan rejects.
main.parallel_query_sort reads named rows back with query_get_value, which takes
the Nth row as the client received it and so asserts the order rather than the
multiset, ascending and descending, and again once the container has spilled to
disk. main.parallel_query_trace used ORDER BY as its example of a declined query
shape and now uses LIMIT, which is still declined.
A capped SELECT must not run in the workers
With SET sql_select_limit=3, a query the gate accepted returned every row --
500 where the serial plan sends 3. The serial executor enforces a row cap in
end_send(), against unit->lim; the manager's drain sends every row a worker
ships and never consults it. So the gate must refuse any select whose unit
carries a cap, and it was testing the syntax instead of the cap:
limit_params.explicit_limit is only set by a LIMIT clause, while
sql_select_limit is installed by mysql_execute_command() as the default limit
of a top-level SELECT, with explicit_limit still unset.
The gate now tests join->unit->lim.is_unlimited(), which is the very value
end_send() would have enforced. This subsumes the explicit-LIMIT case, and it
is per-unit, so the session cap -- which applies only to the top-level select
-- does not cost a derived table's select its parallel scan.
Found reviewing the ORDER BY commit: its fs->limit != HA_ROWS_MAX check turned
out to be load-bearing for the same reason (the implicit cap becomes a filesort
limit), which raised the question of what protected the unsorted path. Nothing
did.
setup_worker_jointabs() builds each worker tab as a struct copy of the
manager's, and since the manager grew a sort stage the driving tab's copy has
carried the plan's Filesort pointer. Nothing reads it in a worker -- the
driving tab reads through the chunk reader, not join_init_read_record() -- and
worker tabs never run JOIN_TAB::cleanup(), so it is inert today. But
JOIN_TAB::cleanup() deletes the filesort and, through it, the SQL_SELECT
holding the plan's condition, so the copy was one future teardown call away
from a double free. The function's own policy is that manager-owned pointers
are cleared from the copy (select, cache_select, pre_idx_push_select_cond);
filesort and filesort_result now join the list, and pwt_assert_tab_inert()
pins the shape: a filesort is only ever the driving table's, and no sort has
run when workers are set up.
No test: the hazard is latent, reachable only by a teardown path that does not
exist yet. The asserts held over the whole main suite run with
parallel_worker_threads forced on.
This commit was prepared with Claude Code: it built the manager-side sort stage
on the existing container and layout machinery, and found the condition-order
defect above by running the whole main suite with parallel_worker_threads forced
on and separating the row differences from the EXPLAIN ones.
This commit was prepared with Claude Code: it probed the drain path with an
implicit sql_select_limit after noticing that only the sorted shape declined,
and confirmed the serial and parallel row counts diverged.
This commit was prepared with Claude Code, as a hardening it proposed while
reviewing the manager-side ORDER BY commit.
This required adding
virtual pwt_worker_base::on_fatal_error()
which pwt_worker overrides.
pwt_manager_base doesn't track if fatal_error has occurred
Should it? fatal_error is in sql_parallel_workers...
- Move 'reaped' and 'kill_signal' OUT to sql_parallel_workers
- Move 'workers' and 'nworkers' OUT to sql_parallel_workers
- Move server_threads update logic IN.
Separating pwt_worker_base out of pwt_worker dropped the assignment that
nulls the worker's thd on its way out. The exit path used to capture the
pointer and clear the member while holding LOCK_worker, then tear the THD
down from the local copy; after the split it kept a local for the detach but
left the member pointing at the THD it went on to destroy.
The hole survived because nothing reached abort_worker(). Its only caller is
the cleanup_old_workers path, which needs init_parallel_workers() to fail
after at least one worker thread is already running, and no test could
produce that. Add a DBUG injection that fails the last worker as if its
thread could not be created: the last one so that the maximum number are
running when the teardown aborts them, and before create_thread() rather
than after, because cleanup_thread_create closes that worker's tables and
destroys its THD, which is only safe while it has no thread of its own.
This commit was prepared with Claude Code: it found the dropped assignment
while evaluating the TODO tags added by the three preceding commits, traced
which readers depended on it, wrote the injection point and the test, and
confirmed by reverting the fix that the test reproduces the crash.
into sql_parrallel_thread.* (name tentative).
No "unit test" like test yet.
Unfinished things:
pwt_manager::notify_fatal_error() should both update a flag
in pwt_manager_base and notify pwt_manager..
pwt_manager_base::{reaped, kill_signal} are not isolated yet.
Each parallel worker now runs the join and the WHERE over its own chunk of the
driving table and ships the base-table columns of every row that qualified. The
manager copies those columns back into the fields they came from in its own
table instances, so that after a row is drained its records hold what a serial
scan would have left there, and then evaluates the select list against them and
sends the row. This replaces the model where the workers shipped raw source
records and the manager ran the join.
The transport carries columns rather than projected values because anything the
manager does with a row beyond sending it reads a record, and not all of those
reads go through Items that could be re-pointed: create_tmp_table() builds
Copy_field pairs holding raw Field pointers into the base tables.
make_join_readinfo()'s gate, can_run_query_in_workers(), chooses the worker-side
path for an inner select-project[-join] over a parallel-scannable driving table:
no tmp table, no LIMIT/SQL_CALC_FOUND_ROWS/procedure/aggregate, no outer join,
no semijoin strategy the worker's plain nested loop cannot honour, and every
non-driving table reached by eq_ref, ref or full scan. Every expression it hands
a worker has to deep-copy into something that shares no node with the original
and reads no table outside the join, or the copy would carry the manager's own
items onto a worker's tables. do_select() then runs run_worker_side_join();
anything ineligible runs serially.
Each worker opens a private copy of every non-const table, deep-clones and
field-rebinds the conditions and the shipped column list, rebuilds each ref,
scans its chunk, runs its own join and ships the columns it read for each
full match. It runs under the session's query id, so an item holding a
value for the statement recomputes it rather than handing out the one it was
cloned with.
sql_parallel_execution.cc holds all of the above; sql_parallel_workers.cc keeps
the worker threads, the batch channel and their lifetime.
Tests: parallel_query_worker_side, parallel_query_join and parallel_query_clone
compare a parallel result set against a serial one; parallel_query and
parallel_query_oom moved to a plain SELECT and were re-recorded.
.claude/commits/MDEV-40012.md records what came from the 13.0 branch verbatim,
what was adapted to this tree, and what is new here.
This commit was prepared with Claude Code: it ported the work from the 13.0
branch onto this tree's refactored structures and renamed scan API, found and
fixed the two holes this tree's deeper Item copying opens -- an expression
reaching a table the worker does not have, and a statement-lifetime item cached
against query id 0 -- split the file, and ran the suites.
PWT cleanup: move pwt_worker init into pwt_worker::init_worker_thd()
PWT cleanup: comment-out THD::parallel_worker.
Instead, add "pwt_worker *worker" member to class PWT_error_handler.
PWT cleanup: introduce/use pwt_worker::cleanup_worker()
PWT cleanup: in pwt_manager, add notify_message_dropped, fix record_event().
Both of them acquire/release needed mutexes inside the function.
PWT cleanup: rename error_to_queue -> save_error_to_queued_event
PWT Cleanup: Comment-out pwt_worker::COND_worker. It is not used.
PWT Cleanup: Let PWT_error_handler have pwt_manager as argument,
Split the code to pwt_manager::finalize_parallel_workers
PWT Cleanup: Rename LOCK_pwt_thread->LOCK_pwt_manager,
make more members private.
PWT Cleanup: privatise pwt_manager components
Cherry-picked onto a base whose parallel-scan handler API has moved on, and
adapted in three places rather than applied verbatim:
- HA_CAN_PARALLEL_SCAN is gone. table_can_be_parallel_scanned() now asks
parallel_scan_support(), which returns a bitmap; matching a particular bit
against the access method the plan chose is is_parallel_scan_applicable()'s
job, so this only asks whether the engine does it at all.
- parallel_init_coordinator() gained a keynr argument. init_parallel_workers()
derives it from the plan: the quick select's index when there are intervals
to hand over, otherwise MAX_KEY for the whole table.
- The gate stays where the preceding commit put it, at the end of
optimize_stage2(), rather than moving back into make_join_readinfo(). It has
to run last: make_join_readinfo() finishes before test_if_skip_sort_order()
has had its say, and a driving table given an ordered index scan after the
fact is not one that can be handed out in chunks. This commit's own
conditions -- can_run_query_in_workers(), worker_side_parallel, and the
range_scan trace key -- were folded into the gate in its new home, and it no
longer installs a record reader: do_select() dispatches on
worker_side_parallel and the table keeps the serial reader for the fall-back
path.
is_parallel_scan_applicable() and parallel_scan_supports_access() now both
exist and agree: only a plain range over the clustered index is handed to the
coordinator. The first is the gate's, the second is used inside
can_run_query_in_workers() for the tables joined after the driving one.
This commit was prepared with Claude Code: it resolved the six conflicts,
carried the two API changes into the merged code, and checked the result by
running each access method through the gate -- confirming that a full scan and
a primary-key range are taken, and that an index full scan, a secondary-index
range, an aggregate and an ORDER BY are all declined.
A parallel worker runs in its own THD, so it runs in its own transaction and
opened its own read view at its first read. Nothing tied that view to the one
the manager holds: the workers and the manager could each read a different
version of the table, and the chunk boundaries the engine partitioned under
the manager's view did not even belong to the snapshot the workers scanned.
Both directions were visible. Under REPEATABLE READ a worker returned rows
another session had committed after the manager's snapshot was taken, and rows
the manager's own transaction had written but not committed went missing,
because to the worker's view the manager was just another active transaction.
InnoDB implements it by copying the source transaction's read view.
ReadView::clone() installs the source's ReadViewBase state and opens the copy.
It inherits the source's m_creator_trx_id rather than keeping the worker's own,
so rows written by the sharing transaction itself stay visible through the copy
-- that is what changes_visible() uses the creator id for -- and it inherits
the source's isolation level, so a READ UNCOMMITTED source is still read
without consulting a view at all. The source view is read under its m_mutex,
the same protection the purge coordinator takes in ReadView::append_to(), and
from_thd's handlerton data under its LOCK_thd_data.
Introduces parallel_worker_threads variable to control the number
of worker threads created by a parallel execution query.
2 new files, sql_parallel_workers.h sql_parallel_workers.cc which
contain structures for the creation, management and deletion of
parallel worker threads (pwt_ in the name). Main management
class created in the stack in JOIN::exec, implemented for the
top level select.
Threads are registed in server_threads, so are visible in
information_schema.processlist and the show processlist command.
We check that a kill query on a parallel worker is passed onto it's
manager and the query is properly aborted, and that a kill connection
is handled properly in parallel_worker.test.
