mirror of
https://github.com/MariaDB/server.git
synced 2026-10-23 18:49:29 +03:00
f1a72e2cca
`save_window_function_values()` did not take into account that one can get `HA_ERR_RECORD_FILE_FULL` on `ha_update_row()`. With blob support in heap, it can now happen more easily (expressions wider than 512 characters are promoted to TEXT in tmp tables, and each update of a blob result column allocates a continuation record). However, it could also happen with Aria tables, which is apparently not tested. The error was returned as a plain `true` with an empty diagnostics area: debug builds hit `Assertion '0'` in `Protocol::end_statement()`, release builds send a bare OK packet after the result set metadata, which the client mis-parses (appears as a hang / lost connection). Errors of the computation are now exposed: `save_window_function_values()` calls `handler::print_error()` for any failure it does not recover from, the previously ignored `ha_rnd_pos()` return values are checked and reported, and `compute_window_func()` distinguishes a read error from EOF and stops the row scan as soon as the per-row loop fails. The overflow itself is recovered from by converting the tmp table to the disk-based tmp engine in place: the rows keep their identity and their contents, only their positions change, and those are translated in the rowid sequence that the computation is built on. The filesort result of a window sort is a sequence of row positions that covers every row of the table exactly once (the sort is set up without a limit) and is the only place where the positions are kept: the row scan and all frame cursors read the rows through it. The conversion (`Window_rowid_remapper`) therefore copies the rows in the order of that sequence, which makes the new position of a row known as soon as the row has been written, and stores it back into the very slot the old position was read from. For the new position to always fit into the slot, a window sort stores the row positions in slots of a fixed 8 bytes (`Filesort::min_ref_length`, set to the new `TMP_TABLE_MAX_REF_LENGTH`), which holds the position of any engine an internal tmp table can use: a pointer into memory or a data file offset. `make_sortkey()` zero-pads a shorter position to its slot, and `SORT_INFO::ref_length` carries the slot width to the readers, so `init_read_record()` no longer re-derives it from `handler::ref_length`, which changes when the table is converted. Sorts that do not set a minimum, and engines whose positions are wider than 8 bytes, are unchanged byte for byte. A sequence held in memory is thereby rewritten in place, keeping its layout, so the frame cursors' places in it stay valid. A sequence held in a temporary file is written and read through an `IO_CACHE`, which encrypts the file when tmp file encryption is enabled, so the file can not be rewritten in place: the old sequence is streamed out of its cache and the translated sequence into a new cached temporary file, which then replaces the old one under the cache. The frame cursors' slave caches stay linked to the master through `next_file_user` across the replacement, and are re-created from the new master afterwards (`Frame_cursor::rowids_rewritten()`). Should re-creating one fail, the cursor forgets its already-released cache, so that it is not released a second time when the cursor is destroyed, and reports the failure. The row whose update did not fit is written from `record[0]`, which holds its new image, instead of being read from the table, so the conversion applies the update that failed. Blob values of `record[0]` that were read from the HEAP table can point into the handler's shared blob reassembly buffer (`hp_read_blobs()`, blob values that span multiple allocation blocks), which reading the other rows overwrites, so they are first given memory of their own. When the window sort was set up with a deferred filter (a HAVING clause deferred to the final ORDER BY sort), the sequence omits the rows the filter rejected and the conversion drops them: every later reader of the table applies the same filter, either directly or by reading the table through a sort that does, so such rows can never reach the result. `create_internal_tmp_table_from_heap()` gains a `Tmp_table_row_copier` hook for the copy from the HEAP table to the Aria or MyISAM table that replaces it. The plain copy loop and the append of the pending `record[0]` become `Tmp_table_default_copier`, used when the caller passes no copier, so both cases of the conversion run through the same code, defined together with `Window_rowid_remapper` just before the function. `ha_end_bulk_insert()` is called also when the copy fails, so the new handler does not keep bulk insert state when its table is dropped; the pending row is thereby written while the bulk insert is still active, which does not delay its duplicate detection, as bulk insert does not cache unique keys (`maria_init_bulk_insert()` skips `HA_NOSAME` keys). An ignored duplicate of the pending row is passed to the caller in `Tmp_table_row_copier::duplicate_row_error`, which sets `*is_duplicate`. Carrying the running computation over the conversion, instead of re-running it, has two visible consequences: - Compound expressions containing window functions (`items_to_copy`) are evaluated exactly once per row, as they are when the table does not overflow, so the conversion does not have to be refused when such an expression is non-deterministic (`RAND_TABLE_BIT`: a user variable assignment, a non-deterministic stored function); those statements produce their result. - The statement sorts once, so sort related aggregate warnings, e.g. the `max_sort_length` truncation counts, are reported for one pass. The rows are copied in the order of the sequence, so the converted table does not hold them in the order they were inserted in. This is safe even when that order is what the table was built for (`TABLE::keep_row_order`, set for `ROWNUM()` with GROUP BY or ORDER BY): before HEAP supported blobs, such a tmp table was created in the disk engine from the start and these statements simply ran, so refusing here would be a regression against that behavior. 1. `ROWNUM()` values are materialized into the tmp table rows during the fill, before the window computation begins; the copy moves them verbatim, so their pairing with the rows can not change. 2. The rowid sequence is translated in place, so the running window computation continues over exactly the same row order, and tie-sensitive window function values (`ROW_NUMBER`, frames over tied keys) keep the values the interrupted pass had already produced. 3. Consumers that scan the converted table afterwards see the rows in the copy order. Among rows with equal sort keys that order differs from the insertion order, but such tie order is unspecified and already differs between the memory and disk tmp engines. The converted table keeps `keep_row_order= true` so the copy order is also the order the disk engine preserves from then on. Tests, in `heap.blob_window_overflow` unless noted: a single window over all rows and a partitioned window overflow mid-computation and must transparently convert; the side-effecting window expression test verifies that the user variable is assigned exactly once per row, against the same query that does not overflow; a fourth test covers the sequence being an array in memory instead of a merge file; a fifth test covers the `keep_row_order` conversion, verifying that every `ROWNUM()` value keeps its insertion-order pairing with its row across the conversion; a sixth test covers blob values larger than a HEAP allocation block surviving the conversion of the pending row; `heap.blob_window_overflow_encrypt` runs the conversion with encrypted temporary files; `heap.blob_window_overflow_debug` covers the failure to re-create a slave cache of the sequence (`simulate_window_seq_slave_oom`), which previously hung the server in the cursor destructor.
114 lines
3.2 KiB
C++
114 lines
3.2 KiB
C++
/*
|
|
Copyright (c) 2018, 2019 MariaDB
|
|
|
|
This program is free software; you can redistribute it and/or modify
|
|
it under the terms of the GNU General Public License as published by
|
|
the Free Software Foundation; version 2 of the License.
|
|
|
|
This program is distributed in the hope that it will be useful,
|
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
GNU General Public License for more details.
|
|
|
|
You should have received a copy of the GNU General Public License
|
|
along with this program; if not, write to the Free Software
|
|
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02111-1301 USA */
|
|
|
|
#include "mariadb.h"
|
|
#include "sql_priv.h"
|
|
#include "sql_select.h"
|
|
#include "derived_handler.h"
|
|
|
|
|
|
/**
|
|
The methods of the Pushdown_derived class.
|
|
|
|
The objects of this class are used for pushdown of the derived tables
|
|
into engines. The main method of the class is Pushdown_derived::execute()
|
|
that initiates execution of the query specifying a derived by a foreign
|
|
engine, receives the rows of the result set and put them in a temporary
|
|
table on the server side.
|
|
|
|
The method uses only the functions of the derived_handle interface to do
|
|
this. The constructor of the class gets this interface as a parameter.
|
|
|
|
Currently a derived tables pushed into an engine is always materialized.
|
|
It could be changed if the cases when the tables is used as driving table.
|
|
*/
|
|
|
|
|
|
Pushdown_derived::Pushdown_derived(TABLE_LIST *tbl, derived_handler *h)
|
|
: derived(tbl), handler(h)
|
|
{
|
|
}
|
|
|
|
|
|
|
|
int Pushdown_derived::execute()
|
|
{
|
|
int err;
|
|
THD *thd= handler->thd;
|
|
TABLE *table= handler->table;
|
|
TMP_TABLE_PARAM *tmp_table_param= handler->tmp_table_param;
|
|
|
|
DBUG_ENTER("Pushdown_derived::execute");
|
|
|
|
if ((err= handler->init_scan()))
|
|
goto error;
|
|
|
|
while (!(err= handler->next_row()))
|
|
{
|
|
if (unlikely(thd->check_killed()))
|
|
{
|
|
handler->end_scan();
|
|
DBUG_RETURN(-1);
|
|
}
|
|
|
|
if ((err= table->file->ha_write_tmp_row(table->record[0])))
|
|
{
|
|
bool is_duplicate;
|
|
if (likely(!table->file->is_fatal_error(err, HA_CHECK_DUP)))
|
|
continue; // Distinct elimination
|
|
|
|
if (create_internal_tmp_table_from_heap(thd, table,
|
|
tmp_table_param->start_recinfo,
|
|
&tmp_table_param->recinfo,
|
|
err, 1, &is_duplicate, NULL))
|
|
DBUG_RETURN(1);
|
|
if (is_duplicate)
|
|
continue;
|
|
}
|
|
}
|
|
|
|
if (err != 0 && err != HA_ERR_END_OF_FILE)
|
|
goto error;
|
|
|
|
if ((err= handler->end_scan()))
|
|
goto error_2;
|
|
|
|
DBUG_RETURN(0);
|
|
|
|
error:
|
|
handler->end_scan();
|
|
error_2:
|
|
handler->print_error(err, MYF(0));
|
|
DBUG_RETURN(-1); // Error not sent to client
|
|
}
|
|
|
|
|
|
void derived_handler::print_error(int error, myf errflag)
|
|
{
|
|
my_error(ER_GET_ERRNO, MYF(0), error, hton_name(ht)->str);
|
|
}
|
|
|
|
|
|
void derived_handler::set_derived(TABLE_LIST *tbl)
|
|
{
|
|
derived= tbl;
|
|
table= tbl->table;
|
|
unit= tbl->derived;
|
|
select= unit->first_select();
|
|
tmp_table_param= ((select_unit *)(unit->result))->get_tmp_table_param();
|
|
}
|
|
|