1
0
mirror of https://github.com/MariaDB/server.git synced 2026-10-23 18:49:29 +03:00
Files
mariadb/sql
Arcadiy Ivanov 03c04bbd6c MDEV-40781 Duplicate row despite DISTINCT when tmp table converts
`create_internal_tmp_table_from_heap()` writes the pending `record[0]`,
the row whose write filled the in-memory table, into the new table.
Since **MDEV-40376** (`636f154bb49`) that write happens *before*
`ha_end_bulk_insert()` rather than after it.

`ha_maria::start_bulk_insert()` disables **all** indexes of an internal
temporary table that is about to receive at least
`MARIA_MIN_ROWS_TO_DISABLE_INDEXES` (100) rows:

```c
if (file->open_flags & HA_OPEN_INTERNAL_TABLE)
{
  /* Internal table; If we get a duplicate something is very wrong */
  file->update|= HA_STATE_CHANGED;
  index_disabled= share->base.keys > 0;
  maria_clear_all_keys_active(file->s->state.key_map);
}
```

`maria_write()` then skips `_ma_check_unique()` entirely, so the unique
constraint that implements `DISTINCT` for a key too wide to be an index
is not enforced. The rows copied out of the in-memory table are already
distinct and need no checking against each other, but the pending row is
exactly the row whose duplicate status is unknown, and it was written
inside that window. A `SELECT DISTINCT` over wide columns could
therefore return a duplicate row.

Note that the justification given in `636f154bb49` is not the mechanism
at work here. It refers to the bulk insert key *tree*, a different
branch of `ha_maria::start_bulk_insert()`; setting
`bulk_insert_buffer_size=0` does not avoid the problem.

The fix splits the copy in two:

1. `Tmp_table_row_copier` gains a second virtual,
   `write_pending_row()`, defaulting to a no-op.
2. `copy_rows()` now only copies the rows the in-memory table holds.
3. `create_internal_tmp_table_from_heap()` calls
   `ha_end_bulk_insert()` and then `write_pending_row()`, so the
   pending row is written with the indexes of the new table back in
   place and a duplicate of an already copied row is detected.

`Window_rowid_remapper` keeps writing its pending row within
`copy_rows()` and inherits the no-op default. Its new position is only
known once the rows before it have been written, and nothing is lost by
writing it with the indexes still disabled: it replaces a row that is
already in the table rather than adding one, and an update of a window
function value cannot collide with another row, as a deduplicating key
is not built on the columns it changes.

The new test covers `SELECT DISTINCT`, `SELECT DISTINCT ... ORDER BY`,
`GROUP BY`, `UNION` and `INSERT ... SELECT DISTINCT`, and asserts that
the conversion actually happened so that a future sizing change cannot
silently void the coverage.
2026-10-11 18:42:17 +03:00
..
2026-06-15 16:53:03 +02:00
…
2026-09-04 12:21:02 +02:00
2025-07-28 21:29:29 +02:00
2024-11-05 14:00:48 -08:00
2025-09-29 18:25:09 +03:00
…
2025-03-14 12:08:56 +04:00
2026-02-16 21:55:18 +01:00
2025-03-14 12:08:56 +04:00
…
2024-12-02 17:51:17 +02:00
2026-01-31 14:08:54 +01:00
2026-07-20 13:23:00 +03:00
2026-07-20 13:23:00 +03:00
…
2026-01-31 14:08:54 +01:00
2024-12-02 11:35:34 +02:00
2026-03-24 09:49:17 +02:00
2026-03-24 09:49:17 +02:00
…
…
2026-06-15 16:53:03 +02:00
2026-05-26 21:15:58 +02:00
2026-07-29 12:27:46 +02:00
2024-11-29 13:43:17 +02:00
…
…
2026-08-20 08:27:14 +02:00
2026-08-20 08:27:14 +02:00
2024-12-05 10:11:58 +01:00
2026-08-20 08:27:14 +02:00
2026-02-03 14:53:15 +01:00
2026-05-26 21:15:58 +02:00
2026-02-03 14:53:15 +01:00
2026-09-04 12:21:02 +02:00
2026-08-20 08:27:14 +02:00
2026-03-24 09:49:17 +02:00
2026-01-31 14:08:54 +01:00
2026-01-08 12:02:45 +01:00
2026-08-03 13:42:39 +02:00
2024-07-08 22:40:16 +02:00
2026-05-26 21:15:58 +02:00
2026-05-26 21:15:58 +02:00
…
2025-03-14 12:08:56 +04:00
2026-09-04 12:21:02 +02:00
…
2024-10-03 15:59:20 +10:00
…
2025-03-14 12:08:56 +04:00
2026-05-26 21:15:58 +02:00
2026-09-04 12:14:15 +02:00
2026-07-29 12:27:46 +02:00
2026-09-04 12:21:02 +02:00
2026-07-30 11:41:55 +02:00
2026-05-08 08:00:01 +02:00
2026-09-04 12:21:02 +02:00
2026-01-31 14:08:54 +01:00
2026-05-26 21:15:58 +02:00
2026-09-04 12:14:15 +02:00
2026-02-03 14:53:15 +01:00
2025-09-29 18:25:09 +03:00
2024-07-22 15:14:50 +02:00
2026-05-26 21:15:58 +02:00
2026-05-26 21:15:58 +02:00
2026-06-16 12:24:32 +02:00
2026-07-30 11:41:55 +02:00
2026-06-16 12:24:32 +02:00
2026-08-20 08:27:14 +02:00
2026-06-15 16:53:03 +02:00
2026-06-15 16:53:03 +02:00
2025-03-14 12:08:56 +04:00
2026-04-02 09:15:17 +03:00
2026-09-04 12:14:15 +02:00
2026-09-04 12:21:02 +02:00
2026-09-04 12:21:02 +02:00
2026-02-06 17:46:50 +01:00
2025-11-04 12:39:27 +01:00
2026-09-04 12:14:15 +02:00
2026-09-04 12:14:15 +02:00
2026-09-10 09:07:23 +03:00
2026-09-04 12:14:15 +02:00
2026-09-04 12:14:15 +02:00
2026-08-20 08:27:14 +02:00
2025-09-04 18:08:39 +03:00
2024-06-24 13:09:47 +03:00
2026-08-03 13:42:39 +02:00
2026-07-30 11:41:55 +02:00
2026-06-15 15:15:10 +02:00
2026-06-15 15:15:10 +02:00
2026-04-02 09:15:17 +03:00
2026-01-31 14:08:54 +01:00
2026-01-05 20:34:44 +01:00
2026-07-30 11:41:55 +02:00
2026-03-24 09:49:17 +02:00
2026-06-29 07:40:04 +03:00
2026-01-08 12:02:45 +01:00
2026-07-30 16:15:40 +02:00
2026-07-30 16:15:40 +02:00
2026-02-06 17:46:51 +01:00
2026-03-17 18:57:42 +01:00
2026-07-30 16:15:40 +02:00
…
2026-02-06 17:46:51 +01:00
2024-06-27 10:26:09 +03:00
2025-08-04 12:05:53 +02:00
2024-12-02 17:51:17 +02:00
2025-09-19 12:31:14 +03:00
2026-05-26 21:15:58 +02:00
2025-03-14 12:08:56 +04:00
2026-09-10 09:07:23 +03:00
2026-09-04 12:21:02 +02:00
2024-07-03 12:45:30 +02:00
2026-09-04 12:21:02 +02:00
2026-09-04 12:21:02 +02:00
2025-01-09 09:41:38 +02:00
2024-12-02 17:51:17 +02:00
2026-09-04 12:21:02 +02:00
2026-09-04 12:21:02 +02:00
2026-08-20 08:27:14 +02:00
2026-09-04 12:21:02 +02:00
2026-08-19 08:40:05 +02:00
2026-07-28 13:14:38 +02:00
2026-05-26 21:15:58 +02:00
…
2026-09-04 12:14:15 +02:00
2025-03-14 12:08:56 +04:00
2024-12-02 17:51:17 +02:00
2026-08-20 08:27:14 +02:00
2026-09-04 12:21:02 +02:00
2026-08-20 08:27:14 +02:00
2026-08-20 08:27:14 +02:00
2026-07-20 13:23:00 +03:00
2026-03-24 09:49:17 +02:00
2025-09-04 18:08:39 +03:00
2024-06-24 13:09:47 +03:00
2025-03-14 12:08:56 +04:00
2026-05-22 09:52:05 +02:00
2024-12-02 17:51:17 +02:00
2024-12-02 11:35:34 +02:00
2026-09-04 12:14:15 +02:00
2026-09-04 12:21:02 +02:00
2025-03-14 12:08:56 +04:00
2026-06-16 12:24:32 +02:00
2025-11-28 12:56:27 +02:00
2024-06-27 10:26:09 +03:00
2026-09-04 12:21:02 +02:00
2026-06-21 15:32:20 +02:00
2026-08-19 08:40:05 +02:00
2026-06-15 16:53:03 +02:00
2026-04-02 09:15:17 +03:00
2024-10-03 13:24:43 +03:00
2024-09-09 13:50:38 +03:00
2026-09-04 12:21:02 +02:00
2026-02-07 22:12:32 +01:00
2026-08-03 13:42:39 +02:00
2026-07-29 12:27:46 +02:00