1
0
mirror of https://github.com/MariaDB/server.git synced 2026-10-23 18:49:29 +03:00
Files
Rex Johnston 378d5bd10e MDEV-41179 MAX_KEY_PARTS ranges in select causing SEGV
When a select containing 32 ranges is made on a table containing a
compound key with 32 parts, the range optimizer can run off the end
of a stack variable, invalidly overwriting subsequent stack variables.

In the struct st_sel_arg_range_seq, we have an array
  RANGE_SEQ_ENTRY stack[MAX_REF_PARTS];

MAX_REF_PARTS is 32.

check_quick_select
  / sel_arg_range_seq_init
    initialises stack[0] as NOT a key part
  / sel_arg_range_seq_next
    iterates through the key parts, adding key part n to stack[n+1]

key part #32 gets referenced by step_down_to(), setting seq->i off the
end of the array.

Fix: RANGE_SEQ_ENTRY stack[MAX_REF_PARTS+1];

The above change exposed an issue with key length calculation on
MS Windows.  Calling make_prev_keypart_map(32) caused the resultant
bitmap to be calculated as (1UL << 32) - 1.  Using the MSVC compiler
this resulted in an empty key length calculation during
handler::index_read_map, causing an assertion in ha_innobase::index_read().

As we only need 32 bits to represent our key map, we change the type thus
-typedef ulong key_part_map;
+typedef uint32 key_part_map;

We correct make_keypart_map() and make_prev_keypart_map() to call our
overflow safe my_set_bits().  We also correct bka_range_seq_next()
and bkah_range_seq_next() to use make_prev_keypart_map().

We also add some DBUG_ASSERTS in key_part_map processing elsewhere,
exposing some issues in our BNLH implementation.  We cap the number of
keyuse parts here, altering the explain output of 2 of our tests.

Numerous places needed bit shift operations altered to use
make*keymap_part and various format strings needed to be corrected.
2026-10-13 07:06:17 +13:00
..
…
2026-07-23 11:12:27 +02:00
2026-07-23 11:12:27 +02:00
…
2025-09-04 18:08:39 +03:00
2024-01-17 13:35:05 +02:00
…
…
2024-11-29 13:43:17 +02:00
2024-11-29 12:37:46 +02:00
2024-10-29 15:24:38 +01:00
…
2025-09-04 18:08:39 +03:00
2023-12-17 11:20:43 +01:00
…
…
…
2025-02-27 04:02:33 +01:00
2025-09-04 18:08:39 +03:00
2024-11-29 13:43:17 +02:00
…
…
2024-11-29 13:43:17 +02:00
2024-12-05 10:11:58 +01:00
2026-08-19 08:13:54 +02:00
2026-01-28 14:22:20 +01:00
2026-01-28 14:22:20 +01:00
2026-01-28 14:22:20 +01:00
2026-01-28 14:22:20 +01:00
2026-07-23 11:12:27 +02:00
2026-01-28 14:22:20 +01:00
2024-11-29 13:43:17 +02:00
2023-11-08 16:23:30 +01:00
…
2024-06-27 10:26:09 +03:00
2024-06-24 13:09:47 +03:00
…
…
2025-09-04 18:08:39 +03:00
…
2023-11-08 16:23:30 +01:00
2026-08-19 08:13:54 +02:00
2023-10-14 13:36:11 +03:00
2026-01-28 14:22:20 +01:00
2025-09-04 18:08:39 +03:00
2025-09-04 18:08:39 +03:00
2023-02-10 13:43:53 +02:00
2026-04-20 11:07:30 +03:00
2023-12-18 11:19:04 +01:00
2024-11-29 12:37:46 +02:00
…
2024-06-24 13:09:47 +03:00
2023-03-24 12:54:05 +11:00
2025-09-04 18:08:39 +03:00
2025-09-04 18:08:39 +03:00
2025-09-04 18:08:39 +03:00
2024-07-22 15:14:50 +02:00
2026-06-15 10:43:23 +02:00
2026-06-15 10:43:23 +02:00
2025-10-28 17:49:51 +01:00
2025-10-28 17:49:51 +01:00
2026-07-23 11:12:27 +02:00
2024-11-29 12:37:46 +02:00
2026-08-19 08:13:54 +02:00
2025-09-04 18:08:39 +03:00
2025-01-10 13:14:42 +01:00
2024-06-24 13:09:47 +03:00
2025-01-30 11:55:13 +01:00
2025-01-30 11:55:13 +01:00
2023-07-10 11:22:21 +03:00
2024-11-29 13:43:17 +02:00
2024-11-29 13:43:17 +02:00
2026-08-19 08:13:54 +02:00
2025-09-04 18:08:39 +03:00
2024-01-18 19:22:23 +02:00
2025-09-04 18:08:39 +03:00
2024-11-23 08:14:22 -07:00
2026-01-02 11:06:05 +02:00
2026-07-23 11:12:27 +02:00
2026-07-23 11:12:27 +02:00
2026-03-17 18:57:42 +01:00
2026-01-28 14:22:20 +01:00
…
2024-08-20 10:00:39 +02:00
2025-09-04 18:08:39 +03:00
2024-06-27 10:26:09 +03:00
2024-11-29 12:37:46 +02:00
…
2025-09-19 12:31:14 +03:00
2025-09-04 18:08:39 +03:00
2023-05-23 12:25:39 +03:00
2024-06-11 12:50:10 +03:00
2024-04-15 18:54:30 +02:00
2026-01-28 14:22:20 +01:00
2024-07-03 12:45:30 +02:00
2023-08-04 08:01:06 +02:00
2026-07-23 11:12:27 +02:00
2023-12-17 11:20:43 +01:00
2026-07-23 11:12:27 +02:00
2023-12-02 01:02:50 +01:00
2025-09-04 18:08:39 +03:00
2025-09-04 18:08:39 +03:00
2026-03-17 11:05:34 +02:00
2024-03-27 15:00:56 +02:00
2025-09-04 18:08:39 +03:00
2026-07-23 11:12:27 +02:00
2025-01-20 09:57:37 +02:00
2025-03-27 08:01:47 +02:00
2023-09-25 13:06:57 +10:00
2024-01-18 19:22:23 +02:00
2023-08-01 15:08:52 +02:00
2024-11-23 08:14:22 -07:00
2024-11-23 08:14:23 -07:00
2026-08-19 08:13:54 +02:00
2026-07-23 11:12:27 +02:00
2026-07-23 11:12:27 +02:00
2024-01-10 18:01:46 +11:00
2025-09-04 18:08:39 +03:00
2025-09-04 18:08:39 +03:00
2024-06-24 13:09:47 +03:00
…
2025-09-04 18:08:39 +03:00
2023-02-10 12:02:11 +02:00
2024-11-29 13:43:17 +02:00
2024-01-02 17:37:58 +02:00
2024-06-27 10:26:09 +03:00
2026-06-21 15:32:20 +02:00
2024-01-03 15:37:57 +02:00
2024-10-03 10:55:08 +03:00
2026-07-23 11:12:27 +02:00
2026-07-23 11:12:27 +02:00
2026-06-21 15:32:20 +02:00