database/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-11-09 11:41:36 +03:00
Code Activity

Changing all cost calculation to be given in milliseconds

Browse Source 
This makes it easier to compare different costs and also allows
the optimizer to optimizer different storage engines more reliably.

- Added tests/check_costs.pl, a tool to verify optimizer cost calculations.
  - Most engine costs has been found with this program. All steps to
    calculate the new costs are documented in Docs/optimizer_costs.txt

- User optimizer_cost variables are given in microseconds (as individual
  costs can be very small). Internally they are stored in ms.
- Changed DISK_READ_COST (was DISK_SEEK_BASE_COST) from a hard disk cost
  (9 ms) to common SSD cost (400MB/sec).
- Removed cost calculations for hard disks (rotation etc).
- Changed the following handler functions to return IO_AND_CPU_COST.
  This makes it easy to apply different cost modifiers in ha_..time()
  functions for io and cpu costs.
  - scan_time()
  - rnd_pos_time() & rnd_pos_call_time()
  - keyread_time()
- Enhanched keyread_time() to calculate the full cost of reading of a set
  of keys with a given number of ranges and optional number of blocks that
  need to be accessed.
- Removed read_time() as keyread_time() + rnd_pos_time() can do the same
  thing and more.
- Tuned cost for: heap, myisam, Aria, InnoDB, archive and MyRocks.
  Used heap table costs for json_table. The rest are using default engine
  costs.
- Added the following new optimizer variables:
  - optimizer_disk_read_ratio
  - optimizer_disk_read_cost
  - optimizer_key_lookup_cost
  - optimizer_row_lookup_cost
  - optimizer_row_next_find_cost
  - optimizer_scan_cost
- Moved all engine specific cost to OPTIMIZER_COSTS structure.
- Changed costs to use 'records_out' instead of 'records_read' when
  recalculating costs.
- Split optimizer_costs.h to optimizer_costs.h and optimizer_defaults.h.
  This allows one to change costs without having to compile a lot of
  files.
- Updated costs for filter lookup.
- Use a better cost estimate in best_extension_by_limited_search()
  for the sorting cost.
- Fixed previous issues with 'filtered' explain column as we are now
  using 'records_out' (min rows seen for table) to calculate filtering.
  This greatly simplifies the filtering code in
  JOIN_TAB::save_explain_data().

This change caused a lot of queries to be optimized differently than
before, which exposed different issues in the optimizer that needs to
be fixed.  These fixes are in the following commits.  To not have to
change the same test case over and over again, the changes in the test
cases are done in a single commit after all the critical change sets
are done.

InnoDB changes:
- Updated InnoDB to not divide big range cost with 2.
- Added cost for InnoDB (innobase_update_optimizer_costs()).
- Don't mark clustered primary key with HA_KEYREAD_ONLY. This will
  prevent that the optimizer is trying to use index-only scans on
  the clustered key.
- Disabled ha_innobase::scan_time() and ha_innobase::read_time() and
  ha_innobase::rnd_pos_time() as the default engine cost functions now
  works good for InnoDB.

Other things:
- Added  --show-query-costs (\Q) option to mysql.cc to show the query
  cost after each query (good when working with query costs).
- Extended my_getopt with GET_ADJUSTED_VALUE which allows one to adjust
  the value that user is given. This is used to change cost from
  microseconds (user input) to milliseconds (what the server is
  internally using).
- Added include/my_tracker.h  ; Useful include file to quickly test
  costs of a function.
- Use handler::set_table() in all places instead of 'table= arg'.
- Added SHOW_OPTIMIZER_COSTS to sys variables. These are input and
  shown in microseconds for the user but stored as milliseconds.
  This is to make the numbers easier to read for the user (less
  pre-zeros).  Implemented in 'Sys_var_optimizer_cost' class.
- In test_quick_select() do not use index scans if 'no_keyread' is set
  for the table. This is what we do in other places of the server.
- Added THD parameter to Unique::get_use_cost() and
  check_index_intersect_extension() and similar functions to be able
  to provide costs to called functions.
- Changed 'records' to 'rows' in optimizer_trace.
- Write more information to optimizer_trace.
- Added INDEX_BLOCK_FILL_FACTOR_MUL (4) and INDEX_BLOCK_FILL_FACTOR_DIV (3)
  to calculate usage space of keys in b-trees. (Before we used numeric
  constants).
- Removed code that assumed that b-trees has similar costs as binary
  trees. Replaced with engine calls that returns the cost.
- Added Bitmap::find_first_bit()
- Added timings to join_cache for ANALYZE table (patch by Sergei Petrunia).
- Added records_init and records_after_filter to POSITION to remember
  more of what best_access_patch() calculates.
- table_after_join_selectivity() changed to recalculate 'records_out'
  based on the new fields from best_access_patch()

Bug fixes:
- Some queries did not update last_query_cost (was 0). Fixed by moving
  setting thd->...last_query_cost in JOIN::optimize().
- Write '0' as number of rows for const tables with a matching row.

Some internals:
- Engine cost are stored in OPTIMIZER_COSTS structure.  When a
  handlerton is created, we also created a new cost variable for the
  handlerton. We also create a new variable if the user changes a
  optimizer cost for a not yet loaded handlerton either with command
  line arguments or with SET
  @@global.engine.optimizer_cost_variable=xx.
- There are 3 global OPTIMIZER_COSTS variables:
  default_optimizer_costs   The default costs + changes from the
                            command line without an engine specifier.
  heap_optimizer_costs      Heap table costs, used for temporary tables
  tmp_table_optimizer_costs The cost for the default on disk internal
                            temporary table (MyISAM or Aria)
- The engine cost for a table is stored in table_share. To speed up
  accesses the handler has a pointer to this. The cost is copied
  to the table on first access. If one wants to change the cost one
  must first update the global engine cost and then do a FLUSH TABLES.
  This was done to be able to access the costs for an open table
  without any locks.
- When a handlerton is created, the cost are updated the following way:
  See sql/keycaches.cc for details:
  - Use 'default_optimizer_costs' as a base
  - Call hton->update_optimizer_costs() to override with the engines
    default costs.
  - Override the costs that the user has specified for the engine.
  - One handler open, copy the engine cost from handlerton to TABLE_SHARE.
  - Call handler::update_optimizer_costs() to allow the engine to update
    cost for this particular table.
  - There are two costs stored in THD. These are copied to the handler
    when the table is used in a query:
    - optimizer_where_cost
    - optimizer_scan_setup_cost
- Simply code in best_access_path() by storing all cost result in a
  structure. (Idea/Suggestion by Igor)
This commit is contained in:
Monty
2022-08-11 13:05:23 +03:00
committed by Sergei Petrunia
parent 590416e21c
commit b66cdbd1ea
110 changed files with 5373 additions and 1537 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1309
Docs/optimizer_costs.txt Normal file
View File

File diff suppressed because it is too large Load Diff

58
client/mysql.cc
View File

@@ -46,7 +46,7 @@
#include <locale.h>
#endif
const char *VER= "15.1";
const char *VER= "15.2";
/* Don't try to make a nice table if the data is too big */
#define MAX_COLUMN_LENGTH 1024
@@ -246,7 +246,7 @@ static my_bool ignore_errors=0,wait_flag=0,quick=0,
tty_password= 0, opt_nobeep=0, opt_reconnect=1,
opt_secure_auth= 0,
default_pager_set= 0, opt_sigint_ignore= 0,
auto_vertical_output= 0,
auto_vertical_output= 0, show_query_cost= 0,
show_warnings= 0, executing_query= 0,
ignore_spaces= 0, opt_binhex= 0, opt_progress_reports;
static my_bool debug_info_flag, debug_check_flag, batch_abort_on_error;
@@ -324,6 +324,7 @@ static int com_quit(String *str,char*),
com_notee(String *str, char*), com_charset(String *str,char*),
com_prompt(String *str, char*), com_delimiter(String *str, char*),
com_warnings(String *str, char*), com_nowarnings(String *str, char*);
static int com_query_cost(String *str, char*);
#ifdef USE_POPEN
static int com_nopager(String *str, char*), com_pager(String *str, char*),
@@ -395,6 +396,8 @@ static COMMANDS commands[] = {
{ "print", 'p', com_print, 0, "Print current command." },
{ "prompt", 'R', com_prompt, 1, "Change your mysql prompt."},
{ "quit", 'q', com_quit, 0, "Quit mysql." },
{ "costs", 'Q', com_query_cost, 0,
"Toggle showing query costs after each query" },
{ "rehash", '#', com_rehash, 0, "Rebuild completion hash." },
{ "source", '.', com_source, 1,
"Execute an SQL script file. Takes a file name as an argument."},
@@ -1156,6 +1159,7 @@ static void print_table_data_xml(MYSQL_RES *result);
static void print_tab_data(MYSQL_RES *result);
static void print_table_data_vertically(MYSQL_RES *result);
static void print_warnings(void);
static void print_last_query_cost(void);
static void end_timer(ulonglong start_time, char *buff);
static void nice_time(double sec,char *buff,bool part_second);
extern "C" sig_handler mysql_end(int sig) __attribute__ ((noreturn));
@@ -1828,6 +1832,10 @@ static struct my_option my_long_options[] =
{"show-warnings", OPT_SHOW_WARNINGS, "Show warnings after every statement.",
&show_warnings, &show_warnings, 0, GET_BOOL, NO_ARG,
0, 0, 0, 0, 0, 0},
{"show-query-costs", OPT_SHOW_WARNINGS,
"Show query cost after every statement.",
&show_query_cost, &show_query_cost, 0, GET_BOOL, NO_ARG,
0, 0, 0, 0, 0, 0},
{"plugin_dir", OPT_PLUGIN_DIR, "Directory for client-side plugins.",
&opt_plugin_dir, &opt_plugin_dir, 0,
GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
@@ -3586,6 +3594,8 @@ end:
/* Show warnings if any or error occurred */
if (show_warnings == 1 && (warnings >= 1 || error))
print_warnings();
if (show_query_cost)
print_last_query_cost();
if (!error && !status.batch &&
(mysql.server_status & SERVER_STATUS_DB_DROPPED))
@@ -4190,6 +4200,30 @@ end:
}
/* print_last_query_cost */
static void print_last_query_cost()
{
const char *query;
char *end;
MYSQL_RES *result;
MYSQL_ROW cur;
query= "show status like 'last_query_cost'";
mysql_real_query_for_lazy(query, strlen(query));
mysql_store_result_for_lazy(&result);
if (!result)
goto end;
cur= mysql_fetch_row(result);
if (strtod(cur[1], &end) != 0.0)
tee_fprintf(PAGER, "%s: %s\n\n", cur[0], cur[1]);
end:
mysql_free_result(result);
}
static const char *array_value(const char **array, char key)
{
for (; *array; array+= 2)
@@ -4765,6 +4799,18 @@ com_nowarnings(String *buffer __attribute__((unused)),
return 0;
}
static int
com_query_cost(String *buffer __attribute__((unused)),
char *line __attribute__((unused)))
{
show_query_cost= 1 - show_query_cost;
if (show_query_cost)
put_info("Last_query_cost enabled.",INFO_INFO);
else
put_info("Last_query_cost disabled.",INFO_INFO);
return 0;
}
/*
Gets argument from a command on the command line. If mode is not GET_NEXT,
skips the command and returns the first argument. The line is modified by
@@ -5020,6 +5066,10 @@ com_status(String *buffer __attribute__((unused)),
ulonglong id;
MYSQL_RES *UNINIT_VAR(result);
/*
Don't remove "limit 1",
it is protection against SQL_SELECT_LIMIT=0
*/
if (mysql_real_query_for_lazy(
C_STRING_WITH_LEN("select DATABASE(), USER() limit 1")))
return 0;
@@ -5027,10 +5077,6 @@ com_status(String *buffer __attribute__((unused)),
tee_puts("--------------", stdout);
usage(1); /* Print version */
tee_fprintf(stdout, "\nConnection id:\t\t%lu\n",mysql_thread_id(&mysql));
/*
Don't remove "limit 1",
it is protection against SQL_SELECT_LIMIT=0
*/
if (!mysql_store_result_for_lazy(&result))
{
MYSQL_ROW cur=mysql_fetch_row(result);

3
include/my_getopt.h
View File

@@ -40,6 +40,7 @@ C_MODE_START
#define GET_FLAGSET 15
#define GET_BIT 16
#define GET_ADJUST_VALUE 256
#define GET_ASK_ADDR 128
#define GET_AUTO 64
#define GET_TYPE_MASK 63
@@ -100,6 +101,7 @@ typedef my_bool (*my_get_one_option)(const struct my_option *, const char *, con
typedef void *(*my_getopt_value)(const char *, uint, const struct my_option *,
int *);
typedef void (*my_getopt_adjust)(const struct my_option *, void *);
extern char *disabled_my_option;
extern char *autoset_my_option;
@@ -109,6 +111,7 @@ extern my_bool my_getopt_prefix_matching;
extern my_bool my_handle_options_init_variables;
extern my_error_reporter my_getopt_error_reporter;
extern my_getopt_value my_getopt_get_addr;
extern my_getopt_adjust my_getopt_adjust_value;
extern int handle_options (int *argc, char ***argv,
const struct my_option *longopts, my_get_one_option)

1
include/my_global.h
View File

@@ -682,6 +682,7 @@ typedef SOCKET_SIZE_TYPE size_socket;
Io buffer size; Must be a power of 2 and a multiple of 512. May be
smaller what the disk page size. This influences the speed of the
isam btree library. eg to big to slow.
4096 is a common block size on SSDs.
*/
#define IO_SIZE 4096U
/*

41
include/my_tracker.h Normal file
View File

@@ -0,0 +1,41 @@
/* Copyright (c) 2022, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; version 2 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1335 USA */
/*
Trivial framework to add a tracker to a C function
*/
#include "my_rdtsc.h"
struct my_time_tracker
{
ulonglong counter;
ulonglong cycles;
};
#ifdef HAVE_TIME_TRACKING
#define START_TRACKING ulonglong my_start_time= my_timer_cycles()
#define END_TRACKING(var) \
{ \
ulonglong my_end_time= my_timer_cycles(); \
(var)->counter++; \
(var)->cycles+= (unlikely(my_end_time < my_start_time) ? \
my_end_time - my_start_time + ULONGLONG_MAX : \
my_end_time - my_start_time); \
}
#else
#define START_TRACKING
#define END_TRACKING(var) do { } while(0)
#endif

2
include/myisam.h
View File

@@ -435,6 +435,8 @@ int thr_write_keys(MI_SORT_PARAM *sort_param);
int sort_write_record(MI_SORT_PARAM *sort_param);
int _create_index_by_sort(MI_SORT_PARAM *info,my_bool no_messages, ulonglong);
my_bool mi_too_big_key_for_sort(MI_KEYDEF *key, ha_rows rows);
struct OPTIMIZER_COSTS;
void myisam_update_optimizer_costs(struct OPTIMIZER_COSTS *costs);
#ifdef __cplusplus
}

2
mysql-test/include/analyze-format.inc
View File

@@ -1,3 +1,3 @@
# The time on ANALYSE FORMAT=JSON is rather variable
--replace_regex /("(r_total_time_ms|r_table_time_ms|r_other_time_ms|r_buffer_size|r_filling_time_ms|r_query_time_in_progress_ms)": )[^, \n]*/\1"REPLACED"/
--replace_regex /("(r_total_time_ms|r_table_time_ms|r_other_time_ms|r_buffer_size|r_filling_time_ms|r_query_time_in_progress_ms|r_unpack_time_ms)": )[^, \n]*/\1"REPLACED"/

26
mysql-test/main/analyze_format_json.result
View File

@@ -183,7 +183,8 @@ ANALYZE
"buffer_type": "flat",
"buffer_size": "1Kb",
"join_type": "BNL",
"r_filtered": 100
"r_filtered": 100,
"r_unpack_time_ms": "REPLACED"
}
}
]
@@ -233,7 +234,8 @@ ANALYZE
"buffer_size": "1Kb",
"join_type": "BNL",
"attached_condition": "tbl1.c > tbl2.c",
"r_filtered": 15.83333333
"r_filtered": 15.83333333,
"r_unpack_time_ms": "REPLACED"
}
}
]
@@ -769,13 +771,14 @@ ANALYZE
"r_other_time_ms": "REPLACED",
"filtered": 100,
"r_filtered": 0,
"attached_condition": "<in_optimizer>(t2.b,t2.b in (subquery#2))"
"attached_condition": "<in_optimizer>(t2.b,<exists>(subquery#2))"
},
"buffer_type": "flat",
"buffer_size": "65",
"join_type": "BNL",
"attached_condition": "<in_optimizer>(t2.b,t2.b in (subquery#2))",
"r_filtered": null
"attached_condition": "<in_optimizer>(t2.b,<exists>(subquery#2))",
"r_filtered": null,
"r_unpack_time_ms": "REPLACED"
}
}
],
@@ -783,20 +786,21 @@ ANALYZE
{
"query_block": {
"select_id": 2,
"r_loops": 1,
"r_loops": 2,
"r_total_time_ms": "REPLACED",
"nested_loop": [
{
"table": {
"table_name": "t1",
"access_type": "ALL",
"r_loops": 1,
"r_loops": 2,
"rows": 2,
"r_rows": 2,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": 100,
"r_filtered": 100
"r_filtered": 0,
"attached_condition": "4 = t1.a"
}
}
]
@@ -878,7 +882,8 @@ ANALYZE
"buffer_type": "flat",
"buffer_size": "1",
"join_type": "BNL",
"r_filtered": null
"r_filtered": null,
"r_unpack_time_ms": "REPLACED"
}
}
],
@@ -916,7 +921,8 @@ ANALYZE
"buffer_size": "65",
"join_type": "BNL",
"attached_condition": "t2.f2 = t3.f3",
"r_filtered": null
"r_filtered": null,
"r_unpack_time_ms": "REPLACED"
}
}
]

6
mysql-test/main/analyze_stmt_orderby.result
View File

@@ -494,7 +494,8 @@ ANALYZE
"buffer_size": "65",
"join_type": "BNL",
"attached_condition": "t3.a = t0.a",
"r_filtered": 10
"r_filtered": 10,
"r_unpack_time_ms": "REPLACED"
}
}
]
@@ -578,7 +579,8 @@ ANALYZE
"buffer_size": "119",
"join_type": "BNL",
"attached_condition": "t5.a = t6.a",
"r_filtered": 21.42857143
"r_filtered": 21.42857143,
"r_unpack_time_ms": "REPLACED"
}
}
]

2
mysql-test/main/ctype_collate.result
View File

@@ -748,7 +748,7 @@ hex(b)
explain
select hex(b) from t1 where b<'zzz' order by b;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t1 ALL PRIMARY NULL NULL NULL 4 Using where; Using filesort
1 SIMPLE t1 range PRIMARY PRIMARY 34 NULL 4 Using where; Using filesort
select hex(b) from t1 where b<'zzz' order by b;
hex(b)
00

1
mysql-test/main/delete.result
View File

@@ -128,7 +128,6 @@ a b
delete ignore t11.*, t12.* from t11,t12 where t11.a = t12.a and t11.b <> (select b from t2 where t11.a < t2.a);
Warnings:
Warning 1242 Subquery returns more than 1 row
Warning 1242 Subquery returns more than 1 row
select * from t11;
a b
0 10

12
mysql-test/main/except.result
View File

@@ -387,7 +387,8 @@ ANALYZE
"buffer_type": "flat",
"buffer_size": "119",
"join_type": "BNL",
"r_filtered": 100
"r_filtered": 100,
"r_unpack_time_ms": "REPLACED"
}
}
]
@@ -429,7 +430,8 @@ ANALYZE
"buffer_type": "flat",
"buffer_size": "119",
"join_type": "BNL",
"r_filtered": 100
"r_filtered": 100,
"r_unpack_time_ms": "REPLACED"
}
}
]
@@ -505,7 +507,8 @@ ANALYZE
"buffer_type": "flat",
"buffer_size": "119",
"join_type": "BNL",
"r_filtered": 100
"r_filtered": 100,
"r_unpack_time_ms": "REPLACED"
}
}
]
@@ -547,7 +550,8 @@ ANALYZE
"buffer_type": "flat",
"buffer_size": "119",
"join_type": "BNL",
"r_filtered": 100
"r_filtered": 100,
"r_unpack_time_ms": "REPLACED"
}
}
]

12
mysql-test/main/except_all.result
View File

@@ -514,7 +514,8 @@ ANALYZE
"buffer_type": "flat",
"buffer_size": "119",
"join_type": "BNL",
"r_filtered": 100
"r_filtered": 100,
"r_unpack_time_ms": "REPLACED"
}
}
]
@@ -556,7 +557,8 @@ ANALYZE
"buffer_type": "flat",
"buffer_size": "119",
"join_type": "BNL",
"r_filtered": 100
"r_filtered": 100,
"r_unpack_time_ms": "REPLACED"
}
}
]
@@ -631,7 +633,8 @@ ANALYZE
"buffer_type": "flat",
"buffer_size": "119",
"join_type": "BNL",
"r_filtered": 100
"r_filtered": 100,
"r_unpack_time_ms": "REPLACED"
}
}
]
@@ -673,7 +676,8 @@ ANALYZE
"buffer_type": "flat",
"buffer_size": "119",
"join_type": "BNL",
"r_filtered": 100
"r_filtered": 100,
"r_unpack_time_ms": "REPLACED"
}
}
]

3
mysql-test/main/fetch_first.result
View File

@@ -843,6 +843,7 @@ fetch first 2 rows with ties;
first_name last_name
Alice Fowler
Bob Trasc
Silvia Ganush
#
# Test CTE support.
#
@@ -858,7 +859,7 @@ select * from temp_table
order by first_name, last_name;
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2 Using filesort
2 DERIVED t1 range t1_name t1_name 103 NULL 3 Using where; Using index
2 DERIVED t1 range t1_name t1_name 206 NULL 3 Using where; Using index for group-by
with temp_table as (
select first_name, last_name
from t1

1
mysql-test/main/fulltext_order_by.test
View File

@@ -36,6 +36,7 @@ SELECT IF(a=7,'match',IF(a=4,'match', 'no-match')), MATCH (message) AGAINST ('st
# for fulltext searches too
#
alter table t1 add key m (message);
show create table t1;
explain SELECT message FROM t1 WHERE MATCH (message) AGAINST ('steve') ORDER BY message;
SELECT message FROM t1 WHERE MATCH (message) AGAINST ('steve') ORDER BY message desc;

8
mysql-test/main/information_schema_all_engines.result
View File

@@ -42,6 +42,7 @@ INNODB_TRX
KEYWORDS
KEY_CACHES
KEY_COLUMN_USAGE
OPTIMIZER_COSTS
OPTIMIZER_TRACE
PARAMETERS
PARTITIONS
@@ -123,6 +124,7 @@ INNODB_TRX trx_id
KEYWORDS WORD
KEY_CACHES KEY_CACHE_NAME
KEY_COLUMN_USAGE CONSTRAINT_SCHEMA
OPTIMIZER_COSTS ENGINE
OPTIMIZER_TRACE QUERY
PARAMETERS SPECIFIC_SCHEMA
PARTITIONS TABLE_SCHEMA
@@ -204,6 +206,7 @@ INNODB_TRX trx_id
KEYWORDS WORD
KEY_CACHES KEY_CACHE_NAME
KEY_COLUMN_USAGE CONSTRAINT_SCHEMA
OPTIMIZER_COSTS ENGINE
OPTIMIZER_TRACE QUERY
PARAMETERS SPECIFIC_SCHEMA
PARTITIONS TABLE_SCHEMA
@@ -289,6 +292,7 @@ INNODB_TABLESPACES_ENCRYPTION information_schema.INNODB_TABLESPACES_ENCRYPTION 1
INNODB_TRX information_schema.INNODB_TRX 1
KEY_CACHES information_schema.KEY_CACHES 1
KEY_COLUMN_USAGE information_schema.KEY_COLUMN_USAGE 1
OPTIMIZER_COSTS information_schema.OPTIMIZER_COSTS 1
OPTIMIZER_TRACE information_schema.OPTIMIZER_TRACE 1
PARAMETERS information_schema.PARAMETERS 1
PARTITIONS information_schema.PARTITIONS 1
@@ -359,6 +363,7 @@ Database: information_schema
| KEYWORDS |
| KEY_CACHES |
| KEY_COLUMN_USAGE |
| OPTIMIZER_COSTS |
| OPTIMIZER_TRACE |
| PARAMETERS |
| PARTITIONS |
@@ -430,6 +435,7 @@ Database: INFORMATION_SCHEMA
| KEYWORDS |
| KEY_CACHES |
| KEY_COLUMN_USAGE |
| OPTIMIZER_COSTS |
| OPTIMIZER_TRACE |
| PARAMETERS |
| PARTITIONS |
@@ -463,5 +469,5 @@ Wildcard: inf_rmation_schema
| information_schema |
SELECT table_schema, count(*) FROM information_schema.TABLES WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', 'mysqltest') GROUP BY TABLE_SCHEMA;
table_schema count(*)
information_schema 66
information_schema 67
mysql 31

6
mysql-test/main/intersect.result
View File

@@ -462,7 +462,8 @@ ANALYZE
"buffer_type": "flat",
"buffer_size": "256Kb",
"join_type": "BNL",
"r_filtered": 100
"r_filtered": 100,
"r_unpack_time_ms": "REPLACED"
}
}
]
@@ -560,7 +561,8 @@ ANALYZE
"buffer_type": "flat",
"buffer_size": "256Kb",
"join_type": "BNL",
"r_filtered": 100
"r_filtered": 100,
"r_unpack_time_ms": "REPLACED"
}
}
]

6
mysql-test/main/intersect_all.result
View File

@@ -493,7 +493,8 @@ ANALYZE
"buffer_type": "flat",
"buffer_size": "65",
"join_type": "BNL",
"r_filtered": 100
"r_filtered": 100,
"r_unpack_time_ms": "REPLACED"
}
}
]
@@ -591,7 +592,8 @@ ANALYZE
"buffer_type": "flat",
"buffer_size": "65",
"join_type": "BNL",
"r_filtered": 100
"r_filtered": 100,
"r_unpack_time_ms": "REPLACED"
}
}
]

6
mysql-test/main/key.result
View File

@@ -631,19 +631,19 @@ id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t1 ALL NULL NULL NULL NULL 6 Using temporary; Using filesort
SHOW STATUS LIKE 'Last_query_cost';
Variable_name Value
Last_query_cost 8.506592
Last_query_cost 0.014749
EXPLAIN SELECT a, SUM( b ) FROM t1 USE INDEX( a ) GROUP BY a;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t1 ALL NULL NULL NULL NULL 6 Using temporary; Using filesort
SHOW STATUS LIKE 'Last_query_cost';
Variable_name Value
Last_query_cost 8.506592
Last_query_cost 0.014749
EXPLAIN SELECT a, SUM( b ) FROM t1 FORCE INDEX( a ) GROUP BY a;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t1 index NULL a 5 NULL 6
SHOW STATUS LIKE 'Last_query_cost';
Variable_name Value
Last_query_cost 8.506592
Last_query_cost 0.014749
DROP TABLE t1;
#
# MDEV-21480: Unique key using ref access though eq_ref access can be used

10
mysql-test/main/myisam.result
View File

@@ -348,11 +348,11 @@ t1 1 c_2 2 a A 5 NULL NULL BTREE NO
explain select * from t1,t2 where t1.a=t2.a;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t2 ALL a NULL NULL NULL 2
1 SIMPLE t1 ALL a NULL NULL NULL 5 Using where; Using join buffer (flat, BNL join)
1 SIMPLE t1 ref a a 4 test.t2.a 3
explain select * from t1,t2 force index(a) where t1.a=t2.a;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t2 ALL a NULL NULL NULL 2
1 SIMPLE t1 ALL a NULL NULL NULL 5 Using where; Using join buffer (flat, BNL join)
1 SIMPLE t1 ref a a 4 test.t2.a 3
explain select * from t1 force index(a),t2 force index(a) where t1.a=t2.a;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t2 ALL a NULL NULL NULL 2
@@ -388,10 +388,10 @@ t1 1 c_2 2 a A 5 NULL NULL BTREE NO
explain select * from t1,t2 force index(c) where t1.a=t2.a;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t2 ALL NULL NULL NULL NULL 2
1 SIMPLE t1 ALL a NULL NULL NULL 5 Using where; Using join buffer (flat, BNL join)
1 SIMPLE t1 ref a a 4 test.t2.a 3
explain select * from t1 where a=0 or a=2;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t1 ALL a NULL NULL NULL 5 Using where
1 SIMPLE t1 range a a 4 NULL 5 Using index condition
explain select * from t1 force index (a) where a=0 or a=2;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t1 range a a 4 NULL 5 Using index condition
@@ -640,7 +640,7 @@ create table t1 ( a tinytext, b char(1), index idx (a(1),b) );
insert into t1 values (null,''), (null,'');
explain select count(*) from t1 where a is null;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t1 ALL idx NULL NULL NULL 2 Using where
1 SIMPLE t1 ref idx idx 4 const 2 Using where
select count(*) from t1 where a is null;
count(*)
2

45
mysql-test/main/mysqld--help.result
View File

@@ -720,11 +720,15 @@ The following specify which files/extra groups are read (specified before remain
max_connections*5 or max_connections + table_cache*2
(whichever is larger) number of file descriptors
(Automatically configured unless set explicitly)
--optimizer-cache-hit-ratio=#
Expected hit rate of the row and index cache in storage
engines. The value should be an integer between 0 and 99,
where 0 means cache is empty and 99 means that value is
almost always in the cache.
--optimizer-disk-read-cost=#
Cost of reading a block of IO_SIZE (4096) from a disk (in
usec).
--optimizer-disk-read-ratio=#
Chance that we have to do a disk read to find a row or
index entry from the engine cache
(cache_misses/total_cache_requests). 0.0 means that
everything is cached and 1.0 means that nothing is
expected to be in the engine cache.
--optimizer-extra-pruning-depth=#
If the optimizer needs to enumerate join prefix of this
size or larger, then it will try agressively prune away
@@ -737,6 +741,8 @@ The following specify which files/extra groups are read (specified before remain
--optimizer-key-copy-cost=#
Cost of finding the next key in the engine and copying it
to the SQL layer.
--optimizer-key-lookup-cost=#
Cost for finding a key based on a key value
--optimizer-key-next-find-cost=#
Cost of finding the next key and rowid when using
filters.
@@ -753,6 +759,14 @@ The following specify which files/extra groups are read (specified before remain
--optimizer-row-copy-cost=#
Cost of copying a row from the engine or the join cache
to the SQL layer.
--optimizer-row-lookup-cost=#
Cost of finding a row based on a rowid or a clustered
key.
--optimizer-row-next-find-cost=#
Cost of finding the next row when scanning the table.
--optimizer-scan-setup-cost=#
Extra cost added to TABLE and INDEX scans to get
optimizer to prefer index lookups.
--optimizer-search-depth=#
Maximum depth of search performed by the query optimizer.
Values larger than the number of relations in a query
@@ -807,6 +821,8 @@ The following specify which files/extra groups are read (specified before remain
record samples
--optimizer-where-cost=#
Cost of checking the row against the WHERE clause.
Increasing this will have the optimizer to prefer plans
with less row combinations.
--performance-schema
Enable the performance schema.
--performance-schema-accounts-size=#
@@ -1719,22 +1735,27 @@ old-alter-table DEFAULT
old-mode UTF8_IS_UTF8MB3
old-passwords FALSE
old-style-user-limits FALSE
optimizer-cache-hit-ratio 50
optimizer-disk-read-cost 0.01024
optimizer-disk-read-ratio 0.02
optimizer-extra-pruning-depth 8
optimizer-index-block-copy-cost 0.2
optimizer-key-compare-cost 0.05
optimizer-key-copy-cost 0.025
optimizer-key-next-find-cost 0.0125
optimizer-index-block-copy-cost 3.56e-05
optimizer-key-compare-cost 1.1361e-05
optimizer-key-copy-cost 1.5685e-05
optimizer-key-lookup-cost 0.000435777
optimizer-key-next-find-cost 8.2347e-05
optimizer-max-sel-arg-weight 32000
optimizer-prune-level 2
optimizer-row-copy-cost 0.05
optimizer-row-copy-cost 6.0866e-05
optimizer-row-lookup-cost 0.000130839
optimizer-row-next-find-cost 4.5916e-05
optimizer-scan-setup-cost 0.01
optimizer-search-depth 62
optimizer-selectivity-sampling-limit 100
optimizer-switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on
optimizer-trace
optimizer-trace-max-mem-size 1048576
optimizer-use-condition-selectivity 4
optimizer-where-cost 0.2
optimizer-where-cost 3.2e-05
performance-schema FALSE
performance-schema-accounts-size -1
performance-schema-consumer-events-stages-current FALSE

36
mysql-test/main/opt_trace_security.result
View File

@@ -80,8 +80,8 @@ select * from db1.t1 {
"table": "t1",
"table_scan": {
"rows": 3,
"read_cost": 1.002563477,
"read_and_compare_cost": 1.752563477
"read_cost": 0.010373215,
"read_and_compare_cost": 0.010469215
}
}
]
@@ -101,18 +101,18 @@ select * from db1.t1 {
{
"access_type": "scan",
"rows": 3,
"rows_after_scan": 3,
"rows_after_filter": 3,
"cost": 1.752563477,
"rows_out": 3,
"cost": 0.010469215,
"index_only": false,
"chosen": true
}
],
"chosen_access_method": {
"type": "scan",
"records_read": 3,
"records_out": 3,
"cost": 1.752563477,
"rows_read": 3,
"rows_out": 3,
"cost": 0.010469215,
"uses_join_buffering": false
}
}
@@ -123,14 +123,14 @@ select * from db1.t1 {
"plan_prefix": [],
"table": "t1",
"rows_for_plan": 3,
"cost_for_plan": 1.752563477
"cost_for_plan": 0.010469215
}
]
},
{
"best_join_order": ["t1"],
"rows": 3,
"cost": 1.752563477
"cost": 0.010469215
},
{
"attaching_conditions_to_tables": {
@@ -219,8 +219,8 @@ select * from db1.v1 {
"table": "t1",
"table_scan": {
"rows": 3,
"read_cost": 1.002563477,
"read_and_compare_cost": 1.752563477
"read_cost": 0.010373215,
"read_and_compare_cost": 0.010469215
}
}
]
@@ -240,18 +240,18 @@ select * from db1.v1 {
{
"access_type": "scan",
"rows": 3,
"rows_after_scan": 3,
"rows_after_filter": 3,
"cost": 1.752563477,
"rows_out": 3,
"cost": 0.010469215,
"index_only": false,
"chosen": true
}
],
"chosen_access_method": {
"type": "scan",
"records_read": 3,
"records_out": 3,
"cost": 1.752563477,
"rows_read": 3,
"rows_out": 3,
"cost": 0.010469215,
"uses_join_buffering": false
}
}
@@ -262,14 +262,14 @@ select * from db1.v1 {
"plan_prefix": [],
"table": "t1",
"rows_for_plan": 3,
"cost_for_plan": 1.752563477
"cost_for_plan": 0.010469215
}
]
},
{
"best_join_order": ["t1"],
"rows": 3,
"cost": 1.752563477
"cost": 0.010469215
},
{
"attaching_conditions_to_tables": {

2
mysql-test/main/opt_trace_ucs2.result
View File

@@ -38,7 +38,7 @@ JSON_DETAILED(JSON_EXTRACT(trace, '$**.analyzing_range_alternatives'))
"using_mrr": false,
"index_only": false,
"rows": 2,
"cost": 2.022733708,
"cost": 0.003717837,
"chosen": true
}
],

437
mysql-test/main/rowid_filter_innodb.result
View File

@@ -245,7 +245,7 @@ EXPLAIN
"key_length": "4",
"used_key_parts": ["l_shipDATE"],
"rows": 510,
"filtered": 100,
"filtered": 10.07493782,
"index_condition": "lineitem.l_shipDATE between '1997-01-01' and '1997-06-30'",
"attached_condition": "lineitem.l_quantity > 45"
}
@@ -257,7 +257,7 @@ set statement optimizer_switch='rowid_filter=off' for ANALYZE SELECT l_orderkey,
WHERE l_shipdate BETWEEN '1997-01-01' AND '1997-06-30' AND
l_quantity > 45;
id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra
1 SIMPLE lineitem range i_l_shipdate,i_l_quantity i_l_shipdate 4 NULL 510 510.00 100.00 11.76 Using index condition; Using where
1 SIMPLE lineitem range i_l_shipdate,i_l_quantity i_l_shipdate 4 NULL 510 510.00 10.07 11.76 Using index condition; Using where
set statement optimizer_switch='rowid_filter=off' for ANALYZE FORMAT=JSON SELECT l_orderkey, l_linenumber, l_shipdate, l_quantity FROM lineitem
WHERE l_shipdate BETWEEN '1997-01-01' AND '1997-06-30' AND
l_quantity > 45;
@@ -284,7 +284,7 @@ ANALYZE
"r_rows": 510,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": 100,
"filtered": 10.07493782,
"r_filtered": 11.76470588,
"index_condition": "lineitem.l_shipDATE between '1997-01-01' and '1997-06-30'",
"attached_condition": "lineitem.l_quantity > 45"
@@ -362,8 +362,8 @@ FROM orders JOIN lineitem ON o_orderkey=l_orderkey
WHERE l_shipdate BETWEEN '1997-01-01' AND '1997-01-31' AND
o_totalprice between 200000 and 230000;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE lineitem range PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity i_l_shipdate 4 NULL 98 Using where; Using index
1 SIMPLE orders eq_ref PRIMARY,i_o_totalprice PRIMARY 4 dbt3_s001.lineitem.l_orderkey 1 Using where
1 SIMPLE orders range PRIMARY,i_o_totalprice i_o_totalprice 9 NULL 71 Using where; Using index
1 SIMPLE lineitem ref PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity PRIMARY 4 dbt3_s001.orders.o_orderkey 4 Using where
set statement optimizer_switch='rowid_filter=on' for EXPLAIN FORMAT=JSON SELECT o_orderkey, l_linenumber, l_shipdate, o_totalprice
FROM orders JOIN lineitem ON o_orderkey=l_orderkey
WHERE l_shipdate BETWEEN '1997-01-01' AND '1997-01-31' AND
@@ -375,35 +375,35 @@ EXPLAIN
"nested_loop": [
{
"table": {
"table_name": "lineitem",
"table_name": "orders",
"access_type": "range",
"possible_keys": ["PRIMARY", "i_o_totalprice"],
"key": "i_o_totalprice",
"key_length": "9",
"used_key_parts": ["o_totalprice"],
"rows": 71,
"filtered": 100,
"attached_condition": "orders.o_totalprice between 200000 and 230000",
"using_index": true
}
},
{
"table": {
"table_name": "lineitem",
"access_type": "ref",
"possible_keys": [
"PRIMARY",
"i_l_shipdate",
"i_l_orderkey",
"i_l_orderkey_quantity"
],
"key": "i_l_shipdate",
"key_length": "4",
"used_key_parts": ["l_shipDATE"],
"rows": 98,
"filtered": 100,
"attached_condition": "lineitem.l_shipDATE between '1997-01-01' and '1997-01-31'",
"using_index": true
}
},
{
"table": {
"table_name": "orders",
"access_type": "eq_ref",
"possible_keys": ["PRIMARY", "i_o_totalprice"],
"key": "PRIMARY",
"key_length": "4",
"used_key_parts": ["o_orderkey"],
"ref": ["dbt3_s001.lineitem.l_orderkey"],
"rows": 1,
"filtered": 4.733333111,
"attached_condition": "orders.o_totalprice between 200000 and 230000"
"used_key_parts": ["l_orderkey"],
"ref": ["dbt3_s001.orders.o_orderkey"],
"rows": 4,
"filtered": 1.633319736,
"attached_condition": "lineitem.l_shipDATE between '1997-01-01' and '1997-01-31'"
}
}
]
@@ -414,8 +414,8 @@ FROM orders JOIN lineitem ON o_orderkey=l_orderkey
WHERE l_shipdate BETWEEN '1997-01-01' AND '1997-01-31' AND
o_totalprice between 200000 and 230000;
id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra
1 SIMPLE lineitem range PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity i_l_shipdate 4 NULL 98 98.00 100.00 100.00 Using where; Using index
1 SIMPLE orders eq_ref PRIMARY,i_o_totalprice PRIMARY 4 dbt3_s001.lineitem.l_orderkey 1 1.00 4.73 11.22 Using where
1 SIMPLE orders range PRIMARY,i_o_totalprice i_o_totalprice 9 NULL 71 71.00 100.00 100.00 Using where; Using index
1 SIMPLE lineitem ref PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity PRIMARY 4 dbt3_s001.orders.o_orderkey 4 6.70 1.63 2.31 Using where
set statement optimizer_switch='rowid_filter=on' for ANALYZE FORMAT=JSON SELECT o_orderkey, l_linenumber, l_shipdate, o_totalprice
FROM orders JOIN lineitem ON o_orderkey=l_orderkey
WHERE l_shipdate BETWEEN '1997-01-01' AND '1997-01-31' AND
@@ -432,45 +432,45 @@ ANALYZE
"nested_loop": [
{
"table": {
"table_name": "lineitem",
"table_name": "orders",
"access_type": "range",
"possible_keys": ["PRIMARY", "i_o_totalprice"],
"key": "i_o_totalprice",
"key_length": "9",
"used_key_parts": ["o_totalprice"],
"r_loops": 1,
"rows": 71,
"r_rows": 71,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": 100,
"r_filtered": 100,
"attached_condition": "orders.o_totalprice between 200000 and 230000",
"using_index": true
}
},
{
"table": {
"table_name": "lineitem",
"access_type": "ref",
"possible_keys": [
"PRIMARY",
"i_l_shipdate",
"i_l_orderkey",
"i_l_orderkey_quantity"
],
"key": "i_l_shipdate",
"key_length": "4",
"used_key_parts": ["l_shipDATE"],
"r_loops": 1,
"rows": 98,
"r_rows": 98,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": 100,
"r_filtered": 100,
"attached_condition": "lineitem.l_shipDATE between '1997-01-01' and '1997-01-31'",
"using_index": true
}
},
{
"table": {
"table_name": "orders",
"access_type": "eq_ref",
"possible_keys": ["PRIMARY", "i_o_totalprice"],
"key": "PRIMARY",
"key_length": "4",
"used_key_parts": ["o_orderkey"],
"ref": ["dbt3_s001.lineitem.l_orderkey"],
"r_loops": 98,
"rows": 1,
"r_rows": 1,
"used_key_parts": ["l_orderkey"],
"ref": ["dbt3_s001.orders.o_orderkey"],
"r_loops": 71,
"rows": 4,
"r_rows": 6.704225352,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": 4.733333111,
"r_filtered": 11.2244898,
"attached_condition": "orders.o_totalprice between 200000 and 230000"
"filtered": 1.633319736,
"r_filtered": 2.31092437,
"attached_condition": "lineitem.l_shipDATE between '1997-01-01' and '1997-01-31'"
}
}
]
@@ -497,8 +497,8 @@ FROM orders JOIN lineitem ON o_orderkey=l_orderkey
WHERE l_shipdate BETWEEN '1997-01-01' AND '1997-01-31' AND
o_totalprice between 200000 and 230000;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE lineitem range PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity i_l_shipdate 4 NULL 98 Using where; Using index
1 SIMPLE orders eq_ref PRIMARY,i_o_totalprice PRIMARY 4 dbt3_s001.lineitem.l_orderkey 1 Using where
1 SIMPLE orders range PRIMARY,i_o_totalprice i_o_totalprice 9 NULL 71 Using where; Using index
1 SIMPLE lineitem ref PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity PRIMARY 4 dbt3_s001.orders.o_orderkey 4 Using where
set statement optimizer_switch='rowid_filter=off' for EXPLAIN FORMAT=JSON SELECT o_orderkey, l_linenumber, l_shipdate, o_totalprice
FROM orders JOIN lineitem ON o_orderkey=l_orderkey
WHERE l_shipdate BETWEEN '1997-01-01' AND '1997-01-31' AND
@@ -510,35 +510,35 @@ EXPLAIN
"nested_loop": [
{
"table": {
"table_name": "lineitem",
"table_name": "orders",
"access_type": "range",
"possible_keys": ["PRIMARY", "i_o_totalprice"],
"key": "i_o_totalprice",
"key_length": "9",
"used_key_parts": ["o_totalprice"],
"rows": 71,
"filtered": 100,
"attached_condition": "orders.o_totalprice between 200000 and 230000",
"using_index": true
}
},
{
"table": {
"table_name": "lineitem",
"access_type": "ref",
"possible_keys": [
"PRIMARY",
"i_l_shipdate",
"i_l_orderkey",
"i_l_orderkey_quantity"
],
"key": "i_l_shipdate",
"key_length": "4",
"used_key_parts": ["l_shipDATE"],
"rows": 98,
"filtered": 100,
"attached_condition": "lineitem.l_shipDATE between '1997-01-01' and '1997-01-31'",
"using_index": true
}
},
{
"table": {
"table_name": "orders",
"access_type": "eq_ref",
"possible_keys": ["PRIMARY", "i_o_totalprice"],
"key": "PRIMARY",
"key_length": "4",
"used_key_parts": ["o_orderkey"],
"ref": ["dbt3_s001.lineitem.l_orderkey"],
"rows": 1,
"filtered": 4.733333111,
"attached_condition": "orders.o_totalprice between 200000 and 230000"
"used_key_parts": ["l_orderkey"],
"ref": ["dbt3_s001.orders.o_orderkey"],
"rows": 4,
"filtered": 1.633319736,
"attached_condition": "lineitem.l_shipDATE between '1997-01-01' and '1997-01-31'"
}
}
]
@@ -549,8 +549,8 @@ FROM orders JOIN lineitem ON o_orderkey=l_orderkey
WHERE l_shipdate BETWEEN '1997-01-01' AND '1997-01-31' AND
o_totalprice between 200000 and 230000;
id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra
1 SIMPLE lineitem range PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity i_l_shipdate 4 NULL 98 98.00 100.00 100.00 Using where; Using index
1 SIMPLE orders eq_ref PRIMARY,i_o_totalprice PRIMARY 4 dbt3_s001.lineitem.l_orderkey 1 1.00 4.73 11.22 Using where
1 SIMPLE orders range PRIMARY,i_o_totalprice i_o_totalprice 9 NULL 71 71.00 100.00 100.00 Using where; Using index
1 SIMPLE lineitem ref PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity PRIMARY 4 dbt3_s001.orders.o_orderkey 4 6.70 1.63 2.31 Using where
set statement optimizer_switch='rowid_filter=off' for ANALYZE FORMAT=JSON SELECT o_orderkey, l_linenumber, l_shipdate, o_totalprice
FROM orders JOIN lineitem ON o_orderkey=l_orderkey
WHERE l_shipdate BETWEEN '1997-01-01' AND '1997-01-31' AND
@@ -567,45 +567,45 @@ ANALYZE
"nested_loop": [
{
"table": {
"table_name": "lineitem",
"table_name": "orders",
"access_type": "range",
"possible_keys": ["PRIMARY", "i_o_totalprice"],
"key": "i_o_totalprice",
"key_length": "9",
"used_key_parts": ["o_totalprice"],
"r_loops": 1,
"rows": 71,
"r_rows": 71,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": 100,
"r_filtered": 100,
"attached_condition": "orders.o_totalprice between 200000 and 230000",
"using_index": true
}
},
{
"table": {
"table_name": "lineitem",
"access_type": "ref",
"possible_keys": [
"PRIMARY",
"i_l_shipdate",
"i_l_orderkey",
"i_l_orderkey_quantity"
],
"key": "i_l_shipdate",
"key_length": "4",
"used_key_parts": ["l_shipDATE"],
"r_loops": 1,
"rows": 98,
"r_rows": 98,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": 100,
"r_filtered": 100,
"attached_condition": "lineitem.l_shipDATE between '1997-01-01' and '1997-01-31'",
"using_index": true
}
},
{
"table": {
"table_name": "orders",
"access_type": "eq_ref",
"possible_keys": ["PRIMARY", "i_o_totalprice"],
"key": "PRIMARY",
"key_length": "4",
"used_key_parts": ["o_orderkey"],
"ref": ["dbt3_s001.lineitem.l_orderkey"],
"r_loops": 98,
"rows": 1,
"r_rows": 1,
"used_key_parts": ["l_orderkey"],
"ref": ["dbt3_s001.orders.o_orderkey"],
"r_loops": 71,
"rows": 4,
"r_rows": 6.704225352,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": 4.733333111,
"r_filtered": 11.2244898,
"attached_condition": "orders.o_totalprice between 200000 and 230000"
"filtered": 1.633319736,
"r_filtered": 2.31092437,
"attached_condition": "lineitem.l_shipDATE between '1997-01-01' and '1997-01-31'"
}
}
]
@@ -634,7 +634,7 @@ l_quantity > 45 AND
o_totalprice between 180000 and 230000;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE orders range PRIMARY,i_o_totalprice i_o_totalprice 9 NULL 144 Using where; Using index
1 SIMPLE lineitem ref|filter PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity,i_l_quantity i_l_orderkey|i_l_shipdate 4|4 dbt3_s001.orders.o_orderkey 4 (8%) Using where; Using rowid filter
1 SIMPLE lineitem ref PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity,i_l_quantity PRIMARY 4 dbt3_s001.orders.o_orderkey 4 Using where
set statement optimizer_switch='rowid_filter=on' for EXPLAIN FORMAT=JSON SELECT o_orderkey, l_linenumber, l_shipdate, l_quantity, o_totalprice
FROM orders JOIN lineitem ON o_orderkey=l_orderkey
WHERE l_shipdate BETWEEN '1997-01-01' AND '1997-06-30' AND
@@ -670,20 +670,12 @@ EXPLAIN
"i_l_orderkey_quantity",
"i_l_quantity"
],
"key": "i_l_orderkey",
"key": "PRIMARY",
"key_length": "4",
"used_key_parts": ["l_orderkey"],
"ref": ["dbt3_s001.orders.o_orderkey"],
"rowid_filter": {
"range": {
"key": "i_l_shipdate",
"used_key_parts": ["l_shipDATE"]
},
"rows": 510,
"selectivity_pct": 8.492922565
},
"rows": 4,
"filtered": 0.855656624,
"filtered": 0.856362581,
"attached_condition": "lineitem.l_shipDATE between '1997-01-01' and '1997-06-30' and lineitem.l_quantity > 45"
}
}
@@ -697,7 +689,7 @@ l_quantity > 45 AND
o_totalprice between 180000 and 230000;
id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra
1 SIMPLE orders range PRIMARY,i_o_totalprice i_o_totalprice 9 NULL 144 144.00 100.00 100.00 Using where; Using index
1 SIMPLE lineitem ref|filter PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity,i_l_quantity i_l_orderkey|i_l_shipdate 4|4 dbt3_s001.orders.o_orderkey 4 (8%) 0.54 (8%) 0.86 20.51 Using where; Using rowid filter
1 SIMPLE lineitem ref PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity,i_l_quantity PRIMARY 4 dbt3_s001.orders.o_orderkey 4 6.62 0.86 1.68 Using where
set statement optimizer_switch='rowid_filter=on' for ANALYZE FORMAT=JSON SELECT o_orderkey, l_linenumber, l_shipdate, l_quantity, o_totalprice
FROM orders JOIN lineitem ON o_orderkey=l_orderkey
WHERE l_shipdate BETWEEN '1997-01-01' AND '1997-06-30' AND
@@ -743,30 +735,17 @@ ANALYZE
"i_l_orderkey_quantity",
"i_l_quantity"
],
"key": "i_l_orderkey",
"key": "PRIMARY",
"key_length": "4",
"used_key_parts": ["l_orderkey"],
"ref": ["dbt3_s001.orders.o_orderkey"],
"rowid_filter": {
"range": {
"key": "i_l_shipdate",
"used_key_parts": ["l_shipDATE"]
},
"rows": 510,
"selectivity_pct": 8.492922565,
"r_rows": 510,
"r_lookups": 954,
"r_selectivity_pct": 8.176100629,
"r_buffer_size": "REPLACED",
"r_filling_time_ms": "REPLACED"
},
"r_loops": 144,
"rows": 4,
"r_rows": 0.541666667,
"r_rows": 6.625,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": 0.855656624,
"r_filtered": 20.51282051,
"filtered": 0.856362581,
"r_filtered": 1.677148847,
"attached_condition": "lineitem.l_shipDATE between '1997-01-01' and '1997-06-30' and lineitem.l_quantity > 45"
}
}
@@ -843,7 +822,7 @@ EXPLAIN
"used_key_parts": ["l_orderkey"],
"ref": ["dbt3_s001.orders.o_orderkey"],
"rows": 4,
"filtered": 0.855656624,
"filtered": 0.856362581,
"attached_condition": "lineitem.l_shipDATE between '1997-01-01' and '1997-06-30' and lineitem.l_quantity > 45"
}
}
@@ -912,7 +891,7 @@ ANALYZE
"r_rows": 6.625,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": 0.855656624,
"filtered": 0.856362581,
"r_filtered": 1.677148847,
"attached_condition": "lineitem.l_shipDATE between '1997-01-01' and '1997-06-30' and lineitem.l_quantity > 45"
}
@@ -995,7 +974,7 @@ EXPLAIN
"used_key_parts": ["l_orderkey"],
"ref": ["dbt3_s001.orders.o_orderkey"],
"rows": 4,
"filtered": 8.492922783,
"filtered": 8.499929428,
"attached_condition": "lineitem.l_shipDATE between '1997-01-01' and '1997-06-30'"
}
}
@@ -1008,7 +987,7 @@ WHERE l_shipdate BETWEEN '1997-01-01' AND '1997-06-30' AND
o_totalprice between 200000 and 230000;
id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra
1 SIMPLE orders range PRIMARY,i_o_totalprice i_o_totalprice 9 NULL 71 71.00 100.00 100.00 Using where; Using index
1 SIMPLE lineitem ref PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity PRIMARY 4 dbt3_s001.orders.o_orderkey 4 6.70 8.49 7.77 Using where
1 SIMPLE lineitem ref PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity PRIMARY 4 dbt3_s001.orders.o_orderkey 4 6.70 8.50 7.77 Using where
set statement optimizer_switch='rowid_filter=on' for ANALYZE FORMAT=JSON SELECT o_orderkey, l_linenumber, l_shipdate, o_totalprice
FROM orders JOIN lineitem ON o_orderkey=l_orderkey
WHERE l_shipdate BETWEEN '1997-01-01' AND '1997-06-30' AND
@@ -1061,7 +1040,7 @@ ANALYZE
"r_rows": 6.704225352,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": 8.492922783,
"filtered": 8.499929428,
"r_filtered": 7.773109244,
"attached_condition": "lineitem.l_shipDATE between '1997-01-01' and '1997-06-30'"
}
@@ -1156,7 +1135,7 @@ EXPLAIN
"used_key_parts": ["l_orderkey"],
"ref": ["dbt3_s001.orders.o_orderkey"],
"rows": 4,
"filtered": 8.492922783,
"filtered": 8.499929428,
"attached_condition": "lineitem.l_shipDATE between '1997-01-01' and '1997-06-30'"
}
}
@@ -1169,7 +1148,7 @@ WHERE l_shipdate BETWEEN '1997-01-01' AND '1997-06-30' AND
o_totalprice between 200000 and 230000;
id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra
1 SIMPLE orders range PRIMARY,i_o_totalprice i_o_totalprice 9 NULL 71 71.00 100.00 100.00 Using where; Using index
1 SIMPLE lineitem ref PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity PRIMARY 4 dbt3_s001.orders.o_orderkey 4 6.70 8.49 7.77 Using where
1 SIMPLE lineitem ref PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity PRIMARY 4 dbt3_s001.orders.o_orderkey 4 6.70 8.50 7.77 Using where
set statement optimizer_switch='rowid_filter=off' for ANALYZE FORMAT=JSON SELECT o_orderkey, l_linenumber, l_shipdate, o_totalprice
FROM orders JOIN lineitem ON o_orderkey=l_orderkey
WHERE l_shipdate BETWEEN '1997-01-01' AND '1997-06-30' AND
@@ -1222,7 +1201,7 @@ ANALYZE
"r_rows": 6.704225352,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": 8.492922783,
"filtered": 8.499929428,
"r_filtered": 7.773109244,
"attached_condition": "lineitem.l_shipDATE between '1997-01-01' and '1997-06-30'"
}
@@ -1314,7 +1293,7 @@ EXPLAIN
"key_length": "4",
"used_key_parts": ["l_receiptDATE"],
"rows": 18,
"filtered": 100,
"filtered": 0.566194832,
"index_condition": "lineitem.l_receiptDATE between '1996-10-05' and '1996-10-10'",
"attached_condition": "lineitem.l_shipDATE between '1996-10-01' and '1996-10-10'"
}
@@ -1343,7 +1322,7 @@ l_shipdate BETWEEN '1996-10-01' AND '1996-10-10' AND
l_receiptdate BETWEEN '1996-10-05' AND '1996-10-10' AND
o_totalprice BETWEEN 200000 AND 250000;
id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra
1 SIMPLE lineitem range PRIMARY,i_l_shipdate,i_l_receiptdate,i_l_orderkey,i_l_orderkey_quantity i_l_receiptdate 4 NULL 18 18.00 100.00 38.89 Using index condition; Using where
1 SIMPLE lineitem range PRIMARY,i_l_shipdate,i_l_receiptdate,i_l_orderkey,i_l_orderkey_quantity i_l_receiptdate 4 NULL 18 18.00 0.57 38.89 Using index condition; Using where
1 SIMPLE orders eq_ref PRIMARY,i_o_totalprice PRIMARY 4 dbt3_s001.lineitem.l_orderkey 1 1.00 5.67 14.29 Using where
set statement optimizer_switch='rowid_filter=on' for ANALYZE FORMAT=JSON SELECT l_shipdate, l_receiptdate, o_totalprice
FROM orders, lineitem
@@ -1380,7 +1359,7 @@ ANALYZE
"r_rows": 18,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": 100,
"filtered": 0.566194832,
"r_filtered": 38.88888889,
"index_condition": "lineitem.l_receiptDATE between '1996-10-05' and '1996-10-10'",
"attached_condition": "lineitem.l_shipDATE between '1996-10-01' and '1996-10-10'"
@@ -1451,7 +1430,7 @@ EXPLAIN
"key_length": "4",
"used_key_parts": ["l_receiptDATE"],
"rows": 18,
"filtered": 100,
"filtered": 0.566194832,
"index_condition": "lineitem.l_receiptDATE between '1996-10-05' and '1996-10-10'",
"attached_condition": "lineitem.l_shipDATE between '1996-10-01' and '1996-10-10'"
}
@@ -1480,7 +1459,7 @@ l_shipdate BETWEEN '1996-10-01' AND '1996-10-10' AND
l_receiptdate BETWEEN '1996-10-05' AND '1996-10-10' AND
o_totalprice BETWEEN 200000 AND 250000;
id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra
1 SIMPLE lineitem range PRIMARY,i_l_shipdate,i_l_receiptdate,i_l_orderkey,i_l_orderkey_quantity i_l_receiptdate 4 NULL 18 18.00 100.00 38.89 Using index condition; Using where
1 SIMPLE lineitem range PRIMARY,i_l_shipdate,i_l_receiptdate,i_l_orderkey,i_l_orderkey_quantity i_l_receiptdate 4 NULL 18 18.00 0.57 38.89 Using index condition; Using where
1 SIMPLE orders eq_ref PRIMARY,i_o_totalprice PRIMARY 4 dbt3_s001.lineitem.l_orderkey 1 1.00 5.67 14.29 Using where
set statement optimizer_switch='rowid_filter=off' for ANALYZE FORMAT=JSON SELECT l_shipdate, l_receiptdate, o_totalprice
FROM orders, lineitem
@@ -1517,7 +1496,7 @@ ANALYZE
"r_rows": 18,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": 100,
"filtered": 0.566194832,
"r_filtered": 38.88888889,
"index_condition": "lineitem.l_receiptDATE between '1996-10-05' and '1996-10-10'",
"attached_condition": "lineitem.l_shipDATE between '1996-10-01' and '1996-10-10'"
@@ -1568,7 +1547,7 @@ o_totalprice BETWEEN 200000 AND 220000 AND
l_shipdate BETWEEN '1996-10-01' AND '1996-12-01';
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE orders range PRIMARY,i_o_totalprice,i_o_totaldiscount i_o_totaldiscount 9 NULL 41 Using index condition; Using where
1 SIMPLE lineitem ref|filter PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity i_l_orderkey|i_l_shipdate 4|4 dbt3_s001.orders.o_orderkey 4 (3%) Using where; Using rowid filter
1 SIMPLE lineitem ref PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity PRIMARY 4 dbt3_s001.orders.o_orderkey 4 Using where
set statement optimizer_switch='rowid_filter=on' for EXPLAIN FORMAT=JSON SELECT o_totaldiscount, o_totalprice, l_shipdate
FROM orders, lineitem
WHERE o_orderkey=l_orderkey AND
@@ -1589,7 +1568,7 @@ EXPLAIN
"key_length": "9",
"used_key_parts": ["o_totaldiscount"],
"rows": 41,
"filtered": 100,
"filtered": 3.333333254,
"index_condition": "orders.o_totaldiscount between 18000 and 20000",
"attached_condition": "orders.o_totalprice between 200000 and 220000"
}
@@ -1604,20 +1583,12 @@ EXPLAIN
"i_l_orderkey",
"i_l_orderkey_quantity"
],
"key": "i_l_orderkey",
"key": "PRIMARY",
"key_length": "4",
"used_key_parts": ["l_orderkey"],
"ref": ["dbt3_s001.orders.o_orderkey"],
"rowid_filter": {
"range": {
"key": "i_l_shipdate",
"used_key_parts": ["l_shipDATE"]
},
"rows": 183,
"selectivity_pct": 3.04746045
},
"rows": 4,
"filtered": 3.047460556,
"filtered": 3.04997468,
"attached_condition": "lineitem.l_shipDATE between '1996-10-01' and '1996-12-01'"
}
}
@@ -1631,8 +1602,8 @@ o_totaldiscount BETWEEN 18000 AND 20000 AND
o_totalprice BETWEEN 200000 AND 220000 AND
l_shipdate BETWEEN '1996-10-01' AND '1996-12-01';
id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra
1 SIMPLE orders range PRIMARY,i_o_totalprice,i_o_totaldiscount i_o_totaldiscount 9 NULL 41 41.00 100.00 2.44 Using index condition; Using where
1 SIMPLE lineitem ref|filter PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity i_l_orderkey|i_l_shipdate 4|4 dbt3_s001.orders.o_orderkey 4 (3%) 4.00 (66%) 3.05 100.00 Using where; Using rowid filter
1 SIMPLE orders range PRIMARY,i_o_totalprice,i_o_totaldiscount i_o_totaldiscount 9 NULL 41 41.00 3.33 2.44 Using index condition; Using where
1 SIMPLE lineitem ref PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity PRIMARY 4 dbt3_s001.orders.o_orderkey 4 6.00 3.05 66.67 Using where
set statement optimizer_switch='rowid_filter=on' for ANALYZE FORMAT=JSON SELECT o_totaldiscount, o_totalprice, l_shipdate
FROM orders, lineitem
WHERE o_orderkey=l_orderkey AND
@@ -1662,7 +1633,7 @@ ANALYZE
"r_rows": 41,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": 100,
"filtered": 3.333333254,
"r_filtered": 2.43902439,
"index_condition": "orders.o_totaldiscount between 18000 and 20000",
"attached_condition": "orders.o_totalprice between 200000 and 220000"
@@ -1678,30 +1649,17 @@ ANALYZE
"i_l_orderkey",
"i_l_orderkey_quantity"
],
"key": "i_l_orderkey",
"key": "PRIMARY",
"key_length": "4",
"used_key_parts": ["l_orderkey"],
"ref": ["dbt3_s001.orders.o_orderkey"],
"rowid_filter": {
"range": {
"key": "i_l_shipdate",
"used_key_parts": ["l_shipDATE"]
},
"rows": 183,
"selectivity_pct": 3.04746045,
"r_rows": 183,
"r_lookups": 6,
"r_selectivity_pct": 66.66666667,
"r_buffer_size": "REPLACED",
"r_filling_time_ms": "REPLACED"
},
"r_loops": 1,
"rows": 4,
"r_rows": 4,
"r_rows": 6,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": 3.047460556,
"r_filtered": 100,
"filtered": 3.04997468,
"r_filtered": 66.66666667,
"attached_condition": "lineitem.l_shipDATE between '1996-10-01' and '1996-12-01'"
}
}
@@ -1748,7 +1706,7 @@ EXPLAIN
"key_length": "9",
"used_key_parts": ["o_totaldiscount"],
"rows": 41,
"filtered": 100,
"filtered": 3.333333254,
"index_condition": "orders.o_totaldiscount between 18000 and 20000",
"attached_condition": "orders.o_totalprice between 200000 and 220000"
}
@@ -1768,7 +1726,7 @@ EXPLAIN
"used_key_parts": ["l_orderkey"],
"ref": ["dbt3_s001.orders.o_orderkey"],
"rows": 4,
"filtered": 3.047460556,
"filtered": 3.04997468,
"attached_condition": "lineitem.l_shipDATE between '1996-10-01' and '1996-12-01'"
}
}
@@ -1782,7 +1740,7 @@ o_totaldiscount BETWEEN 18000 AND 20000 AND
o_totalprice BETWEEN 200000 AND 220000 AND
l_shipdate BETWEEN '1996-10-01' AND '1996-12-01';
id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra
1 SIMPLE orders range PRIMARY,i_o_totalprice,i_o_totaldiscount i_o_totaldiscount 9 NULL 41 41.00 100.00 2.44 Using index condition; Using where
1 SIMPLE orders range PRIMARY,i_o_totalprice,i_o_totaldiscount i_o_totaldiscount 9 NULL 41 41.00 3.33 2.44 Using index condition; Using where
1 SIMPLE lineitem ref PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity PRIMARY 4 dbt3_s001.orders.o_orderkey 4 6.00 3.05 66.67 Using where
set statement optimizer_switch='rowid_filter=off' for ANALYZE FORMAT=JSON SELECT o_totaldiscount, o_totalprice, l_shipdate
FROM orders, lineitem
@@ -1813,7 +1771,7 @@ ANALYZE
"r_rows": 41,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": 100,
"filtered": 3.333333254,
"r_filtered": 2.43902439,
"index_condition": "orders.o_totaldiscount between 18000 and 20000",
"attached_condition": "orders.o_totalprice between 200000 and 220000"
@@ -1838,7 +1796,7 @@ ANALYZE
"r_rows": 6,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": 3.047460556,
"filtered": 3.04997468,
"r_filtered": 66.66666667,
"attached_condition": "lineitem.l_shipDATE between '1996-10-01' and '1996-12-01'"
}
@@ -1868,7 +1826,7 @@ o_totalprice BETWEEN 200000 AND 220000 AND
l_shipdate BETWEEN '1996-10-01' AND '1996-12-01';
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE orders range PRIMARY,i_o_orderdate,i_o_totalprice,i_o_totaldiscount i_o_totaldiscount 9 NULL 41 Using index condition; Using where
1 SIMPLE lineitem ref|filter PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity i_l_orderkey|i_l_shipdate 4|4 dbt3_s001.orders.o_orderkey 4 (3%) Using where; Using rowid filter
1 SIMPLE lineitem ref PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity PRIMARY 4 dbt3_s001.orders.o_orderkey 4 Using where
set statement optimizer_switch='rowid_filter=on' for EXPLAIN FORMAT=JSON SELECT o_totaldiscount, o_totalprice, l_shipdate
FROM v1, lineitem
WHERE o_orderkey=l_orderkey AND
@@ -1909,18 +1867,10 @@ EXPLAIN
"i_l_orderkey",
"i_l_orderkey_quantity"
],
"key": "i_l_orderkey",
"key": "PRIMARY",
"key_length": "4",
"used_key_parts": ["l_orderkey"],
"ref": ["dbt3_s001.orders.o_orderkey"],
"rowid_filter": {
"range": {
"key": "i_l_shipdate",
"used_key_parts": ["l_shipDATE"]
},
"rows": 183,
"selectivity_pct": 3.04746045
},
"rows": 4,
"filtered": "REPLACED",
"attached_condition": "lineitem.l_shipDATE between '1996-10-01' and '1996-12-01'"
@@ -1937,7 +1887,7 @@ o_totalprice BETWEEN 200000 AND 220000 AND
l_shipdate BETWEEN '1996-10-01' AND '1996-12-01';
id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra
1 SIMPLE orders range PRIMARY,i_o_orderdate,i_o_totalprice,i_o_totaldiscount i_o_totaldiscount 9 NULL 41 41.00 # 2.44 Using index condition; Using where
1 SIMPLE lineitem ref|filter PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity i_l_orderkey|i_l_shipdate 4|4 dbt3_s001.orders.o_orderkey 4 (3%) 4.00 (66%) # 100.00 Using where; Using rowid filter
1 SIMPLE lineitem ref PRIMARY,i_l_shipdate,i_l_orderkey,i_l_orderkey_quantity PRIMARY 4 dbt3_s001.orders.o_orderkey 4 6.00 # 66.67 Using where
set statement optimizer_switch='rowid_filter=on' for ANALYZE FORMAT=JSON SELECT o_totaldiscount, o_totalprice, l_shipdate
FROM v1, lineitem
WHERE o_orderkey=l_orderkey AND
@@ -1988,30 +1938,17 @@ ANALYZE
"i_l_orderkey",
"i_l_orderkey_quantity"
],
"key": "i_l_orderkey",
"key": "PRIMARY",
"key_length": "4",
"used_key_parts": ["l_orderkey"],
"ref": ["dbt3_s001.orders.o_orderkey"],
"rowid_filter": {
"range": {
"key": "i_l_shipdate",
"used_key_parts": ["l_shipDATE"]
},
"rows": 183,
"selectivity_pct": 3.04746045,
"r_rows": 183,
"r_lookups": 6,
"r_selectivity_pct": 66.66666667,
"r_buffer_size": "REPLACED",
"r_filling_time_ms": "REPLACED"
},
"r_loops": 1,
"rows": 4,
"r_rows": 4,
"r_rows": 6,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": "REPLACED",
"r_filtered": 100,
"r_filtered": 66.66666667,
"attached_condition": "lineitem.l_shipDATE between '1996-10-01' and '1996-12-01'"
}
}
@@ -2250,7 +2187,7 @@ EXPLAIN EXTENDED
SELECT * FROM t1 HAVING (7, 9) IN (SELECT t2.i1, t2.i2 FROM t2 WHERE t2.i1 = 3);
id select_type table type possible_keys key key_len ref rows filtered Extra
1 PRIMARY NULL NULL NULL NULL NULL NULL NULL NULL Impossible HAVING
2 SUBQUERY t2 ref i1,i2 i1 5 const 1 100.00 Using index condition; Using where
2 SUBQUERY t2 ref i1,i2 i1 5 const 1 10.00 Using index condition; Using where
Warnings:
Note 1003 /* select#1 */ select `test`.`t1`.`pk` AS `pk` from `test`.`t1` having 0
DROP TABLE t1,t2;
@@ -2327,17 +2264,9 @@ EXPLAIN
"key_length": "5",
"used_key_parts": ["a1"],
"ref": ["test.t2.a2"],
"rowid_filter": {
"range": {
"key": "b1",
"used_key_parts": ["b1"]
},
"rows": 115,
"selectivity_pct": 28.75
},
"rows": 36,
"filtered": 28.75,
"attached_condition": "t1.b1 <= (subquery#2) and t1.pk1 + 1 = t2.pk2 + 2"
"rows": 1,
"filtered": 87,
"attached_condition": "t1.b1 <= (subquery#2)"
}
}
],
@@ -2401,7 +2330,7 @@ explain
select * from t1
where el_index like '10%' and (el_index_60 like '10%' or el_index_60 like '20%');
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t1 range el_index,el_index_60 el_index 62 NULL 1000 Using where
1 SIMPLE t1 ALL el_index,el_index_60 NULL NULL NULL 10000 Using where
drop table t10, t11, t1;
#
# MDEV-22160: SIGSEGV in st_join_table::save_explain_data on SELECT
@@ -2900,8 +2829,8 @@ pk a b c
7 5 k 5
explain SELECT * FROM t1 JOIN t2 WHERE a = c AND pk BETWEEN 4 AND 7 AND a BETWEEN 2 AND 12 AND b != 'foo';
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t2 ALL NULL NULL NULL NULL 6 Using where
1 SIMPLE t1 ref|filter PRIMARY,a,a_2 a|PRIMARY 5|4 test.t2.c 3 (4%) Using where; Using rowid filter
1 SIMPLE t1 range|filter PRIMARY,a,a_2 PRIMARY|a 4|5 NULL 4 (11%) Using index condition; Using where; Using rowid filter
1 SIMPLE t2 ALL NULL NULL NULL NULL 6 Using where; Using join buffer (flat, BNL join)
SET optimizer_switch='rowid_filter=off';
SELECT * FROM t1 JOIN t2 WHERE a = c AND pk BETWEEN 4 AND 7 AND a BETWEEN 2 AND 12 AND b != 'foo';
pk a b c
@@ -2992,7 +2921,7 @@ EXPLAIN
]
},
"rows": 1,
"filtered": 100,
"filtered": 1.587301612,
"attached_condition": "t1.f1 is null and t1.f2 is null and (t1.f2 between 'a' and 'z' or t1.f1 = 'a')"
}
}
@@ -3027,7 +2956,7 @@ EXPLAIN
]
},
"rows": 1,
"filtered": 100,
"filtered": 1.587301612,
"attached_condition": "t1.f1 is null and t1.f2 is null and (t1.f2 between 'a' and 'z' or t1.f1 = 'a')"
}
}
@@ -3054,7 +2983,7 @@ id y x
1 2 1
explain extended select * from t1 join t2 on t1.id = t2.x where t2.y = 2 and t1.id = 1;
id select_type table type possible_keys key key_len ref rows filtered Extra
1 SIMPLE t1 const PRIMARY PRIMARY 4 const 1 # Using index
1 SIMPLE t1 const PRIMARY PRIMARY 4 const 1 #
1 SIMPLE t2 index_merge x,y y,x 5,5 NULL 1 # Using intersect(y,x); Using where; Using index
Warnings:
Note 1003 select 1 AS `id`,`test`.`t2`.`y` AS `y`,`test`.`t2`.`x` AS `x` from `test`.`t1` join `test`.`t2` where `test`.`t2`.`y` = 2 and `test`.`t2`.`x` = 1
@@ -3076,7 +3005,7 @@ count(*)
5
explain extended select count(*) from t1 where a between 21 and 30 and b=2;
id select_type table type possible_keys key key_len ref rows filtered Extra
1 SIMPLE t1 ref b,a b 5 const 24 9.60 Using where
1 SIMPLE t1 range b,a a 5 NULL 33 5.90 Using index condition; Using where
Warnings:
Note 1003 select count(0) AS `count(*)` from `test`.`t1` where `test`.`t1`.`b` = 2 and `test`.`t1`.`a` between 21 and 30
select * from t1 where a between 21 and 30 and b=2;
@@ -3177,11 +3106,11 @@ t1.id2 = t1.id);
id select_type table type possible_keys key key_len ref rows filtered Extra
1 PRIMARY t3 ALL NULL NULL NULL NULL 1 100.00 Using where
2 DEPENDENT SUBQUERY t1 ALL NULL NULL NULL NULL 10 100.00 Using where
3 MATERIALIZED t2 range PRIMARY,col_date_key,ch2,id2 ch2 4 NULL 2 100.00 Using where; Using index
3 MATERIALIZED bt1 ALL NULL NULL NULL NULL 10 100.00 Using where; Using join buffer (flat, BNL join)
3 DEPENDENT SUBQUERY t2 range PRIMARY,col_date_key,ch2,id2 ch2 4 NULL 2 100.00 Using where; Using index
3 DEPENDENT SUBQUERY bt1 ALL NULL NULL NULL NULL 10 100.00 Using where; Using join buffer (flat, BNL join)
Warnings:
Note 1276 Field or reference 'test.t3.id' of SELECT #2 was resolved in SELECT #1
Note 1003 /* select#1 */ select 1 AS `1` from `test`.`t3` where <in_optimizer>(1,<expr_cache><`test`.`t3`.`id`>(exists(/* select#2 */ select 1 from `test`.`t1` where <expr_cache><`test`.`t3`.`id`>(<in_optimizer>(`test`.`t3`.`id`,`test`.`t3`.`id` in ( <materialize> (/* select#3 */ select `test`.`bt1`.`id` from `test`.`t2` join `test`.`t1` `bt1` where `test`.`bt1`.`id` = `test`.`t2`.`pk` and `test`.`t2`.`ch2` <= 'g' ), <primary_index_lookup>(`test`.`t3`.`id` in <temporary table> on distinct_key where `test`.`t3`.`id` = `<subquery3>`.`id`)))) or `test`.`t1`.`id2` = `test`.`t1`.`id` limit 1)))
Note 1003 /* select#1 */ select 1 AS `1` from `test`.`t3` where <in_optimizer>(1,<expr_cache><`test`.`t3`.`id`>(exists(/* select#2 */ select 1 from `test`.`t1` where <expr_cache><`test`.`t3`.`id`>(<in_optimizer>(`test`.`t3`.`id`,<exists>(/* select#3 */ select `test`.`bt1`.`id` from `test`.`t2` join `test`.`t1` `bt1` where `test`.`bt1`.`id` = `test`.`t2`.`pk` and `test`.`t2`.`ch2` <= 'g' and <cache>(`test`.`t3`.`id`) = `test`.`bt1`.`id`))) or `test`.`t1`.`id2` = `test`.`t1`.`id` limit 1)))
SELECT 1 FROM t3
WHERE EXISTS ( SELECT 1 FROM t1
WHERE t3.id IN ( SELECT bt1.id FROM t2, t1 AS bt1
@@ -3200,11 +3129,11 @@ t1.id2 = t1.id);
id select_type table type possible_keys key key_len ref rows filtered Extra
1 PRIMARY t3 ALL NULL NULL NULL NULL 1 100.00 Using where
2 DEPENDENT SUBQUERY t1 ALL NULL NULL NULL NULL 10 100.00 Using where
3 MATERIALIZED t2 range PRIMARY,col_date_key,ch2,id2 ch2 4 NULL 2 100.00 Using where; Using index
3 MATERIALIZED bt1 ALL NULL NULL NULL NULL 10 100.00 Using where; Using join buffer (flat, BNL join)
3 DEPENDENT SUBQUERY t2 range PRIMARY,col_date_key,ch2,id2 ch2 4 NULL 2 100.00 Using where; Using index
3 DEPENDENT SUBQUERY bt1 ALL NULL NULL NULL NULL 10 100.00 Using where; Using join buffer (flat, BNL join)
Warnings:
Note 1276 Field or reference 'test.t3.id' of SELECT #2 was resolved in SELECT #1
Note 1003 /* select#1 */ select 1 AS `1` from `test`.`t3` where <in_optimizer>(1,<expr_cache><`test`.`t3`.`id`>(exists(/* select#2 */ select 1 from `test`.`t1` where <expr_cache><`test`.`t3`.`id`>(<in_optimizer>(`test`.`t3`.`id`,`test`.`t3`.`id` in ( <materialize> (/* select#3 */ select `test`.`bt1`.`id` from `test`.`t2` join `test`.`t1` `bt1` where `test`.`bt1`.`ch` = `test`.`t2`.`ch2` and `test`.`bt1`.`id` = `test`.`t2`.`pk` and `test`.`t2`.`ch2` <= 'g' ), <primary_index_lookup>(`test`.`t3`.`id` in <temporary table> on distinct_key where `test`.`t3`.`id` = `<subquery3>`.`id`)))) or `test`.`t1`.`id2` = `test`.`t1`.`id` limit 1)))
Note 1003 /* select#1 */ select 1 AS `1` from `test`.`t3` where <in_optimizer>(1,<expr_cache><`test`.`t3`.`id`>(exists(/* select#2 */ select 1 from `test`.`t1` where <expr_cache><`test`.`t3`.`id`>(<in_optimizer>(`test`.`t3`.`id`,<exists>(/* select#3 */ select `test`.`bt1`.`id` from `test`.`t2` join `test`.`t1` `bt1` where `test`.`bt1`.`ch` = `test`.`t2`.`ch2` and `test`.`bt1`.`id` = `test`.`t2`.`pk` and `test`.`t2`.`ch2` <= 'g' and <cache>(`test`.`t3`.`id`) = `test`.`bt1`.`id`))) or `test`.`t1`.`id2` = `test`.`t1`.`id` limit 1)))
SELECT 1 FROM t3
WHERE EXISTS ( SELECT 1 FROM t1
WHERE t3.id IN ( SELECT bt1.id FROM t2, t1 AS bt1
@@ -3241,7 +3170,7 @@ test.t1 analyze status OK
explain extended select count(0) from t1
where id=15066 and (match s against ('+"fttest"' in boolean mode));
id select_type table type possible_keys key key_len ref rows filtered Extra
1 SIMPLE t1 fulltext id,s s 0 1 1.64 Using where
1 SIMPLE t1 ref id,s id 5 const 1 100.00 Using where
Warnings:
Note 1003 select count(0) AS `count(0)` from `test`.`t1` where `test`.`t1`.`id` = 15066 and (match `test`.`t1`.`s` against ('+"fttest"' in boolean mode))
select count(0) from t1
@@ -3554,7 +3483,7 @@ WHERE 1 = 1 AND domain = 'www.mailhost.i-dev.fr' AND
timestamp >= DATE_ADD('2017-01-30 08:24:51', INTERVAL -1 MONTH)
ORDER BY timestamp DESC;
id select_type table type possible_keys key key_len ref rows filtered Extra
1 SIMPLE t1 range|filter ixEventWhoisDomainDomain,ixEventWhoisDomainTimestamp ixEventWhoisDomainTimestamp|ixEventWhoisDomainDomain 4|98 NULL 20 (67%) 66.67 Using where; Using rowid filter
1 SIMPLE t1 range ixEventWhoisDomainDomain,ixEventWhoisDomainTimestamp ixEventWhoisDomainTimestamp 4 NULL 1 28.57 Using where
Warnings:
Note 1003 select `test`.`t1`.`id` AS `id`,`test`.`t1`.`domain` AS `domain`,`test`.`t1`.`registrant_name` AS `registrant_name`,`test`.`t1`.`registrant_organization` AS `registrant_organization`,`test`.`t1`.`registrant_street1` AS `registrant_street1`,`test`.`t1`.`registrant_street2` AS `registrant_street2`,`test`.`t1`.`registrant_street3` AS `registrant_street3`,`test`.`t1`.`registrant_street4` AS `registrant_street4`,`test`.`t1`.`registrant_street5` AS `registrant_street5`,`test`.`t1`.`registrant_city` AS `registrant_city`,`test`.`t1`.`registrant_postal_code` AS `registrant_postal_code`,`test`.`t1`.`registrant_country` AS `registrant_country`,`test`.`t1`.`registrant_email` AS `registrant_email`,`test`.`t1`.`registrant_telephone` AS `registrant_telephone`,`test`.`t1`.`administrative_name` AS `administrative_name`,`test`.`t1`.`administrative_organization` AS `administrative_organization`,`test`.`t1`.`administrative_street1` AS `administrative_street1`,`test`.`t1`.`administrative_street2` AS `administrative_street2`,`test`.`t1`.`administrative_street3` AS `administrative_street3`,`test`.`t1`.`administrative_street4` AS `administrative_street4`,`test`.`t1`.`administrative_street5` AS `administrative_street5`,`test`.`t1`.`administrative_city` AS `administrative_city`,`test`.`t1`.`administrative_postal_code` AS `administrative_postal_code`,`test`.`t1`.`administrative_country` AS `administrative_country`,`test`.`t1`.`administrative_email` AS `administrative_email`,`test`.`t1`.`administrative_telephone` AS `administrative_telephone`,`test`.`t1`.`technical_name` AS `technical_name`,`test`.`t1`.`technical_organization` AS `technical_organization`,`test`.`t1`.`technical_street1` AS `technical_street1`,`test`.`t1`.`technical_street2` AS `technical_street2`,`test`.`t1`.`technical_street3` AS `technical_street3`,`test`.`t1`.`technical_street4` AS `technical_street4`,`test`.`t1`.`technical_street5` AS `technical_street5`,`test`.`t1`.`technical_city` AS `technical_city`,`test`.`t1`.`technical_postal_code` AS `technical_postal_code`,`test`.`t1`.`technical_country` AS `technical_country`,`test`.`t1`.`technical_email` AS `technical_email`,`test`.`t1`.`technical_telephone` AS `technical_telephone`,`test`.`t1`.`json` AS `json`,`test`.`t1`.`timestamp` AS `timestamp` from `test`.`t1` where `test`.`t1`.`domain` = 'www.mailhost.i-dev.fr' and `test`.`t1`.`timestamp` >= <cache>('2017-01-30 08:24:51' + interval -1 month) order by `test`.`t1`.`timestamp` desc
SET optimizer_switch=@save_optimizer_switch;
@@ -3601,7 +3530,7 @@ SELECT * FROM t1
WHERE (a BETWEEN 9 AND 10 OR a IS NULL) AND (b BETWEEN 9 AND 10 OR b = 9)
ORDER BY pk LIMIT 1;
id select_type table type possible_keys key key_len ref rows filtered Extra
1 SIMPLE t1 index a,b PRIMARY 4 NULL 73 100.00 Using where
1 SIMPLE t1 index a,b PRIMARY 4 NULL 73 56.05 Using where
Warnings:
Note 1003 select `test`.`t1`.`pk` AS `pk`,`test`.`t1`.`a` AS `a`,`test`.`t1`.`b` AS `b` from `test`.`t1` where (`test`.`t1`.`a` between 9 and 10 or `test`.`t1`.`a` is null) and (`test`.`t1`.`b` between 9 and 10 or `test`.`t1`.`b` = 9) order by `test`.`t1`.`pk` limit 1
ANALYZE
@@ -3609,7 +3538,7 @@ SELECT * FROM t1
WHERE (a BETWEEN 9 AND 10 OR a IS NULL) AND (b BETWEEN 9 AND 10 OR b = 9)
ORDER BY pk LIMIT 1;
id select_type table type possible_keys key key_len ref rows r_rows filtered r_filtered Extra
1 SIMPLE t1 index a,b PRIMARY 4 NULL 3008 3008.00 6.38 0.00 Using where
1 SIMPLE t1 index a,b PRIMARY 4 NULL 3008 3008.00 1.36 0.00 Using where
DROP TABLE t1;
SET global innodb_stats_persistent= @stats.save;
#
@@ -3746,7 +3675,7 @@ fi.fh in (6311439873746261694,-397087483897438286,
id select_type table type possible_keys key key_len ref rows filtered Extra
1 SIMPLE t index_merge PRIMARY,acli_rid,acli_tp acli_tp,acli_rid 2,767 NULL 2 100.00 Using intersect(acli_tp,acli_rid); Using where; Using index
1 SIMPLE a ref PRIMARY,acei_aclid acei_aclid 8 test.t.id 1 100.00 Using where
1 SIMPLE fi ref filt_aceid,filt_fh filt_aceid 8 test.a.id 24 14.46 Using where
1 SIMPLE fi ref filt_aceid,filt_fh filt_aceid 8 test.a.id 1 26.09 Using where
Warnings:
Note 1003 select `test`.`t`.`id` AS `id`,`test`.`fi`.`id` AS `id`,`test`.`fi`.`aceid` AS `aceid`,`test`.`fi`.`clid` AS `clid`,`test`.`fi`.`fh` AS `fh` from `test`.`acli` `t` join `test`.`acei` `a` join `test`.`filt` `fi` where `test`.`t`.`tp` = 121 and `test`.`a`.`atp` = 1 and `test`.`fi`.`aceid` = `test`.`a`.`id` and `test`.`a`.`aclid` = `test`.`t`.`id` and `test`.`t`.`rid` = 'B5FCC8C7111E4E3CBC21AAF5012F59C2' and `test`.`fi`.`fh` in (6311439873746261694,-397087483897438286,8518228073041491534,-5420422472375069774)
set statement optimizer_switch='rowid_filter=off' for select t.id, fi.*
@@ -3803,7 +3732,7 @@ fi.fh in (6311439873746261694,-397087483897438286,
id select_type table type possible_keys key key_len ref rows filtered Extra
1 SIMPLE t index_merge PRIMARY,acli_rid,acli_tp acli_tp,acli_rid 2,767 NULL 2 100.00 Using intersect(acli_tp,acli_rid); Using where; Using index
1 SIMPLE a ref PRIMARY,acei_aclid acei_aclid 8 test.t.id 1 100.00 Using where
1 SIMPLE fi ref|filter filt_aceid,filt_fh filt_aceid|filt_fh 8|8 test.a.id 24 (14%) 14.46 Using where; Using rowid filter
1 SIMPLE fi ref filt_aceid,filt_fh filt_aceid 8 test.a.id 1 26.09 Using where
Warnings:
Note 1003 select `test`.`t`.`id` AS `id`,`test`.`fi`.`id` AS `id`,`test`.`fi`.`aceid` AS `aceid`,`test`.`fi`.`clid` AS `clid`,`test`.`fi`.`fh` AS `fh` from `test`.`acli` `t` join `test`.`acei` `a` join `test`.`filt` `fi` where `test`.`t`.`tp` = 121 and `test`.`a`.`atp` = 1 and `test`.`fi`.`aceid` = `test`.`a`.`id` and `test`.`a`.`aclid` = `test`.`t`.`id` and `test`.`t`.`rid` = 'B5FCC8C7111E4E3CBC21AAF5012F59C2' and `test`.`fi`.`fh` in (6311439873746261694,-397087483897438286,8518228073041491534,-5420422472375069774)
set statement optimizer_switch='rowid_filter=on' for select t.id, fi.*
@@ -3862,7 +3791,7 @@ fi.fh in (6311439873746261694,-397087483897438286,
id select_type table type possible_keys key key_len ref rows filtered Extra
1 SIMPLE t index_merge PRIMARY,acli_rid,acli_tp acli_tp,acli_rid 2,767 NULL 2 100.00 Using intersect(acli_tp,acli_rid); Using where; Using index
1 SIMPLE a ref PRIMARY,acei_aclid acei_aclid 8 test.t.id 1 100.00 Using where; Using join buffer (flat, BKA join); Rowid-ordered scan
1 SIMPLE fi ref filt_aceid,filt_fh filt_aceid 8 test.a.id 24 14.46 Using where; Using join buffer (incremental, BKA join); Rowid-ordered scan
1 SIMPLE fi ref filt_aceid,filt_fh filt_aceid 8 test.a.id 1 26.09 Using where; Using join buffer (incremental, BKA join); Rowid-ordered scan
Warnings:
Note 1003 select `test`.`t`.`id` AS `id`,`test`.`fi`.`id` AS `id`,`test`.`fi`.`aceid` AS `aceid`,`test`.`fi`.`clid` AS `clid`,`test`.`fi`.`fh` AS `fh` from `test`.`acli` `t` join `test`.`acei` `a` join `test`.`filt` `fi` where `test`.`t`.`tp` = 121 and `test`.`a`.`atp` = 1 and `test`.`fi`.`aceid` = `test`.`a`.`id` and `test`.`a`.`aclid` = `test`.`t`.`id` and `test`.`t`.`rid` = 'B5FCC8C7111E4E3CBC21AAF5012F59C2' and `test`.`fi`.`fh` in (6311439873746261694,-397087483897438286,8518228073041491534,-5420422472375069774)
set statement optimizer_switch='rowid_filter=off' for select t.id, fi.*
@@ -3919,7 +3848,7 @@ fi.fh in (6311439873746261694,-397087483897438286,
id select_type table type possible_keys key key_len ref rows filtered Extra
1 SIMPLE t index_merge PRIMARY,acli_rid,acli_tp acli_tp,acli_rid 2,767 NULL 2 100.00 Using intersect(acli_tp,acli_rid); Using where; Using index
1 SIMPLE a ref PRIMARY,acei_aclid acei_aclid 8 test.t.id 1 100.00 Using where; Using join buffer (flat, BKA join); Rowid-ordered scan
1 SIMPLE fi ref|filter filt_aceid,filt_fh filt_aceid|filt_fh 8|8 test.a.id 24 (14%) 14.46 Using where; Using join buffer (incremental, BKA join); Rowid-ordered scan; Using rowid filter
1 SIMPLE fi ref filt_aceid,filt_fh filt_aceid 8 test.a.id 1 26.09 Using where; Using join buffer (incremental, BKA join); Rowid-ordered scan
Warnings:
Note 1003 select `test`.`t`.`id` AS `id`,`test`.`fi`.`id` AS `id`,`test`.`fi`.`aceid` AS `aceid`,`test`.`fi`.`clid` AS `clid`,`test`.`fi`.`fh` AS `fh` from `test`.`acli` `t` join `test`.`acei` `a` join `test`.`filt` `fi` where `test`.`t`.`tp` = 121 and `test`.`a`.`atp` = 1 and `test`.`fi`.`aceid` = `test`.`a`.`id` and `test`.`a`.`aclid` = `test`.`t`.`id` and `test`.`t`.`rid` = 'B5FCC8C7111E4E3CBC21AAF5012F59C2' and `test`.`fi`.`fh` in (6311439873746261694,-397087483897438286,8518228073041491534,-5420422472375069774)
set statement optimizer_switch='rowid_filter=on' for select t.id, fi.*
@@ -4039,7 +3968,8 @@ ANALYZE
"join_type": "BKA",
"mrr_type": "Rowid-ordered scan",
"attached_condition": "a.atp = 1",
"r_filtered": 100
"r_filtered": 100,
"r_unpack_time_ms": "REPLACED"
}
},
{
@@ -4070,7 +4000,7 @@ ANALYZE
"r_rows": 32,
"r_table_time_ms": "REPLACED",
"r_other_time_ms": "REPLACED",
"filtered": 14.46428585,
"filtered": 26.08628654,
"r_filtered": 100
},
"buffer_type": "incremental",
@@ -4078,7 +4008,8 @@ ANALYZE
"join_type": "BKA",
"mrr_type": "Rowid-ordered scan",
"attached_condition": "fi.fh in (6311439873746261694,-397087483897438286,8518228073041491534,-5420422472375069774)",
"r_filtered": 100
"r_filtered": 40,
"r_unpack_time_ms": "REPLACED"
}
}
]
@@ -4185,9 +4116,9 @@ EXPLAIN EXTENDED SELECT * FROM t1
WHERE t1.c1 NOT IN (SELECT t2.c1 FROM t2, t1 AS a1
WHERE t2.i1 = t1.pk AND t2.i1 BETWEEN 3 AND 5);
id select_type table type possible_keys key key_len ref rows filtered Extra
1 PRIMARY t1 ALL NULL NULL NULL NULL 60 100.00 Using where
2 DEPENDENT SUBQUERY t2 ref|filter c1,i1 c1|i1 3|5 func 38 (25%) 25.00 Using where; Full scan on NULL key; Using rowid filter
2 DEPENDENT SUBQUERY a1 ALL NULL NULL NULL NULL 60 100.00 Using join buffer (flat, BNL join)
1 PRIMARY t1 ALL NULL NULL NULL NULL 15 100.00 Using where
2 DEPENDENT SUBQUERY t2 ref|filter c1,i1 c1|i1 3|5 func 6 (33%) 11.11 Using where; Full scan on NULL key; Using rowid filter
2 DEPENDENT SUBQUERY a1 ALL NULL NULL NULL NULL 15 100.00 Using join buffer (flat, BNL join)
Warnings:
Note 1276 Field or reference 'test.t1.pk' of SELECT #2 was resolved in SELECT #1
Note 1003 /* select#1 */ select `test`.`t1`.`pk` AS `pk`,`test`.`t1`.`c1` AS `c1` from `test`.`t1` where !<expr_cache><`test`.`t1`.`c1`,`test`.`t1`.`pk`>(<in_optimizer>(`test`.`t1`.`c1`,<exists>(/* select#2 */ select `test`.`t2`.`c1` from `test`.`t2` join `test`.`t1` `a1` where `test`.`t2`.`i1` = `test`.`t1`.`pk` and `test`.`t2`.`i1` between 3 and 5 and trigcond(<cache>(`test`.`t1`.`c1`) = `test`.`t2`.`c1`))))

6
mysql-test/main/show_explain_json.result
View File

@@ -51,13 +51,13 @@ SHOW EXPLAIN
{
"table": {
"table_name": "t1",
"access_type": "index",
"access_type": "range",
"possible_keys": ["a"],
"key": "a",
"key_length": "5",
"used_key_parts": ["a"],
"rows": 1000,
"filtered": 99.90000153,
"rows": 999,
"filtered": 100,
"attached_condition": "t1.a < 100000",
"using_index": true
}

2
mysql-test/main/sp.result
View File

@@ -2173,8 +2173,8 @@ create procedure bug3734 (param1 varchar(100))
select * from t3 where match (title,body) against (param1)|
call bug3734('database')|
id title body
5 MySQL vs. YourSQL In the following database comparison ...
1 MySQL Tutorial DBMS stands for DataBase ...
5 MySQL vs. YourSQL In the following database comparison ...
call bug3734('Security')|
id title body
6 MySQL Security When configured properly, MySQL ...

18
mysql-test/main/status.result
View File

@@ -71,10 +71,10 @@ a
6
show status like 'last_query_cost';
Variable_name Value
Last_query_cost 13.542725
Last_query_cost 0.017820
show status like 'last_query_cost';
Variable_name Value
Last_query_cost 13.542725
Last_query_cost 0.017820
select 1;
1
1
@@ -134,20 +134,20 @@ a
1
SHOW SESSION STATUS LIKE 'Last_query_cost';
Variable_name Value
Last_query_cost 1.501709
Last_query_cost 0.010313
EXPLAIN SELECT a FROM t1;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t1 ALL NULL NULL NULL NULL 2
SHOW SESSION STATUS LIKE 'Last_query_cost';
Variable_name Value
Last_query_cost 1.501709
Last_query_cost 0.010313
SELECT a FROM t1 UNION SELECT a FROM t1 ORDER BY a;
a
1
2
SHOW SESSION STATUS LIKE 'Last_query_cost';
Variable_name Value
Last_query_cost 0.000000
Last_query_cost 0.010313
EXPLAIN SELECT a FROM t1 UNION SELECT a FROM t1 ORDER BY a;
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY t1 ALL NULL NULL NULL NULL 2
@@ -155,25 +155,25 @@ id select_type table type possible_keys key key_len ref rows Extra
NULL UNION RESULT <union1,2> ALL NULL NULL NULL NULL NULL Using filesort
SHOW SESSION STATUS LIKE 'Last_query_cost';
Variable_name Value
Last_query_cost 0.000000
Last_query_cost 0.010313
SELECT a IN (SELECT a FROM t1) FROM t1 LIMIT 1;
a IN (SELECT a FROM t1)
1
SHOW SESSION STATUS LIKE 'Last_query_cost';
Variable_name Value
Last_query_cost 0.000000
Last_query_cost 0.010313
SELECT (SELECT a FROM t1 LIMIT 1) x FROM t1 LIMIT 1;
x
1
SHOW SESSION STATUS LIKE 'Last_query_cost';
Variable_name Value
Last_query_cost 0.000000
Last_query_cost 0.010313
SELECT * FROM t1 a, t1 b LIMIT 1;
a a
1 1
SHOW SESSION STATUS LIKE 'Last_query_cost';
Variable_name Value
Last_query_cost 3.953418
Last_query_cost 0.021119
DROP TABLE t1;
connect con1,localhost,root,,;
show status like 'com_show_status';

11
mysql-test/main/table_elim.result
View File

@@ -337,7 +337,7 @@ id select_type table type possible_keys key key_len ref rows Extra
explain select t1.a from t1 left join t2 on t2.pk between 0.5 and 1.5;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t1 ALL NULL NULL NULL NULL 4
1 SIMPLE t2 index PRIMARY PRIMARY 4 NULL 2 Using where; Using index
1 SIMPLE t2 range PRIMARY PRIMARY 4 NULL 2 Using where; Using index
explain select t1.a from t1 left join t2 on t2.pk between 10 and 10;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t1 ALL NULL NULL NULL NULL 4
@@ -408,7 +408,7 @@ select t1.*
from t1 left join t2 on t2.pk=3 or t2.pk= 4;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t1 ALL NULL NULL NULL NULL 4
1 SIMPLE t2 index PRIMARY PRIMARY 4 NULL 2 Using where; Using index
1 SIMPLE t2 range PRIMARY PRIMARY 4 NULL 2 Using where; Using index
explain
select t1.*
from t1 left join t2 on t2.pk=3 or t2.pk= 3;
@@ -419,7 +419,7 @@ select t1.*
from t1 left join t2 on (t2.pk=3 and t2.b=3) or (t2.pk= 4 and t2.b=3);
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t1 ALL NULL NULL NULL NULL 4
1 SIMPLE t2 ALL PRIMARY NULL NULL NULL 2 Using where
1 SIMPLE t2 range PRIMARY PRIMARY 4 NULL 2 Using where
drop table t1, t2;
#
# LPBUG#523593: Running RQG optimizer_no_subquery crashes MariaDB
@@ -562,7 +562,10 @@ LEFT JOIN t1 ON t4.f1 = t1.f1
JOIN t5 ON t4.f3 ON t3.f1 = t5.f5 ON t2.f4 = t3.f4
WHERE t3.f2 ;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables
1 SIMPLE t3 ALL NULL NULL NULL NULL 2 Using where
1 SIMPLE t5 ref f5 f5 5 test.t3.f1 2 Using where; Using index
1 SIMPLE t4 ALL NULL NULL NULL NULL 3 Using where
1 SIMPLE t2 ref f4 f4 1003 test.t3.f4 2 Using where
# ^^ The above must not produce a QEP of t3,t5,t2,t4
# as that violates the "no interleaving of outer join nests" rule.
DROP TABLE t1,t2,t3,t4,t5;

2
mysql-test/main/type_ranges.result
View File

@@ -173,12 +173,12 @@ PRIMARY KEY (auto)
);
INSERT IGNORE INTO t2 (string,mediumblob_col,new_field) SELECT string,mediumblob_col,new_field from t1 where auto > 10;
Warnings:
Warning 1265 Data truncated for column 'new_field' at row 1
Warning 1265 Data truncated for column 'new_field' at row 2
Warning 1265 Data truncated for column 'new_field' at row 3
Warning 1265 Data truncated for column 'new_field' at row 4
Warning 1265 Data truncated for column 'new_field' at row 5
Warning 1265 Data truncated for column 'new_field' at row 6
Warning 1265 Data truncated for column 'new_field' at row 7
select * from t2;
auto string mediumblob_col new_field
1 2 2 ne

2
mysql-test/main/type_time_6065.result
View File

@@ -2267,7 +2267,7 @@ outr.col_varchar_key IS NULL
);
id select_type table type possible_keys key key_len ref rows filtered Extra
1 PRIMARY outr system col_datetime_key NULL NULL NULL 1 100.00
1 PRIMARY innr ref col_int_key col_int_key 4 const 2 100.00 Using where; FirstMatch(outr)
1 PRIMARY innr ref col_int_key col_int_key 4 const 2 50.00 Using where; FirstMatch(outr)
1 PRIMARY outr2 index col_time_key col_time_key 4 NULL 20 100.00 Using where; Using index; Using join buffer (flat, BNL join)
Warnings:
Note 1003 select 1 AS `col_int_nokey` from `test`.`t3` `outr2` semi join (`test`.`t1` `innr`) where `test`.`innr`.`col_int_key` = 1 and `test`.`innr`.`pk` >= `test`.`innr`.`col_int_nokey` and `test`.`outr2`.`col_time_key` > '2001-11-04 19:07:55'

4
mysql-test/main/user_var.result
View File

@@ -22,7 +22,7 @@ i @vv1:=if(sv1.i,1,0) @vv2:=if(sv2.i,1,0) @vv3:=if(sv3.i,1,0) @vv1+@vv2+@vv3
2 1 0 0 1
explain select * from t1 where i=@vv1;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t1 ALL i NULL NULL NULL 3 Using where
1 SIMPLE t1 ref i i 4 const 2
select @vv1,i,v from t1 where i=@vv1;
@vv1 i v
1 1 1
@@ -35,7 +35,7 @@ id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t1 index NULL i 4 NULL 3 Using where; Using index
explain select * from t1 where i=@vv1;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t1 ALL i NULL NULL NULL 3 Using where
1 SIMPLE t1 ref i i 4 const 2
drop table t1,t2;
set @a=0,@b=0;
select @a:=10, @b:=1, @a > @b, @a < @b;

6
mysys/my_getopt.c
View File

@@ -86,6 +86,7 @@ my_bool my_getopt_prefix_matching= 1;
my_bool my_handle_options_init_variables = 1;
my_getopt_value my_getopt_get_addr= 0;
my_getopt_adjust my_getopt_adjust_value= 0;
static void default_reporter(enum loglevel level, const char *format, ...)
{
@@ -897,7 +898,12 @@ static int setval(const struct my_option *opts, void *value, char *argument,
goto ret;
};
}
if (opts->var_type & GET_ADJUST_VALUE)
(*my_getopt_adjust_value)(opts, value);
validate_value(opts->name, argument, option_file);
DBUG_RETURN(0);
ret:

2
sql/CMakeLists.txt
View File

@@ -174,7 +174,7 @@ SET (SQL_SOURCE
sql_tvc.cc sql_tvc.h
opt_split.cc
rowid_filter.cc rowid_filter.h
optimizer_costs.h
optimizer_costs.h optimizer_defaults.h
opt_trace.cc
table_cache.cc encryption.cc temporary_tables.cc
json_table.cc

2
sql/filesort.cc
View File

@@ -1132,8 +1132,6 @@ write_keys(Sort_param *param, SORT_INFO *fs_info, uint count,
for (uint ix= 0; ix < count; ++ix)
{
uchar *record= fs_info->get_sorted_record(ix);
if (my_b_write(tempfile, record, param->get_record_length(record)))
DBUG_RETURN(1); /* purecov: inspected */
}

21
sql/filesort_utils.cc
View File

@@ -19,7 +19,7 @@
#include "sql_const.h"
#include "sql_sort.h"
#include "table.h"
#include "optimizer_defaults.h"
PSI_memory_key key_memory_Filesort_buffer_sort_keys;
@@ -58,7 +58,6 @@ const LEX_CSTRING filesort_names[]=
Cost of the operation.
*/
static
double get_qsort_sort_cost(ha_rows num_rows, bool with_addon_fields)
{
const double row_copy_cost= with_addon_fields ? DEFAULT_ROW_COPY_COST :
@@ -106,12 +105,13 @@ double get_pq_sort_cost(size_t num_rows, size_t queue_size,
static
double get_merge_cost(ha_rows num_elements, ha_rows num_buffers,
size_t elem_size, double compare_cost)
size_t elem_size, double compare_cost,
double disk_read_cost)
{
/* 2 -> 1 read + 1 write */
const double io_cost= (2.0 * (num_elements * elem_size +
DISK_CHUNK_SIZE - 1) /
DISK_CHUNK_SIZE);
DISK_CHUNK_SIZE) * disk_read_cost;
/* 2 -> 1 insert, 1 pop for the priority queue used to merge the buffers. */
const double cpu_cost= (2.0 * num_elements * log2(1.0 + num_buffers) *
compare_cost) * PQ_SORT_SLOWNESS_CORRECTION_FACTOR;
@@ -131,6 +131,7 @@ double get_merge_many_buffs_cost_fast(ha_rows num_rows,
ha_rows num_keys_per_buffer,
size_t elem_size,
double key_compare_cost,
double disk_read_cost,
bool with_addon_fields)
{
DBUG_ASSERT(num_keys_per_buffer != 0);
@@ -162,7 +163,7 @@ double get_merge_many_buffs_cost_fast(ha_rows num_rows,
total_cost+=
num_merge_calls *
get_merge_cost(num_keys_per_buffer * MERGEBUFF, MERGEBUFF, elem_size,
key_compare_cost);
key_compare_cost, disk_read_cost);
// # of records in remaining buffers.
last_n_elems+= num_remaining_buffs * num_keys_per_buffer;
@@ -170,7 +171,7 @@ double get_merge_many_buffs_cost_fast(ha_rows num_rows,
// Cost of merge sort of remaining buffers.
total_cost+=
get_merge_cost(last_n_elems, 1 + num_remaining_buffs, elem_size,
key_compare_cost);
key_compare_cost, disk_read_cost);
num_buffers= num_merge_calls;
num_keys_per_buffer*= MERGEBUFF;
@@ -179,7 +180,7 @@ double get_merge_many_buffs_cost_fast(ha_rows num_rows,
// Simulate final merge_buff call.
last_n_elems+= num_keys_per_buffer * num_buffers;
total_cost+= get_merge_cost(last_n_elems, 1 + num_buffers, elem_size,
key_compare_cost);
key_compare_cost, disk_read_cost);
return total_cost;
}
@@ -238,7 +239,7 @@ void Sort_costs::compute_pq_sort_costs(Sort_param *param, ha_rows num_rows,
{
costs[PQ_SORT_ORDER_BY_FIELDS]=
get_pq_sort_cost(num_rows, queue_size, false) +
param->sort_form->file->ha_rnd_pos_time(MY_MIN(queue_size - 1, num_rows));
param->sort_form->file->ha_rnd_pos_call_time(MY_MIN(queue_size - 1, num_rows));
}
/* Calculate cost with addon fields */
@@ -272,8 +273,9 @@ void Sort_costs::compute_merge_sort_costs(Sort_param *param,
costs[MERGE_SORT_ORDER_BY_FIELDS]=
get_merge_many_buffs_cost_fast(num_rows, num_available_keys,
row_length, DEFAULT_KEY_COMPARE_COST,
default_optimizer_costs.disk_read_cost,
false) +
param->sort_form->file->ha_rnd_pos_time(MY_MIN(param->limit_rows,
param->sort_form->file->ha_rnd_pos_call_time(MY_MIN(param->limit_rows,
num_rows));
if (with_addon_fields)
@@ -286,6 +288,7 @@ void Sort_costs::compute_merge_sort_costs(Sort_param *param,
costs[MERGE_SORT_ALL_FIELDS]=
get_merge_many_buffs_cost_fast(num_rows, num_available_keys,
row_length, DEFAULT_KEY_COMPARE_COST,
DISK_READ_COST_THD(thd),
true);
}

1
sql/filesort_utils.h
View File

@@ -352,6 +352,7 @@ extern const LEX_CSTRING filesort_names[];
double cost_of_filesort(TABLE *table, ORDER *order_by, ha_rows rows_to_read,
ha_rows limit_rows, enum sort_type *used_sort_type);
double get_qsort_sort_cost(ha_rows num_rows, bool with_addon_fields);
int compare_packed_sort_keys(void *sort_keys, unsigned char **a,
unsigned char **b);
qsort2_cmp get_packed_keys_compare_ptr();

127
sql/ha_partition.cc
View File

@@ -9737,24 +9737,27 @@ uint ha_partition::get_biggest_used_partition(uint *part_index)
time for scan
*/
double ha_partition::scan_time()
IO_AND_CPU_COST ha_partition::scan_time()
{
double scan_time= 0;
IO_AND_CPU_COST scan_time= {0,0};
uint i;
DBUG_ENTER("ha_partition::scan_time");
for (i= bitmap_get_first_set(&m_part_info->read_partitions);
i < m_tot_parts;
i= bitmap_get_next_set(&m_part_info->read_partitions, i))
scan_time+= m_file[i]->scan_time();
{
IO_AND_CPU_COST cost= m_file[i]->scan_time();
scan_time.io+= cost.io;
scan_time.cpu+= cost.cpu;
}
if (m_tot_parts)
{
/*
Add TABLE_SCAN_SETUP_COST for partitions to make cost similar to
in ha_scan_time()
*/
scan_time+= (TABLE_SCAN_SETUP_COST * avg_io_cost() * (m_tot_parts - 1) /
optimizer_cache_cost);
scan_time.cpu+= TABLE_SCAN_SETUP_COST * (m_tot_parts - 1);
}
DBUG_RETURN(scan_time);
}
@@ -9769,34 +9772,78 @@ double ha_partition::scan_time()
@return time for scanning index inx
*/
double ha_partition::key_scan_time(uint inx)
IO_AND_CPU_COST ha_partition::key_scan_time(uint inx, ha_rows rows)
{
double scan_time= 0;
IO_AND_CPU_COST scan_time= {0,0};
uint i;
uint partitions= bitmap_bits_set(&m_part_info->read_partitions);
ha_rows rows_per_part;
DBUG_ENTER("ha_partition::key_scan_time");
if (partitions == 0)
DBUG_RETURN(scan_time);
set_if_bigger(rows, 1);
rows_per_part= (rows + partitions - 1)/partitions;
for (i= bitmap_get_first_set(&m_part_info->read_partitions);
i < m_tot_parts;
i= bitmap_get_next_set(&m_part_info->read_partitions, i))
scan_time+= m_file[i]->key_scan_time(inx);
{
IO_AND_CPU_COST cost= m_file[i]->key_scan_time(inx, rows_per_part);
scan_time.io+= cost.io;
scan_time.cpu+= cost.cpu;
}
DBUG_RETURN(scan_time);
}
double ha_partition::keyread_time(uint inx, uint ranges, ha_rows rows)
IO_AND_CPU_COST ha_partition::keyread_time(uint inx, ulong ranges, ha_rows rows,
ulonglong blocks)
{
double read_time= 0;
IO_AND_CPU_COST read_time= {0,0};
uint i;
uint partitions= bitmap_bits_set(&m_part_info->read_partitions);
DBUG_ENTER("ha_partition::keyread_time");
if (!ranges)
DBUG_RETURN(handler::keyread_time(inx, ranges, rows));
if (partitions == 0)
DBUG_RETURN(read_time);
ha_rows rows_per_part= (rows + partitions - 1)/partitions;
for (i= bitmap_get_first_set(&m_part_info->read_partitions);
i < m_tot_parts;
i= bitmap_get_next_set(&m_part_info->read_partitions, i))
read_time+= m_file[i]->keyread_time(inx, ranges, rows);
{
IO_AND_CPU_COST cost= m_file[i]->keyread_time(inx, ranges, rows_per_part,
blocks);
read_time.io+= cost.io;
read_time.cpu+= cost.cpu;
}
/* Add that we have to do a key lookup for all ranges in all partitions */
read_time.cpu= (partitions-1) * ranges * KEY_LOOKUP_COST;
DBUG_RETURN(read_time);
}
IO_AND_CPU_COST ha_partition::rnd_pos_time(ha_rows rows)
{
IO_AND_CPU_COST read_time= {0,0};
uint i;
uint partitions= bitmap_bits_set(&m_part_info->read_partitions);
if (partitions == 0)
return read_time;
ha_rows rows_per_part= (rows + partitions - 1)/partitions;
for (i= bitmap_get_first_set(&m_part_info->read_partitions);
i < m_tot_parts;
i= bitmap_get_next_set(&m_part_info->read_partitions, i))
{
IO_AND_CPU_COST cost= m_file[i]->rnd_pos_time(rows_per_part);
read_time.io+= cost.io;
read_time.cpu+= cost.cpu;
}
return read_time;
}
/**
Find number of records in a range.
@param inx Index number
@@ -9853,6 +9900,8 @@ ha_rows ha_partition::records_in_range(uint inx, const key_range *min_key,
if (estimated_rows && checked_rows &&
checked_rows >= min_rows_to_check)
{
/* We cannot use page ranges when there is more than one partion */
*pages= unused_page_range;
DBUG_PRINT("info",
("records_in_range(inx %u): %lu (%lu * %lu / %lu)",
inx,
@@ -9866,6 +9915,8 @@ ha_rows ha_partition::records_in_range(uint inx, const key_range *min_key,
DBUG_PRINT("info", ("records_in_range(inx %u): %lu",
inx,
(ulong) estimated_rows));
/* We cannot use page ranges when there is more than one partion */
*pages= unused_page_range;
DBUG_RETURN(estimated_rows);
}
@@ -9896,33 +9947,6 @@ ha_rows ha_partition::estimate_rows_upper_bound()
}
/*
Get time to read
SYNOPSIS
read_time()
index Index number used
ranges Number of ranges
rows Number of rows
RETURN VALUE
time for read
DESCRIPTION
This will be optimised later to include whether or not the index can
be used with partitioning. To achieve we need to add another parameter
that specifies how many of the index fields that are bound in the ranges.
Possibly added as a new call to handlers.
*/
double ha_partition::read_time(uint index, uint ranges, ha_rows rows)
{
DBUG_ENTER("ha_partition::read_time");
DBUG_RETURN(get_open_file_sample()->read_time(index, ranges, rows));
}
/**
Number of rows in table. see handler.h
@@ -12145,6 +12169,18 @@ ha_partition::can_convert_nocopy(const Field &field,
return true;
}
/*
Get table costs for the current statement that should be stored in
handler->cost variables.
When we want to support many different table handlers, we should set
m_file[i]->costs to point to an unique cost structure per open
instance and call something similar as
TABLE_SHARE::update_optimizer_costs(handlerton *hton) and
handler::update_optimizer_costs(&costs) on it.
*/
void ha_partition::set_optimizer_costs(THD *thd)
{
handler::set_optimizer_costs(thd);
@@ -12154,6 +12190,17 @@ void ha_partition::set_optimizer_costs(THD *thd)
m_file[i]->set_optimizer_costs(thd);
}
/*
Get unique table costs for the first instance of the handler and store
in table->share
*/
void ha_partition::update_optimizer_costs(OPTIMIZER_COSTS *costs)
{
uint i= bitmap_get_first_set(&m_part_info->read_partitions);
m_file[i]->update_optimizer_costs(costs);
}
struct st_mysql_storage_engine partition_storage_engine=
{ MYSQL_HANDLERTON_INTERFACE_VERSION };

14
sql/ha_partition.h
View File

@@ -1031,16 +1031,14 @@ public:
/*
Called in test_quick_select to determine if indexes should be used.
*/
double scan_time() override;
IO_AND_CPU_COST scan_time() override;
double key_scan_time(uint inx) override;
IO_AND_CPU_COST key_scan_time(uint inx, ha_rows rows) override;
double keyread_time(uint inx, uint ranges, ha_rows rows) override;
IO_AND_CPU_COST keyread_time(uint inx, ulong ranges, ha_rows rows,
ulonglong blocks) override;
IO_AND_CPU_COST rnd_pos_time(ha_rows rows) override;
/*
The next method will never be called if you do not implement indexes.
*/
double read_time(uint index, uint ranges, ha_rows rows) override;
/*
For the given range how many records are estimated to be in this range.
Used by optimiser to calculate cost of using a particular index.
@@ -1638,5 +1636,7 @@ public:
bool can_convert_nocopy(const Field &field,
const Column_definition &new_field) const override;
void set_optimizer_costs(THD *thd) override;
void update_optimizer_costs(OPTIMIZER_COSTS *costs) override;
};
#endif /* HA_PARTITION_INCLUDED */

216
sql/handler.cc
View File

@@ -46,6 +46,7 @@
#include "ha_sequence.h"
#include "rowid_filter.h"
#include "mysys_err.h"
#include "optimizer_defaults.h"
#ifdef WITH_PARTITION_STORAGE_ENGINE
#include "ha_partition.h"
@@ -621,8 +622,44 @@ int ha_finalize_handlerton(st_plugin_int *plugin)
}
const char *hton_no_exts[]= { 0 };
/*
Get a pointer to the global engine optimizer costs (like
innodb.disk_read_cost) and store the pointer in the handlerton.
This is called once when a handlerton is created.
We also update the not set global costs with the default costs
to allow information_schema to print the real used values.
*/
static bool update_optimizer_costs(handlerton *hton)
{
OPTIMIZER_COSTS costs= default_optimizer_costs;
LEX_CSTRING *name= hton_name(hton);
if (hton->update_optimizer_costs)
hton->update_optimizer_costs(&costs);
mysql_mutex_lock(&LOCK_optimizer_costs);
hton->optimizer_costs= get_or_create_optimizer_costs(name->str,
name->length);
if (!hton->optimizer_costs)
{
mysql_mutex_unlock(&LOCK_optimizer_costs);
return 1; // OOM
}
/* Update not set values from current default costs */
for (uint i=0 ; i < sizeof(OPTIMIZER_COSTS)/sizeof(double) ; i++)
{
double *var= ((double*) hton->optimizer_costs)+i;
if (*var == OPTIMIZER_COST_UNDEF)
*var= ((double*) &costs)[i];
}
mysql_mutex_unlock(&LOCK_optimizer_costs);
return 0;
}
const char *hton_no_exts[]= { 0 };
int ha_initialize_handlerton(st_plugin_int *plugin)
{
@@ -725,6 +762,12 @@ int ha_initialize_handlerton(st_plugin_int *plugin)
hton->savepoint_offset= savepoint_alloc_size;
savepoint_alloc_size+= tmp;
hton2plugin[hton->slot]=plugin;
if (plugin->plugin->type == MYSQL_STORAGE_ENGINE_PLUGIN &&
!(hton->flags & HTON_HIDDEN) &&
update_optimizer_costs(hton))
goto err_deinit;
if (hton->prepare)
{
total_ha_2pc++;
@@ -764,7 +807,6 @@ int ha_initialize_handlerton(st_plugin_int *plugin)
resolve_sysvar_table_options(hton);
update_discovery_counters(hton, 1);
DBUG_RETURN(0);
err_deinit:
@@ -3209,6 +3251,7 @@ handler *handler::clone(const char *name, MEM_ROOT *mem_root)
if (new_handler->ha_open(table, name, table->db_stat,
HA_OPEN_IGNORE_IF_LOCKED, mem_root))
goto err;
new_handler->set_optimizer_costs(ha_thd());
return new_handler;
@@ -3240,58 +3283,97 @@ LEX_CSTRING *handler::engine_name()
return hton_name(ht);
}
/*
It is assumed that the value of the parameter 'ranges' can be only 0 or 1.
If ranges == 1 then the function returns the cost of index only scan
by index 'keyno' of one range containing 'rows' key entries.
If ranges == 0 then the function returns only the cost of copying
those key entries into the engine buffers.
Calculate cost for an index scan for given index and number of records.
This function doesn't take in account into copying the key to record
(KEY_COPY_COST) or comparing the key to the where clause (WHERE_COST)
@param index Index to use
@param ranges Number of ranges (b-tree dives in case of b-tree).
Used by partition engine
@param rows Number of expected rows
@param blocks Number of disk blocks to read (from range optimizer).
0 if not known
This function does not take in account into looking up the key,
copying the key to record and finding the next key. These cost are
handled in ha_keyread_time()
*/
double handler::keyread_time(uint index, uint ranges, ha_rows rows)
IO_AND_CPU_COST handler::keyread_time(uint index, ulong ranges, ha_rows rows,
ulonglong blocks)
{
size_t len;
double cost;
DBUG_ASSERT(ranges == 0 || ranges == 1);
len= table->key_info[index].key_length + ref_length;
if (table->file->is_clustering_key(index))
len= table->s->stored_rec_length;
IO_AND_CPU_COST cost;
ulonglong io_blocks= 0;
DBUG_ASSERT(ranges > 0);
cost= ((double)rows*len/(stats.block_size+1) *
INDEX_BLOCK_COPY_COST(table->in_use));
/*
We divide the cost with optimizer_cache_cost as ha_keyread_time()
and ha_key_scan_time() will multiply the result value with
optimizer_cache_cost and we want to keep the above 'memory operation'
cost unaffected by this multiplication.
*/
cost/= optimizer_cache_cost;
if (ranges)
/* memory engine has stats.block_size == 0 */
if (stats.block_size)
{
uint keys_per_block= (uint) (stats.block_size*3/4/len+1);
/*
We let the cost grow slowly in proportion to number of rows to
promote indexes with less rows.
We do not calculate exact number of block reads as then index
only reads will be more costly than normal reads, especially
compared to InnoDB clustered keys.
KEY_LOOKUP_COST is the cost of finding the first key in the
range. Finding the next key is usually a fast operation so we
don't count it here, it is taken into account in
ha_keyread_and_copy_time()
*/
cost+= (((double) (rows / keys_per_block) + KEY_LOOKUP_COST) *
avg_io_cost());
if (!blocks)
{
/* Estimate length of index data */
if (rows <= 1) // EQ_REF optimization
{
blocks= 1;
io_blocks= (stats.block_size + IO_SIZE - 1)/ IO_SIZE;
}
else
{
size_t len= table->key_storage_length(index);
blocks= ((ulonglong) ((rows * len / INDEX_BLOCK_FILL_FACTOR_DIV *
INDEX_BLOCK_FILL_FACTOR_MUL +
stats.block_size-1)) / stats.block_size +
(ranges - 1));
io_blocks= blocks * stats.block_size / IO_SIZE;
}
}
else
io_blocks= blocks * stats.block_size / IO_SIZE;
}
cost.io= (double) io_blocks * avg_io_cost();
cost.cpu= blocks * INDEX_BLOCK_COPY_COST;
return cost;
}
/*
Cost of doing a set of range scans and finding the key position.
This function is used both with index scans (in which case there should be
an additional KEY_COPY_COST) and when normal index + fetch row scan,
in which case there should an additional rnd_pos_time() cost.
*/
double handler::ha_keyread_time(uint index, ulong ranges, ha_rows rows,
ulonglong blocks)
{
if (rows < ranges)
rows= ranges;
IO_AND_CPU_COST cost= keyread_time(index, ranges, rows, blocks);
return (cost.io * DISK_READ_RATIO +
cost.cpu + ranges * KEY_LOOKUP_COST +
(rows - ranges) * KEY_NEXT_FIND_COST);
}
/*
Read a row from a clustered index
Cost is similar to ha_rnd_pos_call_time() as a index_read() on a clusterd
key has identical code as rnd_pos() (At least in InnoDB:)
*/
double handler::ha_keyread_clustered_and_copy_time(uint index, ulong ranges,
ha_rows rows,
ulonglong blocks)
{
if (rows < ranges)
rows= ranges;
IO_AND_CPU_COST cost= keyread_time(index, ranges, rows, blocks);
return (cost.io * DISK_READ_RATIO +
cost.cpu + ranges * ROW_LOOKUP_COST +
(rows - ranges) * ROW_NEXT_FIND_COST +
rows * ROW_COPY_COST);
}
THD *handler::ha_thd(void) const
{
DBUG_ASSERT(!table || !table->in_use || table->in_use == current_thd);
@@ -3364,7 +3446,7 @@ int handler::ha_open(TABLE *table_arg, const char *name, int mode,
name, ht->db_type, table_arg->db_stat, mode,
test_if_locked));
table= table_arg;
set_table(table_arg);
DBUG_ASSERT(table->s == table_share);
DBUG_ASSERT(m_lock_type == F_UNLCK);
DBUG_PRINT("info", ("old m_lock_type: %d F_UNLCK %d", m_lock_type, F_UNLCK));
@@ -3414,14 +3496,15 @@ int handler::ha_open(TABLE *table_arg, const char *name, int mode,
else
dup_ref=ref+ALIGN_SIZE(ref_length);
cached_table_flags= table_flags();
if (!table->s->optimizer_costs_inited)
{
table->s->optimizer_costs_inited=1;
/* Copy data from global 'engine'.optimizer_costs to TABLE_SHARE */
table->s->update_optimizer_costs(partition_ht());
/* Update costs depend on table structure */
update_optimizer_costs(&table->s->optimizer_costs);
}
/* Copy current optimizer costs. Needed in case clone() is used */
set_optimizer_costs(table->in_use);
DBUG_ASSERT(optimizer_key_copy_cost >= 0.0);
DBUG_ASSERT(optimizer_key_next_find_cost >= 0.0);
DBUG_ASSERT(optimizer_row_copy_cost >= 0.0);
DBUG_ASSERT(optimizer_where_cost >= 0.0);
DBUG_ASSERT(optimizer_key_cmp_cost >= 0.0);
reset_statistics();
}
internal_tmp_table= MY_TEST(test_if_locked & HA_OPEN_INTERNAL_TABLE);
@@ -3453,6 +3536,15 @@ int handler::ha_close(void)
DBUG_RETURN(close());
}
void handler::change_table_ptr(TABLE *table_arg, TABLE_SHARE *share)
{
DBUG_ASSERT(table_arg->s == share);
table= table_arg;
table_share= share;
costs= &share->optimizer_costs;
reset_statistics();
}
int handler::ha_rnd_next(uchar *buf)
{
@@ -8767,27 +8859,19 @@ Table_scope_and_contents_source_st::fix_period_fields(THD *thd,
}
/*
Copy common optimizer cost variables to the engine
Copy upper level cost to the engine as part of start statement
This is needed to provide fast acccess to these variables during
optimization (as we refer to them multiple times).
This is needed to provide fast access to these variables during
optimization (as we refer to them multiple times during one query).
The other option would be to access them from thd, but that
would require a function call (as we cannot access THD from
an inline handler function) and two extra memory accesses
for each variable.
index_block_copy_cost is not copied as it is used so seldom.
The other option would be to access them from THD, but that would
require a function call (as we cannot easily access THD from an
inline handler function) and two extra memory accesses for each
variable.
*/
void handler::set_optimizer_costs(THD *thd)
{
optimizer_key_copy_cost= thd->variables.optimizer_key_copy_cost;
optimizer_key_next_find_cost=
thd->variables.optimizer_key_next_find_cost;
optimizer_row_copy_cost= thd->variables.optimizer_row_copy_cost;
optimizer_where_cost= thd->variables.optimizer_where_cost;
optimizer_key_cmp_cost= thd->variables.optimizer_key_cmp_cost;
set_optimizer_cache_cost(thd->optimizer_cache_hit_ratio);
optimizer_scan_setup_cost= thd->variables.optimizer_scan_setup_cost;
}

246
sql/handler.h
View File

@@ -26,9 +26,9 @@
#endif
#include "sql_const.h"
#include "optimizer_costs.h"
#include "sql_basic_types.h"
#include "mysqld.h" /* server_id */
#include "optimizer_costs.h"
#include "sql_plugin.h" /* plugin_ref, st_plugin_int, plugin */
#include "thr_lock.h" /* thr_lock_type, THR_LOCK_DATA */
#include "sql_cache.h"
@@ -36,6 +36,7 @@
#include "sql_array.h" /* Dynamic_array<> */
#include "mdl.h"
#include "vers_string.h"
#include "optimizer_costs.h"
#include "sql_analyze_stmt.h" // for Exec_time_tracker
@@ -1046,6 +1047,7 @@ enum enum_schema_tables
SCH_KEY_CACHES,
SCH_KEY_COLUMN_USAGE,
SCH_OPEN_TABLES,
SCH_OPTIMIZER_COSTS,
SCH_OPT_TRACE,
SCH_PARAMETERS,
SCH_PARTITIONS,
@@ -1496,6 +1498,10 @@ struct handlerton
/* Called for all storage handlers after ddl recovery is done */
void (*signal_ddl_recovery_done)(handlerton *hton);
/* Called at startup to update default engine costs */
void (*update_optimizer_costs)(OPTIMIZER_COSTS *costs);
void *optimizer_costs; /* Costs are stored here */
/*
Optional clauses in the CREATE/ALTER TABLE
*/
@@ -3085,6 +3091,21 @@ enum class Compare_keys : uint32_t
NotEqual
};
/* Cost for reading a row through an index */
struct INDEX_READ_COST
{
double read_cost;
double index_only_cost;
};
/* Separated costs for IO and CPU. For handler::keyread_time() */
struct IO_AND_CPU_COST
{
double io;
double cpu;
};
/**
The handler class is the interface for dynamically loadable
storage engines. Do not add ifdefs and take care when adding or
@@ -3146,6 +3167,7 @@ protected:
handler *lookup_handler;
public:
handlerton *ht; /* storage engine of this handler */
OPTIMIZER_COSTS *costs; /* Points to table->share->costs */
uchar *ref; /* Pointer to current row */
uchar *dup_ref; /* Pointer to duplicate row */
uchar *lookup_buffer;
@@ -3220,15 +3242,6 @@ public:
ulonglong rows_changed;
/* One bigger than needed to avoid to test if key == MAX_KEY */
ulonglong index_rows_read[MAX_KEY+1];
/*
Cost of using key/record cache: (100-cache_hit_ratio)/100
Updated from THD in open_tables()
*/
double optimizer_cache_cost;
double optimizer_key_next_find_cost;
double optimizer_row_copy_cost, optimizer_key_copy_cost;
double optimizer_where_cost, optimizer_key_cmp_cost;
ha_copy_info copy_info;
private:
@@ -3347,13 +3360,15 @@ private:
For non partitioned handlers this is &TABLE_SHARE::ha_share.
*/
Handler_share **ha_share;
double optimizer_where_cost; // Copy of THD->...optimzer_where_cost
double optimizer_scan_setup_cost; // Copy of THD->...optimzer_scan_...
public:
handler(handlerton *ht_arg, TABLE_SHARE *share_arg)
:table_share(share_arg), table(0),
estimation_rows_to_insert(0),
lookup_handler(this),
ht(ht_arg), ref(0), lookup_buffer(NULL), end_range(NULL),
ht(ht_arg), costs(0), ref(0), lookup_buffer(NULL), end_range(NULL),
implicit_emptied(0),
mark_trx_read_write_done(0),
check_table_binlog_row_based_done(0),
@@ -3364,7 +3379,6 @@ public:
ref_length(sizeof(my_off_t)),
ft_handler(0), inited(NONE), pre_inited(NONE),
pushed_cond(0), next_insert_id(0), insert_id_for_cur_row(0),
optimizer_cache_cost((100-DEFAULT_CACHE_HIT_RATIO)/100.0),
tracker(NULL),
pushed_idx_cond(NULL),
pushed_idx_cond_keyno(MAX_KEY),
@@ -3378,12 +3392,19 @@ public:
m_psi_numrows(0),
m_psi_locker(NULL),
row_logging(0), row_logging_init(0),
m_lock_type(F_UNLCK), ha_share(NULL)
m_lock_type(F_UNLCK), ha_share(NULL), optimizer_where_cost(0),
optimizer_scan_setup_cost(0)
{
DBUG_PRINT("info",
("handler created F_UNLCK %d F_RDLCK %d F_WRLCK %d",
F_UNLCK, F_RDLCK, F_WRLCK));
reset_statistics();
/*
The following variables should be updated in set_optimizer_costs()
which is to be run as part of setting up the table for the query
*/
MEM_UNDEFINED(&optimizer_where_cost, sizeof(optimizer_where_cost));
MEM_UNDEFINED(&optimizer_scan_setup_cost, sizeof(optimizer_scan_setup_cost));
}
virtual ~handler(void)
{
@@ -3584,22 +3605,22 @@ public:
bzero(&copy_info, sizeof(copy_info));
reset_copy_info();
}
virtual void change_table_ptr(TABLE *table_arg, TABLE_SHARE *share)
{
table= table_arg;
table_share= share;
reset_statistics();
}
virtual void change_table_ptr(TABLE *table_arg, TABLE_SHARE *share);
/*
Time for a full table data scan. To be overrided by engines, should not
be used by the sql level.
*/
protected:
virtual double scan_time()
virtual IO_AND_CPU_COST scan_time()
{
return (((ulonglong2double(stats.data_file_length) / stats.block_size)) *
avg_io_cost());
IO_AND_CPU_COST cost;
ulonglong length= stats.data_file_length;
cost.io= (double) (length / IO_SIZE) * avg_io_cost();
cost.cpu= (!stats.block_size ? 0.0 :
(double) ((length + stats.block_size-1)/stats.block_size) *
INDEX_BLOCK_COPY_COST);
return cost;
}
public:
@@ -3615,147 +3636,149 @@ public:
a few rows and the extra cost has no practical effect.
*/
inline double ha_scan_time()
inline double ha_scan_time(ha_rows rows)
{
return (scan_time() * optimizer_cache_cost +
TABLE_SCAN_SETUP_COST * avg_io_cost());
IO_AND_CPU_COST cost= scan_time();
return (cost.io * DISK_READ_RATIO +
cost.cpu + TABLE_SCAN_SETUP_COST +
(double) rows * (ROW_NEXT_FIND_COST + ROW_COPY_COST));
}
/*
Time for a full table scan, fetching the rows from the table and comparing
the row with the where clause
*/
inline double ha_scan_and_compare_time(ha_rows records)
inline double ha_scan_and_compare_time(ha_rows rows)
{
return (ha_scan_time() +
(double) records * (ROW_COPY_COST + WHERE_COST));
return ha_scan_time(rows) + (double) rows * WHERE_COST;
}
/* Cost of (random) reading a block of IO_SIZE */
virtual double avg_io_cost()
{
return 1.0;
return DISK_READ_COST;
}
/*
Update table->share optimizer costs for this particular table.
Called once when table is opened the first time.
*/
virtual void update_optimizer_costs(OPTIMIZER_COSTS *costs) {}
/*
Set handler optimizer cost variables.
Called for each table used by the statment
This is virtual mainly for the partition engine.
*/
virtual void set_optimizer_costs(THD *thd);
/*
Set cost for finding a row in the engine cache
This allows the handler to override the cost if there is no
caching of rows, like in heap or federatedx.
*/
virtual void set_optimizer_cache_cost(double cost)
{
optimizer_cache_cost= cost;
}
/**
The cost of reading a set of ranges from the table using an index
to access it.
@param index The index number.
@param ranges The number of ranges to be read. If 0, it means that
we calculate separately the cost of reading the key.
@param rows Total number of rows to be read.
This method can be used to calculate the total cost of scanning a table
using an index by calling it using read_time(index, 1, table_size).
This function is to be reimplemented by engines (if needed). The sql_level
should call ha_read_time(), ha_read_and_copy_time() or
ha_read_and_compare_time().
*/
protected:
virtual double read_time(uint index, uint ranges, ha_rows rows)
/*
Cost of reading 'rows' number of rows with a rowid
*/
virtual IO_AND_CPU_COST rnd_pos_time(ha_rows rows)
{
return ((rows2double(rows) * ROW_LOOKUP_COST +
rows2double(ranges) * KEY_LOOKUP_COST) * avg_io_cost());
double r= rows2double(rows);
return
{
r * avg_io_cost() * stats.block_size/IO_SIZE, // Blocks read
r * INDEX_BLOCK_COPY_COST // Copy block from cache
};
}
public:
/* Same as above, but take into account CACHE_COST */
inline double ha_read_time(uint index, uint ranges, ha_rows rows)
{
return read_time(index, ranges, rows) * optimizer_cache_cost;
}
/* Same as above, but take into account also copying of the row to 'record' */
inline double ha_read_and_copy_time(uint index, uint ranges, ha_rows rows)
{
return (ha_read_time(index, ranges, rows) +
rows2double(rows) * ROW_COPY_COST);
}
/* Same as above, but take into account also copying and comparing the row */
inline double ha_read_and_compare_time(uint index, uint ranges, ha_rows rows)
{
return (ha_read_time(index, ranges, rows) +
rows2double(rows) * (ROW_COPY_COST + WHERE_COST));
}
/* Cost of reading a row with rowid */
protected:
virtual double rnd_pos_time(ha_rows rows)
{
return rows2double(rows) * ROW_LOOKUP_COST * avg_io_cost();
}
public:
/*
Same as above, but take into account cache_cost and copying of the row
to 'record'.
Note that this should normally be same as ha_read_time(some_key, 0, rows)
Time for doing and internal rnd_pos() inside the engine. For some
engine, this is more efficient than the SQL layer calling
rnd_pos() as there is no overhead in converting/checking the
rnd_pos_value. This is used when calculating the cost of fetching
a key+row in one go (like when scanning an index and fetching the
row).
*/
inline double ha_rnd_pos_time(ha_rows rows)
{
return (rnd_pos_time(rows) * optimizer_cache_cost +
rows2double(rows) * ROW_COPY_COST);
IO_AND_CPU_COST cost= rnd_pos_time(rows);
return (cost.io * DISK_READ_RATIO +
cost.cpu + rows2double(rows) * (ROW_LOOKUP_COST + ROW_COPY_COST));
}
/*
This cost if when we are calling rnd_pos() explict in the call
For the moment this function is identical to ha_rnd_pos time,
but that may change in the future after we do more cost checks for
more engines.
*/
inline double ha_rnd_pos_call_time(ha_rows rows)
{
IO_AND_CPU_COST cost= rnd_pos_time(rows);
return (cost.io * DISK_READ_RATIO +
cost.cpu + rows2double(rows) * (ROW_LOOKUP_COST + ROW_COPY_COST));
}
inline double ha_rnd_pos_call_and_compare_time(ha_rows rows)
{
return (ha_rnd_pos_call_time(rows) + rows2double(rows) * WHERE_COST);
}
/**
Calculate cost of 'index_only' scan for given index and number of records.
Calculate cost of 'index_only' scan for given index, a number of reanges
and number of records.
@param index Index to read
@param flag If flag == 1 then the function returns the cost of
index only scan by index 'index' of one range containing
'rows' key entries.
If flag == 0 then function returns only the cost of copying
those key entries into the engine buffers.
@param rows #of records to read
@param blocks Number of IO blocks that needs to be accessed.
0 if not known (in which case it's calculated)
*/
protected:
virtual double keyread_time(uint index, uint flag, ha_rows rows);
virtual IO_AND_CPU_COST keyread_time(uint index, ulong ranges, ha_rows rows,
ulonglong blocks);
public:
/*
Calculate cost of 'keyread' scan for given index and number of records
including fetching the key to the 'record' buffer.
*/
inline double ha_keyread_time(uint index, uint flag, ha_rows rows)
{
return (keyread_time(index, flag, rows) * optimizer_cache_cost);
}
double ha_keyread_time(uint index, ulong ranges, ha_rows rows,
ulonglong blocks);
/* Same as above, but take into account copying the key the the SQL layer */
inline double ha_keyread_and_copy_time(uint index, uint flag, ha_rows rows)
inline double ha_keyread_and_copy_time(uint index, ulong ranges,
ha_rows rows, ulonglong blocks)
{
return ha_keyread_time(index, flag, rows) + (double) rows * KEY_COPY_COST;
return (ha_keyread_time(index, ranges, rows, blocks) +
(double) rows * KEY_COPY_COST);
}
inline double ha_keyread_and_compare_time(uint index, ulong ranges,
ha_rows rows, ulonglong blocks)
{
return (ha_keyread_time(index, ranges, rows, blocks) +
(double) rows * (KEY_COPY_COST + WHERE_COST));
}
double ha_keyread_clustered_and_copy_time(uint index, ulong ranges,
ha_rows rows,
ulonglong blocks);
/*
Time for a full table index scan (without copy or compare cost).
To be overrided by engines, sql level should use ha_key_scan_time().
Note that IO_AND_CPU_COST does not include avg_io_cost() !
*/
protected:
virtual double key_scan_time(uint index)
virtual IO_AND_CPU_COST key_scan_time(uint index, ha_rows rows)
{
return keyread_time(index, 1, records());
return keyread_time(index, 1, MY_MAX(rows, 1), 0);
}
public:
/* Cost of doing a full index scan */
inline double ha_key_scan_time(uint index)
inline double ha_key_scan_time(uint index, ha_rows rows)
{
return (key_scan_time(index) * optimizer_cache_cost);
IO_AND_CPU_COST cost= key_scan_time(index, rows);
return (cost.io * DISK_READ_RATIO +
cost.cpu + INDEX_SCAN_SETUP_COST + KEY_LOOKUP_COST +
(double) rows * (KEY_NEXT_FIND_COST + KEY_COPY_COST));
}
/*
@@ -3764,8 +3787,7 @@ public:
*/
inline double ha_key_scan_and_compare_time(uint index, ha_rows rows)
{
return (ha_key_scan_time(index) +
(double) rows * (KEY_COPY_COST + WHERE_COST));
return ha_key_scan_time(index, rows) + (double) rows * WHERE_COST;
}
virtual const key_map *keys_to_use_for_scanning() { return &key_map_empty; }
@@ -5218,7 +5240,7 @@ public:
ha_share= arg_ha_share;
return false;
}
void set_table(TABLE* table_arg) { table= table_arg; }
inline void set_table(TABLE* table_arg);
int get_lock_type() const { return m_lock_type; }
public:
/* XXX to be removed, see ha_partition::partition_ht() */
@@ -5292,7 +5314,7 @@ protected:
void unlock_shared_ha_data();
/*
Mroonga needs to call read_time() directly for it's internal handler
Mroonga needs to call some xxx_time() directly for it's internal handler
methods
*/
friend class ha_mroonga;

2
sql/item_func.cc
View File

@@ -5920,6 +5920,7 @@ bool Item_func_get_system_var::fix_length_and_dec(THD *thd)
decimals=0;
break;
case SHOW_DOUBLE:
case SHOW_OPTIMIZER_COST:
decimals= 6;
collation= DTCollation_numeric();
fix_char_length(DBL_DIG + 6);
@@ -5977,6 +5978,7 @@ const Type_handler *Item_func_get_system_var::type_handler() const
case SHOW_CHAR_PTR:
case SHOW_LEX_STRING:
return &type_handler_varchar;
case SHOW_OPTIMIZER_COST:
case SHOW_DOUBLE:
return &type_handler_double;
default:

5
sql/json_table.cc
View File

@@ -54,6 +54,7 @@ public:
bzero(&m_hton, sizeof(m_hton));
m_hton.tablefile_extensions= hton_no_exts;
m_hton.slot= HA_SLOT_UNDEF;
m_hton.flags= HTON_HIDDEN;
}
};
@@ -245,6 +246,10 @@ public:
int open(const char *name, int mode, uint test_if_locked) override
{ return 0; }
int close(void) override { return 0; }
void update_optimizer_costs(OPTIMIZER_COSTS *costs)
{
memcpy(costs, &heap_optimizer_costs, sizeof(*costs));
}
int rnd_init(bool scan) override;
int rnd_next(uchar *buf) override;
int rnd_pos(uchar * buf, uchar *pos) override;

141
sql/keycaches.cc
View File

@@ -15,6 +15,10 @@
#include "mariadb.h"
#include "keycaches.h"
#include "optimizer_costs.h"
#include "optimizer_defaults.h"
#include "handler.h"
#include "sql_class.h"
/****************************************************************************
Named list handling
@@ -22,10 +26,13 @@
NAMED_ILIST key_caches;
NAMED_ILIST rpl_filters;
NAMED_ILIST linked_optimizer_costs;
extern "C" PSI_memory_key key_memory_KEY_CACHE;
extern PSI_memory_key key_memory_NAMED_ILINK_name;
LEX_CSTRING default_base= {STRING_WITH_LEN("default")};
/**
ilink (intrusive list element) with a name
*/
@@ -46,7 +53,7 @@ public:
}
inline bool cmp(const char *name_cmp, size_t length)
{
return length == name_length && !memcmp(name, name_cmp, length);
return !system_charset_info->strnncoll(name, name_length, name_cmp, length);
}
~NAMED_ILINK()
{
@@ -72,7 +79,8 @@ uchar* find_named(I_List<NAMED_ILINK> *list, const char *name, size_t length,
}
bool NAMED_ILIST::delete_element(const char *name, size_t length, void (*free_element)(const char *name, void*))
bool NAMED_ILIST::delete_element(const char *name, size_t length,
void (*free_element)(const char *name, void*))
{
I_List_iterator<NAMED_ILINK> it(*this);
NAMED_ILINK *element;
@@ -104,14 +112,12 @@ void NAMED_ILIST::delete_elements(void (*free_element)(const char *name, void*))
/* Key cache functions */
LEX_CSTRING default_key_cache_base= {STRING_WITH_LEN("default")};
KEY_CACHE zero_key_cache; ///< @@nonexistent_cache.param->value_ptr() points here
KEY_CACHE *get_key_cache(const LEX_CSTRING *cache_name)
{
if (!cache_name || ! cache_name->length)
cache_name= &default_key_cache_base;
cache_name= &default_base;
return ((KEY_CACHE*) find_named(&key_caches,
cache_name->str, cache_name->length, 0));
}
@@ -234,3 +240,128 @@ void free_all_rpl_filters()
{
rpl_filters.delete_elements(free_rpl_filter);
}
/******************************************************************************
Optimizer costs functions
******************************************************************************/
LEX_CSTRING default_costs_base= {STRING_WITH_LEN("default")};
OPTIMIZER_COSTS default_optimizer_costs=
{
DEFAULT_DISK_READ_COST, // disk_read_cost
DEFAULT_INDEX_BLOCK_COPY_COST, // index_block_copy_cost
DEFAULT_WHERE_COST/4, // key_cmp_cost
DEFAULT_KEY_COPY_COST, // key_copy_cost
DEFAULT_KEY_LOOKUP_COST, // key_lookup_cost
DEFAULT_KEY_NEXT_FIND_COST, // key_next_find_cost
DEFAULT_DISK_READ_RATIO, // disk_read_ratio
DEFAULT_ROW_COPY_COST, // row_copy_cost
DEFAULT_ROW_LOOKUP_COST, // row_lookup_cost
DEFAULT_ROW_NEXT_FIND_COST, // row_next_find_cost
DEFAULT_ROWID_COMPARE_COST, // rowid_compare_cost
DEFAULT_ROWID_COPY_COST, // rowid_copy_cost
1 // Cannot be deleted
};
OPTIMIZER_COSTS heap_optimizer_costs, tmp_table_optimizer_costs;
OPTIMIZER_COSTS *get_optimizer_costs(const LEX_CSTRING *cache_name)
{
if (!cache_name->length)
return &default_optimizer_costs;
return ((OPTIMIZER_COSTS*) find_named(&linked_optimizer_costs,
cache_name->str, cache_name->length,
0));
}
OPTIMIZER_COSTS *create_optimizer_costs(const char *name, size_t length)
{
OPTIMIZER_COSTS *optimizer_costs;
DBUG_ENTER("create_optimizer_costs");
DBUG_PRINT("enter",("name: %.*s", (int) length, name));
if ((optimizer_costs= (OPTIMIZER_COSTS*)
my_malloc(key_memory_KEY_CACHE,
sizeof(OPTIMIZER_COSTS), MYF(MY_ZEROFILL | MY_WME))))
{
if (!new NAMED_ILINK(&linked_optimizer_costs, name, length,
(uchar*) optimizer_costs))
{
my_free(optimizer_costs);
optimizer_costs= 0;
}
else
{
/* Mark that values are not yet set */
for (uint i=0 ; i < sizeof(OPTIMIZER_COSTS)/sizeof(double) ; i++)
((double*) optimizer_costs)[i]= OPTIMIZER_COST_UNDEF;
}
}
DBUG_RETURN(optimizer_costs);
}
OPTIMIZER_COSTS *get_or_create_optimizer_costs(const char *name, size_t length)
{
LEX_CSTRING optimizer_costs_name;
OPTIMIZER_COSTS *optimizer_costs;
optimizer_costs_name.str= name;
optimizer_costs_name.length= length;
if (!(optimizer_costs= get_optimizer_costs(&optimizer_costs_name)))
optimizer_costs= create_optimizer_costs(name, length);
return optimizer_costs;
}
extern "C"
{
bool process_optimizer_costs(process_optimizer_costs_t func, TABLE *param)
{
I_List_iterator<NAMED_ILINK> it(linked_optimizer_costs);
NAMED_ILINK *element;
int res= 0;
while ((element= it++))
{
LEX_CSTRING name= { element->name, element->name_length };
OPTIMIZER_COSTS *costs= (OPTIMIZER_COSTS *) element->data;
res |= func(&name, costs, param);
}
return res != 0;
}
}
bool create_default_optimizer_costs()
{
return (new NAMED_ILINK(&linked_optimizer_costs,
default_base.str, default_base.length,
(uchar*) &default_optimizer_costs)) == 0;
}
/*
Make a copy of heap and tmp_table engine costs to be able to create
internal temporary tables without taking a mutex.
*/
void copy_tmptable_optimizer_costs()
{
memcpy(&heap_optimizer_costs, heap_hton->optimizer_costs,
sizeof(heap_optimizer_costs));
memcpy(&tmp_table_optimizer_costs, TMP_ENGINE_HTON->optimizer_costs,
sizeof(tmp_table_optimizer_costs));
}
static void free_optimizer_costs(const char *name, void *cost)
{
if ((OPTIMIZER_COSTS*) cost != &default_optimizer_costs)
my_free(cost);
}
void free_all_optimizer_costs()
{
linked_optimizer_costs.delete_elements(free_optimizer_costs);
}

2
sql/keycaches.h
View File

@@ -35,7 +35,7 @@ class NAMED_ILIST: public I_List<NAMED_ILINK>
};
/* For key cache */
extern LEX_CSTRING default_key_cache_base;
extern LEX_CSTRING default_base;
extern KEY_CACHE zero_key_cache;
extern NAMED_ILIST key_caches;

106
sql/multi_range_read.cc
View File

@@ -20,6 +20,7 @@
#include "key.h"
#include "sql_statistics.h"
#include "rowid_filter.h"
#include "optimizer_defaults.h"
/****************************************************************************
* Default MRR implementation (MRR to non-MRR converter)
@@ -302,45 +303,36 @@ handler::multi_range_read_info_const(uint keyno, RANGE_SEQ_IF *seq,
if (total_rows != HA_POS_ERROR)
{
double io_cost= avg_io_cost();
double range_lookup_cost= (io_cost * KEY_LOOKUP_COST *
optimizer_cache_cost);
double key_cost;
set_if_smaller(total_rows, max_rows);
/* The following calculation is the same as in multi_range_read_info(): */
*flags |= HA_MRR_USE_DEFAULT_IMPL;
cost->reset();
cost->avg_io_cost= cost->idx_avg_io_cost= io_cost;
cost->avg_io_cost= cost->idx_avg_io_cost= 0; // Not used!
if (!is_clustering_key(keyno))
{
cost->idx_io_count= (double) io_blocks;
key_cost= ha_keyread_time(keyno, n_ranges, total_rows, io_blocks);
cost->idx_cpu_cost= key_cost;
if (!(*flags & HA_MRR_INDEX_ONLY))
{
cost->idx_cpu_cost= (ha_keyread_time(keyno, 1, total_rows) +
(n_ranges-1) * range_lookup_cost);
cost->cpu_cost= ha_read_time(keyno, 0, total_rows);
cost->copy_cost= rows2double(total_rows) * ROW_COPY_COST;
/* ha_rnd_pos_time includes ROW_COPY_COST */
cost->cpu_cost= ha_rnd_pos_time(total_rows);
}
else
{
/* Index only read */
cost->idx_cpu_cost= (ha_keyread_time(keyno, 1, total_rows) +
(n_ranges-1) * range_lookup_cost);
cost->copy_cost= rows2double(total_rows) * KEY_COPY_COST;
}
}
else
{
/*
Clustered index
If all index dives are to a few blocks, then limit the
ranges used by read_time to the number of dives.
*/
/* Clustered index */
io_blocks+= unassigned_single_point_ranges;
uint limited_ranges= (uint) MY_MIN((ulonglong) n_ranges, io_blocks);
cost->idx_cpu_cost= limited_ranges * range_lookup_cost;
cost->cpu_cost= ha_read_time(keyno, 0, total_rows);
key_cost= ha_keyread_time(keyno, n_ranges, total_rows, io_blocks);
cost->idx_cpu_cost= key_cost;
cost->copy_cost= rows2double(total_rows) * ROW_COPY_COST;
}
cost->comp_cost= (rows2double(total_rows) * WHERE_COST +
@@ -378,7 +370,7 @@ handler::multi_range_read_info_const(uint keyno, RANGE_SEQ_IF *seq,
@param keyno Index number
@param n_ranges Estimated number of ranges (i.e. intervals) in the
range sequence.
@param n_rows Estimated total number of records contained within all
@param total_rows Estimated total number of records contained within all
of the ranges
@param bufsz INOUT IN: Size of the buffer available for use
OUT: Size of the buffer that will be actually used, or
@@ -393,7 +385,7 @@ handler::multi_range_read_info_const(uint keyno, RANGE_SEQ_IF *seq,
other Error or can't perform the requested scan
*/
ha_rows handler::multi_range_read_info(uint keyno, uint n_ranges, uint n_rows,
ha_rows handler::multi_range_read_info(uint keyno, uint n_ranges, uint total_rows,
uint key_parts, uint *bufsz,
uint *flags, Cost_estimate *cost)
{
@@ -410,38 +402,27 @@ ha_rows handler::multi_range_read_info(uint keyno, uint n_ranges, uint n_rows,
/* Produce the same cost as non-MRR code does */
if (!is_clustering_key(keyno))
{
double range_lookup_cost= (avg_io_cost() * KEY_LOOKUP_COST *
optimizer_cache_cost);
/*
idx_io_count could potentially be increased with the number of
index leaf blocks we have to read for finding n_rows.
*/
cost->idx_io_count= n_ranges;
double key_cost= ha_keyread_time(keyno, n_ranges, total_rows, 0);
cost->idx_cpu_cost= key_cost;
if (!(*flags & HA_MRR_INDEX_ONLY))
{
cost->idx_cpu_cost= (keyread_time(keyno, 1, n_rows) +
(n_ranges-1) * range_lookup_cost);
cost->cpu_cost= read_time(keyno, 0, n_rows);
cost->copy_cost= rows2double(n_rows) * ROW_COPY_COST;
/* ha_rnd_pos_time includes ROW_COPY_COST */
cost->cpu_cost= ha_rnd_pos_time(total_rows);
}
else
{
/*
Same as above, but take into account copying the key to the upper
level.
*/
cost->idx_cpu_cost= (keyread_time(keyno, 1, n_rows) +
(n_ranges-1) * range_lookup_cost);
cost->copy_cost= rows2double(n_rows) * KEY_COPY_COST;
/* Index only read */
cost->copy_cost= rows2double(total_rows) * KEY_COPY_COST;
}
}
else
{
/* Clustering key */
cost->cpu_cost= read_time(keyno, n_ranges, n_rows);
cost->copy_cost= rows2double(n_rows) * ROW_COPY_COST;
cost->cpu_cost= ha_keyread_time(keyno, n_ranges, total_rows, 0);
cost->copy_cost= rows2double(total_rows) * ROW_COPY_COST;
}
cost->comp_cost= rows2double(n_rows) * WHERE_COST;
cost->comp_cost= rows2double(total_rows) * WHERE_COST;
return 0;
}
@@ -2043,7 +2024,7 @@ bool DsMrr_impl::get_disk_sweep_mrr_cost(uint keynr, ha_rows rows, uint flags,
cost->mem_cost= (double)rows_in_last_step * elem_size;
/* Total cost of all index accesses */
index_read_cost= primary_file->ha_keyread_and_copy_time(keynr, 1, rows);
index_read_cost= primary_file->ha_keyread_and_copy_time(keynr, 1, rows, 0);
cost->add_io(index_read_cost, 1 /* Random seeks */);
return FALSE;
}
@@ -2081,42 +2062,6 @@ void get_sort_and_sweep_cost(TABLE *table, ha_rows nrows, Cost_estimate *cost)
/**
Get cost of reading nrows table records in a "disk sweep"
A disk sweep read is a sequence of handler->rnd_pos(rowid) calls that made
for an ordered sequence of rowids.
We assume hard disk IO. The read is performed as follows:
1. The disk head is moved to the needed cylinder
2. The controller waits for the plate to rotate
3. The data is transferred
Time to do #3 is insignificant compared to #2+#1.
Time to move the disk head is proportional to head travel distance.
Time to wait for the plate to rotate depends on whether the disk head
was moved or not.
If disk head wasn't moved, the wait time is proportional to distance
between the previous block and the block we're reading.
If the head was moved, we don't know how much we'll need to wait for the
plate to rotate. We assume the wait time to be a variate with a mean of
0.5 of full rotation time.
Our cost units are "random disk seeks". The cost of random disk seek is
actually not a constant, it depends one range of cylinders we're going
to access. We make it constant by introducing a fuzzy concept of "typical
datafile length" (it's fuzzy as it's hard to tell whether it should
include index file, temp.tables etc). Then random seek cost is:
1 = half_rotation_cost + move_cost * 1/3 * typical_data_file_length
We define half_rotation_cost as DISK_SEEK_BASE_COST=0.9.
If handler::avg_io_cost() < 1.0, then we will trust the handler
when it comes to the average cost (this is for example true for HEAP).
@param table Table to be accessed
@param nrows Number of rows to retrieve
@param interrupted TRUE <=> Assume that the disk sweep will be
@@ -2131,8 +2076,7 @@ void get_sweep_read_cost(TABLE *table, ha_rows nrows, bool interrupted,
cost->reset();
#ifndef OLD_SWEEP_COST
cost->cpu_cost= table->file->ha_rnd_pos_time(nrows);
cost->avg_io_cost= table->file->avg_io_cost();
cost->cpu_cost= table->file->ha_rnd_pos_call_time(nrows);
#else
if (table->file->pk_is_clustering_key(table->s->primary_key))
{

111
sql/mysqld.cc
View File

@@ -52,6 +52,7 @@
#include "sql_expression_cache.h" // subquery_cache_miss, subquery_cache_hit
#include "sys_vars_shared.h"
#include "ddl_log.h"
#include "optimizer_defaults.h"
#include <m_ctype.h>
#include <my_dir.h>
@@ -733,7 +734,7 @@ mysql_mutex_t LOCK_prepared_stmt_count;
#ifdef HAVE_OPENSSL
mysql_mutex_t LOCK_des_key_file;
#endif
mysql_mutex_t LOCK_backup_log;
mysql_mutex_t LOCK_backup_log, LOCK_optimizer_costs;
mysql_rwlock_t LOCK_grant, LOCK_sys_init_connect, LOCK_sys_init_slave;
mysql_rwlock_t LOCK_ssl_refresh;
mysql_rwlock_t LOCK_all_status_vars;
@@ -903,7 +904,7 @@ PSI_mutex_key key_BINLOG_LOCK_index, key_BINLOG_LOCK_xid_list,
key_LOCK_crypt, key_LOCK_delayed_create,
key_LOCK_delayed_insert, key_LOCK_delayed_status, key_LOCK_error_log,
key_LOCK_gdl, key_LOCK_global_system_variables,
key_LOCK_manager, key_LOCK_backup_log,
key_LOCK_manager, key_LOCK_backup_log, key_LOCK_optimizer_costs,
key_LOCK_prepared_stmt_count,
key_LOCK_rpl_status, key_LOCK_server_started,
key_LOCK_status, key_LOCK_temp_pool,
@@ -966,6 +967,7 @@ static PSI_mutex_info all_server_mutexes[]=
{ &key_hash_filo_lock, "hash_filo::lock", 0},
{ &key_LOCK_active_mi, "LOCK_active_mi", PSI_FLAG_GLOBAL},
{ &key_LOCK_backup_log, "LOCK_backup_log", PSI_FLAG_GLOBAL},
{ &key_LOCK_optimizer_costs, "LOCK_optimizer_costs", PSI_FLAG_GLOBAL},
{ &key_LOCK_temp_pool, "LOCK_temp_pool", PSI_FLAG_GLOBAL},
{ &key_LOCK_thread_id, "LOCK_thread_id", PSI_FLAG_GLOBAL},
{ &key_LOCK_crypt, "LOCK_crypt", PSI_FLAG_GLOBAL},
@@ -2006,6 +2008,7 @@ static void clean_up(bool print_message)
mdl_destroy();
dflt_key_cache= 0;
key_caches.delete_elements(free_key_cache);
free_all_optimizer_costs();
wt_end();
multi_keycache_free();
sp_cache_end();
@@ -2128,6 +2131,7 @@ static void clean_up_mutexes()
mysql_mutex_destroy(&LOCK_active_mi);
mysql_rwlock_destroy(&LOCK_ssl_refresh);
mysql_mutex_destroy(&LOCK_backup_log);
mysql_mutex_destroy(&LOCK_optimizer_costs);
mysql_mutex_destroy(&LOCK_temp_pool);
mysql_rwlock_destroy(&LOCK_sys_init_connect);
mysql_rwlock_destroy(&LOCK_sys_init_slave);
@@ -4502,6 +4506,8 @@ static int init_thread_environment()
mysql_mutex_init(key_LOCK_commit_ordered, &LOCK_commit_ordered,
MY_MUTEX_INIT_SLOW);
mysql_mutex_init(key_LOCK_backup_log, &LOCK_backup_log, MY_MUTEX_INIT_FAST);
mysql_mutex_init(key_LOCK_optimizer_costs, &LOCK_optimizer_costs,
MY_MUTEX_INIT_FAST);
mysql_mutex_init(key_LOCK_temp_pool, &LOCK_temp_pool, MY_MUTEX_INIT_FAST);
#ifdef HAVE_OPENSSL
@@ -4998,10 +5004,10 @@ static int init_server_components()
/* need to configure logging before initializing storage engines */
if (!opt_bin_log_used && !WSREP_ON)
{
if (opt_log_slave_updates)
if (opt_log_slave_updates && (global_system_variables.log_warnings > 1))
sql_print_warning("You need to use --log-bin to make "
"--log-slave-updates work.");
if (binlog_format_used)
if (binlog_format_used && (global_system_variables.log_warnings > 1))
sql_print_warning("You need to use --log-bin to make "
"--binlog-format work.");
}
@@ -5432,6 +5438,7 @@ static int init_server_components()
unireg_abort(1);
}
#endif
copy_tmptable_optimizer_costs();
#ifdef WITH_WSREP
/*
@@ -5490,7 +5497,7 @@ static int init_server_components()
}
else
{
if (binlog_expire_logs_seconds)
if (binlog_expire_logs_seconds && (global_system_variables.log_warnings > 1))
sql_print_warning("You need to use --log-bin to make --expire-logs-days "
"or --binlog-expire-logs-seconds work.");
}
@@ -7825,12 +7832,17 @@ static int mysql_init_variables(void)
strnmov(server_version, MYSQL_SERVER_VERSION, sizeof(server_version)-1);
thread_cache.init();
key_caches.empty();
if (!(dflt_key_cache= get_or_create_key_cache(default_key_cache_base.str,
default_key_cache_base.length)))
if (!(dflt_key_cache= get_or_create_key_cache(default_base.str,
default_base.length)))
{
sql_print_error("Cannot allocate the keycache");
return 1;
}
if (create_default_optimizer_costs())
{
sql_print_error("Cannot allocate optimizer_costs");
return 1;
}
/* set key_cache_hash.default_value = dflt_key_cache */
multi_keycache_init();
@@ -8404,11 +8416,14 @@ mysqld_get_one_option(const struct my_option *opt, const char *argument,
}
/** Handle arguments for multiple key caches. */
/**
Handle arguments for multiple key caches, replication_options and
optimizer_costs
*/
C_MODE_START
static void*
static void *
mysql_getopt_value(const char *name, uint length,
const struct my_option *option, int *error)
{
@@ -8446,6 +8461,7 @@ mysql_getopt_value(const char *name, uint length,
}
/* We return in all cases above. Let us silence -Wimplicit-fallthrough */
DBUG_ASSERT(0);
break;
#ifdef HAVE_REPLICATION
/* fall through */
case OPT_REPLICATE_DO_DB:
@@ -8473,11 +8489,87 @@ mysql_getopt_value(const char *name, uint length,
}
return 0;
}
break;
#endif
case OPT_COSTS_DISK_READ_COST:
case OPT_COSTS_INDEX_BLOCK_COPY_COST:
case OPT_COSTS_KEY_CMP_COST:
case OPT_COSTS_KEY_COPY_COST:
case OPT_COSTS_KEY_LOOKUP_COST:
case OPT_COSTS_KEY_NEXT_FIND_COST:
case OPT_COSTS_DISK_READ_RATIO:
case OPT_COSTS_ROW_COPY_COST:
case OPT_COSTS_ROW_LOOKUP_COST:
case OPT_COSTS_ROW_NEXT_FIND_COST:
case OPT_COSTS_ROWID_CMP_COST:
case OPT_COSTS_ROWID_COPY_COST:
{
OPTIMIZER_COSTS *costs;
if (unlikely(!(costs= get_or_create_optimizer_costs(name, length))))
{
if (error)
*error= EXIT_OUT_OF_MEMORY;
return 0;
}
switch (option->id) {
case OPT_COSTS_DISK_READ_COST:
return &costs->disk_read_cost;
case OPT_COSTS_INDEX_BLOCK_COPY_COST:
return &costs->index_block_copy_cost;
case OPT_COSTS_KEY_CMP_COST:
return &costs->key_cmp_cost;
case OPT_COSTS_KEY_COPY_COST:
return &costs->key_copy_cost;
case OPT_COSTS_KEY_LOOKUP_COST:
return &costs->key_lookup_cost;
case OPT_COSTS_KEY_NEXT_FIND_COST:
return &costs->key_next_find_cost;
case OPT_COSTS_DISK_READ_RATIO:
return &costs->disk_read_ratio;
case OPT_COSTS_ROW_COPY_COST:
return &costs->row_copy_cost;
case OPT_COSTS_ROW_LOOKUP_COST:
return &costs->row_lookup_cost;
case OPT_COSTS_ROW_NEXT_FIND_COST:
return &costs->row_next_find_cost;
case OPT_COSTS_ROWID_CMP_COST:
return &costs->rowid_cmp_cost;
case OPT_COSTS_ROWID_COPY_COST:
return &costs->rowid_copy_cost;
default:
DBUG_ASSERT(0);
}
}
}
return option->value;
}
static void
mariadb_getopt_adjust_value(const struct my_option *option, void *value)
{
switch (option->id) {
case OPT_COSTS_DISK_READ_COST:
case OPT_COSTS_INDEX_BLOCK_COPY_COST:
case OPT_COSTS_KEY_CMP_COST:
case OPT_COSTS_KEY_COPY_COST:
case OPT_COSTS_KEY_LOOKUP_COST:
case OPT_COSTS_KEY_NEXT_FIND_COST:
case OPT_COSTS_DISK_READ_RATIO:
case OPT_COSTS_ROW_COPY_COST:
case OPT_COSTS_ROW_LOOKUP_COST:
case OPT_COSTS_ROW_NEXT_FIND_COST:
case OPT_COSTS_ROWID_CMP_COST:
case OPT_COSTS_ROWID_COPY_COST:
/* Value from command is line given in usec. Convert to ms */
*(double*) value= *(double*) value/1000.0;
break;
default:
break;
}
}
static void option_error_reporter(enum loglevel level, const char *format, ...)
{
va_list args;
@@ -8516,6 +8608,7 @@ static int get_options(int *argc_ptr, char ***argv_ptr)
my_getopt_get_addr= mysql_getopt_value;
my_getopt_error_reporter= option_error_reporter;
my_getopt_adjust_value= mariadb_getopt_adjust_value;
/* prepare all_options array */
my_init_dynamic_array(PSI_INSTRUMENT_ME, &all_options, sizeof(my_option),

17
sql/mysqld.h
View File

@@ -330,7 +330,7 @@ extern PSI_mutex_key key_BINLOG_LOCK_index, key_BINLOG_LOCK_xid_list,
key_LOCK_logger, key_LOCK_manager,
key_LOCK_prepared_stmt_count,
key_LOCK_rpl_status, key_LOCK_server_started,
key_LOCK_status,
key_LOCK_status, key_LOCK_optimizer_costs,
key_LOCK_thd_data, key_LOCK_thd_kill,
key_LOCK_user_conn, key_LOG_LOCK_log,
key_master_info_data_lock, key_master_info_run_lock,
@@ -759,7 +759,8 @@ extern mysql_mutex_t
LOCK_error_log, LOCK_delayed_insert, LOCK_short_uuid_generator,
LOCK_delayed_status, LOCK_delayed_create, LOCK_crypt, LOCK_timezone,
LOCK_active_mi, LOCK_manager, LOCK_user_conn,
LOCK_prepared_stmt_count, LOCK_error_messages, LOCK_backup_log;
LOCK_prepared_stmt_count, LOCK_error_messages, LOCK_backup_log,
LOCK_optimizer_costs;
extern MYSQL_PLUGIN_IMPORT mysql_mutex_t LOCK_global_system_variables;
extern mysql_rwlock_t LOCK_all_status_vars;
extern mysql_mutex_t LOCK_start_thread;
@@ -794,6 +795,18 @@ enum options_mysqld
OPT_BINLOG_IGNORE_DB,
OPT_BIN_LOG,
OPT_BOOTSTRAP,
OPT_COSTS_DISK_READ_COST,
OPT_COSTS_INDEX_BLOCK_COPY_COST,
OPT_COSTS_KEY_CMP_COST,
OPT_COSTS_KEY_COPY_COST,
OPT_COSTS_KEY_LOOKUP_COST,
OPT_COSTS_KEY_NEXT_FIND_COST,
OPT_COSTS_DISK_READ_RATIO,
OPT_COSTS_ROW_COPY_COST,
OPT_COSTS_ROW_LOOKUP_COST,
OPT_COSTS_ROW_NEXT_FIND_COST,
OPT_COSTS_ROWID_CMP_COST,
OPT_COSTS_ROWID_COPY_COST,
OPT_EXPIRE_LOGS_DAYS,
OPT_BINLOG_EXPIRE_LOGS_SECONDS,
OPT_CONSOLE,

115
sql/opt_range.cc
View File

@@ -2740,7 +2740,7 @@ int SQL_SELECT::test_quick_select(THD *thd, key_map keys_to_use,
table_info.add_table_name(head);
Json_writer_object trace_range(thd, "range_analysis");
if (unlikely(thd->trace_started()))
if (unlikely(thd->trace_started()) && read_time != DBL_MAX)
{
Json_writer_object table_rec(thd, "table_scan");
table_rec.add("rows", records).add("cost", read_time);
@@ -2867,10 +2867,11 @@ int SQL_SELECT::test_quick_select(THD *thd, key_map keys_to_use,
thd->mem_root= &alloc;
/* Calculate cost of full index read for the shortest covering index */
if (!force_quick_range && !head->covering_keys.is_clear_all())
if (!force_quick_range && !head->covering_keys.is_clear_all() &&
!head->no_keyread)
{
int key_for_use= find_shortest_key(head, &head->covering_keys);
double key_read_time;
uint key_for_use= find_shortest_key(head, &head->covering_keys);
key_read_time= head->file->ha_key_scan_and_compare_time(key_for_use,
records);
DBUG_PRINT("info", ("'all'+'using index' scan will be using key %d, "
@@ -3057,7 +3058,8 @@ int SQL_SELECT::test_quick_select(THD *thd, key_map keys_to_use,
param.table->set_opt_range_condition_rows(group_trp->records);
DBUG_PRINT("info", ("table_rows: %llu opt_range_condition_rows: %llu "
"group_trp->records: %ull",
table_records, param.table->opt_range_condition_rows,
table_records,
param.table->opt_range_condition_rows,
group_trp->records));
Json_writer_object grp_summary(thd, "best_group_range_summary");
@@ -5079,7 +5081,7 @@ static double get_sweep_read_cost(const PARAM *param, ha_rows records,
{
DBUG_ENTER("get_sweep_read_cost");
#ifndef OLD_SWEEP_COST
double cost= (param->table->file->ha_rnd_pos_time(records) +
double cost= (param->table->file->ha_rnd_pos_call_time(records) +
(add_time_for_compare ?
records * param->thd->variables.optimizer_where_cost : 0));
DBUG_PRINT("return", ("cost: %g", cost));
@@ -5095,7 +5097,7 @@ static double get_sweep_read_cost(const PARAM *param, ha_rows records,
We are using the primary key to find the rows.
Calculate the cost for this.
*/
result= table->file->ha_rnd_pos_time(records);
result= table->file->ha_rnd_pos_call_time(records);
}
else
{
@@ -5133,7 +5135,7 @@ static double get_sweep_read_cost(const PARAM *param, ha_rows records,
*/
result= busy_blocks;
}
result+= rows2double(n_rows) * ROW_COPY_COST_THD(param->table->thd);
result+= rows2double(n_rows) * param->table->file->ROW_COPY_COST);
}
DBUG_PRINT("return",("cost: %g", result));
DBUG_RETURN(result);
@@ -5347,7 +5349,7 @@ TABLE_READ_PLAN *get_best_disjunct_quick(PARAM *param, SEL_IMERGE *imerge,
is done in QUICK_RANGE_SELECT::row_in_ranges)
*/
double rid_comp_cost= (rows2double(non_cpk_scan_records) *
ROWID_COMPARE_COST_THD(param->thd));
default_optimizer_costs.rowid_cmp_cost);
imerge_cost+= rid_comp_cost;
trace_best_disjunct.add("cost_of_mapping_rowid_in_non_clustered_pk_scan",
rid_comp_cost);
@@ -5359,7 +5361,7 @@ TABLE_READ_PLAN *get_best_disjunct_quick(PARAM *param, SEL_IMERGE *imerge,
double sweep_cost= get_sweep_read_cost(param, non_cpk_scan_records, 0);
imerge_cost+= sweep_cost;
trace_best_disjunct.
add("records", non_cpk_scan_records).
add("rows", non_cpk_scan_records).
add("cost_sort_rowid_and_read_disk", sweep_cost).
add("cost", imerge_cost);
}
@@ -5389,7 +5391,7 @@ TABLE_READ_PLAN *get_best_disjunct_quick(PARAM *param, SEL_IMERGE *imerge,
}
{
const double dup_removal_cost= Unique::get_use_cost(
const double dup_removal_cost= Unique::get_use_cost(thd,
param->imerge_cost_buff, (uint)non_cpk_scan_records,
param->table->file->ref_length,
(size_t)param->thd->variables.sortbuff_size,
@@ -5463,10 +5465,9 @@ skip_to_ror_scan:
double cost;
if ((*cur_child)->is_ror)
{
/* Ok, we have index_only cost, now get full rows scan cost */
/* Ok, we have index_only cost, now get full rows lokoup cost */
cost= param->table->file->
ha_read_and_compare_time(param->real_keynr[(*cur_child)->key_idx], 1,
(*cur_child)->records);
ha_rnd_pos_call_and_compare_time((*cur_child)->records);
}
else
cost= read_time;
@@ -5935,7 +5936,7 @@ bool prepare_search_best_index_intersect(PARAM *param,
continue;
}
cost= table->opt_range[(*index_scan)->keynr].index_only_fetch_cost(thd);
cost= table->opt_range[(*index_scan)->keynr].index_only_fetch_cost(table);
idx_scan.add("cost", cost);
@@ -6041,7 +6042,7 @@ bool prepare_search_best_index_intersect(PARAM *param,
selected_idx.add("index", key_info->name);
print_keyparts(thd, key_info, (*scan_ptr)->used_key_parts);
selected_idx.
add("records", (*scan_ptr)->records).
add("rows", (*scan_ptr)->records).
add("filtered_records", (*scan_ptr)->filtered_out);
}
}
@@ -6058,7 +6059,7 @@ bool prepare_search_best_index_intersect(PARAM *param,
selected_idx.add("index", key_info->name);
print_keyparts(thd, key_info, (*scan_ptr)->used_key_parts);
selected_idx.
add("records", (*scan_ptr)->records).
add("rows", (*scan_ptr)->records).
add("filtered_records", (*scan_ptr)->filtered_out);
}
}
@@ -6324,7 +6325,8 @@ double get_cpk_filter_cost(ha_rows filtered_records,
*/
static
bool check_index_intersect_extension(PARTIAL_INDEX_INTERSECT_INFO *curr,
bool check_index_intersect_extension(THD *thd,
PARTIAL_INDEX_INTERSECT_INFO *curr,
INDEX_SCAN_INFO *ext_index_scan,
PARTIAL_INDEX_INTERSECT_INFO *next)
{
@@ -6371,7 +6373,7 @@ bool check_index_intersect_extension(PARTIAL_INDEX_INTERSECT_INFO *curr,
size_t max_memory_size= common_info->max_memory_size;
records_sent_to_unique+= ext_index_scan_records;
cost= Unique::get_use_cost(buff_elems, (size_t) records_sent_to_unique,
cost= Unique::get_use_cost(thd, buff_elems, (size_t) records_sent_to_unique,
key_size,
max_memory_size, compare_factor, TRUE,
&next->in_memory);
@@ -6382,7 +6384,7 @@ bool check_index_intersect_extension(PARTIAL_INDEX_INTERSECT_INFO *curr,
double cost2;
bool in_memory2;
ha_rows records2= records_sent_to_unique-records_filtered_out_by_cpk;
cost2= Unique::get_use_cost(buff_elems, (size_t) records2, key_size,
cost2= Unique::get_use_cost(thd, buff_elems, (size_t) records2, key_size,
max_memory_size, compare_factor, TRUE,
&in_memory2);
cost2+= get_cpk_filter_cost(ext_index_scan_records, common_info->cpk_scan,
@@ -6442,7 +6444,8 @@ bool check_index_intersect_extension(PARTIAL_INDEX_INTERSECT_INFO *curr,
*/
static
void find_index_intersect_best_extension(PARTIAL_INDEX_INTERSECT_INFO *curr)
void find_index_intersect_best_extension(THD *thd,
PARTIAL_INDEX_INTERSECT_INFO *curr)
{
PARTIAL_INDEX_INTERSECT_INFO next;
COMMON_INDEX_INTERSECT_INFO *common_info= curr->common_info;
@@ -6475,8 +6478,9 @@ void find_index_intersect_best_extension(PARTIAL_INDEX_INTERSECT_INFO *curr)
{
*rem_first_index_scan_ptr= *index_scan_ptr;
*index_scan_ptr= rem_first_index_scan;
if (check_index_intersect_extension(curr, *rem_first_index_scan_ptr, &next))
find_index_intersect_best_extension(&next);
if (check_index_intersect_extension(thd, curr, *rem_first_index_scan_ptr,
&next))
find_index_intersect_best_extension(thd, &next);
*index_scan_ptr= *rem_first_index_scan_ptr;
*rem_first_index_scan_ptr= rem_first_index_scan;
}
@@ -6528,7 +6532,7 @@ TRP_INDEX_INTERSECT *get_best_index_intersect(PARAM *param, SEL_TREE *tree,
read_time))
DBUG_RETURN(NULL);
find_index_intersect_best_extension(&init);
find_index_intersect_best_extension(thd, &init);
if (common.best_length <= 1 && !common.best_uses_cpk)
DBUG_RETURN(NULL);
@@ -6697,7 +6701,7 @@ ROR_SCAN_INFO *make_ror_scan(const PARAM *param, int idx, SEL_ARG *sel_arg)
*/
ror_scan->index_read_cost=
param->table->file->ha_keyread_and_copy_time(ror_scan->keynr, 1,
ror_scan->records);
ror_scan->records, 0);
DBUG_RETURN(ror_scan);
}
@@ -13885,10 +13889,10 @@ get_best_group_min_max(PARAM *param, SEL_TREE *tree, double read_time)
cause= "not single_table";
else if (join->select_lex->olap == ROLLUP_TYPE) /* Check (B3) for ROLLUP */
cause= "rollup";
else if (table->s->keys == 0) /* There are no indexes to use. */
else if (table->s->keys == 0) // There are no indexes to use.
cause= "no index";
else if (join->conds && join->conds->used_tables()
& OUTER_REF_TABLE_BIT) /* Cannot execute with correlated conditions. */
& OUTER_REF_TABLE_BIT) // Cannot execute with correlated conditions.
cause= "correlated conditions";
if (cause)
@@ -14093,7 +14097,8 @@ get_best_group_min_max(PARAM *param, SEL_TREE *tree, double read_time)
does not qualify as covering in our case. If this is the case, below
we check that all query fields are indeed covered by 'cur_index'.
*/
if (cur_index_info->user_defined_key_parts == table->actual_n_key_parts(cur_index_info)
if (cur_index_info->user_defined_key_parts ==
table->actual_n_key_parts(cur_index_info)
&& pk < MAX_KEY && cur_index != pk &&
(table->file->ha_table_flags() & HA_PRIMARY_KEY_IN_READ_INDEX))
{
@@ -14136,7 +14141,8 @@ get_best_group_min_max(PARAM *param, SEL_TREE *tree, double read_time)
first Item? If so, then why? What is the array for?
*/
/* Above we already checked that all group items are fields. */
DBUG_ASSERT((*tmp_group->item)->real_item()->type() == Item::FIELD_ITEM);
DBUG_ASSERT((*tmp_group->item)->real_item()->type() ==
Item::FIELD_ITEM);
Item_field *group_field= (Item_field *) (*tmp_group->item)->real_item();
if (group_field->field->eq(cur_part->field))
{
@@ -15000,24 +15006,28 @@ void cost_group_min_max(TABLE* table, KEY *index_info, uint used_key_parts,
bool have_min, bool have_max,
double *read_cost, ha_rows *records)
{
uint keys_per_block, key_length;
ha_rows table_records;
ha_rows num_groups;
ha_rows num_blocks;
uint keys_per_block;
ha_rows keys_per_group;
ha_rows keys_per_subgroup; /* Average number of keys in sub-groups */
/* formed by a key infix. */
double p_overlap; /* Probability that a sub-group overlaps two blocks. */
double quick_prefix_selectivity;
double io_cost;
handler *file= table->file;
DBUG_ENTER("cost_group_min_max");
/* Same code as in handler::key_read_time() */
table_records= table->stat_records();
/* Assume block is 75 % full */
keys_per_block= (uint) (table->file->stats.block_size * 3 / 4 /
(index_info->key_length + table->file->ref_length)
+ 1);
num_blocks= (ha_rows)(table_records / keys_per_block) + 1;
key_length= (index_info->key_length + file->ref_length);
num_blocks= (table_records * key_length / INDEX_BLOCK_FILL_FACTOR_DIV *
INDEX_BLOCK_FILL_FACTOR_MUL) / file->stats.block_size + 1;
keys_per_block= (file->stats.block_size /
(key_length * INDEX_BLOCK_FILL_FACTOR_MUL /
INDEX_BLOCK_FILL_FACTOR_DIV) +
1);
/* Compute the number of keys in a group. */
if (!group_key_parts)
@@ -15035,7 +15045,10 @@ void cost_group_min_max(TABLE* table, KEY *index_info, uint used_key_parts,
keys_per_group= (table_records / 10) + 1;
}
}
if (keys_per_group > 1)
num_groups= (table_records / keys_per_group) + 1;
else
num_groups= table_records;
/* Apply the selectivity of the quick select for group prefixes. */
if (range_tree && (quick_prefix_records != HA_POS_ERROR))
@@ -15059,8 +15072,7 @@ void cost_group_min_max(TABLE* table, KEY *index_info, uint used_key_parts,
/* There cannot be more groups than matched records */
set_if_smaller(num_groups, quick_prefix_records);
}
/* Ensure we don't have more groups than rows in table */
set_if_smaller(num_groups, table_records);
DBUG_ASSERT(num_groups <= table_records);
if (used_key_parts > group_key_parts)
{
@@ -15081,39 +15093,22 @@ void cost_group_min_max(TABLE* table, KEY *index_info, uint used_key_parts,
io_cost= (double) MY_MIN(num_groups * (1 + p_overlap), num_blocks);
}
else
io_cost= (keys_per_group > keys_per_block) ?
io_cost= ((keys_per_group > keys_per_block) ?
(have_min && have_max) ? (double) (num_groups + 1) :
(double) num_groups :
(double) num_blocks;
(double) num_blocks);
/*
CPU cost must be comparable to that of an index scan as computed
in SQL_SELECT::test_quick_select(). When the groups are small,
e.g. for a unique index, using index scan will be cheaper since it
reads the next record without having to re-position to it on every
group. To make the CPU cost reflect this, we estimate the CPU cost
as the sum of:
1. Cost for evaluating the condition for each num_group
KEY_COMPARE_COST (similarly as for index scan).
2. Cost for navigating the index structure (assuming a b-tree).
Note: We only add the cost for one index comparision per block. For a
b-tree the number of comparisons will be larger. However the cost
is low as all of the upper level b-tree blocks should be in
memory.
TODO: This cost should be provided by the storage engine.
3. Cost for comparing the row with the where clause
group.
*/
const THD *thd= table->in_use;
const double tree_traversal_cost=
ceil(log(static_cast<double>(table_records))/
log(static_cast<double>(keys_per_block))) *
thd->variables.optimizer_key_cmp_cost;
const double cpu_cost= (num_groups *
(tree_traversal_cost +
thd->variables.optimizer_where_cost));
*read_cost= io_cost + cpu_cost;
uint keyno= (uint) (index_info - table->key_info);
*read_cost= file->ha_keyread_and_compare_time(keyno, (ulong) num_groups,
num_groups,
io_cost);
*records= num_groups;
DBUG_PRINT("info",

33
sql/opt_split.cc
View File

@@ -188,6 +188,7 @@
#include "mariadb.h"
#include "sql_select.h"
#include "opt_trace.h"
#include "optimizer_defaults.h"
/* Info on a splitting field */
struct SplM_field_info
@@ -665,6 +666,8 @@ add_ext_keyuses_for_splitting_field(Dynamic_array<KEYUSE_EXT> *ext_keyuses,
/*
@brief
Cost of the post join operation used in specification of splittable table
This does not include the cost of creating the temporary table as this
operation can be executed many times for the same temporary table.
*/
static
@@ -673,13 +676,18 @@ double spl_postjoin_oper_cost(THD *thd, double join_record_count, uint rec_len)
double cost;
TMPTABLE_COSTS tmp_cost= get_tmp_table_costs(thd, join_record_count,
rec_len, 0, 1);
// cost to fill tmp table
cost= tmp_cost.create + tmp_cost.write * join_record_count;
// cost to perform post join operation used here
/* cost to fill tmp table */
cost= tmp_cost.write * join_record_count;
/* cost to perform post join operation used here */
cost+= tmp_cost.lookup * join_record_count;
cost+= (join_record_count == 0 ? 0 :
/* cost to preform sorting */
/* QQQ
We should use cost_of_filesort() for computing sort.
Do we always preform sorting ? If not, this should be done conditionally
*/
cost+= ((join_record_count == 0 ? 0 :
join_record_count * log2 (join_record_count)) *
SORT_INDEX_CMP_COST; // cost to perform sorting
SORT_INDEX_CMP_COST);
return cost;
}
@@ -873,7 +881,7 @@ void reset_validity_vars_for_keyuses(KEYUSE_EXT *key_keyuse_ext_start,
splitting the function set it as the true plan of materialization
of the table T.
The function caches the found plans for materialization of table T
together if the info what key was used for splitting. Next time when
together with the info what key was used for splitting. Next time when
the optimizer prefers to use the same key the plan is taken from
the cache of plans
@@ -1004,12 +1012,11 @@ SplM_plan_info * JOIN_TAB::choose_best_splitting(double record_count,
spl_opt_info->unsplit_card : 1);
uint rec_len= table->s->rec_buff_length;
double split_card= spl_opt_info->unsplit_card * spl_plan->split_sel;
double oper_cost= split_card *
spl_postjoin_oper_cost(thd, split_card, rec_len);
spl_plan->cost= join->best_positions[join->table_count-1].read_time +
+ oper_cost;
double oper_cost= (split_card *
spl_postjoin_oper_cost(thd, split_card, rec_len));
spl_plan->cost= (join->best_positions[join->table_count-1].read_time +
oper_cost);
if (unlikely(thd->trace_started()))
{
@@ -1030,7 +1037,7 @@ SplM_plan_info * JOIN_TAB::choose_best_splitting(double record_count,
}
if (spl_plan)
{
if(record_count * spl_plan->cost < spl_opt_info->unsplit_cost - 0.01)
if (record_count * spl_plan->cost + COST_EPS < spl_opt_info->unsplit_cost)
{
/*
The best plan that employs splitting is cheaper than
@@ -1054,7 +1061,7 @@ SplM_plan_info * JOIN_TAB::choose_best_splitting(double record_count,
trace.
add("startup_cost", startup_cost).
add("splitting_cost", spl_plan->cost).
add("records", records);
add("rows", records);
}
}
else

92
sql/opt_subselect.cc
View File

@@ -35,6 +35,7 @@
#include "sql_test.h"
#include <my_bit.h>
#include "opt_trace.h"
#include "optimizer_defaults.h"
/*
This file contains optimizations for semi-join subqueries.
@@ -1471,8 +1472,8 @@ void get_delayed_table_estimates(TABLE *table,
hash_sj_engine->tmp_table->s->reclength);
/* Do like in handler::ha_scan_and_compare_time, but ignore the where cost */
*scan_time= ((data_size/table->file->stats.block_size+2) *
table->file->avg_io_cost()) + *out_rows * file->ROW_COPY_COST;
*scan_time= ((data_size/IO_SIZE * table->file->avg_io_cost()) +
*out_rows * file->ROW_COPY_COST);
}
@@ -2595,11 +2596,9 @@ bool optimize_semijoin_nests(JOIN *join, table_map all_table_map)
We don't need to check the where clause for each row, so no
WHERE_COST is needed.
*/
scan_cost= (TABLE_SCAN_SETUP_COST +
(cost.block_size == 0 ? 0 :
((rowlen * (double) sjm->rows) / cost.block_size +
TABLE_SCAN_SETUP_COST)));
scan_cost= (rowlen * (double) sjm->rows) / cost.block_size;
total_cost= (scan_cost * cost.cache_hit_ratio * cost.avg_io_cost +
TABLE_SCAN_SETUP_COST_THD(thd) +
row_copy_cost * sjm->rows);
sjm->scan_cost.convert_from_cost(total_cost);
@@ -2699,8 +2698,6 @@ get_tmp_table_costs(THD *thd, double row_count, uint row_size, bool blobs_used,
bool add_copy_cost)
{
TMPTABLE_COSTS cost;
double row_copy_cost= add_copy_cost ? ROW_COPY_COST_THD(thd) : 0;
/* From heap_prepare_hp_create_info(), assuming one hash key used */
row_size+= sizeof(char*)*2;
row_size= MY_ALIGN(MY_MAX(row_size, sizeof(char*)) + 1, sizeof(char*));
@@ -2708,24 +2705,31 @@ get_tmp_table_costs(THD *thd, double row_count, uint row_size, bool blobs_used,
if (row_count > thd->variables.max_heap_table_size / (double) row_size ||
blobs_used)
{
double row_copy_cost= (add_copy_cost ?
tmp_table_optimizer_costs.row_copy_cost :
0);
/* Disk based table */
cost.lookup= ((DISK_TEMPTABLE_LOOKUP_COST *
thd->optimizer_cache_hit_ratio)) + row_copy_cost;
cost.write= cost.lookup + row_copy_cost;
cost.lookup= ((tmp_table_optimizer_costs.key_lookup_cost *
tmp_table_optimizer_costs.disk_read_ratio) +
row_copy_cost);
cost.write= cost.lookup;
cost.create= DISK_TEMPTABLE_CREATE_COST;
cost.block_size= DISK_TEMPTABLE_BLOCK_SIZE;
cost.avg_io_cost= 1.0;
cost.cache_hit_ratio= thd->optimizer_cache_hit_ratio;
cost.avg_io_cost= tmp_table_optimizer_costs.disk_read_cost;
cost.cache_hit_ratio= tmp_table_optimizer_costs.disk_read_ratio;
}
else
{
/* Values are as they are in heap.h */
double row_copy_cost= (add_copy_cost ?
heap_optimizer_costs.row_copy_cost :
0);
cost.lookup= HEAP_TEMPTABLE_LOOKUP_COST + row_copy_cost;
cost.write= cost.lookup + row_copy_cost;
cost.write= cost.lookup;
cost.create= HEAP_TEMPTABLE_CREATE_COST;
cost.block_size= 0;
cost.avg_io_cost= HEAP_TEMPTABLE_LOOKUP_COST;
cost.cache_hit_ratio= 1.0;
cost.block_size= 1;
cost.avg_io_cost= 0;
cost.cache_hit_ratio= 0;
}
return cost;
}
@@ -3196,7 +3200,7 @@ bool Sj_materialization_picker::check_qep(JOIN *join,
if (unlikely(trace.trace_started()))
{
trace.
add("records", *record_count).
add("rows", *record_count).
add("cost", *read_time);
}
return TRUE;
@@ -3250,7 +3254,7 @@ bool Sj_materialization_picker::check_qep(JOIN *join,
best_access_path(join, join->positions[i].table, rem_tables,
join->positions, i,
disable_jbuf, prefix_rec_count, &curpos, &dummy);
prefix_rec_count= COST_MULT(prefix_rec_count, curpos.records_read);
prefix_rec_count= COST_MULT(prefix_rec_count, curpos.records_out);
prefix_cost= COST_ADD(prefix_cost, curpos.read_time);
//TODO: take into account join condition selectivity here
}
@@ -3277,7 +3281,7 @@ bool Sj_materialization_picker::check_qep(JOIN *join,
if (unlikely(trace.trace_started()))
{
trace.
add("records", *record_count).
add("rows", *record_count).
add("cost", *read_time);
}
return TRUE;
@@ -3378,7 +3382,7 @@ bool LooseScan_picker::check_qep(JOIN *join,
if (unlikely(trace.trace_started()))
{
trace.
add("records", *record_count).
add("rows", *record_count).
add("cost", *read_time);
}
return TRUE;
@@ -3476,7 +3480,7 @@ bool Firstmatch_picker::check_qep(JOIN *join,
- remove fanout added by the last table
*/
if (*record_count)
*record_count /= join->positions[idx].records_read;
*record_count /= join->positions[idx].records_out;
}
else
{
@@ -3497,7 +3501,7 @@ bool Firstmatch_picker::check_qep(JOIN *join,
if (unlikely(trace.trace_started()))
{
trace.
add("records", *record_count).
add("rows", *record_count).
add("cost", *read_time);
}
return TRUE;
@@ -3624,21 +3628,22 @@ bool Duplicate_weedout_picker::check_qep(JOIN *join,
*/
uint first_tab= first_dupsweedout_table;
double dups_cost;
double prefix_rec_count;
double first_weedout_table_rec_count;
double sj_inner_fanout= 1.0;
double sj_outer_fanout= 1.0;
uint temptable_rec_size;
if (first_tab == join->const_tables)
{
prefix_rec_count= 1.0;
first_weedout_table_rec_count= 1.0;
temptable_rec_size= 0;
dups_cost= 0.0;
}
else
{
dups_cost= join->positions[first_tab - 1].prefix_cost;
prefix_rec_count= join->positions[first_tab - 1].prefix_record_count;
first_weedout_table_rec_count=
join->positions[first_tab - 1].prefix_record_count;
temptable_rec_size= 8; /* This is not true but we'll make it so */
}
@@ -3677,17 +3682,14 @@ bool Duplicate_weedout_picker::check_qep(JOIN *join,
sj_outer_fanout,
temptable_rec_size,
0, 0);
double write_cost=
COST_ADD(one_cost.create,
COST_MULT(join->positions[first_tab].prefix_record_count,
sj_outer_fanout * one_cost.write));
double full_lookup_cost=
COST_MULT(join->positions[first_tab].prefix_record_count,
COST_MULT(sj_outer_fanout,
sj_inner_fanout * one_cost.lookup));
*read_time= COST_ADD(dups_cost, COST_ADD(write_cost, full_lookup_cost));
double prefix_record_count= join->positions[first_tab].prefix_record_count;
double write_cost= (one_cost.create +
prefix_record_count * sj_outer_fanout * one_cost.write);
double full_lookup_cost= (prefix_record_count * sj_outer_fanout *
sj_inner_fanout * one_cost.lookup);
*read_time= dups_cost + write_cost + full_lookup_cost;
*record_count= prefix_rec_count * sj_outer_fanout;
*record_count= first_weedout_table_rec_count * sj_outer_fanout;
*handled_fanout= dups_removed_fanout;
*strategy= SJ_OPT_DUPS_WEEDOUT;
if (unlikely(join->thd->trace_started()))
@@ -3695,7 +3697,10 @@ bool Duplicate_weedout_picker::check_qep(JOIN *join,
Json_writer_object trace(join->thd);
trace.
add("strategy", "DuplicateWeedout").
add("records", *record_count).
add("prefix_row_count", prefix_record_count).
add("tmp_table_rows", sj_outer_fanout).
add("sj_inner_fanout", sj_inner_fanout).
add("rows", *record_count).
add("dups_cost", dups_cost).
add("write_cost", write_cost).
add("full_lookup_cost", full_lookup_cost).
@@ -3899,7 +3904,7 @@ static void recalculate_prefix_record_count(JOIN *join, uint start, uint end)
prefix_count= 1.0;
else
prefix_count= COST_MULT(join->best_positions[j-1].prefix_record_count,
join->best_positions[j-1].records_read);
join->best_positions[j-1].records_out);
join->best_positions[j].prefix_record_count= prefix_count;
}
@@ -4051,7 +4056,7 @@ void fix_semijoin_strategies_for_picked_join_order(JOIN *join)
join->best_positions, i,
FALSE, prefix_rec_count,
join->best_positions + i, &dummy);
prefix_rec_count *= join->best_positions[i].records_read;
prefix_rec_count *= join->best_positions[i].records_out;
rem_tables &= ~join->best_positions[i].table->table->map;
}
}
@@ -4093,7 +4098,7 @@ void fix_semijoin_strategies_for_picked_join_order(JOIN *join)
TRUE /* no jbuf */,
record_count, join->best_positions + idx, &dummy);
}
record_count *= join->best_positions[idx].records_read;
record_count *= join->best_positions[idx].records_out;
rem_tables &= ~join->best_positions[idx].table->table->map;
}
}
@@ -4151,7 +4156,7 @@ void fix_semijoin_strategies_for_picked_join_order(JOIN *join)
}
}
rem_tables &= ~join->best_positions[idx].table->table->map;
record_count *= join->best_positions[idx].records_read;
record_count *= join->best_positions[idx].records_out;
}
first_pos->sj_strategy= SJ_OPT_LOOSE_SCAN;
first_pos->n_sj_tables= my_count_bits(first_pos->table->emb_sj_nest->sj_inner_tables);
@@ -5368,7 +5373,8 @@ int setup_semijoin_dups_elimination(JOIN *join, ulonglong options,
Got a table that's not within any semi-join nest. This is a case
like this:
SELECT * FROM ot1, nt1 WHERE ot1.col IN (SELECT expr FROM it1, it2)
SELECT * FROM ot1, nt1 WHERE
ot1.col IN (SELECT expr FROM it1, it2)
with a join order of
@@ -6780,7 +6786,7 @@ bool JOIN::choose_subquery_plan(table_map join_tables)
Json_writer_object trace_wrapper(thd);
Json_writer_object trace_subquery(thd, "subquery_plan");
trace_subquery.
add("records", inner_record_count_1).
add("rows", inner_record_count_1).
add("materialization_cost", materialize_strategy_cost).
add("in_exist_cost", in_exists_strategy_cost).
add("choosen", strategy);

8
sql/opt_subselect.h
View File

@@ -226,15 +226,17 @@ public:
if (!(found_part & 1 ) && /* no usable ref access for 1st key part */
s->table->covering_keys.is_set(key))
{
double records, read_time;
part1_conds_met= TRUE;
DBUG_PRINT("info", ("Can use full index scan for LooseScan"));
/* Calculate the cost of complete loose index scan. */
double records= rows2double(s->table->file->stats.records);
records= rows2double(s->table->file->stats.records);
/* The cost is entire index scan cost (divided by 2) */
double read_time= s->table->file->ha_keyread_and_copy_time(key, 1,
(ha_rows) records);
read_time= s->table->file->ha_keyread_and_copy_time(key, 1,
(ha_rows) records,
0);
/*
Now find out how many different keys we will get (for now we

4
sql/opt_trace.cc
View File

@@ -696,8 +696,8 @@ void print_best_access_for_table(THD *thd, POSITION *pos)
Json_writer_object obj(thd, "chosen_access_method");
obj.
add("type", pos->type == JT_ALL ? "scan" : join_type_str[pos->type]).
add("records_read", pos->records_read).
add("records_out", pos->records_out).
add("rows_read", pos->records_read).
add("rows_out", pos->records_out).
add("cost", pos->read_time).
add("uses_join_buffering", pos->use_join_buffer);
if (pos->range_rowid_filter_info)

218
sql/optimizer_costs.h
View File

@@ -18,41 +18,79 @@
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1335 USA
*/
/* This file includes costs variables used by the optimizer */
/*
This file defines costs structures and cost functions used by the optimizer
*/
/*
The table/index cache hit ratio in %. 0 means that a searched for key or row
will never be in the cache while 100 means it always in the cache.
According to folklore, one need at least 80 % hit rate in the cache for
MariaDB to run very well. We set CACHE_HIT_RATIO to a bit smaller
as there is still a cost involved in finding the row in the B tree, hash
or other seek structure.
Increasing CACHE_HIT_RATIO will make MariaDB prefer key lookups over
table scans as the impact of ROW_COPY_COST and INDEX_COPY cost will
have a larger impact when more rows are exmined..
Note that avg_io_cost() is multipled with this constant!
OPTIMIZER_COSTS stores cost variables for each engine. They are stored
in linked_optimizer_costs (pointed to by handlerton) and TABLE_SHARE.
*/
#define DEFAULT_CACHE_HIT_RATIO 50
/* Convert ratio to cost */
static inline double cache_hit_ratio(uint ratio)
#define OPTIMIZER_COST_UNDEF -1.0
struct OPTIMIZER_COSTS
{
return (((double) (100 - ratio)) / 100.0);
}
double disk_read_cost;
double index_block_copy_cost;
double key_cmp_cost;
double key_copy_cost;
double key_lookup_cost;
double key_next_find_cost;
double disk_read_ratio;
double row_copy_cost;
double row_lookup_cost;
double row_next_find_cost;
double rowid_cmp_cost;
double rowid_copy_cost;
double initialized; // Set if default or connected with handlerton
};
/* Default optimizer costs */
extern OPTIMIZER_COSTS default_optimizer_costs;
/*
These are used to avoid taking mutex while creating tmp tables
These are created once after the server is started so they are
not dynamic.
*/
extern OPTIMIZER_COSTS heap_optimizer_costs, tmp_table_optimizer_costs;
/*
Base cost for finding keys and rows from the engine is 1.0
All other costs should be proportional to these
Interface to the engine cost variables. See optimizer_defaults.h for
the default values.
*/
/* Cost for finding the first key in a key scan */
#define KEY_LOOKUP_COST ((double) 1.0)
/* Cost of finding a key from a row_ID (not used for clustered keys) */
#define ROW_LOOKUP_COST ((double) 1.0)
#define DISK_READ_RATIO costs->disk_read_ratio
#define KEY_LOOKUP_COST costs->key_lookup_cost
#define ROW_LOOKUP_COST costs->row_lookup_cost
#define INDEX_BLOCK_COPY_COST costs->index_block_copy_cost
#define KEY_COPY_COST costs->key_copy_cost
#define ROW_COPY_COST costs->row_copy_cost
#define ROW_COPY_COST_THD(THD) default_optimizer_costs.row_copy_cost
#define KEY_NEXT_FIND_COST costs->key_next_find_cost
#define ROW_NEXT_FIND_COST costs->row_next_find_cost
#define KEY_COMPARE_COST costs->key_cmp_cost
#define SORT_INDEX_CMP_COST default_optimizer_costs.key_cmp_cost
#define DISK_READ_COST costs->disk_read_cost
#define DISK_READ_COST_THD(thd) default_optimizer_costs.disk_read_cost
/* Cost of comparing two rowids. This is set relative to KEY_COMPARE_COST */
#define ROWID_COMPARE_COST costs->rowid_cmp_cost
#define ROWID_COMPARE_COST_THD(THD) default_optimizer_costs.rowid_cmp_cost
/* Cost of comparing two rowids. This is set relative to KEY_COPY_COST */
#define ROWID_COPY_COST costs->rowid_copy_cost
/* Engine unrelated costs. Stored in THD so that the user can change them */
#define WHERE_COST optimizer_where_cost
#define WHERE_COST_THD(THD) ((THD)->variables.optimizer_where_cost)
#define TABLE_SCAN_SETUP_COST optimizer_scan_setup_cost
#define TABLE_SCAN_SETUP_COST_THD(THD) (THD)->variables.optimizer_scan_setup_cost
#define INDEX_SCAN_SETUP_COST optimizer_scan_setup_cost/2
/* Default fill factors of an (b-tree) index block is assumed to be 0.75 */
#define INDEX_BLOCK_FILL_FACTOR_DIV 3
#define INDEX_BLOCK_FILL_FACTOR_MUL 4
/*
These constants impact the cost of QSORT and priority queue sorting,
@@ -68,94 +106,13 @@ static inline double cache_hit_ratio(uint ratio)
*/
#define QSORT_SORT_SLOWNESS_CORRECTION_FACTOR (0.1)
#define PQ_SORT_SLOWNESS_CORRECTION_FACTOR (0.1)
/*
Cost of finding and copying keys from the storage engine index cache to
an internal cache as part of an index scan.
Used in handler::keyread_time()
*/
#define DEFAULT_INDEX_BLOCK_COPY_COST ((double) 1 / 5.0)
#define INDEX_BLOCK_COPY_COST(THD) ((THD)->variables.optimizer_index_block_copy_cost)
/*
Cost of finding the next row during table scan and copying it to
'table->record'.
If this is too small, then table scans will be prefered over 'ref'
as with table scans there are no key read (KEY_LOOKUP_COST), fewer
disk reads but more record copying and row comparisions. If it's
too big then MariaDB will used key lookup even when table scan is
better.
*/
#define DEFAULT_ROW_COPY_COST ((double) 1.0 / 20.0)
#define ROW_COPY_COST optimizer_row_copy_cost
#define ROW_COPY_COST_THD(THD) ((THD)->variables.optimizer_row_copy_cost)
/*
Creating a record from the join cache is faster than getting a row from
the engine. JOIN_CACHE_ROW_COPY_COST_FACTOR is the factor used to
take this into account. This is multiplied with ROW_COPY_COST.
*/
#define JOIN_CACHE_ROW_COPY_COST_FACTOR 0.75
/*
Cost of finding the next key during index scan and copying it to
'table->record'
If this is too small, then index scans will be prefered over 'ref'
as with table scans there are no key read (KEY_LOOKUP_COST) and
fewer disk reads.
*/
#define DEFAULT_KEY_COPY_COST ((double) 1.0 / 40.0)
#define KEY_COPY_COST optimizer_key_copy_cost
#define KEY_COPY_COST_THD(THD) ((THD)->variables.optimizer_key_copy_cost)
/*
Cost of finding the next index entry and checking it against filter
This cost is very low as it's done inside the storage engine.
Should be smaller than KEY_COPY_COST.
*/
#define DEFAULT_KEY_NEXT_FIND_COST ((double) 1.0 / 80.0)
#define KEY_NEXT_FIND_COST optimizer_next_find_cost
/**
The following is used to decide if MariaDB should use table scanning
instead of reading with keys. The number says how many evaluation of the
WHERE clause is comparable to reading one extra row from a table.
*/
#define DEFAULT_WHERE_COST (1 / 5.0)
#define WHERE_COST optimizer_where_cost
#define WHERE_COST_THD(THD) ((THD)->variables.optimizer_where_cost)
#define DEFAULT_KEY_COMPARE_COST (1 / 20.0)
#define KEY_COMPARE_COST optimizer_key_cmp_cost
/*
Cost of comparing two rowids. This is set relative to KEY_COMPARE_COST
This is usally just a memcmp!
*/
#define ROWID_COMPARE_COST KEY_COMPARE_COST/10.0
#define ROWID_COMPARE_COST_THD(THD) ((THD)->variables.KEY_COMPARE_COST / 10.0)
/*
Setup cost for different operations
*/
/* Extra cost for doing a range scan. Used to prefer 'ref' over range */
#define MULTI_RANGE_READ_SETUP_COST (double) (1.0 / 50.0)
/*
These costs are mainly to handle small tables, like the one we have in the
mtr test suite
*/
/* Extra cost for full table scan. Used to prefer range over table scans */
#define TABLE_SCAN_SETUP_COST 1.0
/* Extra cost for full index scan. Used to prefer range over index scans */
#define INDEX_SCAN_SETUP_COST 1.0
/*
The lower bound of accepted rows when using filter.
This is used to ensure that filters are not too agressive.
*/
#define MIN_ROWS_AFTER_FILTERING 1.0
#define JOIN_CACHE_ROW_COPY_COST_FACTOR(thd) 1.0
/*
cost1 is better that cost2 only if cost1 + COST_EPS < cost2
@@ -163,33 +120,8 @@ static inline double cache_hit_ratio(uint ratio)
when there are identical plans. Without COST_EPS some plans in the
test suite would vary depending on floating point calculations done
in different paths.
*/
#define COST_EPS 0.0001
/*
For sequential disk seeks the cost formula is:
DISK_SEEK_BASE_COST + DISK_SEEK_PROP_COST * #blocks_to_skip
The cost of average seek
DISK_SEEK_BASE_COST + DISK_SEEK_PROP_COST*BLOCKS_IN_AVG_SEEK =1.0.
*/
#define DISK_SEEK_BASE_COST ((double)0.9)
#define BLOCKS_IN_AVG_SEEK 128
#define DISK_SEEK_PROP_COST ((double)0.1/BLOCKS_IN_AVG_SEEK)
/*
Subquery materialization-related constants
*/
/* This should match ha_heap::read_time() */
#define HEAP_TEMPTABLE_LOOKUP_COST 0.05
#define HEAP_TEMPTABLE_CREATE_COST 1.0
#define DISK_TEMPTABLE_LOOKUP_COST 1.0
#define DISK_TEMPTABLE_CREATE_COST TMPFILE_CREATE_COST*2 /* 2 tmp tables */
#define DISK_TEMPTABLE_BLOCK_SIZE 8192
#define SORT_INDEX_CMP_COST 0.02
#define COST_EPS 0.0000001
#define COST_MAX (DBL_MAX * (1.0 - DBL_EPSILON))
@@ -207,4 +139,22 @@ static inline double COST_MULT(double c, double f)
return (COST_MAX / (f) > (c) ? (c) * (f) : COST_MAX);
}
OPTIMIZER_COSTS *get_optimizer_costs(const LEX_CSTRING *cache_name);
OPTIMIZER_COSTS *create_optimizer_costs(const char *name, size_t length);
OPTIMIZER_COSTS *get_or_create_optimizer_costs(const char *name,
size_t length);
bool create_default_optimizer_costs();
void copy_tmptable_optimizer_costs();
void free_all_optimizer_costs();
struct TABLE;
extern "C"
{
typedef int (*process_optimizer_costs_t) (const LEX_CSTRING *,
const OPTIMIZER_COSTS *,
TABLE *);
bool process_optimizer_costs(process_optimizer_costs_t func, TABLE *param);
}
#endif /* OPTIMIZER_COSTS_INCLUDED */

183
sql/optimizer_defaults.h Normal file
View File

@@ -0,0 +1,183 @@
#ifndef OPTIMIZER_DEFAULTS_INCLUDED
#define OPTIMIZER_DEFAULTS_INCLUDED
/*
Copyright (c) 2022, MariaDB AB
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; version 2 of
the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1335 USA
*/
/*
This file contains costs constants used by the optimizer
All costs should be based on milliseconds (1 cost = 1 ms)
*/
/* Cost for finding the first key in a key scan */
#define DEFAULT_KEY_LOOKUP_COST ((double) 0.000435777)
/* Cost of finding a row based on row_ID */
#define DEFAULT_ROW_LOOKUP_COST ((double) 0.000130839)
/*
Cost of finding and copying key and row blocks from the storage
engine index cache to an internal cache as part of an index
scan. This includes all mutexes that needs to be taken to get
exclusive access to a page. The number is taken from accessing an
existing blocks from Aria page cache.
Used in handler::scan_time() and handler::keyread_time()
*/
#define DEFAULT_INDEX_BLOCK_COPY_COST ((double) 3.56e-05)
/*
Cost of copying a row to 'table->record'.
Used by scan_time() and rnd_pos_time() methods.
If this is too small, then table scans will be prefered over 'ref'
as with table scans there are no key read (KEY_LOOKUP_COST), fewer
disk reads but more record copying and row comparisions. If it's
too big then MariaDB will used key lookup even when table scan is
better.
*/
#define DEFAULT_ROW_COPY_COST ((double) 0.000060866)
/*
Cost of copying the key to 'table->record'
If this is too small, then, for small tables, index scans will be
prefered over 'ref' as with index scans there are fewer disk reads.
*/
#define DEFAULT_KEY_COPY_COST ((double) 0.000015685)
/*
Cost of finding the next index entry and checking its rowid against filter
This cost is very low as it's done inside the storage engine.
Should be smaller than KEY_COPY_COST.
*/
#define DEFAULT_KEY_NEXT_FIND_COST ((double) 0.000082347)
/* Cost of finding the next row when scanning a table */
#define DEFAULT_ROW_NEXT_FIND_COST ((double) 0.000045916)
/**
The cost of executing the WHERE clause as part of any row check.
Increasing this would force the optimizer to use row combinations
that reads fewer rows.
The default cost comes from recording times from a simple where clause that
compares two fields (date and a double) with constants.
*/
#define DEFAULT_WHERE_COST ((double) 3.2e-05)
/* The cost of comparing a key when using range access or sorting */
#define DEFAULT_KEY_COMPARE_COST 0.000011361
/* Rowid compare is usually just a single memcmp of a short string */
#define DEFAULT_ROWID_COMPARE_COST 0.000002653
/* Rowid copy is usually just a single memcpy of a short string */
#define DEFAULT_ROWID_COPY_COST 0.000002653
/*
Average disk seek time on a hard disk is 8-10 ms, which is also
about the time to read a IO_SIZE (8192) block.
A medium ssd is about 400MB/second, which gives us the time for
reading an IO_SIZE block to IO_SIZE/400000000 = 0.0000204 sec= 0.02 ms.
*/
#define DEFAULT_DISK_READ_COST ((double) IO_SIZE / 400000000.0 * 1000)
/*
The follwoing is an old comment for hard-disks, please ignore the
following, except if you like history:
For sequential hard disk seeks the cost formula is:
DISK_SEEK_BASE_COST + DISK_SEEK_PROP_COST * #blocks_to_skip
The cost of average seek
DISK_SEEK_BASE_COST + DISK_SEEK_PROP_COST*BLOCKS_IN_AVG_SEEK = 10.
*/
/*
The table/index cache_miss/total_cache_request ratio.
1.0 means that a searched for key or row will never be in the cache while
0.0 means it always in the cache (and we don't have to do any disk reads).
According to folklore, one should not have to access disk for more
than 20% of the cache request for MariaDB to run very well.
However in practice when we read rows or keys in a query, we will often
read the same row over and over again. Because of this we set
DEFAULT_DISK_READ_RATIO to 0.20/10 = 0.02.
Increasing DISK_READ_RATIO will make MariaDB prefer key lookup over
table scans as the impact of ROW_COPY_COST and INDEX_COPY cost will
have a larger impact when more rows are examined..
We are not yet taking into account cache usage statistics as this
could confuse users as the EXPLAIN and costs for a query would change
between to query calls, which may confuse users (and also make the
mtr tests very unpredictable).
Note that the engine's avg_io_cost() (DEFAULT_DISK_READ_COST by default)
is multiplied with this constant!
*/
#define DEFAULT_DISK_READ_RATIO 0.02
/*
The following costs are mainly to ensure we don't do table and index
scans for small tables, like the one we have in the mtr test suite.
This is mostly to keep the mtr tests use indexes (as the optimizer would
if the tables are large). It will also ensure that EXPLAIN is showing
more key user for users where they are testing queries with small tables
at the start of projects.
This is probably OK for most a the execution time difference between table
scan and index scan compared to key lookups so small when using small
tables. It also helps to fill the index cache which will help mitigate
the speed difference.
*/
/*
Extra cost for full table and index scan. Used to prefer key and range
over index and table scans
INDEX_SCAN_SETUP_COST (defined in optimizer_costs.h) is half of
table_scan_setup_cost to get the optimizer to prefer index scans to table
scans as key copy is faster than row copy and index blocks provides
more information in the cache.
This will also help MyISAM as with MyISAM the table scans has a cost
very close to index scans (they are fast but require a read call
that we want to avoid even if it's small).
10 usec is about 10 MyISAM row lookups with optimizer_disk_read_ratio= 0.02
*/
#define DEFAULT_TABLE_SCAN_SETUP_COST 0.01 // 10 usec
/* Extra cost for doing a range scan. Used to prefer 'ref' over range */
#define MULTI_RANGE_READ_SETUP_COST KEY_LOOKUP_COST
/*
Temporary file and temporary table related costs
Used with subquery materialization, derived tables etc
*/
#define TMPFILE_CREATE_COST 0.5 // Cost of creating and deleting files
#define HEAP_TEMPTABLE_CREATE_COST 0.025 // ms
/* Cost taken from HEAP_LOOKUP_COST in ha_heap.cc */
#define HEAP_TEMPTABLE_LOOKUP_COST (0.00016097*1000 + heap_optimizer_costs.row_copy_cost)
#define DISK_TEMPTABLE_LOOKUP_COST(thd) (tmp_table_optimizer_costs.key_lookup_cost + tmp_table_optimizer_costs.row_lookup_cost + tmp_table_optimizer_costs.row_copy_cost)
#define DISK_TEMPTABLE_CREATE_COST TMPFILE_CREATE_COST*2 // 2 tmp tables
#define DISK_TEMPTABLE_BLOCK_SIZE IO_SIZE
#endif /* OPTIMIZER_DEFAULTS_INCLUDED */

19
sql/rowid_filter.cc
View File

@@ -32,7 +32,7 @@ lookup_cost(Rowid_filter_container_type cont_type)
{
switch (cont_type) {
case SORTED_ARRAY_CONTAINER:
return log(est_elements)*0.01+key_next_find_cost;
return log(est_elements) * rowid_compare_cost + base_lookup_cost;
default:
DBUG_ASSERT(0);
return 0;
@@ -125,11 +125,13 @@ void Range_rowid_filter_cost_info::init(Rowid_filter_container_type cont_type,
key_no= idx;
est_elements= (ulonglong) table->opt_range[key_no].rows;
cost_of_building_range_filter= build_cost(container_type);
where_cost= tab->in_use->variables.optimizer_where_cost;
key_next_find_cost= tab->in_use->variables.optimizer_key_next_find_cost;
base_lookup_cost= tab->file->ROW_NEXT_FIND_COST;
rowid_compare_cost= tab->file->ROWID_COMPARE_COST;
selectivity= est_elements/((double) table->stat_records());
gain= avg_access_and_eval_gain_per_row(container_type,
tab->file->optimizer_cache_cost);
tab->file->ROW_LOOKUP_COST);
if (gain > 0)
cross_x= cost_of_building_range_filter/gain;
else
@@ -147,15 +149,18 @@ double
Range_rowid_filter_cost_info::build_cost(Rowid_filter_container_type cont_type)
{
double cost;
OPTIMIZER_COSTS *costs= &table->s->optimizer_costs;
DBUG_ASSERT(table->opt_range_keys.is_set(key_no));
cost= table->opt_range[key_no].index_only_fetch_cost(table->in_use);
/* Cost of fetching keys */
cost= table->opt_range[key_no].index_only_fetch_cost(table);
switch (cont_type) {
case SORTED_ARRAY_CONTAINER:
cost+= ARRAY_WRITE_COST * est_elements; /* cost filling the container */
cost+= ARRAY_SORT_C * est_elements * log(est_elements); /* sorting cost */
/* Add cost of filling container and cost of sorting */
cost= (est_elements *
(costs->rowid_copy_cost + // Copying rowid
costs->rowid_cmp_cost * log2(est_elements))); // Sort
break;
default:
DBUG_ASSERT(0);

17
sql/rowid_filter.h
View File

@@ -143,20 +143,6 @@ class SQL_SELECT;
class Rowid_filter_container;
class Range_rowid_filter_cost_info;
/*
Cost to write rowid into array. Assume inserting 1000 row id's into the
array has same cost as a 'disk io' or key fetch
*/
#define ARRAY_WRITE_COST 0.001
/*
Factor used to calculate cost of sorting rowids in array
This is multiplied by 'elements * log(elements)', so this factor
has a very high cost weight!
A value of 0.001 will have 200 rows have a cost of 1.05 and
1000 rows a cost of 6.90.
*/
#define ARRAY_SORT_C 0.001
typedef enum
{
SORTED_ARRAY_CONTAINER,
@@ -406,7 +392,8 @@ class Range_rowid_filter_cost_info final: public Sql_alloc
/* The index whose range scan would be used to build the range filter */
uint key_no;
double cost_of_building_range_filter;
double where_cost, key_next_find_cost;
double where_cost, base_lookup_cost, rowid_compare_cost;
/*
(gain*row_combinations)-cost_of_building_range_filter yields the gain of
the filter for 'row_combinations' key tuples of the index key_no

8
sql/set_var.cc
View File

@@ -310,7 +310,13 @@ do { \
case SHOW_HA_ROWS: do_num_val (ha_rows,CMD);
#define case_for_double(CMD) \
case SHOW_DOUBLE: do_num_val (double,CMD)
case SHOW_DOUBLE: do_num_val (double,CMD); \
case SHOW_OPTIMIZER_COST: \
{ \
double val= ((*(double*) value) == OPTIMIZER_COST_UNDEF ? OPTIMIZER_COST_UNDEF : \
(*(double*) value) * 1000); \
CMD; \
} while (0)
#define case_get_string_as_lex_string \
case SHOW_CHAR: \

2
sql/set_var.h
View File

@@ -84,7 +84,7 @@ protected:
typedef bool (*on_update_function)(sys_var *self, THD *thd, enum_var_type type);
int flags; ///< or'ed flag_enum values
const SHOW_TYPE show_val_type; ///< what value_ptr() returns for sql_show.cc
SHOW_TYPE show_val_type; ///< what value_ptr() returns for sql_show.cc
PolyLock *guard; ///< *second* lock that protects the variable
ptrdiff_t offset; ///< offset to the value from global_system_variables
on_check_function on_check;

12
sql/sql_bitmap.h
View File

@@ -270,13 +270,21 @@ public:
{
return buffer[0];
}
uint bits_set()
uint bits_set() const
{
uint res= 0;
for (size_t i= 0; i < ARRAY_ELEMENTS; i++)
res += my_count_bits(buffer[i]);
if (buffer[i])
res+= my_count_bits(buffer[i]);
return res;
}
uint find_first_bit() const
{
for (size_t i= 0; i < ARRAY_ELEMENTS; i++)
if (buffer[i])
return (uint)i*BITS_PER_ELEMENT + my_find_first_bit(buffer[i]);
return width;
}
class Iterator
{
const Bitmap& map;

1
sql/sql_class.cc
View File

@@ -1236,7 +1236,6 @@ void THD::init()
*/
variables.pseudo_thread_id= thread_id;
variables.default_master_connection.str= default_master_connection_buff;
optimizer_cache_hit_ratio= cache_hit_ratio(variables.optimizer_cache_hit_ratio);
::strmake(default_master_connection_buff,
global_system_variables.default_master_connection.str,
variables.default_master_connection.length);

14
sql/sql_class.h
View File

@@ -692,9 +692,7 @@ typedef struct system_variables
ulonglong slave_skip_counter;
ulonglong max_relay_log_size;
double optimizer_index_block_copy_cost, optimizer_key_next_find_cost;
double optimizer_row_copy_cost, optimizer_key_copy_cost;
double optimizer_where_cost, optimizer_key_cmp_cost;
double optimizer_where_cost, optimizer_scan_setup_cost;
double long_query_time_double, max_statement_time_double;
double sample_percentage;
@@ -793,7 +791,6 @@ typedef struct system_variables
uint group_concat_max_len;
uint eq_range_index_dive_limit;
uint optimizer_cache_hit_ratio; // Stored in handler::optimizer_cache_cost
uint idle_transaction_timeout;
uint idle_readonly_transaction_timeout;
uint idle_write_transaction_timeout;
@@ -831,7 +828,6 @@ typedef struct system_variables
my_bool session_track_user_variables;
#endif // USER_VAR_TRACKING
my_bool tcp_nodelay;
plugin_ref table_plugin;
plugin_ref tmp_table_plugin;
plugin_ref enforced_table_plugin;
@@ -2674,7 +2670,6 @@ public:
struct system_status_var org_status_var; // For user statistics
struct system_status_var *initial_status_var; /* used by show status */
THR_LOCK_INFO lock_info; // Locking info of this thread
double optimizer_cache_hit_ratio; // From optimizer_cache_hit_ratio
/**
Protects THD data accessed from other threads:
@@ -7424,6 +7419,13 @@ inline void handler::decrement_statistics(ulong SSV::*offset) const
status_var_decrement(table->in_use->status_var.*offset);
}
/* Update references in the handler to the table */
inline void handler::set_table(TABLE* table_arg)
{
table= table_arg;
costs= &table_arg->s->optimizer_costs;
}
inline int handler::ha_ft_read(uchar *buf)
{

14
sql/sql_const.h
View File

@@ -121,11 +121,11 @@
/*
This is used when reading large blocks, sequential read.
We assume that reading this much will be the same cost as 1 seek / fetching
one row from the storage engine.
We assume that reading this much will be roughly the same cost as 1
seek / fetching one row from the storage engine.
Cost of one read of DISK_CHUNK_SIZE is DISK_SEEK_BASE_COST (ms).
*/
#define DISK_CHUNK_SIZE (uint) (65536) /* Size of diskbuffer for tmpfiles */
#define TMPFILE_CREATE_COST 2.0 /* Creating and deleting tmp file */
#define FRM_VER_TRUE_VARCHAR (FRM_VER+4) /* 10 */
#define FRM_VER_EXPRESSSIONS (FRM_VER+5) /* 11 */
@@ -204,8 +204,14 @@
#define MIN_ROWS_TO_USE_TABLE_CACHE 100
#define MIN_ROWS_TO_USE_BULK_INSERT 100
/*
The lower bound of accepted rows when using filter.
This is used to ensure that filters are not too agressive.
*/
#define MIN_ROWS_AFTER_FILTERING 1.0
/**
Number of rows in a reference table when refereed through a not unique key.
Number of rows in a reference table when refered through a not unique key.
This value is only used when we don't know anything about the key
distribution.
*/

9
sql/sql_explain.cc
View File

@@ -1379,10 +1379,12 @@ double Explain_table_access::get_r_filtered()
}
int Explain_table_access::print_explain(select_result_sink *output, uint8 explain_flags,
int Explain_table_access::print_explain(select_result_sink *output,
uint8 explain_flags,
bool is_analyze,
uint select_id, const char *select_type,
bool using_temporary, bool using_filesort)
bool using_temporary,
bool using_filesort)
{
THD *thd= output->thd; // note: for SHOW EXPLAIN, this is target thd.
MEM_ROOT *mem_root= thd->mem_root;
@@ -2011,6 +2013,9 @@ void Explain_table_access::print_explain_json(Explain_query *query,
writer->add_double(jbuf_tracker.get_filtered_after_where()*100.0);
else
writer->add_null();
writer->add_member("r_unpack_time_ms");
writer->add_double(jbuf_unpack_tracker.get_time_ms());
}
}

4
sql/sql_explain.h
View File

@@ -753,7 +753,7 @@ public:
class Explain_table_access : public Sql_alloc
{
public:
Explain_table_access(MEM_ROOT *root) :
Explain_table_access(MEM_ROOT *root, bool timed) :
derived_select_number(0),
non_merged_sjm_number(0),
extra_tags(root),
@@ -766,6 +766,7 @@ public:
pushed_index_cond(NULL),
sjm_nest(NULL),
pre_join_sort(NULL),
jbuf_unpack_tracker(timed),
rowid_filter(NULL)
{}
~Explain_table_access() { delete sjm_nest; }
@@ -874,6 +875,7 @@ public:
Gap_time_tracker extra_time_tracker;
Table_access_tracker jbuf_tracker;
Time_and_counter_tracker jbuf_unpack_tracker;
Explain_rowid_filter *rowid_filter;

2
sql/sql_join_cache.cc
View File

@@ -1600,6 +1600,7 @@ bool JOIN_CACHE::put_record()
bool JOIN_CACHE::get_record()
{
bool res;
ANALYZE_START_TRACKING(thd(), join_tab->jbuf_unpack_tracker);
uchar *prev_rec_ptr= 0;
if (with_length)
pos+= size_of_rec_len;
@@ -1615,6 +1616,7 @@ bool JOIN_CACHE::get_record()
if (prev_cache)
prev_cache->get_record_by_pos(prev_rec_ptr);
}
ANALYZE_STOP_TRACKING(thd(), join_tab->jbuf_unpack_tracker);
return res;
}

1
sql/sql_plugin.h
View File

@@ -24,6 +24,7 @@
#define SHOW_always_last SHOW_KEY_CACHE_LONG, \
SHOW_HAVE, SHOW_MY_BOOL, SHOW_HA_ROWS, SHOW_SYS, \
SHOW_LONG_NOFLUSH, SHOW_LEX_STRING, SHOW_ATOMIC_COUNTER_UINT32_T, \
SHOW_OPTIMIZER_COST, \
/* SHOW_*_STATUS must be at the end, SHOW_LONG_STATUS being first */ \
SHOW_LONG_STATUS, SHOW_DOUBLE_STATUS, SHOW_LONGLONG_STATUS, \
SHOW_UINT32_STATUS

709
sql/sql_select.cc
View File

File diff suppressed because it is too large Load Diff

42
sql/sql_select.h
View File

@@ -309,6 +309,7 @@ typedef struct st_join_table {
Table_access_tracker *tracker;
Table_access_tracker *jbuf_tracker;
Time_and_counter_tracker *jbuf_unpack_tracker;
// READ_RECORD::Setup_func materialize_table;
READ_RECORD::Setup_func read_first_record;
@@ -341,6 +342,9 @@ typedef struct st_join_table {
*/
double read_time;
/* Copy of POSITION::records_init, set by get_best_combination() */
double records_init;
/* Copy of POSITION::records_read, set by get_best_combination() */
double records_read;
@@ -356,7 +360,6 @@ typedef struct st_join_table {
double partial_join_cardinality;
/* set by estimate_scan_time() */
double cached_scan_time;
double cached_scan_and_compare_time;
double cached_forced_index_cost;
@@ -959,21 +962,44 @@ public:
/* The table that's put into join order */
JOIN_TAB *table;
/* number of rows that will be read from the table */
double records_init;
/*
The number of rows that will be read from the table
Number of rows left after filtering, calculated in best_access_path()
In case of use_cond_selectivity > 1 it contains rows after the used
rowid filter (if such one exists).
If use_cond_selectivity <= 1 it contains the minimum rows of any
rowid filtering or records_init if no filter exists.
*/
double records_after_filter;
/*
Number of expected rows before applying the full WHERE clause. This
includes rowid filter and table->cond_selectivity if
use_cond_selectivity > 1. See matching_candidates_in_table().
Should normally not be used.
*/
double records_read;
/*
The "fanout": number of output rows that will be produced (after
The number of rows after applying the WHERE clause.
Same as the "fanout": number of output rows that will be produced (after
pushed down selection condition is applied) per each row combination of
previous tables.
This takes into account table->cond_selectivity, the WHERE clause
related to this table calculated in
calculate_cond_selectivity_for_table(), and the used rowid filter but
does not take into account the WHERE clause involving preceding tables
calculated in table_after_join_selectivity().
In best_access_path() it is set to the minum number of accepted rows
for any possible access method or filter:
records_out takes into account table->cond_selectivity, the WHERE clause
related to this table calculated in calculate_cond_selectivity_for_table(),
and the used rowid filter.
After best_access_path() records_out it does not yet take into
account the part of the WHERE clause involving preceding tables.
records_out is updated in best_extension_by_limited_search() to take these
tables into account by calling table_after_join_selectivity().
*/
double records_out;

68
sql/sql_show.cc
View File

@@ -50,6 +50,7 @@
#include "authors.h"
#include "contributors.h"
#include "sql_partition.h"
#include "optimizer_defaults.h"
#ifdef HAVE_EVENT_SCHEDULER
#include "events.h"
#include "event_data_objects.h"
@@ -3668,6 +3669,9 @@ const char* get_one_variable(THD *thd,
/* 6 is the default precision for '%f' in sprintf() */
end= buff + my_fcvt(*value.as_double, 6, buff, NULL);
break;
case SHOW_OPTIMIZER_COST: // Stored in 1ms, displayed in us
end= buff + my_fcvt(*value.as_double*1000, 6, buff, NULL);
break;
case SHOW_LONG_STATUS:
value.as_char= status_var_value.as_char + value.as_intptr;
/* fall through */
@@ -9196,6 +9200,49 @@ int fill_key_cache_tables(THD *thd, TABLE_LIST *tables, COND *cond)
}
/* Ensure we return 'OPTIMIZER_COST_UNDEF' if cost < 0 */
static double fix_cost(double cost)
{
return cost < 0 ? OPTIMIZER_COST_UNDEF : cost;
}
static int run_fill_optimizer_costs_tables(const LEX_CSTRING *name,
const OPTIMIZER_COSTS *costs,
TABLE *table)
{
THD *thd= table->in_use;
DBUG_ENTER("run_fill_optimizer_costs_tables");
restore_record(table, s->default_values);
table->field[0]->store(name->str, name->length, system_charset_info);
table->field[1]->store(fix_cost(costs->disk_read_cost*1000.0));
table->field[2]->store(fix_cost(costs->index_block_copy_cost*1000.0));
table->field[3]->store(fix_cost(costs->key_cmp_cost*1000.0));
table->field[4]->store(fix_cost(costs->key_copy_cost*1000.0));
table->field[5]->store(fix_cost(costs->key_lookup_cost*1000.0));
table->field[6]->store(fix_cost(costs->key_next_find_cost*1000.0));
table->field[7]->store(fix_cost(costs->disk_read_ratio));
table->field[8]->store(fix_cost(costs->row_copy_cost*1000.0));
table->field[9]->store(fix_cost(costs->row_lookup_cost*1000.0));
table->field[10]->store(fix_cost(costs->row_next_find_cost*1000.0));
table->field[11]->store(fix_cost(costs->rowid_cmp_cost*1000.0));
table->field[12]->store(fix_cost(costs->rowid_copy_cost*1000.0));
DBUG_RETURN(schema_table_store_record(thd, table));
}
int fill_optimizer_costs_tables(THD *thd, TABLE_LIST *tables, COND *cond)
{
DBUG_ENTER("fill_optimizer_costs_tables");
int res= process_optimizer_costs(run_fill_optimizer_costs_tables,
tables->table);
DBUG_RETURN(res);
}
namespace Show {
ST_FIELD_INFO schema_fields_info[]=
@@ -9824,6 +9871,25 @@ ST_FIELD_INFO keycache_fields_info[]=
};
ST_FIELD_INFO optimizer_costs_fields_info[]=
{
Column("ENGINE", Varchar(NAME_LEN),NOT_NULL),
Column("OPTIMIZER_DISK_READ_COST", Decimal(906), NOT_NULL),
Column("OPTIMIZER_INDEX_BLOCK_COPY_COST", Decimal(906), NOT_NULL),
Column("OPTIMIZER_KEY_COMPARE_COST", Decimal(906), NOT_NULL),
Column("OPTIMIZER_KEY_COPY_COST", Decimal(906), NOT_NULL),
Column("OPTIMIZER_KEY_LOOKUP_COST", Decimal(906), NOT_NULL),
Column("OPTIMIZER_KEY_NEXT_FIND_COST", Decimal(906), NOT_NULL),
Column("OPTIMIZER_DISK_READ_RATIO", Decimal(906), NOT_NULL),
Column("OPTIMIZER_ROW_COPY_COST", Decimal(906), NOT_NULL),
Column("OPTIMIZER_ROW_LOOKUP_COST", Decimal(906), NOT_NULL),
Column("OPTIMIZER_ROW_NEXT_FIND_COST", Decimal(906), NOT_NULL),
Column("OPTIMIZER_ROWID_COMPARE_COST", Decimal(906), NOT_NULL),
Column("OPTIMIZER_ROWID_COPY_COST", Decimal(906), NOT_NULL),
CEnd()
};
ST_FIELD_INFO show_explain_tabular_fields_info[]=
{
Column("id", SLonglong(3), NULLABLE, "id"),
@@ -9962,6 +10028,8 @@ ST_SCHEMA_TABLE schema_tables[]=
OPTIMIZE_I_S_TABLE|OPEN_TABLE_ONLY},
{"OPEN_TABLES", Show::open_tables_fields_info, 0,
fill_open_tables, make_old_format, 0, -1, -1, 1, 0},
{"OPTIMIZER_COSTS", Show::optimizer_costs_fields_info, 0,
fill_optimizer_costs_tables, 0, 0, -1,-1, 0, 0},
{"OPTIMIZER_TRACE", Show::optimizer_trace_info, 0,
fill_optimizer_trace_info, NULL, NULL, -1, -1, false, 0},
{"PARAMETERS", Show::parameters_fields_info, 0,

2
sql/sql_yacc.yy
View File

@@ -8277,7 +8277,7 @@ assign_to_keycache_parts:
key_cache_name:
ident { $$= $1; }
| DEFAULT { $$ = default_key_cache_base; }
| DEFAULT { $$ = default_base; }
;
preload:

146
sql/sys_vars.cc
View File

@@ -53,8 +53,9 @@
#include "debug_sync.h" // DEBUG_SYNC
#include "sql_show.h"
#include "opt_trace_context.h"
#include "log_event.h"
#include "optimizer_defaults.h"
#ifdef WITH_PERFSCHEMA_STORAGE_ENGINE
#include "../storage/perfschema/pfs_server.h"
#endif /* WITH_PERFSCHEMA_STORAGE_ENGINE */
@@ -6977,68 +6978,111 @@ static Sys_var_ulong Sys_optimizer_max_sel_arg_weight(
SESSION_VAR(optimizer_max_sel_arg_weight), CMD_LINE(REQUIRED_ARG),
VALID_RANGE(0, ULONG_MAX), DEFAULT(SEL_ARG::MAX_WEIGHT), BLOCK_SIZE(1));
/*
We don't allow 100 for optimizer_cache_cost as there is always a small
cost of finding the key, on cached pages, that we have to take into account.
*/
static bool update_optimizer_cache_hit_ratio(sys_var *self, THD *thd,
enum_var_type type)
{
if (type == OPT_SESSION)
thd->optimizer_cache_hit_ratio=
cache_hit_ratio(thd->variables.optimizer_cache_hit_ratio);
return 0;
}
static Sys_var_engine_optimizer_cost Sys_optimizer_disk_read_ratio(
"optimizer_disk_read_ratio",
"Chance that we have to do a disk read to find a row or index entry from "
"the engine cache (cache_misses/total_cache_requests). 0.0 means that "
"everything is cached and 1.0 means that nothing is expected to be in the "
"engine cache.",
COST_VAR(disk_read_ratio),
CMD_LINE(REQUIRED_ARG, OPT_COSTS_DISK_READ_RATIO),
VALID_RANGE(0.0, 1.0), DEFAULT(DEFAULT_DISK_READ_RATIO), COST_ADJUST(1));
static Sys_var_uint Sys_optimizer_cache_hit_ratio(
"optimizer_cache_hit_ratio",
"Expected hit rate of the row and index cache in storage engines. "
"The value should be an integer between 0 and 99, where 0 means cache is "
"empty and 99 means that value is almost always in the cache.",
SESSION_VAR(optimizer_cache_hit_ratio), CMD_LINE(REQUIRED_ARG),
VALID_RANGE(0, 99), DEFAULT(DEFAULT_CACHE_HIT_RATIO), 1, NO_MUTEX_GUARD,
NOT_IN_BINLOG, ON_CHECK(0), ON_UPDATE(update_optimizer_cache_hit_ratio));
static Sys_var_engine_optimizer_cost Sys_optimizer_key_lookup_cost(
"optimizer_key_lookup_cost",
"Cost for finding a key based on a key value",
COST_VAR(key_lookup_cost),
CMD_LINE(REQUIRED_ARG, OPT_COSTS_KEY_LOOKUP_COST),
VALID_RANGE(0, 1000), DEFAULT(DEFAULT_KEY_LOOKUP_COST), COST_ADJUST(1000));
static Sys_var_double Sys_optimizer_key_copy_cost(
static Sys_var_engine_optimizer_cost Sys_optimizer_row_lookup_cost(
"optimizer_row_lookup_cost",
"Cost of finding a row based on a rowid or a clustered key.",
COST_VAR(row_lookup_cost),
CMD_LINE(REQUIRED_ARG, OPT_COSTS_ROW_LOOKUP_COST),
VALID_RANGE(0, 1000), DEFAULT(DEFAULT_ROW_LOOKUP_COST), COST_ADJUST(1000));
static Sys_var_engine_optimizer_cost Sys_optimizer_disk_read_cost(
"optimizer_disk_read_cost",
"Cost of reading a block of IO_SIZE (4096) from a disk (in usec).",
COST_VAR(disk_read_cost),
CMD_LINE(REQUIRED_ARG, OPT_COSTS_DISK_READ_COST),
VALID_RANGE(0, 10000), DEFAULT(DEFAULT_DISK_READ_COST), COST_ADJUST(1000));
static Sys_var_engine_optimizer_cost Sys_optimizer_key_copy_cost(
"optimizer_key_copy_cost",
"Cost of finding the next key in the engine and copying it to the SQL layer.",
SESSION_VAR(optimizer_key_copy_cost), CMD_LINE(REQUIRED_ARG),
VALID_RANGE(0, 1), DEFAULT(DEFAULT_KEY_COPY_COST), NO_MUTEX_GUARD,
NOT_IN_BINLOG);
"Cost of finding the next key in the engine and copying it to the SQL "
"layer.",
COST_VAR(key_copy_cost),
CMD_LINE(REQUIRED_ARG, OPT_COSTS_KEY_COPY_COST),
VALID_RANGE(0, 1000), DEFAULT(DEFAULT_KEY_COPY_COST), COST_ADJUST(1000));
static Sys_var_double Sys_optimizer_index_block_copy_cost(
static Sys_var_engine_optimizer_cost Sys_optimizer_index_block_copy_cost(
"optimizer_index_block_copy_cost",
"Cost of copying a key block from the cache to intern storage as part of an "
"index scan.",
SESSION_VAR(optimizer_index_block_copy_cost), CMD_LINE(REQUIRED_ARG),
VALID_RANGE(0, 1), DEFAULT(DEFAULT_INDEX_BLOCK_COPY_COST), NO_MUTEX_GUARD,
NOT_IN_BINLOG);
"Cost of copying a key block from the cache to intern storage as part of "
"an index scan.",
COST_VAR(index_block_copy_cost),
CMD_LINE(REQUIRED_ARG, OPT_COSTS_INDEX_BLOCK_COPY_COST),
VALID_RANGE(0, 1000), DEFAULT(DEFAULT_INDEX_BLOCK_COPY_COST), COST_ADJUST(1000));
static Sys_var_double Sys_optimizer_key_next_find_cost(
static Sys_var_engine_optimizer_cost Sys_optimizer_row_next_find_cost(
"optimizer_row_next_find_cost",
"Cost of finding the next row when scanning the table.",
COST_VAR(row_next_find_cost),
CMD_LINE(REQUIRED_ARG, OPT_COSTS_ROW_NEXT_FIND_COST),
VALID_RANGE(0, 1000), DEFAULT(DEFAULT_ROW_NEXT_FIND_COST), COST_ADJUST(1000));
static Sys_var_engine_optimizer_cost Sys_optimizer_key_next_find_cost(
"optimizer_key_next_find_cost",
"Cost of finding the next key and rowid when using filters.",
SESSION_VAR(optimizer_key_next_find_cost), CMD_LINE(REQUIRED_ARG),
VALID_RANGE(0, 1), DEFAULT(DEFAULT_KEY_NEXT_FIND_COST), NO_MUTEX_GUARD,
NOT_IN_BINLOG);
COST_VAR(key_next_find_cost),
CMD_LINE(REQUIRED_ARG, OPT_COSTS_KEY_NEXT_FIND_COST),
VALID_RANGE(0, 1000), DEFAULT(DEFAULT_KEY_NEXT_FIND_COST), COST_ADJUST(1000));
static Sys_var_double Sys_optimizer_row_copy_cost(
static Sys_var_engine_optimizer_cost Sys_optimizer_row_copy_cost(
"optimizer_row_copy_cost",
"Cost of copying a row from the engine or the join cache to the SQL layer.",
SESSION_VAR(optimizer_row_copy_cost), CMD_LINE(REQUIRED_ARG),
VALID_RANGE(0, 1), DEFAULT(DEFAULT_ROW_COPY_COST), NO_MUTEX_GUARD,
NOT_IN_BINLOG);
COST_VAR(row_copy_cost),
CMD_LINE(REQUIRED_ARG, OPT_COSTS_ROW_COPY_COST),
VALID_RANGE(0, 1000), DEFAULT(DEFAULT_ROW_COPY_COST), COST_ADJUST(1000));
static Sys_var_double Sys_optimizer_where_cost(
"optimizer_where_cost",
"Cost of checking the row against the WHERE clause.",
SESSION_VAR(optimizer_where_cost), CMD_LINE(REQUIRED_ARG),
VALID_RANGE(0, 1), DEFAULT(DEFAULT_WHERE_COST), NO_MUTEX_GUARD,
NOT_IN_BINLOG);
static Sys_var_double Sys_optimizer_key_cmp_cost(
static Sys_var_engine_optimizer_cost Sys_optimizer_key_cmp_cost(
"optimizer_key_compare_cost",
"Cost of checking a key against the end key condition.",
SESSION_VAR(optimizer_key_cmp_cost), CMD_LINE(REQUIRED_ARG),
VALID_RANGE(0, 1), DEFAULT(DEFAULT_KEY_COMPARE_COST), NO_MUTEX_GUARD,
NOT_IN_BINLOG);
COST_VAR(key_cmp_cost),
CMD_LINE(REQUIRED_ARG, OPT_COSTS_KEY_CMP_COST),
VALID_RANGE(0, 1000), DEFAULT(DEFAULT_KEY_COMPARE_COST), COST_ADJUST(1000));
static Sys_var_engine_optimizer_cost Sys_optimizer_rowid_cmp_cost(
"optimizer_rowid_compare_cost",
"Cost of comparing two rowid's",
COST_VAR(rowid_cmp_cost),
CMD_LINE(REQUIRED_ARG, OPT_COSTS_ROWID_CMP_COST),
VALID_RANGE(0, 1000), DEFAULT(DEFAULT_ROWID_COMPARE_COST), COST_ADJUST(1000));
static Sys_var_engine_optimizer_cost Sys_optimizer_rowid_copy_cost(
"optimizer_rowid_copy_cost",
"Cost of copying a rowid",
COST_VAR(rowid_copy_cost),
CMD_LINE(REQUIRED_ARG, OPT_COSTS_ROWID_COPY_COST),
VALID_RANGE(0, 1000), DEFAULT(DEFAULT_ROWID_COPY_COST), COST_ADJUST(1000));
/* The following costs are stored in THD and handler */
static Sys_var_optimizer_cost Sys_optimizer_where_cost(
"optimizer_where_cost",
"Cost of checking the row against the WHERE clause. Increasing this will "
"have the optimizer to prefer plans with less row combinations.",
SESSION_VAR(optimizer_where_cost),
CMD_LINE(REQUIRED_ARG),
VALID_RANGE(0, 100000), DEFAULT(DEFAULT_WHERE_COST), COST_ADJUST(1000));
static Sys_var_optimizer_cost Sys_optimizer_scan_cost(
"optimizer_scan_setup_cost",
"Extra cost added to TABLE and INDEX scans to get optimizer to prefer "
"index lookups.",
SESSION_VAR(optimizer_scan_setup_cost),
CMD_LINE(REQUIRED_ARG),
VALID_RANGE(0, 100000000), DEFAULT(DEFAULT_TABLE_SCAN_SETUP_COST),
COST_ADJUST(1000));

141
sql/sys_vars.inl
View File

@@ -32,6 +32,7 @@
#include "rpl_mi.h" // For Multi-Source Replication
#include "debug_sync.h"
#include "sql_acl.h" // check_global_access()
#include "optimizer_defaults.h" // create_optimizer_costs
/*
a set of mostly trivial (as in f(X)=X) defines below to make system variable
@@ -40,6 +41,7 @@
#define VALID_RANGE(X,Y) X,Y
#define DEFAULT(X) X
#define BLOCK_SIZE(X) X
#define COST_ADJUST(X) X
#define GLOBAL_VAR(X) sys_var::GLOBAL, (((char*)&(X))-(char*)&global_system_variables), sizeof(X)
#define SESSION_VAR(X) sys_var::SESSION, offsetof(SV, X), sizeof(((SV *)0)->X)
#define SESSION_ONLY(X) sys_var::ONLY_SESSION, offsetof(SV, X), sizeof(((SV *)0)->X)
@@ -1048,7 +1050,7 @@ public:
/* If no basename, assume it's for the key cache named 'default' */
if (!base_name->length)
base_name= &default_key_cache_base;
base_name= &default_base;
key_cache= get_key_cache(base_name);
@@ -1228,6 +1230,143 @@ public:
{ var->save_result.double_value= getopt_ulonglong2double(option.def_value); }
};
/*
Optimizer costs
Stored as cost factor (1 cost = 1 ms).
Given and displayed as microsconds (as most values are very small)
*/
class Sys_var_optimizer_cost: public Sys_var_double
{
public:
double cost_adjust;
Sys_var_optimizer_cost(const char *name_arg,
const char *comment, int flag_args, ptrdiff_t off, size_t size,
CMD_LINE getopt,
double min_val, double max_val, double def_val,
ulong arg_cost_adjust, PolyLock *lock=0,
enum binlog_status_enum binlog_status_arg=VARIABLE_NOT_IN_BINLOG,
on_check_function on_check_func=0,
on_update_function on_update_func=0,
const char *substitute=0)
:Sys_var_double(name_arg, comment, flag_args, off, size, getopt,
min_val, max_val, def_val, lock,
binlog_status_arg,
on_check_func,
on_update_func,
substitute)
{
if (arg_cost_adjust == 1000)
{
show_val_type= SHOW_OPTIMIZER_COST; // For select @@var
option.var_type|= GET_ADJUST_VALUE;
}
cost_adjust= (double) arg_cost_adjust;
global_var(double)= (double)option.def_value/cost_adjust; // To usec
}
bool session_update(THD *thd, set_var *var)
{
session_var(thd, double)= var->save_result.double_value/cost_adjust;
return false;
}
bool global_update(THD *thd, set_var *var)
{
global_var(double)= var->save_result.double_value/cost_adjust;
return false;
}
void session_save_default(THD *thd, set_var *var)
{ var->save_result.double_value= global_var(double) * cost_adjust; }
void global_save_default(THD *thd, set_var *var)
{
var->save_result.double_value= getopt_ulonglong2double(option.def_value)*
cost_adjust;
}
};
/*
The class for optimzer costs with structured names, unique for each engine.
Used as 'engine.variable_name'
Class specific constructor arguments:
everything derived from Sys_var_optimizer_cost
Backing store: double
@note these variables can be only GLOBAL
*/
#define COST_VAR(X) GLOBAL_VAR(default_optimizer_costs.X)
#define cost_var_ptr(KC, OFF) (((uchar*)(KC))+(OFF))
#define cost_var(KC, OFF) (*(double*)cost_var_ptr(KC, OFF))
typedef bool (*cost_update_function)(THD *, OPTIMIZER_COSTS *, ptrdiff_t,
double, double);
static bool update_cost(THD *thd, OPTIMIZER_COSTS *key_cache,
ptrdiff_t offset, double new_value, double cost_adjust)
{
cost_var(key_cache, offset)= new_value / cost_adjust;
return 0;
}
class Sys_var_engine_optimizer_cost: public Sys_var_optimizer_cost
{
cost_update_function cost_update;
public:
Sys_var_engine_optimizer_cost(const char *name_arg,
const char *comment, int flag_args, ptrdiff_t off, size_t size,
CMD_LINE getopt,
double min_val, double max_val, double def_val,
long cost_adjust, PolyLock *lock= 0,
cost_update_function on_update_func= update_cost,
const char *substitute=0)
: Sys_var_optimizer_cost(name_arg, comment, flag_args, off, size,
getopt, min_val, max_val, def_val, cost_adjust,
lock, VARIABLE_NOT_IN_BINLOG, 0,
0, substitute),
cost_update(on_update_func)
{
option.var_type|= GET_ASK_ADDR;
option.value= (uchar**)1; // crash me, please
// fix an offset from global_system_variables to be an offset in KEY_CACHE
offset= global_var_ptr() - (uchar*) &default_optimizer_costs;
SYSVAR_ASSERT(scope() == GLOBAL);
}
bool global_update(THD *thd, set_var *var)
{
double new_value= var->save_result.double_value;
LEX_CSTRING *base_name= &var->base;
OPTIMIZER_COSTS *optimizer_costs;
bool res;
/* If no basename, assume it's for the default costs */
if (!base_name->length)
base_name= &default_base;
mysql_mutex_lock(&LOCK_optimizer_costs);
if (!(optimizer_costs= get_or_create_optimizer_costs(base_name->str,
base_name->length)))
{
mysql_mutex_unlock(&LOCK_optimizer_costs);
return true;
}
res= cost_update(thd, optimizer_costs, offset, new_value, cost_adjust);
mysql_mutex_unlock(&LOCK_optimizer_costs);
return res;
}
const uchar *global_value_ptr(THD *thd, const LEX_CSTRING *base) const
{
OPTIMIZER_COSTS *optimizer_costs= get_optimizer_costs(base);
if (!optimizer_costs)
optimizer_costs= &default_optimizer_costs;
return cost_var_ptr(optimizer_costs, offset);
}
};
/**
The class for the @max_user_connections.
It's derived from Sys_var_uint, but non-standard session value

43
sql/table.cc
View File

@@ -2792,6 +2792,8 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write,
DBUG_ASSERT((null_pos + (null_bit_pos + 7) / 8) <= share->field[0]->ptr);
}
share->primary_key= MAX_KEY;
/* Fix key->name and key_part->field */
if (key_parts)
{
@@ -2923,6 +2925,11 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write,
}
}
/* Primary key must be set early as engine may use it in index_flag() */
share->primary_key= (primary_key < MAX_KEY &&
share->keys_in_use.is_set(primary_key) ?
primary_key : MAX_KEY);
key_first_info= keyinfo;
for (uint key=0 ; key < keys ; key++,keyinfo++)
{
@@ -3165,7 +3172,7 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write,
if (primary_key < MAX_KEY &&
(share->keys_in_use.is_set(primary_key)))
{
share->primary_key= primary_key;
DBUG_ASSERT(share->primary_key == primary_key);
/*
If we are using an integer as the primary key then allow the user to
refer to it as '_rowid'
@@ -3182,10 +3189,10 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write,
}
}
else
share->primary_key = MAX_KEY; // we do not have a primary key
{
DBUG_ASSERT(share->primary_key == MAX_KEY);
}
}
else
share->primary_key= MAX_KEY;
if (new_field_pack_flag <= 1)
{
/* Old file format with default as not null */
@@ -3413,6 +3420,27 @@ err:
}
/*
Make a copy of optimizer costs to be able to access these without any locks
and to allow the engine to update costs.
*/
void TABLE_SHARE::update_optimizer_costs(handlerton *hton)
{
if (hton != view_pseudo_hton && !(hton->flags & HTON_HIDDEN))
{
mysql_mutex_lock(&LOCK_optimizer_costs);
memcpy(&optimizer_costs, hton->optimizer_costs, sizeof(optimizer_costs));
mysql_mutex_unlock(&LOCK_optimizer_costs);
}
else
{
bzero(&optimizer_costs, sizeof(optimizer_costs));
MEM_UNDEFINED(&optimizer_costs, sizeof(optimizer_costs));
}
}
static bool sql_unusable_for_discovery(THD *thd, handlerton *engine,
const char *sql)
{
@@ -5673,7 +5701,6 @@ void TABLE::init(THD *thd, TABLE_LIST *tl)
no_cache= false;
initialize_opt_range_structures();
/*
Update optimizer_costs to ensure that a SET STATEMENT of the
variables it will work.
@@ -10438,10 +10465,10 @@ inline void TABLE::initialize_opt_range_structures()
}
double TABLE::OPT_RANGE::index_only_fetch_cost(THD *thd)
double TABLE::OPT_RANGE::index_only_fetch_cost(TABLE *table)
{
return (index_only_cost + (double) rows *
thd->variables.optimizer_key_copy_cost);
return (index_only_cost +
(double) rows * table->s->optimizer_costs.key_copy_cost);
}

11
sql/table.h
View File

@@ -813,6 +813,7 @@ struct TABLE_SHARE
return is_view ? view_pseudo_hton :
db_plugin ? plugin_hton(db_plugin) : NULL;
}
OPTIMIZER_COSTS optimizer_costs; /* Copy of get_optimizer_costs() */
enum row_type row_type; /* How rows are stored */
enum Table_type table_type;
enum tmp_table_type tmp_table;
@@ -888,6 +889,7 @@ struct TABLE_SHARE
bool has_update_default_function;
bool can_do_row_logging; /* 1 if table supports RBR */
bool long_unique_table;
bool optimizer_costs_inited;
ulong table_map_id; /* for row-based replication */
@@ -1194,6 +1196,7 @@ struct TABLE_SHARE
void set_overlapped_keys();
void set_ignored_indexes();
key_map usable_indexes(THD *thd);
void update_optimizer_costs(handlerton *hton);
};
/* not NULL, but cannot be dereferenced */
@@ -1420,7 +1423,7 @@ public:
Cost of fetching keys with index only read and returning them to the
sql level.
*/
double index_only_fetch_cost(THD *thd);
double index_only_fetch_cost(TABLE *table);
} *opt_range;
/*
Bitmaps of key parts that =const for the duration of join execution. If
@@ -1745,6 +1748,12 @@ public:
uint actual_n_key_parts(KEY *keyinfo);
ulong actual_key_flags(KEY *keyinfo);
int update_virtual_field(Field *vf, bool ignore_warnings);
inline size_t key_storage_length(uint index)
{
if (file->is_clustering_key(index))
return s->stored_rec_length;
return key_info[index].key_length + file->ref_length;
}
int update_virtual_fields(handler *h, enum_vcol_update_mode update_mode);
int update_default_fields(bool ignore_errors);
void evaluate_update_default_function();

31
sql/uniques.cc
View File

@@ -159,7 +159,7 @@ inline double log2_n_fact(double x)
total_buf_elems* log2(n_buffers) * ROWID_COMPARE_COST;
*/
static double get_merge_buffers_cost(uint *buff_elems, uint elem_size,
static double get_merge_buffers_cost(THD *thd, uint *buff_elems, uint elem_size,
uint *first, uint *last,
double compare_factor)
{
@@ -171,7 +171,8 @@ static double get_merge_buffers_cost(uint *buff_elems, uint elem_size,
size_t n_buffers= last - first + 1;
/* Using log2(n)=log(n)/log(2) formula */
return (2*((double)total_buf_elems*elem_size) / IO_SIZE +
return (2*((double)total_buf_elems*elem_size) / IO_SIZE *
default_optimizer_costs.disk_read_cost +
total_buf_elems*log((double) n_buffers) * compare_factor / M_LN2);
}
@@ -185,6 +186,7 @@ static double get_merge_buffers_cost(uint *buff_elems, uint elem_size,
SYNOPSIS
get_merge_many_buffs_cost()
thd THD, used to get disk_read_cost
buffer buffer space for temporary data, at least
Unique::get_cost_calc_buff_size bytes
maxbuffer # of full buffers
@@ -203,7 +205,8 @@ static double get_merge_buffers_cost(uint *buff_elems, uint elem_size,
Cost of merge in disk seeks.
*/
static double get_merge_many_buffs_cost(uint *buffer,
static double get_merge_many_buffs_cost(THD *thd,
uint *buffer,
uint maxbuffer, uint max_n_elems,
uint last_n_elems, int elem_size,
double compare_factor)
@@ -231,13 +234,13 @@ static double get_merge_many_buffs_cost(uint *buffer,
uint lastbuff= 0;
for (i = 0; i <= (int) maxbuffer - MERGEBUFF*3/2; i += MERGEBUFF)
{
total_cost+=get_merge_buffers_cost(buff_elems, elem_size,
total_cost+=get_merge_buffers_cost(thd, buff_elems, elem_size,
buff_elems + i,
buff_elems + i + MERGEBUFF-1,
compare_factor);
lastbuff++;
}
total_cost+=get_merge_buffers_cost(buff_elems, elem_size,
total_cost+=get_merge_buffers_cost(thd, buff_elems, elem_size,
buff_elems + i,
buff_elems + maxbuffer,
compare_factor);
@@ -246,7 +249,7 @@ static double get_merge_many_buffs_cost(uint *buffer,
}
/* Simulate final merge_buff call. */
total_cost += get_merge_buffers_cost(buff_elems, elem_size,
total_cost += get_merge_buffers_cost(thd, buff_elems, elem_size,
buff_elems, buff_elems + maxbuffer,
compare_factor);
return total_cost;
@@ -304,7 +307,7 @@ static double get_merge_many_buffs_cost(uint *buffer,
these will be random seeks.
*/
double Unique::get_use_cost(uint *buffer, size_t nkeys, uint key_size,
double Unique::get_use_cost(THD *thd, uint *buffer, size_t nkeys, uint key_size,
size_t max_in_memory_size,
double compare_factor,
bool intersect_fl, bool *in_memory)
@@ -312,7 +315,7 @@ double Unique::get_use_cost(uint *buffer, size_t nkeys, uint key_size,
size_t max_elements_in_tree;
size_t last_tree_elems;
size_t n_full_trees; /* number of trees in unique - 1 */
double result;
double result, disk_read_cost;
max_elements_in_tree= ((size_t) max_in_memory_size /
ALIGN_SIZE(sizeof(TREE_ELEMENT)+key_size));
@@ -345,14 +348,15 @@ double Unique::get_use_cost(uint *buffer, size_t nkeys, uint key_size,
First, add cost of writing all trees to disk, assuming that all disk
writes are sequential.
*/
result += DISK_SEEK_BASE_COST * n_full_trees *
ceil(((double) key_size)*max_elements_in_tree / IO_SIZE);
result += DISK_SEEK_BASE_COST * ceil(((double) key_size)*last_tree_elems / IO_SIZE);
disk_read_cost= DISK_READ_COST_THD(thd);
result += disk_read_cost * n_full_trees *
ceil(((double) key_size)*max_elements_in_tree / DISK_CHUNK_SIZE);
result += disk_read_cost * ceil(((double) key_size)*last_tree_elems / DISK_CHUNK_SIZE);
/* Cost of merge */
if (intersect_fl)
key_size+= sizeof(element_count);
double merge_cost= get_merge_many_buffs_cost(buffer, (uint)n_full_trees,
double merge_cost= get_merge_many_buffs_cost(thd, buffer, (uint)n_full_trees,
(uint)max_elements_in_tree,
(uint)last_tree_elems, key_size,
compare_factor);
@@ -361,7 +365,8 @@ double Unique::get_use_cost(uint *buffer, size_t nkeys, uint key_size,
Add cost of reading the resulting sequence, assuming there were no
duplicate elements.
*/
result += ceil((double)key_size*nkeys/IO_SIZE);
result+= (ceil((double)key_size*nkeys/IO_SIZE) *
default_optimizer_costs.disk_read_cost);
return result;
}

2
sql/uniques.h
View File

@@ -78,7 +78,7 @@ public:
return log((double) tree_elems) * compare_factor / M_LN2;
}
static double get_use_cost(uint *buffer, size_t nkeys, uint key_size,
static double get_use_cost(THD *thd, uint *buffer, size_t nkeys, uint key_size,
size_t max_in_memory_size, double compare_factor,
bool intersect_fl, bool *in_memory);
inline static int get_cost_calc_buff_size(size_t nkeys, uint key_size,

60
storage/archive/ha_archive.cc
View File

@@ -132,7 +132,8 @@ extern "C" PSI_file_key arch_key_file_data;
static handler *archive_create_handler(handlerton *hton,
TABLE_SHARE *table,
MEM_ROOT *mem_root);
int archive_discover(handlerton *hton, THD* thd, TABLE_SHARE *share);
static int archive_discover(handlerton *hton, THD* thd, TABLE_SHARE *share);
static void archive_update_optimizer_costs(OPTIMIZER_COSTS *costs);
/*
Number of rows that will force a bulk insert.
@@ -205,6 +206,7 @@ static const char *ha_archive_exts[] = {
NullS
};
int archive_db_init(void *p)
{
DBUG_ENTER("archive_db_init");
@@ -217,10 +219,10 @@ int archive_db_init(void *p)
archive_hton= (handlerton *)p;
archive_hton->db_type= DB_TYPE_ARCHIVE_DB;
archive_hton->create= archive_create_handler;
archive_hton->flags= HTON_NO_FLAGS;
archive_hton->discover_table= archive_discover;
archive_hton->tablefile_extensions= ha_archive_exts;
archive_hton->update_optimizer_costs= archive_update_optimizer_costs;
archive_hton->flags= HTON_NO_FLAGS;
DBUG_RETURN(0);
}
@@ -267,7 +269,7 @@ ha_archive::ha_archive(handlerton *hton, TABLE_SHARE *table_arg)
archive_reader_open= FALSE;
}
int archive_discover(handlerton *hton, THD* thd, TABLE_SHARE *share)
static int archive_discover(handlerton *hton, THD* thd, TABLE_SHARE *share)
{
DBUG_ENTER("archive_discover");
DBUG_PRINT("archive_discover", ("db: '%s' name: '%s'", share->db.str,
@@ -1092,6 +1094,54 @@ int ha_archive::index_init(uint keynr, bool sorted)
DBUG_RETURN(0);
}
#define ARCHIVE_DECOMPRESS_TIME 0.081034543792841 // See optimizer_costs.txt
static void archive_update_optimizer_costs(OPTIMIZER_COSTS *costs)
{
costs->disk_read_ratio= 0.20; // Assume 80 % of data is cached by system
costs->row_lookup_cost= 0; // See rnd_pos_time
costs->key_lookup_cost= 0; // See key_read_time
costs->key_next_find_cost= 0; // Only unique indexes
costs->index_block_copy_cost= 0;
}
IO_AND_CPU_COST ha_archive::scan_time()
{
IO_AND_CPU_COST cost;
ulonglong blocks;
DBUG_ENTER("ha_archive::scan_time");
blocks= stats.data_file_length / IO_SIZE;
cost.io= 0; // No cache
cost.cpu= (blocks * DISK_READ_COST * DISK_READ_RATIO +
blocks* ARCHIVE_DECOMPRESS_TIME);
DBUG_RETURN(cost);
}
IO_AND_CPU_COST ha_archive::keyread_time(uint index, ulong ranges, ha_rows rows,
ulonglong blocks)
{
IO_AND_CPU_COST cost= scan_time();
/*
As these is an unique indexe, assume that we have to scan half the file for
each range to find the row.
*/
cost.cpu= cost.cpu * ranges / 2;
return cost;
}
IO_AND_CPU_COST ha_archive::rnd_pos_time(ha_rows rows)
{
IO_AND_CPU_COST cost;
/* We have to do one azseek() for each row */
cost.io= rows2double(rows);
cost.cpu= rows * (DISK_READ_COST * DISK_READ_RATIO + ARCHIVE_DECOMPRESS_TIME);
return cost;
}
/*
No indexes, so if we get a request for an index search since we tell
@@ -1116,8 +1166,6 @@ int ha_archive::index_read_idx(uchar *buf, uint index, const uchar *key,
current_k_offset= mkey->key_part->offset;
current_key= key;
current_key_len= key_len;
DBUG_ENTER("ha_archive::index_read_idx");
rc= rnd_init(TRUE);

4
storage/archive/ha_archive.h
View File

@@ -111,6 +111,10 @@ public:
uint max_supported_key_length() const { return sizeof(ulonglong); }
uint max_supported_key_part_length() const { return sizeof(ulonglong); }
ha_rows records() { return share->rows_recorded; }
IO_AND_CPU_COST scan_time() override;
IO_AND_CPU_COST keyread_time(uint index, ulong ranges, ha_rows rows,
ulonglong blocks) override;
IO_AND_CPU_COST rnd_pos_time(ha_rows rows) override;
int index_init(uint keynr, bool sorted);
virtual int index_read(uchar * buf, const uchar * key,
uint key_len, enum ha_rkey_function find_flag);

11
storage/connect/ha_connect.h
View File

@@ -308,13 +308,18 @@ public:
/** @brief
Called in test_quick_select to determine if indexes should be used.
*/
virtual double scan_time() { return (double) (stats.records+stats.deleted) / 20.0+10; }
virtual IO_AND_CPU_COST scan_time()
{ return { 0, (double) (stats.records+stats.deleted) * avg_io_cost() }; };
/** @brief
This method will never be called if you do not implement indexes.
*/
virtual double read_time(uint, uint, ha_rows rows)
{ return (double) rows / 20.0+1; }
virtual IO_AND_CPU_COST keyread_time(uint index, ulong ranges, ha_rows rows,
ulonglong blocks)
{
return { 0, (double) rows * 0.001 };
}
/*
Everything below are methods that we implement in ha_connect.cc.

7
storage/csv/ha_tina.h
View File

@@ -124,7 +124,12 @@ public:
/*
Called in test_quick_select to determine if indexes should be used.
*/
virtual double scan_time() { return (double) (stats.records+stats.deleted) / 20.0+10; }
virtual IO_AND_CPU_COST scan_time()
{
return { (double) ((share->saved_data_file_length + IO_SIZE-1))/ IO_SIZE *
avg_io_cost(),
(stats.records+stats.deleted) * ROW_NEXT_FIND_COST };
}
/* The next method will never be called */
virtual bool fast_key_read() { return 1;}
/*

33
storage/example/ha_example.h
View File

@@ -150,15 +150,40 @@ public:
uint max_supported_key_length() const { return 0; }
/** @brief
Called in test_quick_select to determine if indexes should be used.
Called in test_quick_select to determine cost of table scan
*/
virtual double scan_time() { return (double) (stats.records+stats.deleted) / 20.0+10; }
virtual IO_AND_CPU_COST scan_time()
{
IO_AND_CPU_COST cost;
/* 0 blocks, 0.001 ms / row */
cost.io= (double) (stats.records+stats.deleted) * avg_io_cost();
cost.cpu= 0;
return cost;
}
/** @brief
This method will never be called if you do not implement indexes.
*/
virtual double read_time(uint, uint, ha_rows rows)
{ return (double) rows / 20.0+1; }
virtual IO_AND_CPU_COST keyread_time(uint, ulong, ha_rows rows,
ulonglong blocks)
{
IO_AND_CPU_COST cost;
cost.io= blocks * avg_io_cost();
cost.cpu= (double) rows * 0.001;
return cost;
}
/** @brief
Cost of fetching 'rows' records through rnd_pos()
*/
virtual IO_AND_CPU_COST rnd_pos_time(ha_rows rows)
{
IO_AND_CPU_COST cost;
/* 0 blocks, 0.001 ms / row */
cost.io= 0;
cost.cpu= (double) rows * avg_io_cost();
return cost;
}
/*
Everything below are methods that we implement in ha_example.cc.

24
storage/federated/ha_federated.cc
View File

@@ -459,6 +459,20 @@ static void init_federated_psi_keys(void)
}
#endif /* HAVE_PSI_INTERFACE */
/*
Federated doesn't need costs.disk_read_ratio as everything is one a
remote server and nothing is cached locally
*/
static void federated_update_optimizer_costs(OPTIMIZER_COSTS *costs)
{
/*
Setting disk_read_ratios to 1.0, ensures we are using the costs
from rnd_pos_time() and scan_time()
*/
costs->disk_read_ratio= 1.0;
}
/*
Initialize the federated handler.
@@ -485,6 +499,7 @@ int federated_db_init(void *p)
federated_hton->rollback= federated_rollback;
federated_hton->create= federated_create_handler;
federated_hton->drop_table= [](handlerton *, const char*) { return -1; };
federated_hton->update_optimizer_costs= federated_update_optimizer_costs;
federated_hton->flags= HTON_ALTER_NOT_SUPPORTED | HTON_NO_PARTITION;
/*
@@ -905,19 +920,10 @@ ha_federated::ha_federated(handlerton *hton,
:handler(hton, table_arg),
mysql(0), stored_result(0)
{
optimizer_cache_cost= 1;
trx_next= 0;
bzero(&bulk_insert, sizeof(bulk_insert));
}
/*
Federated doesn't need optimizer_cache_cost as everything is one a
remote server and nothing is cached locally
*/
void ha_federated::set_optimizer_cache_cost(double cost)
{}
/*
Convert MySQL result set row to handler internal format

21
storage/federated/ha_federated.h
View File

@@ -180,20 +180,25 @@ public:
The reason for "records * 1000" is that such a large number forces
this to use indexes "
*/
virtual double scan_time()
IO_AND_CPU_COST scan_time()
{
DBUG_PRINT("info", ("records %lu", (ulong) stats.records));
return (double)(stats.records*1000);
}
virtual double read_time(uint index, uint ranges, ha_rows rows)
return
{
return rows2double(rows) + rows2double(ranges);
(double) (stats.mean_rec_length * stats.records)/IO_SIZE * avg_io_cost(),
0
};
}
virtual double rnd_pos_time(ha_rows rows)
IO_AND_CPU_COST rnd_pos_time(ha_rows rows)
{
return rows2double(rows);
return { (double) stats.records * avg_io_cost(), 0 };
}
IO_AND_CPU_COST keyread_time(uint index, ulong ranges, ha_rows rows,
ulonglong blocks)
{
return { (double) (ranges + rows) * avg_io_cost(), 0 };
}
virtual void set_optimizer_cache_cost(double cost);
const key_map *keys_to_use_for_scanning() { return &key_map_full; }
/*

23
storage/federatedx/ha_federatedx.cc
View File

@@ -411,6 +411,20 @@ create_federatedx_derived_handler(THD* thd, TABLE_LIST *derived);
static select_handler*
create_federatedx_select_handler(THD* thd, SELECT_LEX *sel);
/*
Federated doesn't need costs.disk_read_ratio as everything is one a remote
server and nothing is cached locally
*/
static void federatedx_update_optimizer_costs(OPTIMIZER_COSTS *costs)
{
/*
Setting disk_read_ratios to 1.0, ensures we are using the costs
from rnd_pos_time() and scan_time()
*/
costs->disk_read_ratio= 0.0;
}
/*
Initialize the federatedx handler.
@@ -443,6 +457,7 @@ int federatedx_db_init(void *p)
federatedx_hton->flags= HTON_ALTER_NOT_SUPPORTED;
federatedx_hton->create_derived= create_federatedx_derived_handler;
federatedx_hton->create_select= create_federatedx_select_handler;
federatedx_hton->update_optimizer_costs= federatedx_update_optimizer_costs;
if (mysql_mutex_init(fe_key_mutex_federatedx,
&federatedx_mutex, MY_MUTEX_INIT_FAST))
@@ -842,17 +857,9 @@ ha_federatedx::ha_federatedx(handlerton *hton,
:handler(hton, table_arg),
txn(0), io(0), stored_result(0)
{
optimizer_cache_cost= 1;
bzero(&bulk_insert, sizeof(bulk_insert));
}
/*
Federated doesn't need optimizer_cache_cost as everything is one a remote server and
nothing is cached locally
*/
void ha_federatedx::set_optimizer_cache_cost(double cost)
{}
/*
Convert MySQL result set row to handler internal format

20
storage/federatedx/ha_federatedx.h
View File

@@ -367,20 +367,24 @@ public:
The reason for "records * 1000" is that such a large number forces
this to use indexes "
*/
double scan_time()
IO_AND_CPU_COST scan_time()
{
DBUG_PRINT("info", ("records %lu", (ulong) stats.records));
return (double)(stats.records*1000);
}
double read_time(uint index, uint ranges, ha_rows rows)
return
{
return rows2double(rows) + rows2double(ranges);
(double) (stats.mean_rec_length * stats.records)/8192 * avg_io_cost(),
0
};
}
virtual double rnd_pos_time(ha_rows rows)
IO_AND_CPU_COST keyread_time(uint index, ulong ranges, ha_rows rows,
ulonglong blocks)
{
return rows2double(rows);
return { (double) (ranges + rows) * avg_io_cost(), 0 };
}
IO_AND_CPU_COST rnd_pos_time(ha_rows rows)
{
return { (double) rows * avg_io_cost(), 0 };
}
virtual void set_optimizer_cache_cost(double cost);
const key_map *keys_to_use_for_scanning() { return &key_map_full; }
/*

59
storage/heap/ha_heap.cc
View File

@@ -42,6 +42,28 @@ static int heap_drop_table(handlerton *hton, const char *path)
return error == ENOENT ? -1 : error;
}
/* See optimizer_costs.txt for how the following values where calculated */
#define HEAP_ROW_NEXT_FIND_COST 8.0166e-06 // For table scan
#define BTREE_KEY_NEXT_FIND_COST 0.00007739 // For binary tree scan
#define HEAP_LOOKUP_COST 0.00016097 // Heap lookup cost
static void heap_update_optimizer_costs(OPTIMIZER_COSTS *costs)
{
/*
A lot of values are 0 as heap supports all needed xxx_time() functions
*/
costs->disk_read_cost=0; // All data in memory
costs->disk_read_ratio= 0.0; // All data in memory
costs->key_next_find_cost= 0;
costs->key_copy_cost= 0; // Set in keyread_time()
costs->row_copy_cost= 2.334e-06; // This is small as its just a memcpy
costs->row_lookup_cost= 0; // Direct pointer
costs->row_next_find_cost= 0;
costs->key_lookup_cost= 0;
costs->key_next_find_cost= 0;
costs->index_block_copy_cost= 0;
}
int heap_init(void *p)
{
handlerton *heap_hton;
@@ -53,6 +75,7 @@ int heap_init(void *p)
heap_hton->create= heap_create_handler;
heap_hton->panic= heap_panic;
heap_hton->drop_table= heap_drop_table;
heap_hton->update_optimizer_costs= heap_update_optimizer_costs;
heap_hton->flags= HTON_CAN_RECREATE;
return 0;
@@ -74,7 +97,6 @@ ha_heap::ha_heap(handlerton *hton, TABLE_SHARE *table_arg)
:handler(hton, table_arg), file(0), records_changed(0), key_stat_version(0),
internal_table(0)
{
optimizer_cache_cost= 1.0;
}
/*
@@ -230,6 +252,41 @@ void ha_heap::update_key_stats()
}
IO_AND_CPU_COST ha_heap::keyread_time(uint index, ulong ranges, ha_rows rows,
ulonglong blocks)
{
KEY *key=table->key_info+index;
if (key->algorithm == HA_KEY_ALG_BTREE)
{
double lookup_cost;
lookup_cost= ranges * costs->key_cmp_cost * log2(stats.records+1);
return {0, ranges * lookup_cost + (rows-ranges) * BTREE_KEY_NEXT_FIND_COST };
}
else
{
return {0, (ranges * HEAP_LOOKUP_COST +
(rows-ranges) * BTREE_KEY_NEXT_FIND_COST) };
}
}
IO_AND_CPU_COST ha_heap::scan_time()
{
return {0, (double) (stats.records+stats.deleted) * HEAP_ROW_NEXT_FIND_COST };
}
IO_AND_CPU_COST ha_heap::rnd_pos_time(ha_rows rows)
{
/*
The row pointer is a direct pointer to the block. Thus almost instant
in practice.
Note that ha_rnd_pos_time() will add ROW_COPY_COST to this result
*/
return { 0, 0 };
}
int ha_heap::write_row(const uchar * buf)
{
int res;

27
storage/heap/ha_heap.h
View File

@@ -62,22 +62,13 @@ public:
const key_map *keys_to_use_for_scanning() { return &btree_keys; }
uint max_supported_keys() const { return MAX_KEY; }
uint max_supported_key_part_length() const { return MAX_KEY_LENGTH; }
double scan_time() override
{ return (double) (stats.records+stats.deleted) / 20.0+10; }
double read_time(uint index, uint ranges, ha_rows rows) override
{ return (double) (rows +1)/ 20.0; }
double keyread_time(uint index, uint ranges, ha_rows rows) override
{ return (double) (rows + ranges) / 20.0 ; }
double avg_io_cost()
{ return 0.05; } /* 1/20 */
double rnd_pos_time(ha_rows rows) override
{
return (double) rows/ 20.0;
}
/*
Heap doesn't need optimizer_cache_cost as everything is in memory and
it supports all needed _time() functions
*/
IO_AND_CPU_COST scan_time() override;
IO_AND_CPU_COST keyread_time(uint index, ulong ranges, ha_rows rows,
ulonglong blocks) override;
IO_AND_CPU_COST rnd_pos_time(ha_rows rows) override;
/* 0 for avg_io_cost ensures that there are no read-block calculations */
double avg_io_cost() override { return 0.0; }
int open(const char *name, int mode, uint test_if_locked);
int close(void);
void set_keys_for_scanning(void);
@@ -88,10 +79,6 @@ public:
ulonglong nb_desired_values,
ulonglong *first_value,
ulonglong *nb_reserved_values);
void set_optimizer_cache_cost(double cost) override
{
optimizer_cache_cost= 1.0;
}
int index_read_map(uchar * buf, const uchar * key, key_part_map keypart_map,
enum ha_rkey_function find_flag);
int index_read_last_map(uchar *buf, const uchar *key, key_part_map keypart_map);

5
storage/innobase/btr/btr0cur.cc
View File

@@ -5134,6 +5134,7 @@ search_loop:
DBUG_EXECUTE_IF("bug14007649", DBUG_RETURN(n_rows););
#ifdef NOT_USED
/* Do not estimate the number of rows in the range to over 1 / 2 of the
estimated rows in the whole table */
@@ -5148,6 +5149,10 @@ search_loop:
if (n_rows == 0)
n_rows= table_n_rows;
}
#else
if (n_rows > table_n_rows)
n_rows= table_n_rows;
#endif
DBUG_RETURN(n_rows);

53
storage/innobase/handler/ha_innodb.cc
View File

@@ -4019,6 +4019,26 @@ skip_buffering_tweak:
DBUG_RETURN(0);
}
/*********************************************************************//**
Setup costs factors for InnoDB to be able to approximate how many
ms different opperations takes. See cost functions in handler.h how
the different variables are used */
static void innobase_update_optimizer_costs(OPTIMIZER_COSTS *costs)
{
/*
The following number was found by check_costs.pl when using 1M rows
and all rows are cached. See optimizer_costs.txt for details
*/
costs->row_next_find_cost= 0.00007013;
costs->row_lookup_cost= 0.00076597;
costs->key_next_find_cost= 0.00009900;
costs->key_lookup_cost= 0.00079112;
costs->row_copy_cost= 0.00006087;
}
/** Initialize the InnoDB storage engine plugin.
@param[in,out] p InnoDB handlerton
@return error code
@@ -4086,6 +4106,8 @@ static int innodb_init(void* p)
innobase_hton->prepare_commit_versioned
= innodb_prepare_commit_versioned;
innobase_hton->update_optimizer_costs= innobase_update_optimizer_costs;
innodb_remember_check_sysvar_funcs();
compile_time_assert(DATA_MYSQL_TRUE_VARCHAR == MYSQL_TYPE_VARCHAR);
@@ -5017,10 +5039,10 @@ ha_innobase::index_flags(
}
ulong flags= key == table_share->primary_key
? HA_CLUSTERED_INDEX : 0;
? HA_CLUSTERED_INDEX : HA_KEYREAD_ONLY;
flags |= HA_READ_NEXT | HA_READ_PREV | HA_READ_ORDER
| HA_READ_RANGE | HA_KEYREAD_ONLY
| HA_READ_RANGE
| HA_DO_INDEX_COND_PUSHDOWN
| HA_DO_RANGE_FILTER_PUSHDOWN;
@@ -14287,13 +14309,15 @@ ha_innobase::estimate_rows_upper_bound()
DBUG_RETURN((ha_rows) estimate);
}
/*********************************************************************//**
How many seeks it will take to read through the table. This is to be
comparable to the number returned by records_in_range so that we can
decide if we should scan the table or use keys.
@return estimated time measured in disk seeks */
double
#ifdef NOT_USED
IO_AND_CPU_COST
ha_innobase::scan_time()
/*====================*/
{
@@ -14313,24 +14337,28 @@ ha_innobase::scan_time()
TODO: This will be further improved to return some approximate
estimate but that would also needs pre-population of stats
structure. As of now approach is in sync with MyISAM. */
return(ulonglong2double(stats.data_file_length) / IO_SIZE + 2);
return { (ulonglong2double(stats.data_file_length) / IO_SIZE * avg_io_cost()), 0.0 };
}
ulint stat_clustered_index_size;
IO_AND_CPU_COST cost;
ut_a(m_prebuilt->table->stat_initialized);
stat_clustered_index_size =
m_prebuilt->table->stat_clustered_index_size;
return((double) stat_clustered_index_size);
cost.io= (double) stat_clustered_index_size * avg_io_cost();
cost.cpu= 0;
return(cost);
}
#endif
/******************************************************************//**
Calculate the time it takes to read a set of ranges through an index
This enables us to optimise reads for clustered indexes.
@return estimated time measured in disk seeks */
#ifdef NOT_USED
double
ha_innobase::read_time(
/*===================*/
@@ -14355,14 +14383,14 @@ ha_innobase::read_time(
return(time_for_scan);
}
return(ranges + (double) rows / (double) total_rows * time_for_scan);
return(ranges * KEY_LOOKUP_COST + (double) rows / (double) total_rows * time_for_scan);
}
/******************************************************************//**
Calculate the time it takes to read a set of rows with primary key.
*/
double
IO_AND_CPU_COST
ha_innobase::rnd_pos_time(ha_rows rows)
{
ha_rows total_rows;
@@ -14370,15 +14398,18 @@ ha_innobase::rnd_pos_time(ha_rows rows)
/* Assume that the read time is proportional to the scan time for all
rows + at most one seek per range. */
double time_for_scan = scan_time();
IO_AND_CPU_COST time_for_scan = scan_time();
if ((total_rows = estimate_rows_upper_bound()) < rows) {
return(time_for_scan);
}
return((double) rows + (double) rows / (double) total_rows * time_for_scan);
double frac= (double) rows + (double) rows / (double) total_rows;
time_for_scan.io*= frac;
time_for_scan.cpu*= frac;
return(time_for_scan);
}
#endif
/*********************************************************************//**
Calculates the key number used inside MySQL for an Innobase index.

8
storage/innobase/handler/ha_innodb.h
View File

@@ -105,12 +105,10 @@ public:
int close(void) override;
double scan_time() override;
double read_time(uint index, uint ranges, ha_rows rows) override;
#ifdef NOT_USED
IO_AND_CPU_COST scan_time() override;
double rnd_pos_time(ha_rows rows) override;
#endif
int write_row(const uchar * buf) override;
int update_row(const uchar * old_data, const uchar * new_data) override;

47
storage/maria/ha_maria.cc
View File

@@ -1101,14 +1101,44 @@ ulong ha_maria::index_flags(uint inx, uint part, bool all_parts) const
}
double ha_maria::scan_time()
/*
Update costs that are unique for this TABLE instance
*/
void ha_maria::update_optimizer_costs(OPTIMIZER_COSTS *costs)
{
if (file->s->data_file_type == BLOCK_RECORD)
return (ulonglong2double(stats.data_file_length - file->s->block_size) /
file->s->block_size) + 2;
return handler::scan_time();
/*
Default costs for Aria with BLOCK_FORMAT is the same as MariaDB default
costs.
*/
if (file->s->data_file_type != BLOCK_RECORD)
{
/*
MyISAM format row lookup costs are slow as the row data is on a not
cached file. Costs taken from ha_myisam.cc
*/
costs->row_next_find_cost= 0.000063539;
costs->row_lookup_cost= 0.001014818;
}
}
IO_AND_CPU_COST ha_maria::rnd_pos_time(ha_rows rows)
{
IO_AND_CPU_COST cost= handler::rnd_pos_time(rows);
/* file may be 0 if this is an internal temporary file that is not yet opened */
if (file && file->s->data_file_type != BLOCK_RECORD)
{
/*
Row data is not cached. costs.row_lookup_cost includes the cost of
the reading the row from system (probably cached by the OS).
*/
cost.io= 0;
}
return cost;
}
/*
We need to be able to store at least 2 keys on an index page as the
splitting algorithms depends on this. (With only one key on a page
@@ -3839,6 +3869,12 @@ bool ha_maria::is_changed() const
return file->state->changed;
}
static void aria_update_optimizer_costs(OPTIMIZER_COSTS *costs)
{
costs->rowid_copy_cost= 0.000001; // Just a short memcopy
costs->rowid_cmp_cost= 0.000001; // Just a short memcmp
}
static int ha_maria_init(void *p)
{
@@ -3871,6 +3907,7 @@ static int ha_maria_init(void *p)
maria_hton->show_status= maria_show_status;
maria_hton->prepare_for_backup= maria_prepare_for_backup;
maria_hton->end_backup= maria_end_backup;
maria_hton->update_optimizer_costs= aria_update_optimizer_costs;
/* TODO: decide if we support Maria being used for log tables */
maria_hton->flags= (HTON_CAN_RECREATE | HTON_SUPPORT_LOG_TABLES |

4
storage/maria/ha_maria.h
View File

@@ -77,8 +77,6 @@ public:
{ return max_supported_key_length(); }
enum row_type get_row_type() const override final;
void change_table_ptr(TABLE *table_arg, TABLE_SHARE *share) override final;
virtual double scan_time() override final;
int open(const char *name, int mode, uint test_if_locked) override;
int close(void) override final;
int write_row(const uchar * buf) override;
@@ -114,6 +112,8 @@ public:
int remember_rnd_pos() override final;
int restart_rnd_next(uchar * buf) override final;
void position(const uchar * record) override final;
void update_optimizer_costs(OPTIMIZER_COSTS *costs) override final;
IO_AND_CPU_COST rnd_pos_time(ha_rows rows) override final;
int info(uint) override final;
int info(uint, my_bool);
int extra(enum ha_extra_function operation) override final;

2
storage/maria/ma_pagecache.c
View File

@@ -3876,7 +3876,7 @@ restart:
{
pagecache_pthread_mutex_unlock(&pagecache->cache_lock);
DBUG_ASSERT(0);
return (uchar*) 0;
DBUG_RETURN((uchar*) 0);
}
}
/*

72
storage/mroonga/ha_mroonga.cpp
View File

@@ -4308,6 +4308,7 @@ int ha_mroonga::wrapper_open(const char *name, int mode, uint open_options)
wrap_handler->set_ha_share_ref(&table->s->ha_share);
#endif
error = wrap_handler->ha_open(table, name, mode, open_options);
wrap_handler->set_optimizer_costs(ha_thd());
} else {
if (!(wrap_handler = parent_for_clone->wrap_handler->clone(name,
mem_root_for_clone)))
@@ -13008,9 +13009,9 @@ int ha_mroonga::truncate()
DBUG_RETURN(error);
}
double ha_mroonga::wrapper_scan_time()
IO_AND_CPU_COST ha_mroonga::wrapper_scan_time()
{
double res;
IO_AND_CPU_COST res;
MRN_DBUG_ENTER_METHOD();
MRN_SET_WRAP_SHARE_KEY(share, table->s);
MRN_SET_WRAP_TABLE_KEY(this, table);
@@ -13020,17 +13021,16 @@ double ha_mroonga::wrapper_scan_time()
DBUG_RETURN(res);
}
double ha_mroonga::storage_scan_time()
IO_AND_CPU_COST ha_mroonga::storage_scan_time()
{
MRN_DBUG_ENTER_METHOD();
double time = handler::scan_time();
DBUG_RETURN(time);
DBUG_RETURN(handler::scan_time());
}
double ha_mroonga::scan_time()
IO_AND_CPU_COST ha_mroonga::scan_time()
{
MRN_DBUG_ENTER_METHOD();
double time;
IO_AND_CPU_COST time;
if (share->wrapper_mode)
{
time = wrapper_scan_time();
@@ -13040,51 +13040,87 @@ double ha_mroonga::scan_time()
DBUG_RETURN(time);
}
double ha_mroonga::wrapper_read_time(uint index, uint ranges, ha_rows rows)
IO_AND_CPU_COST ha_mroonga::wrapper_rnd_pos_time(ha_rows rows)
{
double res;
IO_AND_CPU_COST res;
MRN_DBUG_ENTER_METHOD();
MRN_SET_WRAP_SHARE_KEY(share, table->s);
MRN_SET_WRAP_TABLE_KEY(this, table);
res = wrap_handler->rnd_pos_time(rows);
MRN_SET_BASE_SHARE_KEY(share, table->s);
MRN_SET_BASE_TABLE_KEY(this, table);
DBUG_RETURN(res);
}
IO_AND_CPU_COST ha_mroonga::storage_rnd_pos_time(ha_rows rows)
{
MRN_DBUG_ENTER_METHOD();
IO_AND_CPU_COST time = handler::rnd_pos_time(rows);
DBUG_RETURN(time);
}
IO_AND_CPU_COST ha_mroonga::rnd_pos_time(ha_rows rows)
{
MRN_DBUG_ENTER_METHOD();
IO_AND_CPU_COST time;
if (share->wrapper_mode)
{
time = wrapper_rnd_pos_time(rows);
} else {
time = storage_rnd_pos_time(rows);
}
DBUG_RETURN(time);
}
IO_AND_CPU_COST ha_mroonga::wrapper_keyread_time(uint index, ulong ranges,
ha_rows rows, ulonglong blocks)
{
IO_AND_CPU_COST res;
MRN_DBUG_ENTER_METHOD();
if (index < MAX_KEY) {
KEY *key_info = &(table->key_info[index]);
if (mrn_is_geo_key(key_info)) {
res = handler::read_time(index, ranges, rows);
res = handler::keyread_time(index, ranges, rows, blocks);
DBUG_RETURN(res);
}
MRN_SET_WRAP_SHARE_KEY(share, table->s);
MRN_SET_WRAP_TABLE_KEY(this, table);
res = wrap_handler->read_time(share->wrap_key_nr[index], ranges, rows);
res = wrap_handler->keyread_time(share->wrap_key_nr[index], ranges, rows, blocks);
MRN_SET_BASE_SHARE_KEY(share, table->s);
MRN_SET_BASE_TABLE_KEY(this, table);
} else {
MRN_SET_WRAP_SHARE_KEY(share, table->s);
MRN_SET_WRAP_TABLE_KEY(this, table);
res = wrap_handler->read_time(index, ranges, rows);
res = wrap_handler->keyread_time(index, ranges, rows, blocks);
MRN_SET_BASE_SHARE_KEY(share, table->s);
MRN_SET_BASE_TABLE_KEY(this, table);
}
DBUG_RETURN(res);
}
double ha_mroonga::storage_read_time(uint index, uint ranges, ha_rows rows)
IO_AND_CPU_COST ha_mroonga::storage_keyread_time(uint index, ulong ranges, ha_rows rows, ulonglong blocks)
{
MRN_DBUG_ENTER_METHOD();
double time = handler::read_time(index, ranges, rows);
IO_AND_CPU_COST time = handler::keyread_time(index, ranges, rows, blocks);
DBUG_RETURN(time);
}
double ha_mroonga::read_time(uint index, uint ranges, ha_rows rows)
IO_AND_CPU_COST ha_mroonga::keyread_time(uint index, ulong ranges, ha_rows rows, ulonglong blocks)
{
MRN_DBUG_ENTER_METHOD();
double time;
IO_AND_CPU_COST time;
if (share->wrapper_mode)
{
time = wrapper_read_time(index, ranges, rows);
time = wrapper_keyread_time(index, ranges, rows, blocks);
} else {
time = storage_read_time(index, ranges, rows);
time = storage_keyread_time(index, ranges, rows, blocks);
}
DBUG_RETURN(time);
}
#ifdef MRN_HANDLER_HAVE_KEYS_TO_USE_FOR_SCANNING
const key_map *ha_mroonga::wrapper_keys_to_use_for_scanning()
{

15
storage/mroonga/ha_mroonga.hpp
View File

@@ -531,8 +531,9 @@ public:
int end_bulk_insert() mrn_override;
int delete_all_rows() mrn_override;
int truncate() mrn_override;
double scan_time() mrn_override;
double read_time(uint index, uint ranges, ha_rows rows) mrn_override;
IO_AND_CPU_COST scan_time() mrn_override;
IO_AND_CPU_COST rnd_pos_time(ha_rows rows) mrn_override;
IO_AND_CPU_COST keyread_time(uint index, ulong ranges, ha_rows rows, ulonglong blocks) mrn_override;
#ifdef MRN_HANDLER_HAVE_KEYS_TO_USE_FOR_SCANNING
const key_map *keys_to_use_for_scanning() mrn_override;
#endif
@@ -1106,10 +1107,12 @@ private:
int wrapper_truncate_index();
int storage_truncate();
int storage_truncate_index();
double wrapper_scan_time();
double storage_scan_time();
double wrapper_read_time(uint index, uint ranges, ha_rows rows);
double storage_read_time(uint index, uint ranges, ha_rows rows);
IO_AND_CPU_COST wrapper_scan_time();
IO_AND_CPU_COST storage_scan_time();
IO_AND_CPU_COST wrapper_rnd_pos_time(ha_rows rows);
IO_AND_CPU_COST storage_rnd_pos_time(ha_rows rows);
IO_AND_CPU_COST wrapper_keyread_time(uint index, ulong ranges, ha_rows rows, ulonglong blocks);
IO_AND_CPU_COST storage_keyread_time(uint index, ulong ranges, ha_rows rows, ulonglong blocks);
#ifdef MRN_HANDLER_HAVE_KEYS_TO_USE_FOR_SCANNING
const key_map *wrapper_keys_to_use_for_scanning();
const key_map *storage_keys_to_use_for_scanning();

28
storage/myisam/ha_myisam.cc
View File

@@ -804,6 +804,17 @@ ulong ha_myisam::index_flags(uint inx, uint part, bool all_parts) const
return flags;
}
IO_AND_CPU_COST ha_myisam::rnd_pos_time(ha_rows rows)
{
IO_AND_CPU_COST cost= handler::rnd_pos_time(rows);
/*
Row data is not cached. costs.row_lookup_cost includes the cost of
the reading the row from system (probably cached by the OS).
*/
cost.io= 0;
return cost;
}
/* Name is here without an extension */
int ha_myisam::open(const char *name, int mode, uint test_if_locked)
@@ -2577,6 +2588,22 @@ static int myisam_drop_table(handlerton *hton, const char *path)
return mi_delete_table(path);
}
void myisam_update_optimizer_costs(OPTIMIZER_COSTS *costs)
{
/*
MyISAM row lookup costs are slow as the row data is not cached
The following numbers where found by check_costs.pl when using 1M rows
and all rows are cached. See optimizer_costs.txt
*/
costs->row_next_find_cost= 0.000063539;
costs->row_lookup_cost= 0.001014818;
costs->key_next_find_cost= 0.000090585;
costs->key_lookup_cost= 0.000550142;
costs->key_copy_cost= 0.000015685;
}
static int myisam_init(void *p)
{
handlerton *hton;
@@ -2596,6 +2623,7 @@ static int myisam_init(void *p)
hton->create= myisam_create_handler;
hton->drop_table= myisam_drop_table;
hton->panic= myisam_panic;
hton->update_optimizer_costs= myisam_update_optimizer_costs;
hton->flags= HTON_CAN_RECREATE | HTON_SUPPORT_LOG_TABLES;
hton->tablefile_extensions= ha_myisam_exts;
mi_killed= mi_killed_in_mariadb;

5
storage/myisam/ha_myisam.h
View File

@@ -82,14 +82,14 @@ class ha_myisam final : public handler
int index_first(uchar * buf);
int index_last(uchar * buf);
int index_next_same(uchar *buf, const uchar *key, uint keylen);
int ft_init()
int ft_init() override
{
if (!ft_handler)
return 1;
ft_handler->please->reinit_search(ft_handler);
return 0;
}
FT_INFO *ft_init_ext(uint flags, uint inx,String *key)
FT_INFO *ft_init_ext(uint flags, uint inx,String *key) override
{
return ft_init_search(flags,file,inx,
(uchar *)key->ptr(), key->length(), key->charset(),
@@ -102,6 +102,7 @@ class ha_myisam final : public handler
int remember_rnd_pos();
int restart_rnd_next(uchar *buf);
void position(const uchar *record);
IO_AND_CPU_COST rnd_pos_time(ha_rows rows) override;
int info(uint);
int extra(enum ha_extra_function operation);
int extra_opt(enum ha_extra_function operation, ulong cache_size);

34
storage/myisammrg/ha_myisammrg.cc
View File

@@ -339,6 +339,32 @@ static void myrg_set_external_ref(MYRG_INFO *m_info, void *ext_ref_arg)
}
}
IO_AND_CPU_COST ha_myisammrg::rnd_pos_time(ha_rows rows)
{
IO_AND_CPU_COST cost= handler::rnd_pos_time(rows);
/*
Row data is notcached. costs.row_lookup_cost includes the cost of
the reading the row from system (probably cached by the OS).
*/
cost.io= 0;
return cost;
}
IO_AND_CPU_COST ha_myisammrg::keyread_time(uint index, ulong ranges,
ha_rows rows,
ulonglong blocks)
{
IO_AND_CPU_COST cost= handler::keyread_time(index, ranges, rows, blocks);
if (!blocks)
{
cost.io*= file->tables;
cost.cpu*= file->tables;
}
/* Add the cost of having to do a key lookup in all trees */
cost.cpu+= (file->tables-1) * (ranges * KEY_LOOKUP_COST);
return cost;
}
/**
Open a MERGE parent table, but not its children.
@@ -1744,6 +1770,12 @@ int myisammrg_panic(handlerton *hton, ha_panic_function flag)
return myrg_panic(flag);
}
static void myisammrg_update_optimizer_costs(OPTIMIZER_COSTS *costs)
{
myisam_update_optimizer_costs(costs);
}
static int myisammrg_init(void *p)
{
handlerton *myisammrg_hton;
@@ -1759,7 +1791,7 @@ static int myisammrg_init(void *p)
myisammrg_hton->panic= myisammrg_panic;
myisammrg_hton->flags= HTON_NO_PARTITION;
myisammrg_hton->tablefile_extensions= ha_myisammrg_exts;
myisammrg_hton->update_optimizer_costs= myisammrg_update_optimizer_costs;
return 0;
}

14
storage/myisammrg/ha_myisammrg.h
View File

@@ -102,9 +102,17 @@ public:
uint max_supported_keys() const { return MI_MAX_KEY; }
uint max_supported_key_length() const { return HA_MAX_KEY_LENGTH; }
uint max_supported_key_part_length() const { return HA_MAX_KEY_LENGTH; }
double scan_time()
{ return ulonglong2double(stats.data_file_length) / IO_SIZE + file->tables; }
IO_AND_CPU_COST scan_time() override
{
IO_AND_CPU_COST cost;
cost.io= (ulonglong2double(stats.data_file_length) / IO_SIZE +
file->tables) * avg_io_cost();
cost.cpu= records() * ROW_NEXT_FIND_COST;
return cost;
}
IO_AND_CPU_COST rnd_pos_time(ha_rows rows) override;
IO_AND_CPU_COST keyread_time(uint index, ulong ranges, ha_rows rows,
ulonglong blocks) override;
int open(const char *name, int mode, uint test_if_locked);
int add_children_list(void);
int attach_children(void);

7
storage/oqgraph/ha_oqgraph.h
View File

@@ -74,9 +74,10 @@ public:
const char **bas_ext() const;
uint max_supported_keys() const { return MAX_KEY; }
uint max_supported_key_part_length() const { return MAX_KEY_LENGTH; }
double scan_time() { return (double) 1000000000; }
double read_time(uint index, uint ranges, ha_rows rows)
{ return 1; }
IO_AND_CPU_COST scan_time()
{ return { (double) 1000000000, (double) 1000000000 }; }
IO_AND_CPU_COST rnd_pos_time(ha_rows rows)
{ return { (double) rows, (double) rows }; }
// Doesn't make sense to change the engine on a virtual table.
virtual bool can_switch_engines() { return false; }

Some files were not shown because too many files have changed in this diff Show More