This patch contains a full implementation of the optimization
that allows to use in-memory rowid / primary filters built for range
conditions over indexes. In many cases usage of such filters reduce
the number of disk seeks spent for fetching table rows.
In this implementation the choice of what possible filter to be applied
(if any) is made purely on cost-based considerations.
This implementation re-achitectured the partial implementation of
the feature pushed by Galina Shalygina in the commit
8d5a11122c.
Besides this patch contains a better implementation of the generic
handler function handler::multi_range_read_info_const() that
takes into account gaps between ranges when calculating the cost of
range index scans. It also contains some corrections of the
implementation of the handler function records_in_range() for MyISAM.
This patch supports the feature for InnoDB and MyISAM.
Part of MDEV-5336 Implement LOCK FOR BACKUP
The idea is that instead of waiting in close_cached_tables() for all
tables to be closed, we instead call flush_tables() that does:
- Flush not used objects in table cache to free memory
- Collect all tables that are open
- Call HA_EXTRA_FLUSH on the objects, to get them into "closed state"
- Added HA_EXTRA_FLUSH support to archive and CSV
- Added multi-user protection to HA_EXTRA_FLUSH in MyISAM and Aria
The benefit compared to old code is:
- FTWRL doesn't have to wait for long running read operations or
open HANDLER's
Modern compilers (such as GCC 8) emit warnings that the
'register' keyword is deprecated and not valid C++17.
Let us remove most use of the 'register' keyword.
Code in 'extra/' is not touched.
Handle string length as size_t, consistently (almost always:))
Change function prototypes to accept size_t, where in the past
ulong or uint were used. change local/member variables to size_t
when appropriate.
This fix excludes rocksdb, spider,spider, sphinx and connect for now.
- Added sql/mariadb.h file that should be included first by files in sql
directory, if sql_plugin.h is not used (sql_plugin.h adds SHOW variables
that must be done before my_global.h is included)
- Removed a lot of include my_global.h from include files
- Removed include's of some files that my_global.h automatically includes
- Removed duplicated include's of my_sys.h
- Replaced include my_config.h with my_global.h
This was done to make it clear that a update_row() should not change the
row.
This was not done for handler::write_row() as this function still needs
to update auto_increment values in the row. This should at some point
be moved to handler::ha_write_row() after which write_row can also have
const arguments.
Also, implement MDEV-11027 a little differently from 5.5 and 10.0:
recv_apply_hashed_log_recs(): Change the return type back to void
(DB_SUCCESS was always returned).
Report progress also via systemd using sd_notifyf().
Define my_thread_id as an unsigned type, to avoid mismatch with
ulonglong. Change some parameters to this type.
Use size_t in a few more places.
Declare many flag constants as unsigned to avoid sign mismatch
when shifting bits or applying the unary ~ operator.
When applying the unary ~ operator to enum constants, explictly
cast the result to an unsigned type, because enum constants can
be treated as signed.
In InnoDB, change the source code line number parameters from
ulint to unsigned type. Also, make some InnoDB functions return
a narrower type (unsigned or uint32_t instead of ulint;
bool instead of ibool).
* don't issue an error for ER_KEY_BASED_ON_GENERATED_VIRTUAL_COLUMN
* support keyread on vcols
* callback into the server to compute vcol values from mi_check/mi_repair
* DMLs just work. Automatically.
- If one specifies --force twice to myisamchk and aria_chk, then we will try to finnish the repair even if sort_buffer would be too small.
This was done by dynamically allocate buffer handler objects as long as memory lasts.
- New option for myisamchk and aria_chk: create-missing-keys
- Changed default size of myisam_sort_buffer_size from 8M to 128M.
- Changed default size of sort_buffer_size in aria_chk from 128M to 256M.
- Increased information in error message about 'sort_buffer_size' beeing to small.
- Print also to 'show warnings' if repair was retried.
- Increased size of internal sort-buffer-readers from 16K to 128K
- Changed printing of 'number of records' to use %ll instead of casting to long
- Changed buffer sizes for myisam and aria to use MY_ALIGN_DOWN() to get same number of bytes allocated on different machines.
include/my_global.h:
Added MY_ALIGN_DOWN() to get previous alignment (for big memory areas)
include/myisam.h:
Increased size of types to be able to handle more records
include/myisamchk.h:
Increased size of types to be able to handle more records
Added T_FORCE_SORT_MEMORY to force repair to work even if sort_buffer would not be big enough
mysql-test/r/myisam.result:
Updated result
mysql-test/r/mysqld--help.result:
Updated result
mysql-test/r/repair.result:
Updated result
mysql-test/suite/maria/maria.result:
Added test cases for some fixed bugs in MyISAM to verify that Aria doesn't have them too.
mysql-test/suite/maria/maria.test:
Added test cases for some fixed bugs in MyISAM to verify that Aria doesn't have them too.
mysql-test/suite/maria/maria3.result:
Updated result after sort buffer size increase
mysql-test/suite/sys_vars/r/aria_sort_buffer_size_basic.result:
Updated result after sort buffer size increase
mysql-test/suite/sys_vars/r/myisam_sort_buffer_size_basic.result:
Updated result after sort buffer size increase
mysql-test/suite/sys_vars/t/myisam_sort_buffer_size_basic.test:
Updated result after sort buffer size increase
mysql-test/t/myisam.test:
Fixed error messages to not print system specific data
mysql-test/t/repair.test:
Fixed error messages to not print system specific data
storage/maria/ha_maria.cc:
Print also to 'show warnings' if repair was retried
Changed default size of sort_buffer_size from 128M to 256M (same as in mysqld)
storage/maria/ma_check.c:
Renamed USE_BUFFER_INIT -> PAGE_BUFFER_INIT
storage/maria/ma_sort.c:
Increased size of internal sort-buffer-readers from 16K to 128K
Increased size of types to be able to handle more records
Added support for T_FORCE_SORT_MEMORY
Don't allocate too many extra BUFFPEK at a time (they are probably not needed)
Improved error message for "sort_buffer_size is too small"
Changed printing of 'number of records' to use %ll instead of casting to long
Fixed bug where maria_update_key_parts() was called too early.
Fixed bug in detecting result from read_to_buffer().
Added 'out of memory' checking when calling 'alloc_dynamic()'.
storage/maria/maria_chk.c:
Added --create-missing-keys
If one specifies --force twice then we will try to finnish the repair even if sort_buffer would be too small.
check_param.sort_buffer_length varialble was used with wrong type.
storage/maria/maria_def.h:
Increased size of types to be able to handle more records
Use MY_ALIGN_DOWN() to get same number of bytes allocated on different machines
Renamed USE_BUFFER_INIT -> PAGE_BUFFER_INIT
storage/maria/maria_ftdump.c:
Renamed USE_BUFFER_INIT -> PAGE_BUFFER_INIT
storage/maria/maria_read_log.c:
Use PAGE_BUFFER_INIT for page cache
storage/myisam/ha_myisam.cc:
Changed default size of myisam_sort_buffer_size from 8M to 128M
storage/myisam/mi_check.c:
Renamed USE_BUFFER_INIT -> KEY_BUFFER_INIT
storage/myisam/myisam_ftdump.c:
Renamed USE_BUFFER_INIT -> KEY_BUFFER_INIT
storage/myisam/myisamchk.c:
Added --create-missing-keys
If one specifies --force twice then we will try to finnish the repair even if sort_buffer would be too small.
check_param.sort_buffer_length varialble was used with wrong type.
Renamed USE_BUFFER_INIT -> KEY_BUFFER_INIT
storage/myisam/myisamdef.h:
Increased SORT_BUFFER_INIT to 64M
(speeds up repair a lot and most machines have nowadays a lot of memory)
Use MY_ALIGN_DOWN() to get same number of bytes allocated on different machines
Renamed USE_BUFFER_INIT -> KEY_BUFFER_INIT
storage/myisam/sort.c:
Increased size of internal sort-buffer-readers from 16K to 128K
Increased size of types to be able to handle more records
Added support for T_FORCE_SORT_MEMORY
Don't allocate too many extra BUFFPEK at a time (they are probably not needed)
Improved error message for "sort_buffer_size is too small"
Changed printing of 'number of records' to use %ll instead of casting to long
Fixed bug in detecting result from read_to_buffer().
Added 'out of memory' checking when calling 'alloc_dynamic()'.
The patch lifts the limitation of the current implementation
of ALTER TABLE that does not allow to build unique/primary
indexes by sort for MyISAM and Aria engines.
mysql-test/suite/innodb/t/group_commit_crash.test:
remove autoincrement to avoid rbr being used for insert ... select
mysql-test/suite/innodb/t/group_commit_crash_no_optimize_thread.test:
remove autoincrement to avoid rbr being used for insert ... select
mysys/my_addr_resolve.c:
a pointer to a buffer is returned to the caller -> the buffer cannot be on the stack
mysys/stacktrace.c:
my_vsnprintf() is ok here, in 5.5
ON 64 BIT MACHINES
PROBLEM: When sorting index during repair of
myisam tables, due to improper casting
of buffer size variables value of myisam_
sort_buffer_size is not set greater than
4GB.
SOLUTION: Proper casting of buffer size variable.
myisam_buffer_size changed to unsigned
long long to handle size > 4GB on
linux as well as windows.
sql/sql_insert.cc:
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
******
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
small cleanup
******
small cleanup
