The main problem is that at ever since
commit aaef2e1d8c removed the
function buf_wait_for_read(), it is not safe to invoke
buf_page_get_low() with RW_NO_LATCH, that is, only buffer-fixing
the page. If a page read (or decryption or decompression) is in
progress, there would be a race condition when executing consistency
checks, and a page would wrongly be flagged as corrupted.
Furthermore, if the page is actually corrupted and the initial
access to it was with RW_NO_LATCH (only buffer-fixing), the
page read handler would likely end up in an infinite loop in
buf_pool_t::corrupted_evict(). It is not safe to invoke
mtr_t::upgrade_buffer_fix() on a block on which a page latch
was not initially acquired in buf_page_get_low().
btr_block_reget(): Remove the constant parameter rw_latch=RW_X_LATCH.
btr_block_get(): Assert that RW_NO_LATCH is not being used,
and change the parameter type of rw_latch.
btr_pcur_move_to_next_page(), innobase_table_is_empty(): Adjust for the
parameter type change of btr_block_get().
btr_root_block_get(): If mode==RW_NO_LATCH, do not check the integrity of
the page, because it is not safe to do so.
btr_page_alloc_low(), btr_page_free(): If the root page latch is not
previously held by the mini-transaction, invoke btr_root_block_get()
again with the proper latching mode.
btr_latch_prev(): Helper function to safely acquire a latch on a
preceding sibling page while holding a latch on a B-tree page.
To avoid deadlocks, we must not wait for the latch while holding
a latch on the current page, because another thread may be waiting
for our page latch when moving to the next page from our preceding
sibling page. If s_lock_try() or x_lock_try() on the preceding page fails,
we must release the current page latch, and wait for the latch on the
preceding page as well as the current page, in that order.
Page splits or merges will be prevented by the parent page latch
that we are holding.
btr_cur_t::search_leaf(): Make use of btr_latch_prev().
btr_cur_t::open_leaf(): Make use of btr_latch_prev(). Do not invoke
mtr_t::upgrade_buffer_fix() (when latch_mode == BTR_MODIFY_TREE),
because we will already have acquired all page latches upfront.
btr_cur_t::pessimistic_search_leaf(): Do acquire an exclusive index latch
before accessing the page. Make use of btr_latch_prev().
row_ins_sec_index_entry_low(): Correct a condition that was
inadvertently inverted
in commit 89ec4b53ac (MDEV-29603).
We are not supposed to buffer INSERT operations into unique indexes,
because duplicate key values would not be checked for. It is only
allowed when using unique_checks=0, and in that case the user is
supposed to guarantee that there are no duplicates.
For the sake of readable diffs this change is done separately from the
previous commit and it changes all man page headers to:
- Use correct syntax \fB and \fR to make titles bold instead of previous
capitalized version that had no effect
- Omit unnecessary \&. syntax, works fine without it
- Bump version to 10.11
Ideally the version would automatically be inherited from the
build/release, but for now at least update it manually to match what is
in the VERSION file of this release branch.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
Add \FR on lines that start with \FB as it is likely useful, but not
stricly necessary. However if there is just \ the line won't render at
all as happened in example below from 'man mysql-test-test.pl.1':
AUTHOR
MariaDB Foundation (http://www.mariadb.org/).
MYSQL-TEST-RUN()
->
AUTHOR
MariaDB Foundation (http://www.mariadb.org/).
MariaDB 10.11 15 May 2020 MYSQL-TEST-RUN(1)
Also use full command name instead of truncated ones.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
The man pages has several places where \' is intended to escape the single
quote and just print '. However, in reality this will print an ` (acute
accent). Fix this by removing excess backslashes. Also remove some excess
& in front of dots, such escaping is also unnecessary.
Also join several lines to avoid lone words in quotes to be read by groff
as macros when they are just text.
In a table (t{}) the content must be on new lines separately, so groff
will try to read them as macros too, so use \(aq that renders to '
(single quote).
This error was originally detected by Lintian:
https://lintian.debian.org/tags/acute-accent-in-manual-page
Example of man page sections before and after:
Make a backup of each table´s data file using the name tbl_name.OLD.
->
Make a backup of each table's data file using the name tbl_name.OLD.
A typical debug_options string is ´d:t:o,file_name´.
The default is ´d:t:o,/tmp/my_print_defaults.trace´.
->
A typical debug_options string is 'd:t:o,file_name'.
The default is 'd:t:o,/tmp/my_print_defaults.trace'.
shell> mysql --xml -uroot -e "SHOW VARIABLES LIKE ´version%´"
<?xml version="1.0"?>
<resultset statement="SHOW VARIABLES LIKE ´version%´" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
->
shell> mysql --xml -uroot -e "SHOW VARIABLES LIKE 'version%'"
<?xml version="1.0"?>
<resultset statement="SHOW VARIABLES LIKE 'version%'" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
This command was used to validate no man pages have syntax errors:
for x in *.?
do
echo "##### $x #####"
LC_ALL=C.UTF-8 MANROFFSEQ='' MANWIDTH=80 man --warnings -E UTF-8 -l -Tutf8 -Z $x >/dev/null
done
The man pages are not perfect after this, there are still a lot of
suboptimal syntax, but this helps towards better man pages.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
ANALYZE FORMAT=JSON output now includes table.r_engine_stats which
has the engine statistics. Only non-zero members are printed.
Internally: EXPLAIN data structures Explain_table_acccess and
Explain_update now have handler* handler_for_stats pointer.
It is used to read statistics from handler_for_stats->handler_stats.
The following applies only to 10.9+, backport doesn't use it:
Explain data structures exist after the tables are closed. We avoid
walking invalid pointers using this:
- SQL layer calls Explain_query::notify_tables_are_closed() before
closing tables.
- After that call, printing of JSON output is disabled. Non-JSON output
can be printed but we don't access handler_for_stats when doing that.
The DBUG_ASSER in HA_CREATE_INFO::resolve_to_charset_collation_context()
didn't take into account that the second execution is possible not only
during a prepared EXECUTE, but also during a CALL.
This issue could only be noticed in very extraordinary circumstances
when trying to rename a temporary table that is not in the file system.
The issue was found when I temporarly disabled check_if_frm_exists()
while searching for another bug.
Item_func_tochar::check_arguments() didn't check if its arguments
each had one column. Failing to make this check and proceeding would
eventually cause either an assertion failure or the execution would
reach "MY_ASSERT_UNREACHABLE();" which would produce a crash with
a misleading stack trace.
* Fixed Item_func_tochar::check_arguments() to do the required check.
* Also fixed MY_ASSERT_UNREACHABLE() to terminate the program. Just
"executing" __builtin_unreachable() used to cause "undefined results",
which in my experience was a crash with corrupted stack trace.
Currently Debian package: mariadb-client-compat and
mariadb-server-compat are missing Extended description
which leads to Lintian error:
* extended-description-is-empty
Commit orders Description to same as rest of
the Debian control file does.
Commit also fix Debian Lintian error:
* obsolete-relation-form-in-source
MariaDB Compression pluging pacakages mariadb-plugin-provider-*
have only one shared object and those are not linked against
libc and it's intentional so supressing error
library-not-linked-against-libc
There is needed dependency for Systemd DH plugin which
makes error missing-build-dependency-for-dh-addon obsolette.
Reworked debian/control that makes most of
version-substvar-for-external-package error not correct
so remove those that are not available anymore and update
those that are still relevant.
NOTE TO MERGER: This is only up-to MariaDB 10.10.
Lintian erros with
missing-build-dependency-for-dh-addon systemd (*)
One of these premises should be installed:
debhelper:any (>= 9.20160709~)
debhelper-compat:any
dh-sequence-systemd:any
dh-systemd:any
As there is package debhelper version 10 or higher required
for build. This is false positive Lintian error which should
be supressed..
NOTE TO MERGER: This is only up-to MariaDB 10.10.
Salsa-CI file has got in malfunction state and
if fails with YAML error:
mariadb-10.3 with Buster backports upgrade job:
undefined need: build buster-backports
Also remove Salsa-CI MariaDB 10.3 double
upgrade target.
There is also several upgrades which makes
test green again.
NOTE TO MERGER: This is only up-to MariaDB 10.10.
Remove
In debian/control file there is several mariadb-*-10.9 Conflicts
or Replaces that which are not neede on MariaDB 10.9 anymore
as there is not suffix anymore.
Package libmariadbclient-dev is part of Ubuntu and it's
not build with official package so there can't be
variable: '${source:Version}' which is used with packages
that are part of package.
NOTE TO MERGER: This is only up-to MariaDB 10.10.
Before MDEV-24671, the wait time was derived from my_interval_timer() /
1000 (nanoseconds converted to microseconds, and not microseconds to
milliseconds like I must have assumed). The lock_sys.wait_time and
lock_sys.wait_time_max are already in milliseconds; we should not divide
them by 1000.
In MDEV-24738 the millisecond counts lock_sys.wait_time and
lock_sys.wait_time_max were changed to a 32-bit type. That would
overflow in 49.7 days. Keep using a 64-bit type for those millisecond
counters.
Reviewed by: Marko Mäkelä
recv_log_recover_10_5(): Make reads aligned by 4096 bytes, to avoid
any trouble in case the file was opened in O_DIRECT mode and
the physical block size is larger than 512 bytes.
Because innodb_log_file_size used to be defined in whole megabytes,
reading multiples of 4096 bytes instead of 512 should not be an issue.
The new statistics is enabled by adding the "engine", "innodb" or "full"
option to --log-slow-verbosity
Example output:
# Pages_accessed: 184 Pages_read: 95 Pages_updated: 0 Old_rows_read: 1
# Pages_read_time: 17.0204 Engine_time: 248.1297
Page_read_time is time doing physical reads inside a storage engine.
(Writes cannot be tracked as these are usually done in the background).
Engine_time is the time spent inside the storage engine for the full
duration of the read/write/update calls. It uses the same code as
'analyze statement' for calculating the time spent.
The engine statistics is done with a generic interface that should be
easy for any engine to use. It can also easily be extended to provide
even more statistics.
Currently only InnoDB has counters for Pages_% and Undo_% status.
Engine_time works for all engines.
Implementation details:
class ha_handler_stats holds all engine stats. This class is included
in handler and THD classes.
While a query is running, all statistics is updated in the handler. In
close_thread_tables() the statistics is added to the THD.
handler::handler_stats is a pointer to where statistics should be
collected. This is set to point to handler::active_handler_stats if
stats are requested. If not, it is set to 0.
handler_stats has also an element, 'active' that is 1 if stats are
requested. This is to allow engines to avoid doing any 'if's while
updating the statistics.
Cloned or partition tables have the pointer set to the base table if
status are requested.
There is a small performance impact when using --log-slow-verbosity=engine:
- All engine calls in 'select' will be timed.
- IO calls for InnoDB reads will be timed.
- Incrementation of counters are done on local variables and accesses
are inline, so these should have very little impact.
- Statistics has to be reset for each statement for the THD and each
used handler. This is only 40 bytes, which should be neglectable.
- For partition tables we have to loop over all partitions to update
the handler_status as part of table_init(). Can be optimized in the
future to only do this is log-slow-verbosity changes. For this to work
we have to update handler_status for all opened partitions and
also for all partitions opened in the future.
Other things:
- Added options 'engine' and 'full' to log-slow-verbosity.
- Some of the new files in the test suite comes from Percona server, which
has similar status information.
- buf_page_optimistic_get(): Do not increment any counter, since we are
only validating a pointer, not performing any buf_pool.page_hash lookup.
- Added THD argument to save_explain_data_intern().
- Switched arguments for save_explain_.*_data() to have
always THD first (generates better code as other functions also have THD
first).
on aarch64 `char` by default is unsigned for performance reasons.
let's adjust checks to work for both signed and unsigned `char`
followup for ef84f8137b
optimizer implicitly assumed that if `a` in `a=b` is not NULL,
then it's safe to convert `a` to the type of `b` and search the
result in the index(b).
which is not always the case, as converting a non-null value
to a different type might produce NULL. And searching for NULL
in the index might find NULL there, so NULL will be equal to NULL,
making `a=b` behave as if it was `a<=>b`
* UUIDs version >= 6 are now stored without byte-swapping
* UUIDs with version >=8 and variant=0 are now considered invalid
* old tables are supported
* old (always byte swapped) and new (swapped for version < 6) UUIDs
can be compared and converted transparently
fil_node_open_file_low(): Always acquire an advisory lock on
the system tablespace. Originally, we already did this in
SysTablespace::open_file(), but SysTablespace::open_or_create()
would release those locks when it is closing the file handles.
This is a 10.5+ specific follow up to
commit 0ee1082bd2 (MDEV-28495).
Thanks to Daniel Black for verifying this bug.
dtype_new_read_for_order_and_null_size(): Correctly assign type->prtype.
This caused the fatal error and crash.
ibuf_merge(): Relax a too strict condition that would result in
[ERROR] InnoDB: Unable to upgrade the change buffer
when there exist buffered changes to redundant secondary indexes, such as
PRIMARY KEY(x), INDEX(x).
ibuf_upgrade(): Modify at most one user tablespace per mini-transaction,
to be crash-safe.
page_cur_insert_rec_zip(), page_cur_delete_rec(): Relax debug assertions
for ibuf_upgrade().
ibuf_log_rebuild_if_needed(): Invoke recv_sys.debug_free() only after
srv_log_rebuild_if_needed() to avoid an assertion failure. This code
is executed when the innodb_log_file_size is changed when upgrading
from 10.x to 11.0.
Tested by: Matthias Leich, Christian Hesse
When InnoDB attempts to buffer a change operation of a secondary index
leaf page (to insert, delete-mark or remove a record) and the
change buffer is too large, InnoDB used to trigger a change buffer merge
that could affect any tables. This could lead to huge variance in
system throughput and potentially unpredictable crashes, in case the
change buffer was corrupted and a crash occurred while attempting to
merge changes to a table that is not being accessed by the current
SQL statement.
ibuf_insert_low(): Simply return DB_STRONG_FAIL when the maximum size
of the change buffer is exceeded.
ibuf_contract_after_insert(): Remove.
ibuf_get_merge_page_nos_func(): Remove a constant parameter.
The function ibuf_contract() will be our only caller, during
shutdown with innodb_fast_shutdown=0.
debug-only issue. the test was doing
set debug_sync='now SIGNAL go3';
...
set debug_sync='reset';
which translated into
add "go3" to the hash of active signals
pthread_broadcast to wake up waiting threads
...
clear the hash of active signals
as a result a waiting thread was awoken, but the hash was emptied
before the thread checked if its signal was in the hash. so the
thread didn't find its signal and went back to sleep.
let's wait until the awoken thread has completely finished
disconnecting and was added to the thread cache.
debug_sync refactoring introduced a statically instantiated object
debug_sync_global of the structure st_debug_sync_globals.
st_debug_sync_globals includes Hash_set<> which allocates memory
in the constructor. sf_malloc() calls _my_thread_var()->dbug_id
which is pthread_getspecific(THR_KEY_mysys), and THR_KEY_mysys is 0
before pthread_key_create(). pthread_getspecific(0) returns a valid
pointer, not EINVAL. And safemalloc dereferences it.
let's statically initialize THR_KEY_mysys to -1, this makes
pthread_getspecific(THR_KEY_mysys) to fail before pthread_key_create()
is called.
followup for 8885225de6
