This patch is the result of running
run-clang-tidy -fix -header-filter=.* -checks='-*,modernize-use-equals-default' .
Code style changes have been done on top. The result of this change
leads to the following improvements:
1. Binary size reduction.
* For a -DBUILD_CONFIG=mysql_release build, the binary size is reduced by
~400kb.
* A raw -DCMAKE_BUILD_TYPE=Release reduces the binary size by ~1.4kb.
2. Compiler can better understand the intent of the code, thus it leads
to more optimization possibilities. Additionally it enabled detecting
unused variables that had an empty default constructor but not marked
so explicitly.
Particular change required following this patch in sql/opt_range.cc
result_keys, an unused template class Bitmap now correctly issues
unused variable warnings.
Setting Bitmap template class constructor to default allows the compiler
to identify that there are no side-effects when instantiating the class.
Previously the compiler could not issue the warning as it assumed Bitmap
class (being a template) would not be performing a NO-OP for its default
constructor. This prevented the "unused variable warning".
MDEV-25604 Atomic DDL: Binlog event written upon recovery does not
have default database
The purpose of this task is to ensure that ALTER TABLE is atomic even if
the MariaDB server would be killed at any point of the alter table.
This means that either the ALTER TABLE succeeds (including that triggers,
the status tables and the binary log are updated) or things should be
reverted to their original state.
If the server crashes before the new version is fully up to date and
commited, it will revert to the original table and remove all
temporary files and tables.
If the new version is commited, crash recovery will use the new version,
and update triggers, the status tables and the binary log.
The one execption is ALTER TABLE .. RENAME .. where no changes are done
to table definition. This one will work as RENAME and roll back unless
the whole statement completed, including updating the binary log (if
enabled).
Other changes:
- Added handlerton->check_version() function to allow the ddl recovery
code to check, in case of inplace alter table, if the table in the
storage engine is of the new or old version.
- Added handler->table_version() so that an engine can report the current
version of the table. This should be changed each time the table
definition changes.
- Added ha_signal_ddl_recovery_done() and
handlerton::signal_ddl_recovery_done() to inform all handlers when
ddl recovery has been done. (Needed by InnoDB).
- Added handlerton call inplace_alter_table_committed, to signal engine
that ddl_log has been closed for the alter table query.
- Added new handerton flag
HTON_REQUIRES_NOTIFY_TABLEDEF_CHANGED_AFTER_COMMIT to signal when we
should call hton->notify_tabledef_changed() during
mysql_inplace_alter_table. This was required as MyRocks and InnoDB
needed the call at different times.
- Added function server_uuid_value() to be able to generate a temporary
xid when ddl recovery writes the query to the binary log. This is
needed to be able to handle crashes during ddl log recovery.
- Moved freeing of the frm definition to end of mysql_alter_table() to
remove duplicate code and have a common exit strategy.
-------
InnoDB part of atomic ALTER TABLE
(Implemented by Marko Mäkelä)
innodb_check_version(): Compare the saved dict_table_t::def_trx_id
to determine whether an ALTER TABLE operation was committed.
We must correctly recover dict_table_t::def_trx_id for this to work.
Before purge removes any trace of DB_TRX_ID from system tables, it
will make an effort to load the user table into the cache, so that
the dict_table_t::def_trx_id can be recovered.
ha_innobase::table_version(): return garbage, or the trx_id that would
be used for committing an ALTER TABLE operation.
In InnoDB, table names starting with #sql-ib will remain special:
they will be dropped on startup. This may be revisited later in
MDEV-18518 when we implement proper undo logging and rollback
for creating or dropping multiple tables in a transaction.
Table names starting with #sql will retain some special meaning:
dict_table_t::parse_name() will not consider such names for
MDL acquisition, and dict_table_rename_in_cache() will treat such
names specially when handling FOREIGN KEY constraints.
Simplify InnoDB DROP INDEX.
Prevent purge wakeup
To ensure that dict_table_t::def_trx_id will be recovered correctly
in case the server is killed before ddl_log_complete(), we will block
the purge of any history in SYS_TABLES, SYS_INDEXES, SYS_COLUMNS
between ha_innobase::commit_inplace_alter_table(commit=true)
(purge_sys.stop_SYS()) and purge_sys.resume_SYS().
The completion callback purge_sys.resume_SYS() must be between
ddl_log_complete() and MDL release.
--------
MyRocks support for atomic ALTER TABLE
(Implemented by Sergui Petrunia)
Implement these SE API functions:
- ha_rocksdb::table_version()
- hton->check_version = rocksdb_check_versionMyRocks data dictionary
now stores table version for each table.
(Absence of table version record is interpreted as table_version=0,
that is, which means no upgrade changes are needed)
- For inplace alter table of a partitioned table, call the underlying
handlerton when checking if the table is ok. This assumes that the
partition engine commits all changes at once.
Insert worked incorrect as well. RocksDB used table->record[0] internally to store some
intermediate results for key conversion, during index searching among other operations.
So table->record[0] is spoiled during ha_rnd_index_map in ha_check_overlaps, so in turn
the broken record data was inserted.
The fix is to store RocksDB intermediate result in its own buffer instead of table->record[0].
`rocksdb` MTR suite is is checked and runs fine.
No need for additional tests. The existing overlaps.test covers the case completely.
However, I am not going to add anything related to rocksdb to suite, to keep it away
from additional dependencies.
To run tests with RocksDB engine, one can add following to engines.combinations:
[rocksdb]
plugin-load=$HA_ROCKSDB_SO
default-storage-engine=rocksdb
rocksdb
- Add SLEEP() calls to the testcase to make it really test that the
time doesn't change.
- Always use .frm file creation time as a creation timestamp (attempts
to re-use older create_time when the table DDL changes are not good
because then create_time will change after server restart)
- Use the same method names as the upstream patch does
- Use std::atomic for m_update_time
Copy of
commit dcd9379eb5707bc7514a2ff4d9127790356505cb
Author: Manuel Ung <mung@fb.com>
Date: Fri Jun 14 10:38:17 2019 -0700
Skip valgrind for rocksdb.force_shutdown
Summary:
This test does unclean shutdown, and leaks memory.
Squash with: D15749084
Reviewed By: hermanlee
Differential Revision: D15828957
fbshipit-source-id: 30541455d74
main.derived_cond_pushdown: Move all 10.3 tests to the end,
trim trailing white space, and add an "End of 10.3 tests" marker.
Add --sorted_result to tests where the ordering is not deterministic.
main.win_percentile: Add --sorted_result to tests where the
ordering is no longer deterministic.
Move up-to this revision in the upstream:
commit de1e8c7bfe7c875ea284b55040e8f3cd3a56fcc2
Author: Abhinav Sharma <abhinavsharma@fb.com>
Date: Thu Aug 23 14:34:39 2018 -0700
Log updates to semi-sync whitelist in the error log
Summary:
Plugin variable changes are not logged in the error log even when
log_global_var_changes is enabled. Logging updates to whitelist will help in
debugging.
Reviewed By: guokeno0
Differential Revision: D9483807
fbshipit-source-id: e111cda773d
commit de1e8c7bfe7c875ea284b55040e8f3cd3a56fcc2
Author: Abhinav Sharma <abhinavsharma@fb.com>
Date: Thu Aug 23 14:34:39 2018 -0700
Log updates to semi-sync whitelist in the error log
Summary:
Plugin variable changes are not logged in the error log even when
log_global_var_changes is enabled. Logging updates to whitelist will help in
debugging.
Reviewed By: guokeno0
Differential Revision: D9483807
fbshipit-source-id: e111cda773d
In reverse-ordered column families, if one wants to start reading at the
logical end of the index, they should Seek() to a key value that is not
covered by the index. This may (and typically does) prevent use of a bloom
filter.
The calls to setup_scan_iterator() that are made for index and table scan
didn't take this into account and passed eq_cond_len=INDEX_NUMBER_SIZE.
Fixed them to compute and pass correct eq_cond_len.
Also, removed an incorrect assert in ha_rocksdb::setup_iterator_bounds.
commit 445e518bc7
Author: Sergei Petrunia <psergey@askmonty.org>
Date: Sat Jan 27 10:18:20 2018 +0000
Copy of
commit f8f364b47f2784f16b401f27658f1c16eaf348ec
Author: Jay Edgar <jkedgar@fb.com>
Date: Tue Oct 17 15:19:31 2017 -0700
Add a hashed, hierarchical, wheel timer implementation
Summary:
In order to implement idle timeouts on detached sessions we need something inside MySQL that is lightweight and can handle calling events in the future wi
By default the timers are grouped into 10ms buckets (the 'hashed' part), though the size of the buckets is configurable at the creation of the timer. Eac
Reviewed By: djwatson
Differential Revision: D6199806
fbshipit-source-id: 5e1590f
commit f8f364b47f2784f16b401f27658f1c16eaf348ec
Author: Jay Edgar <jkedgar@fb.com>
Date: Tue Oct 17 15:19:31 2017 -0700
Add a hashed, hierarchical, wheel timer implementation
Summary:
In order to implement idle timeouts on detached sessions we need something inside MySQL that is lightweight and can handle calling events in the future with very little cost for cancelling or resetting the event. A hashed, hi
By default the timers are grouped into 10ms buckets (the 'hashed' part), though the size of the buckets is configurable at the creation of the timer. Each wheel (the 'wheel' part) maintains 256 buckets and cascades to the whe
Reviewed By: djwatson
Differential Revision: D6199806
fbshipit-source-id: 5e1590f
- Make Rdb_binlog_manager::unpack_value to not have a stack overrun
when it is reading invalid data (which it currently does as we in
MariaDB do not store binlog coordinates under BINLOG_INFO_INDEX_NUMBER,
see comments in MDEV-14892 for details).
- We may need to store these coordinates in the future, so instead of
removing the call of this function, let's make it work properly for
all possible inputs.
The error
"Unsupported collation on string indexed column %s Use
binary collation (latin1_bin, binary, utf8_bin)."
is misleading. Change it:
- It is now a warning
- It is printed only for collations that do not support index-only access
(reversible collations that use unpack_info are ok)
- The new warning text is:
Indexed column %s.%s uses a collation that does not allow index-only
access in secondary key and has reduced disk space efficiency
in primary key.
Upstream cset we are merging from:
commit 184a4a2d82f4f6f3cbcb1015bcdb32bebe73315c
Author: Abhinav Sharma <abhinavsharma@fb.com>
Date: Thu Sep 14 11:40:08 2017 -0700
Bump rocksdb submodule
Summary:
Bump rocksdb to include the fix for rocksdb.trx_info_rpl
Lots of conflicts, got the code to compile but tests are likely to
be broken
This allows basic master crash-safety
- Un-comment and update relevant parts of the code
- Make rocksdb_rpl suite work like other MyRocks testsuites
(load the MyRocks plugin, don't start if it is not compiled in, etc)
- For now, disable all tests in the rocksdb_rpl suite.
- MariaDB-fication of rpl_rocksdb_2p_crash_recover test.
If compiling a non DBUG binary with
-DDBUG_ASSERT_AS_PRINTF asserts will be
changed to printf + stack trace (of stack
trace are enabled).
- Changed #ifndef DBUG_OFF to
#ifdef DBUG_ASSERT_EXISTS
for those DBUG_OFF that was just used to enable
assert
- Assert checking that could greatly impact
performance where changed to DBUG_ASSERT_SLOW which
is not affected by DBUG_ASSERT_AS_PRINTF
- Added one extra option to my_print_stacktrace() to
get more silent in case of stack trace printing as
part of assert.
commit 394d0712d3d46a87a8063e14e998e9c22336e3a6
Author: Anca Agape <anca@fb.com>
Date: Thu Jul 27 15:43:07 2017 -0700
Fix rpl.rpl_4threads_deadlock test broken by D5005670
Summary:
In D5005670 in fill_fields_processlist() function we introduced a point
where we were trying to take the LOCK_thd_data before the
synchronization point used by test
processlist_after_LOCK_thd_count_before_LOCK_thd_data. This was
happening in get_attached_srv_session() function called. Replaced this
with get_attached_srv_session_safe() and moved it after lock is aquired.
Reviewed By: tianx
Differential Revision: D5505992
fbshipit-source-id: bc53924
commit ba00e640f658ad8d0a4dff09a497a51b8a4de935
Author: Herman Lee <herman@fb.com>
Date: Wed Feb 22 06:30:06 2017 -0800
Improve add_index_alter_cardinality test
Summary:
Split add_index_inplace_cardinality test out and add a debug_sync point
to it so that the flush of the memtable occurs while the alter is
running.
Closes https://github.com/facebook/mysql-5.6/pull/539
Reviewed By: alxyang
Differential Revision: D4597887
Pulled By: hermanlee
fbshipit-source-id: faedda2
commit d1bb19b8f751875472211312c8e810143a7ba4b6
Author: Manuel Ung <mung@fb.com>
Date: Fri Feb 3 11:50:34 2017 -0800
Add cardinality stats to information schema
Summary: This adds cardinality stats to the INFORMATION_SCHEMA.ROCKSDB_INDEX_FILE_MAP table. This is the only missing user collected properties from SST files that we don't expose, which is useful for debugging cardinality bugs.
Reviewed By: hermanlee
Differential Revision: D4509156
fbshipit-source-id: 2d3918a
copy of
commit 86587affafe77ef555f7c3839839de44f0f203f3
Author: Tian Xia <tianx@fb.com>
Date: Tue Oct 4 10:01:52 2016 -0700
Allow filtering of show commands through admission control
