1
0
mirror of https://github.com/MariaDB/server.git synced 2025-12-01 17:39:21 +03:00
Commit Graph

261 Commits

Author SHA1 Message Date
Marko Mäkelä
e32212c63c MDEV-19582 Out-of-bounds memory accesses by WolfSSL
Fix errors caught by clang-7 in the encrypted variants of the tests
innodb.innodb-table-online innodb.innodb-index-online
2019-05-27 10:16:23 +03:00
Oleksandr Byelkin
c07325f932 Merge branch '10.3' into 10.4 2019-05-19 20:55:37 +02:00
Marko Mäkelä
198ed24cac MDEV-19513: Rename dict_operation_lock to dict_sys.latch
dict_sys.lock(), dict_sys_lock(): Acquire both mutex and latch.

dict_sys.unlock(), dict_sys_unlock(): Release both mutex and latch.

dict_sys.assert_locked(): Assert that both mutex and latch are held.
2019-05-17 15:26:33 +03:00
Marko Mäkelä
5fd7502e77 MDEV-19513: Allocate dict_sys statically
dict_sys_t::create(): Renamed from dict_init().

dict_sys_t::close(): Renamed from dict_close().

dict_sys_t::add(): Sliced from dict_table_t::add_to_cache().

dict_sys_t::remove(): Renamed from dict_table_remove_from_cache().

dict_sys_t::prevent_eviction(): Renamed from
dict_table_move_from_lru_to_non_lru().

dict_sys_t::acquire(): Replaces dict_move_to_mru() and some more logic.

dict_sys_t::resize(): Renamed from dict_resize().

dict_sys_t::find(): Replaces dict_lru_find_table() and
dict_non_lru_find_table().
2019-05-17 14:32:53 +03:00
Marko Mäkelä
874f8f30f2 Merge 10.2 into 10.3 2019-05-14 17:25:25 +03:00
Marko Mäkelä
be85d3e61b Merge 10.2 into 10.3 2019-05-14 17:18:46 +03:00
Marko Mäkelä
b93ecea65c Remove unnecessary pointer indirection for rw_lock_t
In MySQL 5.7.8 an extra level of pointer indirection was added to
dict_operation_lock and some other rw_lock_t without solid justification,
in mysql/mysql-server@52720f1772.

Let us revert that change and remove the rather useless rw_lock_t
constructor and destructor and the magic_n field. In this way,
some unnecessary pointer dereferences and heap allocation will be avoided
and debugging might be a little easier.
2019-05-13 18:46:12 +03:00
Marko Mäkelä
26a14ee130 Merge 10.1 into 10.2 2019-05-13 17:54:04 +03:00
Oleksandr Byelkin
c51f85f882 Merge branch '10.2' into 10.3 2019-05-12 17:20:23 +02:00
Vicențiu Ciorbaru
c0ac0b8860 Update FSF address 2019-05-11 19:25:02 +03:00
Marko Mäkelä
b2f3755c8e Merge 10.1 into 10.2 2019-05-10 08:02:21 +03:00
Marko Mäkelä
f92749ed36 MDEV-18220: heap-use-after-free in fts_get_table_name_prefix()
fts_table_t::parent: Remove the redundant field. Refer to
table->name.m_name instead.

fts_update_sync_doc_id(), fts_update_next_doc_id(): Remove
the redundant parameter table_name.

fts_get_table_name_prefix(): Access the dict_table_t::name.
FIXME: Ensure that this access is always covered by
dict_sys->mutex.
2019-05-10 07:57:01 +03:00
Marko Mäkelä
d3dcec5d65 Merge 10.3 into 10.4 2019-05-05 15:06:44 +03:00
Marko Mäkelä
b132b8895e Merge 10.3 into 10.4 2019-05-05 10:23:14 +03:00
Marko Mäkelä
b6f4cccd19 Merge 10.2 into 10.3 2019-05-03 20:14:09 +03:00
Marko Mäkelä
ce195987c3 MDEV-19385: Inconsistent definition of dtuple_get_nth_v_field()
The accessor dtuple_get_nth_v_field() was defined differently between
debug and release builds in MySQL 5.7.8 in
mysql/mysql-server@c47e1751b7
and a debug assertion to document or enforce the questionable assumption
tuple->v_fields == &tuple->fields[tuple->n_fields] was missing.

This was apparently no problem until MDEV-11369 introduced instant
ADD COLUMN to MariaDB Server 10.3. With that work present, in one
test case, trx_undo_report_insert_virtual() could in release builds
fetch the wrong value for a virtual column.

We replace many of the dtuple_t accessors with const-preserving
inline functions, and fix missing or misleadingly applied const
qualifiers accordingly.
2019-05-03 20:02:50 +03:00
Marko Mäkelä
4d59f45260 Merge 10.2 into 10.3 2019-04-27 20:41:31 +03:00
Marko Mäkelä
e7029e864f Merge 10.3 into 10.4 2019-04-17 15:59:30 +03:00
Marko Mäkelä
250799f961 Merge 10.2 into 10.3 2019-04-17 15:26:17 +03:00
Marko Mäkelä
376bf4ede5 MDEV-19241 InnoDB fails to write MLOG_INDEX_LOAD upon completing ALTER TABLE
Similar to what was done in commit aa3f7a107c
for FULLTEXT INDEX, we must ensure that MLOG_INDEX_LOAD records will always
be written if redo logging was disabled.

row_merge_build_indexes(): Invoke row_merge_write_redo() also when
online operation is not being executed or an error occurs.
In case of an error, invoke flush_observer->interrupted() so that
the pages will not be flushed but merely evicted from the buffer pool.
Before resuming redo logging, it is crucial for the correctness of
mariabackup and InnoDB crash recovery to flush or evict all affected pages
and to write MLOG_INDEX_LOAD records.
2019-04-17 13:58:22 +03:00
Marko Mäkelä
d8303c3ee7 Merge 10.3 into 10.4 2019-04-08 08:22:34 +03:00
Marko Mäkelä
cc492bfd4f Merge 10.2 into 10.3 2019-04-07 11:49:50 +03:00
Marko Mäkelä
aa3f7a107c MDEV-12699 preparation: Write MLOG_INDEX_LOAD for FTS_ tables
The record MLOG_INDEX_LOAD is supposed to be written to indicate that
some page modifications bypassed redo logging, and that redo logging
is now re-enabled. It was not written for fulltext indexes during
ALTER TABLE.

row_merge_write_redo(): Declare globally. Assert that the index
is neither a spatial nor fulltext index.

recv_mlog_index_load(): Observe a MLOG_INDEX_LOAD operation.

recv_parse_log_recs(): Handle MLOG_INDEX_LOAD also in multi-record
mini-transactions. Because of this omission, we should keep writing
MLOG_INDEX_LOAD in single-record mini-transactions, because older
versions of Mariabackup would fail.

row_fts_merge_insert(): Write MLOG_INDEX_LOAD for the auxiliary
tables of fulltext indexes.
2019-04-06 21:25:43 +03:00
Marko Mäkelä
c8f8d5ceb7 Merge 10.3 into 10.4 2019-04-03 11:43:39 +03:00
Marko Mäkelä
c6b8b05be4 Merge 10.2 into 10.3 2019-04-03 11:22:51 +03:00
Marko Mäkelä
dbc716675b Merge 10.1 into 10.2 2019-04-03 10:32:21 +03:00
Marko Mäkelä
c0fca2863b Fix -Wnonnull-compare
InnoDB and XtraDB had redundant assertions for checking that
function parameters that were declared as nonnull were not NULL.
2019-04-03 09:46:49 +03:00
Marko Mäkelä
5c3ff5cb93 Merge 10.3 into 10.4 2019-04-02 11:04:54 +03:00
Sergei Golubchik
4e1d3f83b7 Merge branch '10.2' into 10.3 2019-03-29 19:41:41 +01:00
Thirunarayanan Balathandayuthapani
0623cc7c16 MDEV-19051 Avoid unnecessary writing MLOG_INDEX_LOAD
1) Avoid writing of MLOG_INDEX_LOAD redo log record during inplace
alter table when the table is empty and also for spatial index.

2) Avoid creation of temporary merge file for spatial index during
index creation process.
2019-03-28 10:55:18 +02:00
Marko Mäkelä
7af0de9fa3 Merge 10.3 into 10.4 2019-03-21 11:22:13 +02:00
Marko Mäkelä
5a780f2ec8 Merge 10.2 into 10.3 2019-03-21 10:14:23 +02:00
Marko Mäkelä
185a5000e6 Silence bogus -Wmaybe-uninitialized 2019-03-21 09:24:03 +02:00
Marko Mäkelä
b4cda8bbbc After-merge fix for GCC
GCC does not like MY_ATTRIBUTE((nonnull)) on a reference-to-pointer
parameter. clang did not flag an issue wit that.
2019-03-07 18:54:53 +02:00
Marko Mäkelä
913e33e423 Merge 10.1 into 10.2
Rewrite the MDEV-13818 fix to prevent heap-use-after-free.

Add a test case for MDEV-18272.
2019-03-07 17:52:27 +02:00
Marko Mäkelä
e3adf96aeb MDEV-13818 CREATE INDEX leaks memory if running out of undo log space
row_merge_create_index_graph(): Relay the internal state
from dict_create_index_step(). Our caller should free the index
only if it was not copied, added to the cache, and freed.

row_merge_create_index(): Free the index template if it was
not added to the cache. This is a safer variant of the logic
that was introduced in 65070beffd in 10.2.

prepare_inplace_alter_table_dict(): Add additional fault injection
to exercise a code path where we have already added an index
to the cache.
2019-03-07 15:35:55 +02:00
Sergei Golubchik
65070beffd MDEV-13818 CREATE INDEX leaks memory if running out of undo log space
free already allocated indexes if row_merge_create_index() fails

This fixes innodb.alter_crash failure
in ASAN_OPTIONS="abort_on_error=1" runs
2019-03-06 15:28:27 +01:00
Marko Mäkelä
2c74799d64 MDEV-18627 Wrong result after instant size change of integer
If we instantly change the size of a fixed-length field
and treat it as kind-of variable-length, then we will need
conversions between old column values and new ones.

I tried adding such a conversion to row_build(), but then I
noticed that more conversions would be needed, because
old values still appeared in a freshly rebuilt secondary index,
causing a mismatch when trying to search with the correct
longer value that was converted in my provisional fix to row_build().

So, we will revert the essential part of
MDEV-15563: Instant ROW_FORMAT=REDUNDANT column extension
(commit 22feb179ae), but not
remove any tests.
2019-02-18 18:30:01 +02:00
Marko Mäkelä
22feb179ae MDEV-15563: Instant ROW_FORMAT=REDUNDANT column extension
This was developed by Aleksey Midenkov based on my design.

In the original InnoDB storage format (that was retroactively named
ROW_FORMAT=REDUNDANT in MySQL 5.0.3), the length of each index field
is stored explicitly.

Because of this, we can and now will allow instant conversion from
VARCHAR to CHAR or VARBINARY to BINARY of equal or greater size,
as well as instant conversion of TINYINT to SMALLINT to MEDIUMINT
to INT to BIGINT (while not changing between signed and unsigned).

Theoretically, we could allow changing from an unsigned integer to
a bigger unsigned integer, as well as changing CHAR to VARCHAR, but
that would require additional metadata and conversions whenever
reading old records.

Field_str::is_equal(), Field_varstring::is_equal(), Field_num::is_equal():
Return the new result IS_EQUAL_PACK_LENGTH_EXT if the table advertises
HA_EXTENDED_TYPES_CONVERSION capability and we are considering the
above-mentioned conversions.

ALTER_COLUMN_EQUAL_PACK_LENGTH_EXT: A new ALTER TABLE flag, similar
to ALTER_COLUMN_EQUAL_PACK_LENGTH but requiring conversions when
reading the data. The Field::is_equal() result IS_EQUAL_PACK_LENGTH_EXT
will map to this flag.

dtype_get_fixed_size_low(): For BINARY, CHAR and integer columns
in ROW_FORMAT=REDUNDANT, return 0 (variable length) from now on.

dtype_get_sql_null_size(): Keep returning the current size for
BINARY, CHAR and integer columns, so that in ROW_FORMAT=REDUNDANT
it will remain possible to update in place between NULL and NOT NULL
values.

btr_index_rec_validate(): Relax a CHECK TABLE length check for
ROW_FORMAT=REDUNDANT tables.

btr_cur_instant_init_low(): No longer trust fixed_len
for ROW_FORMAT=REDUNDANT tables.

We cannot rely on fixed_len anymore because the record can have shorter
length from before instant extension. Note that importing such tablespace
into earlier MariaDB versions produces ER_TABLE_SCHEMA_MISMATCH when
using a .cfg file.
2019-02-13 17:39:05 +02:00
Marko Mäkelä
0a1c3477bf MDEV-18493 Remove page_size_t
MySQL 5.7 introduced the class page_size_t and increased the size of
buffer pool page descriptors by introducing this object to them.

Maybe the intention of this exercise was to prepare for a future
where the buffer pool could accommodate multiple page sizes.
But that future never arrived, not even in MySQL 8.0. It is much
easier to manage a pool of a single page size, and typically all
storage devices of an InnoDB instance benefit from using the same
page size.

Let us remove page_size_t from MariaDB Server. This will make it
easier to remove support for ROW_FORMAT=COMPRESSED (or make it a
compile-time option) in the future, just by removing various
occurrences of zip_size.
2019-02-07 12:21:35 +02:00
Sergey Vojtovich
e60dc209d4 MDEV-17441 - InnoDB transition to C++11 atomics
Almost trivial rw_lock_t::waiters transition. Since C++11 doesn't
seem to allow mixed (atomic and non-atomic) access to atomic variables,
we have to perform atomic initialisation.
2018-12-27 22:46:38 +04:00
Marko Mäkelä
b5763ecd01 Merge 10.3 into 10.4 2018-12-18 11:33:53 +02:00
Marko Mäkelä
45531949ae Merge 10.2 into 10.3 2018-12-18 09:15:41 +02:00
Marko Mäkelä
10e01b56f7 Fix USE_AFTER_FREE (CWE-416)
A static analysis tool suggested that in the function
row_merge_read_clustered_index(), ut_free(nonnull) could
be invoked twice for nonnull!=NULL. While a manual review
of the code disproved this, it should not hurt to clean up
the code so that the static analysis tool will not complain.

index_tuple_info_t::insert(), row_mtuple_cmp(): Remove the
parameter mtr_committed, which duplicated !mtr->is_active().

row_merge_read_clustered_index(): Initialize row_heap = NULL.
Remove a duplicated call mem_heap_empty(row_heap) that was
inadvertently added in commit cb1e76e4de.

Replace a "goto func_exit" with "break", to get consistent error
handling for both failures to create or write a temporary file.

end_of_index: Assign row_heap=NULL and nonnull=NULL to prevent
double freeing.

func_exit: Check for row_heap!=NULL before invoking mem_heap_free().

Closes #959
2018-12-17 17:10:42 +02:00
Marko Mäkelä
7dcbc33db5 Merge 10.3 into 10.4 2018-11-26 17:20:07 +02:00
Marko Mäkelä
06e5f28f9f MDEV-12266: Remove a level of pointer indirection
Replace table->space->id with table->space_id.
2018-11-22 17:10:26 +02:00
Marko Mäkelä
dde2ca4aa1 Merge 10.3 into 10.4 2018-11-19 20:22:33 +02:00
Marko Mäkelä
fd58bb71e2 Merge 10.2 into 10.3 2018-11-19 18:45:53 +02:00
Marko Mäkelä
ff88e4bb8a Remove many redundant #include from InnoDB 2018-11-19 11:42:14 +02:00
Marko Mäkelä
074c684099 Merge 10.3 into 10.4 2018-11-06 16:24:16 +02:00