and make sure that private ca key is not deleted at the end of
the procedure, so that we could generate additional certificates
any time without regenerating everything
Problem was that bpage was referenced after it was already freed
from LRU. Fixed by adding a new variable encrypted that is
passed down to buf_page_check_corrupt() and used in
buf_page_get_gen() to stop processing page read.
This patch should also address following test failures and
bugs:
MDEV-12419: IMPORT should not look up tablespace in
PageConverter::validate(). This is now removed.
MDEV-10099: encryption.innodb_onlinealter_encryption fails
sporadically in buildbot
MDEV-11420: encryption.innodb_encryption-page-compression
failed in buildbot
MDEV-11222: encryption.encrypt_and_grep failed in buildbot on P8
Removed dict_table_t::is_encrypted and dict_table_t::ibd_file_missing
and replaced these with dict_table_t::file_unreadable. Table
ibd file is missing if fil_get_space(space_id) returns NULL
and encrypted if not. Removed dict_table_t::is_corrupted field.
Ported FilSpace class from 10.2 and using that on buf_page_check_corrupt(),
buf_page_decrypt_after_read(), buf_page_encrypt_before_write(),
buf_dblwr_process(), buf_read_page(), dict_stats_save_defrag_stats().
Added test cases when enrypted page could be read while doing
redo log crash recovery. Also added test case for row compressed
blobs.
btr_cur_open_at_index_side_func(),
btr_cur_open_at_rnd_pos_func(): Avoid referencing block that is
NULL.
buf_page_get_zip(): Issue error if page read fails.
buf_page_get_gen(): Use dberr_t for error detection and
do not reference bpage after we hare freed it.
buf_mark_space_corrupt(): remove bpage from LRU also when
it is encrypted.
buf_page_check_corrupt(): @return DB_SUCCESS if page has
been read and is not corrupted,
DB_PAGE_CORRUPTED if page based on checksum check is corrupted,
DB_DECRYPTION_FAILED if page post encryption checksum matches but
after decryption normal page checksum does not match. In read
case only DB_SUCCESS is possible.
buf_page_io_complete(): use dberr_t for error handling.
buf_flush_write_block_low(),
buf_read_ahead_random(),
buf_read_page_async(),
buf_read_ahead_linear(),
buf_read_ibuf_merge_pages(),
buf_read_recv_pages(),
fil_aio_wait():
Issue error if page read fails.
btr_pcur_move_to_next_page(): Do not reference page if it is
NULL.
Introduced dict_table_t::is_readable() and dict_index_t::is_readable()
that will return true if tablespace exists and pages read from
tablespace are not corrupted or page decryption failed.
Removed buf_page_t::key_version. After page decryption the
key version is not removed from page frame. For unencrypted
pages, old key_version is removed at buf_page_encrypt_before_write()
dict_stats_update_transient_for_index(),
dict_stats_update_transient()
Do not continue if table decryption failed or table
is corrupted.
dict0stats.cc: Introduced a dict_stats_report_error function
to avoid code duplication.
fil_parse_write_crypt_data():
Check that key read from redo log entry is found from
encryption plugin and if it is not, refuse to start.
PageConverter::validate(): Removed access to fil_space_t as
tablespace is not available during import.
Fixed error code on innodb.innodb test.
Merged test cased innodb-bad-key-change5 and innodb-bad-key-shutdown
to innodb-bad-key-change2. Removed innodb-bad-key-change5 test.
Decreased unnecessary complexity on some long lasting tests.
Removed fil_inc_pending_ops(), fil_decr_pending_ops(),
fil_get_first_space(), fil_get_next_space(),
fil_get_first_space_safe(), fil_get_next_space_safe()
functions.
fil_space_verify_crypt_checksum(): Fixed bug found using ASAN
where FIL_PAGE_END_LSN_OLD_CHECKSUM field was incorrectly
accessed from row compressed tables. Fixed out of page frame
bug for row compressed tables in
fil_space_verify_crypt_checksum() found using ASAN. Incorrect
function was called for compressed table.
Added new tests for discard, rename table and drop (we should allow them
even when page decryption fails). Alter table rename is not allowed.
Added test for restart with innodb-force-recovery=1 when page read on
redo-recovery cant be decrypted. Added test for corrupted table where
both page data and FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION is corrupted.
Adjusted the test case innodb_bug14147491 so that it does not anymore
expect crash. Instead table is just mostly not usable.
fil0fil.h: fil_space_acquire_low is not visible function
and fil_space_acquire and fil_space_acquire_silent are
inline functions. FilSpace class uses fil_space_acquire_low
directly.
recv_apply_hashed_log_recs() does not return anything.
c3cf7f47f0f4a1ec314001aaf0c3d9c1c1f62097 reverted the patch
for BUG#24487120. After merging the reverting patch from MySQL
to MariaDB the problems described in MDEV-11079 and MDEV-11631 disappeared.
Adding test cases only.
Partially backporting MDEV-9874 from 10.2 to 10.0
READ_INFO::read_field() raised the ER_INVALID_CHARACTER_STRING error
when reading an escape character followed by a multi-byte character.
Raising wellformedness errors in READ_INFO::read_field() was wrong,
because the main goal of READ_INFO::read_field() is to *unescape* the
data which was presumably escaped using mysql_real_escape_string(),
using the same character set with the one specified in
"LOAD DATA INFILE ... CHARACTER SET ..." (or assumed by default).
During LOAD DATA, multi-byte characters are not always scanned as a single
entity! In case of escaped data, parts of a multi-byte character can be
scanned on different loop iterations. So the old code erroneously tested
welformedness in the middle of a multi-byte character.
Moreover, the data after unescaping can go into a BLOB field, not a text field.
Wellformedness tests are meaningless in this case.
Ater this patch, wellformedness is only checked later, during
Field::store(str,length,cs) time. The loop that scans bytes only
makes sure to revert the changes made by mysql_real_escape_string().
Note, in some cases users can supply data which did not really go through
mysql_real_escape_string() and was escaped by some other means,
or was not escaped at all. The file reported in this MDEV contains
the string "\ä", which is an example of such improperly escaped data, as
- either there should be two backslashes: "\\ä"
- or there should be no backslashes at all: "ä"
mysql_real_escape_string() could not generate "\ä".
Two problems:
(1) When pushing warning to sql-layer we need to check that thd != NULL
to avoid NULL-pointer reference.
(2) At tablespace key rotation if used key_id is not found from
encryption plugin tablespace should not be rotated.
LOAD DATA AT MASTER.
Revert "BUG#23080148 - BACKPORT BUG 14653594 AND BUG 20683959 TO"
This reverts commit 1d31f5b3090d129382b50b95512f2f79305715a1.
The commit causes replication incompatibility between minor revisions
and based on discussion with Srinivasarao, the patch is reverted.
The collation customization code for the UCA (Unicode Collation Alrorithm)
based collations now allows to reset to and shift of characters with
implicit weights. Previously reset/shift worked only for the characters
with explicit DUCET weights. An attempt to use reset/shift with
character with implicit weights made the server crash.
MYSQL-5.5
The bug asks for a backport of bug#1463594 and bug#20682959. This
is required because of the fact that if replication is enabled, master
transaction can commit whereas slave can't commit due to not exact
'enviroment'. This manifestation is seen in bug#22024200.
In some cases, MariaDB 10.0 could write a master.info file that was read
incorrectly by 10.1 and could cause server to fail to start after an upgrade.
(If writing a new master.info file that is shorter than the old, extra
junk may remain at the end of the file. This is handled properly in
10.1 with an END_MARKER line, but this line is not written by
10.0. The fix here is to make 10.1 robust at reading the master.info
files written by 10.0).
Fix several things around reading master.info and read_mi_key_from_file():
- read_mi_key_from_file() did not distinguish between a line with and
without an eqals '=' sign.
- If a line was empty, read_mi_key_from_file() would incorrectly return
the key from the previous call.
- An extra using_gtid=X line left-over by MariaDB 10.0 might incorrectly
be read and overwrite the correct value.
- Fix incorrect usage of strncmp() which should be strcmp().
- Add test cases.
DESCRIPTION
===========
Inability of mysql LOAD XML command to handle empty XML
tags i.e. <row><tag/></row>. Also the behaviour is wrong
and (different than above) when there is a space in empty
tag i.e. <row><tag /></row>
ANALYSIS
========
In read_xml() the case where we encounter a close tag ('/')
we're decreasing the 'level' blindly which is wrong.
Actually when its an without-space-empty-tag (succeeding
char is '>'), we need to skip the decrement. In other words
whenever we hit a close tag ('/'), decrease the 'level'
only when (i) It's not an (without space) empty tag i.e.
<tag/> or, (ii) It is of format <row col="val" .../>
FIX
===
The switch case for '/' is modified. We've removed the
blind decrement of 'level'. We do it only when its not an
without-space-empty-tag. Also we are setting 'in_tag' to
false to let program know that we're done reading current
tag (required in the case of format <row col="val" .../>)
Don't use a fixed buffer for X509_NAME_oneline() in the client.
Do as the server does - allocate it dynamically.
For a test - regenerate certificates to have the server cert with
a long subject.
