mirror of
https://github.com/MariaDB/server.git
synced 2025-11-06 13:10:12 +03:00
cc277a7d24
The innodb_encrypt_log=ON subformat of FORMAT_10_8 is inefficient, because a new encryption or decryption context is being set up for every log record payload snippet. An in-place conversion between the old and new innodb_encrypt_log=ON format is technically possible. No such conversion has been implemented, though. There is some overhead with respect to the unencrypted format (innodb_encrypt_log=OFF): At the end of each mini-transaction, right before the CRC-32C, additional 8 bytes will be reserved for a nonce (really, log_sys.get_flushed_lsn()), which forms a part of an initialization vector. log_t::FORMAT_ENC_11: The new format identifier, a UTF-8 encoding of 🗝 U+1F5DD OLD KEY (encryption). In this format, everything except the types and lengths of log records will be encrypted. Thus, unlike in FORMAT_10_8, also page identifiers and FILE_ records will be encrypted. The initialization vector (IV) consists of the 8-byte nonce as well as the type and length byte(s) of the first record of the mini-transaction. Page identifiers will no longer form any part of the IV. The old log_t::FORMAT_ENC_10_8 (innodb_encrypt_log=ON) will be supported both by mariadb-backup and by crash recovery. Downgrade from the new format will only be possible if the new server has been running or restarted with innodb_encrypt_log=OFF. If innodb_encrypt_log=ON, only the new log_t::FORMAT_ENC_11 will be written. log_t::is_recoverable(): A new predicate, which holds for all 3 formats. recv_sys_t::tmp_buf: A heap-allocated buffer for decrypting a mini-transaction, or for making the wrap-around of a memory-mapped log file contiguous. recv_sys_t::start_lsn: The start of the mini-transaction. Updated at the start of parse_tail(). log_decrypt_mtr(): Decrypt a mini-transaction in recv_sys.tmp_buf. Theoretically, when reading the log via pread() rather than a read-only memory mapping, we could modify the contents of log_sys.buf in place. If we did that, we would have to re-read the last log block into log_sys.buf before resuming writes, because otherwise that block could be re-written as a mix of old decrypted data and new encrypted data, which would cause a subsequent recovery failure unless the log checkpoint had been advanced beyond this point. log_decrypt_legacy(): Decrypt a log_t::FORMAT_ENC_10_8 record snippet on stack. Replaces recv_buf::copy_if_needed(). recv_sys_t::get_backup_parser(): Return a recv_sys_t::parser, that is, a pointer to an instantiation of parse_mmap or parse_mtr for the current log format. recv_sys_t::parse_mtr(), recv_sys_t::parse_mmap(): Add a parameter template<uint32_t> for the current log_sys.format. log_parse_start(): Validate the CRC-32C of a mini-transaction. This has been split from the recv_sys_t::parse() template to reduce code duplication. These two are the lowest-level functions that will be instantiated for both recv_buf and recv_ring. recv_sys_t::parse(): Split into ::log_parse_start() and parse_tail(). Add a parameter template<uint32_t format> to specialize for log_sys.format at compilation time. recv_sys_t::parse_tail(): Operate on pointers to contiguous mini-transaction data. Use a parameter template<bool ENC_10_8> for special handling of the old innodb_encrypt_log=ON format. The former recv_buf::get_buf() is being inlined here. Much of the logic is split into non-inline functions, to avoid duplicating a lot of code for every template expansion. log_crypt: Encrypt or decrypt a mini-transaction in place in the new innodb_encrypt_log=ON format. We will use temporary buffers so that encryption_ctx_update() can be invoked on integer multiples of MY_AES_BLOCK_SIZE, except for the last bytes of the encrypted payload, which will be encrypted or decrypted in place thanks to ENCRYPTION_FLAG_NOPAD. log_crypt::append(): Invoke encryption_ctx_update() in MY_AES_BLOCK_SIZE (16-byte) blocks and scatter/gather shorter data blocks as needed. log_crypt::finish(), Handle the last (possibly incomplete) block as a special case, with ENCRYPTION_FLAG_NOPAD. mtr_t::parse_length(): Parse the length of a log record. mtr_t::encrypt(): Use log_crypt instead of the old log_encrypt_buf(). recv_buf::crc32c(): Add a parameter for the initial CRC-32C value. recv_sys_t::rewind(): Operate on pointers to the start of the mini-transaction and to the first skipped record. recv_sys_t::trim(): Declare as ATTRIBUTE_COLD so that this rarely invoked function will not be expanded inline in parse_tail(). recv_sys_t::parse_init(): Handle INIT_PAGE or FREE_PAGE while scanning to the end of the log. recv_sys_t::parse_page0(): Handle WRITE to FSP_SPACE_SIZE and FSP_SPACE_FLAGS. recv_sys_t::parse_store_if_exists(), recv_sys_t::parse_store(), recv_sys_t::parse_oom(): Handle page-level log records. mlog_decode_varint_length(): Make use of __builtin_clz() to avoid a loop when possible. mlog_decode_varint(): Define only on const byte*, as ATTRIBUTE_NOINLINE static because it is a rather large function. recv_buf::decode_varint(): Trivial wrapper for mlog_decode_varint(). recv_ring::decode_varint(): Special implementation. log_page_modify(): Note that a page will be modified in recovery. Split from recv_sys_t::parse_tail(). log_parse_file(): Handle non-page log records. log_record_corrupted(), log_unknown(), log_page_id_corrupted(): Common error reporting functions.
This directory contains test suites for the MariaDB server. To run currently existing test cases, execute ./mysql-test-run in this directory. Some tests are known to fail on some platforms or be otherwise unreliable. In the file collections/smoke_test there is a list of tests that are expected to be stable. In general you do not have to have to do "make install", and you can have a co-existing MariaDB installation, the tests will not conflict with it. To run the tests in a source directory, you must do "make" first. In Red Hat distributions, you should run the script as user "mysql". The user is created with nologin shell, so the best bet is something like # su - # cd /usr/share/mysql-test # su -s /bin/bash mysql -c ./mysql-test-run This will use the installed MariaDB executables, but will run a private copy of the server process (using data files within /usr/share/mysql-test), so you need not start the mysqld service beforehand. You can omit --skip-test-list option if you want to check whether the listed failures occur for you. To clean up afterwards, remove the created "var" subdirectory, e.g. # su -s /bin/bash - mysql -c "rm -rf /usr/share/mysql-test/var" If tests fail on your system, please read the following manual section for instructions on how to report the problem: https://mariadb.com/kb/en/reporting-bugs If you want to use an already running MySQL server for specific tests, use the --extern option to mysql-test-run. Please note that in this mode, you are expected to provide names of the tests to run. For example, here is the command to run the "alias" and "analyze" tests with an external server: # mysql-test-run --extern socket=/tmp/mysql.sock alias analyze To match your setup, you might need to provide other relevant options. With no test names on the command line, mysql-test-run will attempt to execute the default set of tests, which will certainly fail, because many tests cannot run with an external server (they need to control the options with which the server is started, restart the server during execution, etc.) You can create your own test cases. To create a test case, create a new file in the main subdirectory using a text editor. The file should have a .test extension. For example: # xemacs t/test_case_name.test In the file, put a set of SQL statements that create some tables, load test data, and run some queries to manipulate it. Your test should begin by dropping the tables you are going to create and end by dropping them again. This ensures that you can run the test over and over again. If you are using mysqltest commands in your test case, you should create the result file as follows: # mysql-test-run --record test_case_name or # mysqltest --record < t/test_case_name.test If you only have a simple test case consisting of SQL statements and comments, you can create the result file in one of the following ways: # mysql-test-run --record test_case_name # mysql test < t/test_case_name.test > r/test_case_name.result # mysqltest --record --database test --result-file=r/test_case_name.result < t/test_case_name.test When this is done, take a look at r/test_case_name.result. If the result is incorrect, you have found a bug. In this case, you should edit the test result to the correct results so that we can verify that the bug is corrected in future releases. If you want to submit your test case you can send it to developers@lists.mariadb.org or attach it to a bug report on http://mariadb.org/jira/. If the test case is really big or if it contains 'not public' data, then put your .test file and .result file(s) into a tar.gz archive, add a README that explains the problem, ftp the archive to ftp://ftp.mariadb.org/private and submit a report to https://mariadb.org/jira about it. The latest information about mysql-test-run can be found at: https://mariadb.com/kb/en/mariadb/mysqltest/ If you want to create .rdiff files, check https://mariadb.com/kb/en/mariadb/mysql-test-auxiliary-files/