5 Commits

Author	SHA1	Message	Date
Marko Mäkelä	f77329ace9	Bug#13721257 RACE CONDITION IN UPDATES OR INSERTS OF WIDE RECORDS ... This bug was originally filed and fixed as Bug#12612184. The original fix was buggy, and it was patched by Bug#12704861. Also that patch was buggy (potentially breaking crash recovery), and both fixes were reverted. This fix was not ported to the built-in InnoDB of MySQL 5.1, because the function signatures of many core functions are different from InnoDB Plugin and later versions. The block allocation routines and their callers would have to changed so that they handle block descriptors instead of page frames. When a record is updated so that its size grows, non-updated columns can be selected for external (off-page) storage. The bug is that the initially inserted updated record contains an all-zero BLOB pointer to the field that was not updated. Only after the BLOB pages have been allocated and written, the valid pointer can be written to the record. Between the release of the page latch in mtr_commit(mtr) after btr_cur_pessimistic_update() and the re-latching of the page in btr_pcur_restore_position(), other threads can see the invalid BLOB pointer consisting of 20 zero bytes. Moreover, if the system crashes at this point, the situation could persist after crash recovery, and the contents of the non-updated column would be permanently lost. The problem is amplified by the ROW_FORMAT=DYNAMIC and ROW_FORMAT=COMPRESSED that were introduced in innodb_file_format=barracuda in InnoDB Plugin, but the bug does exist in all InnoDB versions. The fix is as follows. After a pessimistic B-tree operation that needs to write out off-page columns, allocate the pages for these columns in the mini-transaction that performed the B-tree operation (btr_mtr), but write the pages in a separate mini-transaction (blob_mtr). Do mtr_commit(blob_mtr) before mtr_commit(btr_mtr). A quirk: Do not reuse pages that were previously freed in btr_mtr. Only write the off-page columns to 'fresh' pages. In this way, crash recovery will see redo log entries for blob_mtr before any redo log entry for btr_mtr. It will apply the BLOB page writes to pages that were marked free at that point. If crash recovery fails to see all of the btr_mtr redo log, there will be some unreachable BLOB data in free pages, but the B-tree will be in a consistent state. btr_page_alloc_low(): Renamed from btr_page_alloc(). Add the parameter init_mtr. Return an allocated block, or NULL. If init_mtr!=mtr but the page was already X-latched in mtr, do not initialize the page. btr_page_alloc(): Wrapper for btr_page_alloc_for_ibuf() and btr_page_alloc_low(). btr_page_free(): Add a debug assertion that the page was a B-tree page. btr_lift_page_up(): Return the father block. btr_compress(), btr_cur_compress_if_useful(): Add the parameter ibool adjust, for adjusting the cursor position. btr_cur_pessimistic_update(): Preserve the cursor position when big_rec will be written and the new flag BTR_KEEP_POS_FLAG is defined. Remove a duplicate rec_get_offsets() call. Keep the X-latch on index->lock when big_rec is needed. btr_store_big_rec_extern_fields(): Replace update_inplace with an operation code, and local_mtr with btr_mtr. When not doing a fresh insert and btr_mtr has freed pages, put aside any pages that were previously X-latched in btr_mtr, and free the pages after writing out all data. The data must be written to 'fresh' pages, because btr_mtr will be committed and written to the redo log after the BLOB writes have been written to the redo log. btr_blob_op_is_update(): Check if an operation passed to btr_store_big_rec_extern_fields() is an update or insert-by-update. fseg_alloc_free_page_low(), fsp_alloc_free_page(), fseg_alloc_free_extent(), fseg_alloc_free_page_general(): Add the parameter init_mtr. Return an allocated block, or NULL. If init_mtr!=mtr but the page was already X-latched in mtr, do not initialize the page. xdes_get_descriptor_with_space_hdr(): Assert that the file space header is being X-latched. fsp_alloc_from_free_frag(): Refactored from fsp_alloc_free_page(). fsp_page_create(): New function, for allocating, X-latching and potentially initializing a page. If init_mtr!=mtr but the page was already X-latched in mtr, do not initialize the page. fsp_free_page(): Add ut_ad(0) to the error outcomes. fsp_free_page(), fseg_free_page_low(): Increment mtr->n_freed_pages. fsp_alloc_seg_inode_page(), fseg_create_general(): Assert that the page was not previously X-latched in the mini-transaction. A file segment or inode page should never be allocated in the middle of an mini-transaction that frees pages, such as btr_cur_pessimistic_delete(). fseg_alloc_free_page_low(): If the hinted page was allocated, skip the check if the tablespace should be extended. Return NULL instead of FIL_NULL on failure. Remove the flag frag_page_allocated. Instead, return directly, because the page would already have been initialized. fseg_find_free_frag_page_slot() would return ULINT_UNDEFINED on error, not FIL_NULL. Correct a bogus assertion. fseg_alloc_free_page(): Redefine as a wrapper macro around fseg_alloc_free_page_general(). buf_block_buf_fix_inc(): Move the definition from the buf0buf.ic to buf0buf.h, so that it can be called from other modules. mtr_t: Add n_freed_pages (number of pages that have been freed). page_rec_get_nth_const(), page_rec_get_nth(): The inverse function of page_rec_get_n_recs_before(), get the nth record of the record list. This is faster than iterating the linked list. Refactored from page_get_middle_rec(). trx_undo_rec_copy(): Add a debug assertion for the length. trx_undo_add_page(): Return a block descriptor or NULL instead of a page number or FIL_NULL. trx_undo_report_row_operation(): Add debug assertions. trx_sys_create_doublewrite_buf(): Assert that each page was not previously X-latched. page_cur_insert_rec_zip_reorg(): Make use of page_rec_get_nth(). row_ins_clust_index_entry_by_modify(): Pass BTR_KEEP_POS_FLAG, so that the repositioning of the cursor can be avoided. row_ins_index_entry_low(): Add DEBUG_SYNC points before and after writing off-page columns. If inserting by updating a delete-marked record, do not reposition the cursor or commit the mini-transaction before writing the off-page columns. row_build(): Tighten a debug assertion about null BLOB pointers. row_upd_clust_rec(): Add DEBUG_SYNC points before and after writing off-page columns. Do not reposition the cursor or commit the mini-transaction before writing the off-page columns. rb:939 approved by Jimmy Yang	2012-02-17 11:42:04 +02:00
Marko Mäkelä	41f229cd9e	Bug#12704861 Corruption after a crash during BLOB update ... The fix of Bug#12612184 broke crash recovery. When a record that contains off-page columns (BLOBs) is updated, we must first write redo log about the BLOB page writes, and only after that write the redo log about the B-tree changes. The buggy fix would log the B-tree changes first, meaning that after recovery, we could end up having a record that contains a null BLOB pointer. Because we will be redo logging the writes off the off-page columns before the B-tree changes, we must make sure that the pages chosen for the off-page columns are free both before and after the B-tree changes. In this way, the worst thing that can happen in crash recovery is that the BLOBs are written to free pages, but the B-tree changes are not applied. The BLOB pages would correctly remain free in this case. To achieve this, we must allocate the BLOB pages in the mini-transaction of the B-tree operation. A further quirk is that BLOB pages are allocated from the same file segment as leaf pages. Because of this, we must temporarily "hide" any leaf pages that were freed during the B-tree operation by "fake allocating" them prior to writing the BLOBs, and freeing them again before the mtr_commit() of the B-tree operation, in btr_mark_freed_leaves(). btr_cur_mtr_commit_and_start(): Remove this faulty function that was introduced in the Bug#12612184 fix. The problem that this function was trying to address was that when we did mtr_commit() the BLOB writes before the mtr_commit() of the update, the new BLOB pages could have overwritten clustered index B-tree leaf pages that were freed during the update. If recovery applied the redo log of the BLOB writes but did not see the log of the record update, the index tree would be corrupted. The correct solution is to make the freed clustered index pages unavailable to the BLOB allocation. This function is also a likely culprit of InnoDB hangs that were observed when testing the Bug#12612184 fix. btr_mark_freed_leaves(): Mark all freed clustered index leaf pages of a mini-transaction allocated (nonfree=TRUE) before storing the BLOBs, or freed (nonfree=FALSE) before committing the mini-transaction. btr_freed_leaves_validate(): A debug function for checking that all clustered index leaf pages that have been marked free in the mini-transaction are consistent (have not been zeroed out). btr_page_alloc_low(): Refactored from btr_page_alloc(). Return the number of the allocated page, or FIL_NULL if out of space. Add the parameter "mtr_t* init_mtr" for specifying the mini-transaction where the page should be initialized, or if this is a "fake allocation" (init_mtr=NULL) by btr_mark_freed_leaves(nonfree=TRUE). btr_page_alloc(): Add the parameter init_mtr, allowing the page to be initialized and X-latched in a different mini-transaction than the one that is used for the allocation. Invoke btr_page_alloc_low(). If a clustered index leaf page was previously freed in mtr, remove it from the memo of previously freed pages. btr_page_free(): Assert that the page is a B-tree page and it has been X-latched by the mini-transaction. If the freed page was a leaf page of a clustered index, link it by a MTR_MEMO_FREE_CLUST_LEAF marker to the mini-transaction. btr_store_big_rec_extern_fields_func(): Add the parameter alloc_mtr, which is NULL (old behaviour in inserts) and the same as local_mtr in updates. If alloc_mtr!=NULL, the BLOB pages will be allocated from it instead of the mini-transaction that is used for writing the BLOBs. fsp_alloc_from_free_frag(): Refactored from fsp_alloc_free_page(). Allocate the specified page from a partially free extent. fseg_alloc_free_page_low(), fseg_alloc_free_page_general(): Add the parameter "mtr_t* init_mtr" for specifying the mini-transaction where the page should be initialized, or NULL if this is a "fake allocation" that prevents the reuse of a previously freed B-tree page for BLOB storage. If init_mtr==NULL, try harder to reallocate the specified page and assert that it succeeded. fsp_alloc_free_page(): Add the parameter "mtr_t* init_mtr" for specifying the mini-transaction where the page should be initialized. Do not allow init_mtr == NULL, because this function is never to be used for "fake allocations". mtr_t: Add the operation MTR_MEMO_FREE_CLUST_LEAF and the flag mtr->freed_clust_leaf for quickly determining if any MTR_MEMO_FREE_CLUST_LEAF operations have been posted. row_ins_index_entry_low(): When columns are being made off-page in insert-by-update, invoke btr_mark_freed_leaves(nonfree=TRUE) and pass the mini-transaction as the alloc_mtr to btr_store_big_rec_extern_fields(). Finally, invoke btr_mark_freed_leaves(nonfree=FALSE) to avoid leaking pages. row_build(): Correct a comment, and add a debug assertion that a record that contains NULL BLOB pointers must be a fresh insert. row_upd_clust_rec(): When columns are being moved off-page, invoke btr_mark_freed_leaves(nonfree=TRUE) and pass the mini-transaction as the alloc_mtr to btr_store_big_rec_extern_fields(). Finally, invoke btr_mark_freed_leaves(nonfree=FALSE) to avoid leaking pages. buf_reset_check_index_page_at_flush(): Remove. The function fsp_init_file_page_low() already sets bpage->check_index_page_at_flush=FALSE. There is a known issue in tablespace extension. If the request to allocate a BLOB page leads to the tablespace being extended, crash recovery could see BLOB writes to pages that are off the tablespace file bounds. This should trigger an assertion failure in fil_io() at crash recovery. The safe thing would be to write redo log about the tablespace extension to the mini-transaction of the BLOB write, not to the mini-transaction of the record update. However, there is no redo log record for file extension in the current redo log format. rb:693 approved by Sunny Bains	2011-08-29 11:16:42 +03:00
Marko Mäkelä	0ff2a182b6	Bug #11766513 - 59641: Prepared XA transaction in system after hard crash ... causes future shutdown hang InnoDB would hang on shutdown if any XA transactions exist in the system in the PREPARED state. This has been masked by the fact that MySQL would roll back any PREPARED transaction on shutdown, in the spirit of Bug #12161 Xa recovery and client disconnection. [mysql-test-run] do_shutdown_server: Interpret --shutdown_server 0 as a request to kill the server immediately without initiating a shutdown procedure. xid_cache_insert(): Initialize XID_STATE::rm_error in order to avoid a bogus error message on XA ROLLBACK of a recovered PREPARED transaction. innobase_commit_by_xid(), innobase_rollback_by_xid(): Free the InnoDB transaction object after rolling back a PREPARED transaction. trx_get_trx_by_xid(): Only consider transactions whose trx->is_prepared flag is set. The MySQL layer seems to prevent attempts to roll back connected transactions that are in the PREPARED state from another connection, but it is better to play it safe. The is_prepared flag was introduced in the InnoDB Plugin. trx_n_prepared: A new counter, counting the number of InnoDB transactions in the PREPARED state. logs_empty_and_mark_files_at_shutdown(): On shutdown, allow trx_n_prepared transactions to exist in the system. trx_undo_free_prepared(), trx_free_prepared(): New functions, to free the memory objects of PREPARED transactions on shutdown. This is not needed in the built-in InnoDB, because it would collect all allocated memory on shutdown. The InnoDB Plugin needs this because of innodb_use_sys_malloc. trx_sys_close(): Invoke trx_free_prepared() on all remaining transactions.	2011-04-07 21:12:54 +03:00
Satya B	a1092e9b66	Applying InnoDB Plugin 1.0.6 snapshot,part 1. Fixes BUG#45992 and BUG#46656 ... Detailed revision comments: r6130 | marko | 2009-11-02 11:42:56 +0200 (Mon, 02 Nov 2009) | 9 lines branches/zip: Free all resources at shutdown. Set pointers to NULL, so that Valgrind will not complain about freed data structures that are reachable via pointers. This addresses Bug #45992 and Bug #46656. This patch is mostly based on changes copied from branches/embedded-1.0, mainly c5432, c3439, c3134, c2994, c2978, but also some other code was copied. Some added cleanup code is specific to MySQL/InnoDB. rb://199 approved by Sunny Bains	2009-11-30 17:02:05 +05:30
Satya B	3945d5e554	Adding innodb_plugin-1.0.4 as storage/innodb_plugin.	2009-05-27 15:15:59 +05:30