rw_lock_create_func(): Initialize lock->writer_thread, so that Valgrind
will not complain even when Valgrind instrumentation is not enabled.
Flag lock->writer_thread uninitialized, so that Valgrind can complain
when it is used uninitialized.
rw_lock_set_writer_id_and_recursion_flag(): Revert the bogus Valgrind
instrumentation that was pushed in the first attempt to fix this bug.
by silencing a bogus Valgrind warning:
==4392== Conditional jump or move depends on uninitialised value(s)
==4392== at 0x5A18416: rw_lock_set_writer_id_and_recursion_flag (sync0rw.ic:283)
==4392== by 0x5A1865C: rw_lock_x_lock_low (sync0rw.c:558)
==4392== by 0x5A18481: rw_lock_x_lock_func (sync0rw.c:617)
==4392== by 0x597EEE6: mtr_x_lock_func (mtr0mtr.ic:271)
==4392== by 0x597EBBD: fsp_header_init (fsp0fsp.c:970)
==4392== by 0x5A15E78: innobase_start_or_create_for_mysql (srv0start.c:1508)
==4392== by 0x598B789: innobase_init(void*) (ha_innodb.cc:2282)
os_compare_and_swap_thread_id() is defined as
__sync_bool_compare_and_swap(). From the GCC doc:
`bool __sync_bool_compare_and_swap (TYPE *ptr, TYPE oldval TYPE newval, ...)'
...
The "bool" version returns true if the comparison is successful and
NEWVAL was written.
So it is not possible that the return value is uninitialized, no matter what
the arguments to os_compare_and_swap_thread_id() are. Probably Valgrind gets
confused by the implementation of the GCC internal function
__sync_bool_compare_and_swap().
This option is known to be broken when tablespaces contain off-page
columns after crash recovery. It has only been tested when creating
the data files from the scratch.
btr_blob_dbg_t: A map from page_no:heap_no:field_no to first_blob_page_no.
This map is instantiated for every clustered index in index->blobs.
It is protected by index->blobs_mutex.
btr_blob_dbg_msg_issue(): Issue a diagnostic message.
Invoked when btr_blob_dbg_msg is set.
btr_blob_dbg_rbt_insert(): Insert a btr_blob_dbg_t into index->blobs.
btr_blob_dbg_rbt_delete(): Remove a btr_blob_dbg_t from index->blobs.
btr_blob_dbg_cmp(): Comparator for btr_blob_dbg_t.
btr_blob_dbg_add_blob(): Add a BLOB reference to the map.
btr_blob_dbg_add_rec(): Add all BLOB references from a record to the map.
btr_blob_dbg_print(): Display the map of BLOB references in an index.
btr_blob_dbg_remove_rec(): Remove all BLOB references of a record from
the map.
btr_blob_dbg_is_empty(): Check that no BLOB references exist to or
from a page. Disowned references from delete-marked records are
tolerated.
btr_blob_dbg_op(): Perform an operation on all BLOB references on a
B-tree page.
btr_blob_dbg_add(): Add all BLOB references from a B-tree page to the
map.
btr_blob_dbg_remove(): Remove all BLOB references from a B-tree page
from the map.
btr_blob_dbg_restore(): Restore the BLOB references after a failed
page reorganize.
btr_blob_dbg_set_deleted_flag(): Modify the 'deleted' flag in the BLOB
references of a record.
btr_blob_dbg_owner(): Own or disown a BLOB reference.
btr_page_create(), btr_page_free_low(): Assert that no BLOB references exist.
btr_create(): Create index->blobs for clustered indexes.
btr_page_reorganize_low(): Invoke btr_blob_dbg_remove() before copying
the records. Invoke btr_blob_dbg_restore() if the operation fails.
btr_page_empty(), btr_lift_page_up(), btr_compress(), btr_discard_page():
Invoke btr_blob_dbg_remove().
btr_cur_del_mark_set_clust_rec(): Invoke btr_blob_dbg_set_deleted_flag().
Other cases of modifying the delete mark are either in the secondary
index or during crash recovery, which we do not promise to support.
btr_cur_set_ownership_of_extern_field(): Invoke btr_blob_dbg_owner().
btr_store_big_rec_extern_fields(): Invoke btr_blob_dbg_add_blob().
btr_free_externally_stored_field(): Invoke btr_blob_dbg_assert_empty()
on the first BLOB page.
page_cur_insert_rec_low(), page_cur_insert_rec_zip(),
page_copy_rec_list_end_to_created_page(): Invoke btr_blob_dbg_add_rec().
page_cur_insert_rec_zip_reorg(), page_copy_rec_list_end(),
page_copy_rec_list_start(): After failure, invoke
btr_blob_dbg_remove() and btr_blob_dbg_add().
page_cur_delete_rec(): Invoke btr_blob_dbg_remove_rec().
page_delete_rec_list_end(): Invoke btr_blob_dbg_op(btr_blob_dbg_remove_rec).
page_zip_reorganize(): Invoke btr_blob_dbg_remove() before copying the records.
page_zip_copy_recs(): Invoke btr_blob_dbg_add().
row_upd_rec_in_place(): Invoke btr_blob_dbg_rbt_delete() and
btr_blob_dbg_rbt_insert().
innobase_start_or_create_for_mysql(): Warn when UNIV_BLOB_DEBUG is enabled.
rb://550 approved by Jimmy Yang
btr_rec_get_field_ref_offs(), btr_rec_get_field_ref(): New functions.
Get the pointer to an externally stored field.
btr_cur_set_ownership_of_extern_field(): Assert that the BLOB has not
already been disowned.
btr_store_big_rec_extern_fields(): Rename to
btr_store_big_rec_extern_fields_func() and add the debug parameter
update_in_place. All pointers to externally stored columns in the
record must either be zero or they must be pointers to inherited
columns, owned by this record or an earlier record version. For any
BLOB that is stored, the BLOB pointer must previously have been
zero. When the function completes, all BLOB pointers must be nonzero
and owned by the record.
rb://549 approved by Jimmy Yang
It was the enabling of UNIV_DEBUG_FILE_ACCESSES that caught Bug #55284
in the first place. This is a very light piece of of debug code, and
there really is no reason why it is not enabled in all debug builds.
rb://551 approved by Jimmy Yang
trx rollback or purge
This patch does not relax the failing debug assertion during purge.
That will be revisited once we have managed to repeat the assertion failure.
row_upd_changes_ord_field_binary_func(): Renamed from
row_upd_changes_ord_field_binary(). Add the parameter que_thr_t* in
UNIV_DEBUG builds. When the off-page column cannot be retrieved,
assert that the current transaction is a recovered one and that it is
the one that is currently being rolled back.
row_upd_changes_ord_field_binary(): A wrapper macro for
row_upd_changes_ord_field_binary_func() that discards the que_thr_t*
parameter unless UNIV_DEBUG is defined.
row_purge_upd_exist_or_extern_func(): Renamed from
row_purge_upd_exist_or_extern(). Add the parameter que_thr_t* in
UNIV_DEBUG builds.
row_purge_upd_exist_or_extern(): A wrapper macro for
row_purge_upd_exist_or_extern_func() that discards the que_thr_t*
parameter unless UNIV_DEBUG is defined.
Make trx_roll_crash_recv_trx const. If there were a 'do not
dereference' attribute, it would be appropriate as well.
rb://588 approved by Jimmy Yang
trx_get_trx_by_xid(): Invalidate trx->xid after a successful lookup,
so that subsequent callers will not find the same transaction.
The only callers of trx_get_trx_by_xid() will be invoking
innobase_commit_low() or innobase_rollback_trx(), and those code paths
should not depend on trx->xid.
rb://584 approved by Jimmy Yang
buf_block_alloc(): ulint zip_size is always 0.
buf_LRU_get_free_block(): ulint zip_size is always 0.
buf_LRU_free_block(): ibool* buf_pool_mutex_released is always NULL.
Remove these parameters.
buf_LRU_get_free_block(): Simplify the initialization of block->page.zip
and release buf_pool_mutex() earlier.
rw_lock_debug_print(): Add parameter FILE* for specifying the output stream.
rw_lock_list_print_info(): Invoke rw_lock_debug_print() on file, not stderr.
"rows examined" estimates". This change implements "innodb_stats_method"
with options of "nulls_equal", "nulls_unequal" and "null_ignored".
rb://553 approved by Marko
This bug fix requires that Bug #58912 be fixed as well (bzr revision id
marko.makela@oracle.com-20101221093919-mcmmgd4zpse9567d). Otherwise,
another double BLOB free could occur when InnoDB would try to perform
an update-in-place as delete-and-insert-by-update-in-place.
row_upd_clust_rec_by_insert(): Do not disown the externally stored
columns from the old record (btr_cur_mark_extern_inherited_fields())
until after checking the foreign key constraints and successfully
inserting the updated record. If a lock wait timeout occurs between
the delete-marking of the old record and the insertion of the updated
record, mark the columns inherited before retrying the insert.
Distinguish the state UPD_NODE_INSERT_BLOB from
UPD_NODE_INSERT_CLUSTERED.
btr_cur_del_mark_set_clust_rec(): Replace the cursor with
block,rec,index,offsets so that the offsets need not be recalculated.
Assert that rec is on a clustered index leaf page.
btr_cur_disown_inherited_fields(): Renamed from
btr_cur_mark_extern_inherited_fields(). Use
upd_get_field_by_field_no(). Assert that there are externally stored
columns. Assert that a mini-transaction is passed. Remove the return
status. (The only caller, row_upd_clust_rec_by_insert(), will have
determined that some fields have changed ownership.)
btr_cur_mark_dtuple_inherited_extern(): Rename to
row_upd_clust_rec_by_insert_inherit_func() and declare as static. Add
the debug parameters rec, offsets. When rec is given, assert that the
off-page columns match those in the inesrt tuple and that the off-page
columns are owned by the record. Assert that the non-updated off-page
columns in the insert tuple are owned, and mark them inherited.
row_upd_clust_rec_by_insert_inherit(): A wrapper macro for
row_upd_clust_rec_by_insert_inherit_func().
row_undo_mod_upd_exist_sec(): Adjust a comment about
row_upd_clust_rec_by_insert().
rb:508 approved by Jimmy Yang
row_upd_changes_ord_field_binary(): Do not return TRUE if the update
vector changes a column that is covered by a prefix index, but does
not change the column prefix. Add the row_ext_t parameter for
determining whether the prefixes of externally stored columns match.
dfield_datas_are_binary_equal(): Add the parameter len, for comparing
column prefixes when len > 0.
innodb.test: Add a test case where the patch of Bug #55284 failed
without this fix.
rb:537 approved by Jimmy Yang
InnoDB AUTOINC code expects the locks to be released in strict reverse order
at the end of the statement. However, nested stored proedures and partition
tables break this rule. We now allow the locks to be deleted from the
trx->autoinc_locks vector in any order but optimise for the common (old) case.
rb://441 Approved by Marko Makela
Also delete dict_update_statistics() and rename dict_update_statistics_low()
to dict_update_statistics() because the only thing that distinguished those
two functions was the removed parameter.
Improve the diagnostics of buffer pool accesses for B-trees,
so that the file names and line numbers of the real calls are shown
instead of the line of the buf_page_get() call in btr_block_get().
btr_page_get(): Replaced with a macro.
btr_block_get_func(): Renamed from btr_block_get(). Add file, line.
btr_block_get(): A macro that passes the __FILE__, __LINE__ to
btr_block_get_func().
dict_truncate_index_tree(): Replace a btr_page_get() call
with btr_block_get(), since we are only latching the page, not accessing it.
This patch doesn't get rid of the need to acquire the dict_sys->mutex but
reduces the need to keep the mutex locked for the duration of the query
to fsp_get_available_space_in_free_extents() from ha_innobase::info().
This is a port of revno:3548 from the builtin to the plugin.
rb://501 Approved by Jimmy Yang and Marko Makela.
Replace the array of mutexes that used to protect
dict_index_t::stat_n_diff_key_vals[] with an array of rw locks that protects
all the stats related members in dict_table_t and all of its indexes.
Approved by: Jimmy (rb://503)
row_search_for_mysql(): When a secondary index record might not be
visible in the current transaction's read view and we consult the
clustered index and optionally some undo log records, return the
relevant columns of the clustered index record to MySQL instead of the
secondary index record.
ibuf_insert_to_index_page_low(): New function, refactored from
ibuf_insert_to_index_page().
ibuf_insert_to_index_page(): When we are inserting a record in place
of a delete-marked record and some fields of the record differ, update
that record just like row_ins_sec_index_entry_by_modify() would do.
btr_cur_update_alloc_zip(): Make the function public.
mysql_row_templ_t: Add clust_rec_field_no.
row_sel_store_mysql_rec(), row_sel_push_cache_row_for_mysql(): Add the
flag rec_clust, for returning data at clust_rec_field_no instead of
rec_field_no. Resurrect the debug assertion that the record not be
marked for deletion. (Bug #55626)
[UNIV_DEBUG || UNIV_IBUF_DEBUG] ibuf_debug, buf_page_get_gen(),
buf_flush_page_try():
Implement innodb_change_buffering_debug=1 for evicting pages from the
buffer pool, so that change buffering will be attempted more
frequently.
TYPE __sync_lock_test_and_set (TYPE *ptr, TYPE value, ...)
it is not documented what happens if the two arguments are of different
type like it was before: the first one was lock_word_t (byte) and the
second one was 1 or 0 (int).
Approved by: Marko (via IRC)
Fix compiler warning:
./include/ut0rnd.ic: In function 'ut_rnd_gen_ulint':
./include/ut0rnd.ic:88:8: error: variable 'n_bits' set but not used [-Werror=unused-but-set-variable]
Currently we do a full validation of AHI whenever check tables is
called on any table. This patch fixes this by only doing this full
check in debug versions.
bug#55716
rb://423
approved by: Marko
The bug is due to a double delete of a BLOB, once via:
rollback -> btr_cur_pessimistic_delete()
and the second time via purge.
The bug is in row_upd_clust_rec_by_insert(). There we relinquish ownership
of the non-updated BLOB columns in btr_cur_mark_extern_inherited_fields()
before building the row entry that will be inserted and whose contents will
be logged in the UNDO log. However, we don't set the BLOB column later to
INHERITED so that a possible rollback will not free the original row's
non-updated BLOB entries. This is because the condition that checks for
that is in :
if (node->upd_ext) {}.
node->upd_ext is non-NULL only if a BLOB column was updated and that column
is part of some key ordering (see row_upd_replace()). This results in the
non-update BLOB columns being deleted during a rollback and subsequently by
purge again.
rb://413
columns
When the server crashes after a record stub has been inserted and
before all its off-page columns have been written, the record will
contain incomplete off-page columns after crash recovery. Such records
may only be accessed at the READ UNCOMMITTED isolation level or when
rolling back a recovered transaction in recv_recovery_rollback_active().
Skip these records at the READ UNCOMMITTED isolation level.
TODO: Add assertions for checking the above assumptions hold when an
incomplete BLOB is encountered.
btr_rec_copy_externally_stored_field(): Return NULL if the field is
incomplete.
row_prebuilt_t::templ_contains_blob: Clarify what "BLOB" means in this
context. Hint: MySQL BLOBs are not the same as InnoDB BLOBs.
row_sel_store_mysql_rec(): Return FALSE if not all columns could be
retrieved. Previously this function always returned TRUE. Assert that
the record is not delete-marked.
row_sel_push_cache_row_for_mysql(): Return FALSE if not all columns
could be retrieved.
row_search_for_mysql(): Skip records containing incomplete off-page
columns. Assert that the transaction isolation level is READ
UNCOMMITTED.
rb://380 approved by Jimmy Yang
and clarifies the invariant in dict_table_get_on_id().
In Mar 2007 Marko observed a crash during recovery, the crash resulted from
an UNDO operation on a system table. His solution was to acquire an X lock on
the data dictionary, this in hindsight was an overkill. It is unclear what
caused the crash, current hypothesis is that it was a memory corruption.
The X lock results in performance issues by when undoing changes due to
rollback during normal operation on regular tables.
Why the change is safe:
======================
The InnoDB code has changed since the original X lock change was made. In the
new code we always lock the data dictionary in X mode during startup when
UNDOing operations on the system tables (this is a given). This ensures that
the crash Marko observed cannot happen as long as all transactions that update
the system tables follow the standard rules by setting the appropriate DICT_OP
flag when writing the log records when they make the changes.
If transactions violate the above mentioned rule then during recovery (at
startup) the rollback code (see trx0roll.c) will not acquire the X lock
and we will see the crash again. This will however be a different bug.
when renaming tables
Allocate the table name using ut_malloc() instead of table->heap because
the latter cannot be freed.
Adjust dict_sys->size calculations all over the code.
Change dict_table_t::name from const char* to char* because we need to
ut_malloc()/ut_free() it.
Reviewed by: Inaam, Marko, Heikki (rb://384)
Approved by: Heikki (rb://384)
