rb://942
approved by: Marko Makela
We don't need to scan LRU for dropping AHI entries when DROPing a table.
AHI entries are already removed when we free up extents for the btree.
WITH LARGE BUFFER POOL
(Note: this a backport of revno:3472 from mysql-trunk)
rb://845
approved by: Marko
When dropping a table (with an .ibd file i.e.: with
innodb_file_per_table set) we scan entire LRU to invalidate pages from
that table. This can be painful in case of large buffer pools as we hold
the buf_pool->mutex for the scan. Note that gravity of the problem does
not depend on the size of the table. Even with an empty table but a
large and filled up buffer pool we'll end up scanning a very long LRU
list.
The fix is to scan flush_list and just remove the blocks belonging to
the table from the flush_list, marking them as non-dirty. The blocks
are left in the LRU list for eventual eviction due to aging. The
flush_list is typically much smaller than the LRU list but for cases
where it is very long we have the solution of releasing the
buf_pool->mutex after scanning 1K pages.
buf_page_[set|unset]_sticky(): Use new IO-state BUF_IO_PIN to ensure
that a block stays in the flush_list and LRU list when we release
buf_pool->mutex. Previously we have been abusing BUF_IO_READ to achieve
this.
Also addressed issues in bug #11745133, where we could mark a table
corrupted instead of crashing the server when found a corrupted buffer/page
if the table created with innodb_file_per_table on.
Rename buf_pool_watch, buf_pool_mutex, buf_pool_zip_mutex
to buf_pool->watch, buf_pool->mutex, buf_pool->zip_mutex
in comments. Refer to buf_pool->flush_list_mutex instead of
flush_list_mutex.
Remove obsolete declarations of buf_pool_mutex and buf_pool_zip_mutex.
row_search_for_mysql(): When a secondary index record might not be
visible in the current transaction's read view and we consult the
clustered index and optionally some undo log records, return the
relevant columns of the clustered index record to MySQL instead of the
secondary index record.
REC_INFO_DELETED_FLAG: Move the definition from rem0rec.ic to rem0rec.h.
ibuf_insert_to_index_page_low(): New function, refactored from
ibuf_insert_to_index_page().
ibuf_insert_to_index_page(): When we are inserting a record in place
of a delete-marked record and some fields of the record differ, update
that record just like row_ins_sec_index_entry_by_modify() would do.
mysql_row_templ_t: Add clust_rec_field_no.
row_sel_store_mysql_rec(), row_sel_push_cache_row_for_mysql(): Add the
flag rec_clust, for returning data at clust_rec_field_no instead of
rec_field_no. Resurrect the debug assertion that the record not be
marked for deletion. (Bug #55626)
buf_LRU_free_block(): Refactored from
buf_LRU_search_and_free_block(). This is needed for the
innodb_change_buffering_debug diagnostics.
[UNIV_DEBUG || UNIV_IBUF_DEBUG] ibuf_debug, buf_page_get_gen(),
buf_flush_page_try():
Implement innodb_change_buffering_debug=1 for evicting pages from the
buffer pool, so that change buffering will be attempted more
frequently.
bzr branch mysql-5.1-performance-version mysql-trunk # Summit
cd mysql-trunk
bzr merge mysql-5.1-innodb_plugin # which is 5.1 + Innodb plugin
bzr rm innobase # remove the builtin
Next step: build, test fixes.
Bug #39939: DROP TABLE/DISCARD TABLESPACE takes long time in buf_LRU_invalidate_tablespace()
Detailed revision comments:
r2742 | inaam | 2008-10-08 22:02:15 +0300 (Wed, 08 Oct 2008) | 11 lines
branches/5.1:
Improve implementation of buf_LRU_invalidate_tablespace by attempting
hash index drop in batches instead of doing it one by one.
Reviewed by: Heikki, Sunny, Marko
Approved by: Heikki
Fixes the following bugs:
Bug #30706: SQL thread on slave is allowed to block client queries when slave load is high
Add (innodb|innobase|srv)_replication_delay MySQL config parameter.
Bug #30888: Innodb table + stored procedure + row deletion = server crash
While adding code for the low level read of the AUTOINC value from the index,
the case for MEDIUM ints which are 3 bytes was missed triggering an
assertion.
Bug #30907: Regression: "--innodb_autoinc_lock_mode=0" (off) not same as older releases
We don't rely on *first_value to be 0 when checking whether
get_auto_increment() has been invoked for the first time in a multi-row
INSERT. We instead use trx_t::n_autoinc_rows. Initialize trx::n_autoinc_rows
inside ha_innobase::start_stmt() too.
Bug #31444: "InnoDB: Error: MySQL is freeing a thd" in innodb_mysql.test
ha_innobase::external_lock(): Update prebuilt->mysql_has_locked and
trx->n_mysql_tables_in_use only after row_lock_table_for_mysql() returns
DB_SUCCESS. A timeout on LOCK TABLES would lead to an inconsistent state,
which would cause trx_free() to print a warning.
Bug #31494: innodb + 5.1 + read committed crash, assertion
Set an error code when a deadlock occurs in semi-consistent read.
Fixes:
- Bug #23710: crash_commit_before fails if innodb_file_per_table=1
- Bug #28254: innodb crash if shutdown during innodb_table_monitor is running
- Bug #28604: innodb_force_recovery restricts data dump
- Bug #29097: fsp_get_available_space_in_free_extents() is capped at 4TB
- Bug #29155: Innodb "Parallel recovery" is not prevented
innodb-5.1-ss1318
innodb-5.1-ss1330
innodb-5.1-ss1332
innodb-5.1-ss1340
Fixes:
- Bug #21409: Incorrect result returned when in READ-COMMITTED with query_cache ON
At low transaction isolation levels we let each consistent read set
its own snapshot.
- Bug #23666: strange Innodb_row_lock_time_% values in show status; also millisecs wrong
On Windows ut_usectime returns secs and usecs relative to the UNIX
epoch (which is Jan, 1 1970).
- Bug #25494: LATEST DEADLOCK INFORMATION is not always cleared
lock_deadlock_recursive(): When the search depth or length is exceeded,
rewind lock_latest_err_file and display the two transactions at the
point of aborting the search.
- Bug #25927: Foreign key with ON DELETE SET NULL on NOT NULL can crash server
Prevent ALTER TABLE ... MODIFY ... NOT NULL on columns for which
there is a foreign key constraint ON ... SET NULL.
- Bug #26835: Repeatable corruption of utf8-enabled tables inside InnoDB
The bug could be reproduced as follows:
Define a table so that the first column of the clustered index is
a VARCHAR or a UTF-8 CHAR in a collation where sequences of bytes
of differing length are considered equivalent.
Insert and delete a record. Before the delete-marked record is
purged, insert another record whose first column is of different
length but equivalent to the first record. Under certain conditions,
the insertion can be incorrectly performed as update-in-place.
Likewise, an operation that could be done as update-in-place can
unnecessarily be performed as delete and insert, but that would not
cause corruption but merely degraded performance.
Bugs fixed:
- Bug #20877: InnoDB data dictionary memory footprint is too big
- Bug #13544: Second delete of same row in transaction illustrates non-optimal locking
- Bug #20791: valgrind errors in InnoDB
Bugs fixed:
- Bug #20791 valgrind errors in InnoDB
Remove Valgrind warning of Bug #20791 : in new database
creation, we read the doublewrite buffer magic number from
uninitialized memory; the code worked because it was extremely
unlikely that the memory would contain the magic number
- Bug #21784 DROP TABLE crashes 5.1.12-pre if concurrent
queries on the table
remove update_thd() in ::store_lock()
Also includes numerous coding style fixes, etc. See file-level
comments for details.
* Fix BUG#15650: "DELETE with LEFT JOIN crashes server with innodb_locks_unsafe_for binlog"
* Fix BUG#17134: "Partitions: uncommitted changes are visible"
* Fix BUG#17992: "Partitions: InnoDB, somehow rotten table after UPDATE"
row0ins.c: MySQL's partitioned table code does not set preduilt->sql_stat_start right
if it does an insert in the same statement after doing a search first in the same
partition table. We now write trx id always to the buffer, not just when flag
sql_stat_start is on. This will waste CPU time very sightly.
* Fix BUG#18077: "InnoDB uses full explicit table locks in stored FUNCTION"
* Fix BUG#18238: "When locks exhaust the buffer pool, InnoDB does not roll back the trx"
* Fix BUG#18252" "Disk space leak in updates of InnoDB BLOB rows in 5.0 and 5.1"
* Fix BUG#18283: "When InnoDB returns error 'lock table full', MySQL can write to binlog too much"
* Fix BUG#18350: "Use consistent read in CREATE ... SELECT ... if innodb_locks_unsafe_for_binlog"
* Fix BUG#18384: "InnoDB memory leak on duplicate key errors in 5.0 if row has many columns"
* Fix BUG#18934: "InnoDB crashes when table uses column names like DB_ROW_ID"
Refuse tables that use reserved column names.
* InnoDB's SQL parser:
- Add support for UNSIGNED types, EXIT keyword, quoted identifiers, user-function callbacks
for processing results of FETCH statements, bound literals, DATA_VARCHAR for bound literals.
- Allow bound literals of type non-INTEGER to be of length 0.
- Add make_flex.sh and update lexer/parser generation documentation.
- Add comment clarifying the difference between 'alias' and 'indirection' fields in sym_node_t.
- Remove never reached duplicate code in pars_set_dfield_type().
- Rewrite pars_info datatypes and APIs, add a few helper functions.
- Since the functions definitions in pars_info_t are accessed after pars_sql() returns
in the query graph execution stage, we can't free pars_info_t in pars_sql(). Instead,
make pars_sql() transfer ownership of pars_info_t to the created query graph, and
make que_graph_free() free it if needed.
- Allow access to system columns like DB_ROW_ID.
* Use bound literals in row_truncate_table_for_mysql, row_drop_table_for_mysql,
row_discard_tablespace_for_mysql, and row_rename_table_for_mysql.
* Setting an isolation level of the transaction to read committed weakens the locks for
this session similarly like the option innodb_locks_unsafe_for binlog. This patch removes
alnost all gap locking (used in next-key locking) and makes MySQL to release the row locks
on the rows which does not belong to result set. Additionally, nonlocking selects on
INSERT INTO SELECT, UPDATE ... (SELECT ...), and CREATE ... SELECT ... use a nonlocking
consistent read. If a binlog is used, then binlog format should be set to row based
binloging to make the execution of the complex SQL statements.
* Disable the statistic variables btr_search_n_hash_fail and n_hash_succ, n_hash_fail,
n_patt_succ, and n_searches of btr_search_t in builds without #ifdef UNIV_SEARCH_PERF_STAT.
* Make innodb.test faster. Group all consistent read test cases to a one test case and
wait their lock timeout after all have been send to the server. Decrease amount of rows
inserted in a certain test - this has no effect on the effectiveness of the test and
reduces the running time by ~10 sec. Remove temporary work-arounds from innodb.result
now that ALTER TABLE DROP FOREIGN KEY works once again.
* Make innodb_unsafe_binlog.test faster. Grout all consistent read test cases to a one
test case amd wait their lock timeout after all have been sent to the server. Remove
unnecessary option --loose_innodb_lock_wait_timeout.
* Print dictionary memory size in SHOW INNODB STATUS.
* Fix memory leaks in row_create_table_for_mysql() in rare corner cases.
* Remove code related to clustered tables. They were never implemented, and the
implementation would be challenging with ROW_FORMAT=COMPACT. Remove the table types
DICT_TABLE_CLUSTER_MEMBER and DICT_TABLE_CLUSTER and all related tests and functions.
dict_table_t: Remove mix_id, mix_len, mix_id_len, mix_id_buf, and cluster_name.
plan_t: Remove mixed_index.
dict_create_sys_tables_tuple(): Set MIX_ID=0, MIX_LEN=0, CLUSTER_NAME=NULL when
inserting into SYS_TABLES.
dict_tree_check_search_tuple(): Enclose in #ifdef UNIV_DEBUG.
* Move calling of thr_local_free() from trx_free_for_mysql() to
innobase_close_connection().
Fixed BUGS:
#3300: "UPDATE statement with no index column in where condition locks
all rows"
Implement semi-consistent read to reduce lock conflicts at the cost
of breaking serializability.
ha_innobase::unlock_row(): reset the "did semi consistent read" flag
ha_innobase::was_semi_consistent_read(),
ha_innobase::try_semi_consistent_read(): new methods
row_prebuilt_t, row_create_prebuilt(): add field row_read_type for
keeping track of semi-consistent reads
row_vers_build_for_semi_consistent_read(),
row_sel_build_committed_vers_for_mysql(): new functions
row_search_for_mysql(): implement semi-consistent reads
#9802: "Foreign key checks disallow alter table".
Added test cases.
#12456: "Cursor shows incorrect data - DML does not affect,
probably caching"
This patch implements a high-granularity read view to be used with
cursors. In this high-granularity consistent read view modifications
done by the creating transaction after the cursor is created or
future transactions are not visible. But those modifications that
transaction did before the cursor was created are visible.
#12701: "Support >4GB buffer pool and log files on 64-bit Windows"
Do not call os_file_create_tmpfile() at runtime. Instead, create all
tempfiles at startup and guard access to them with mutexes.
#13778: "If FOREIGN_KEY_CHECKS=0, one can create inconsistent FOREIGN KEYs".
When FOREIGN_KEY_CHECKS=0 we still need to check that datatypes between
foreign key references are compatible.
#14189: "VARBINARY and BINARY variables: trailing space ignored with InnoDB"
innobase_init(): Assert that
DATA_MYSQL_BINARY_CHARSET_COLL == my_charset_bin.number.
dtype_get_pad_char(): Do not pad VARBINARY or BINARY columns.
row_ins_cascade_calc_update_vec(): Refuse ON UPDATE CASCADE when trying
to change the length of a VARBINARY column that refers to or is referenced
by a BINARY column. BINARY columns are no longer padded on comparison,
and thus they cannot be padded on storage either.
#14747: "Race condition can cause btr_search_drop_page_hash_index() to crash"
Note that buf_block_t::index should be protected by btr_search_latch
or an s-latch or x-latch on the index page.
btr_search_drop_page_hash_index(): Read block->index while holding
btr_search_latch and use the cached value in the loop. Remove some
redundant assertions.
#15108: "mysqld crashes when innodb_log_file_size is set > 4G"
#15308: "Problem of Order with Enum Column in Primary Key"
#15550: "mysqld crashes in printing a FOREIGN KEY error in InnoDB"
row_ins_foreign_report_add_err(): When printing the parent record,
use the index in the parent table rather than the index in the child table.
#15653: "Slow inserts to InnoDB if many thousands of .ibd files"
Keep track on unflushed modifications to file spaces. When there are tens
of thousands of file spaces, flushing all files in fil_flush_file_spaces()
would be very slow.
fil_flush_file_spaces(): Only flush unflushed file spaces.
fil_space_t, fil_system_t: Add a list of unflushed spaces.
#15991: "innodb-file-per-table + symlink database + rename = cr"
os_file_handle_error(): Map the error codes EXDEV, ENOTDIR, and EISDIR
to the new code OS_FILE_PATH_ERROR. Treat this code as OS_FILE_PATH_ERROR.
This fixes the crash on RENAME TABLE when the .ibd file is a symbolic link
to a different file system.
#16157: "InnoDB crashes when main location settings are empty"
This patch is from Heikki.
#16298: "InnoDB segfaults in INSERTs in upgrade of 4.0 -> 5.0 tables
with VARCHAR BINARY"
dict_load_columns(): Set the charset-collation code
DATA_MYSQL_BINARY_CHARSET_COLL for those binary string columns
that lack a charset-collation code, i.e., the tables were created
with an older version of MySQL/InnoDB than 4.1.2.
#16229: "MySQL/InnoDB uses full explicit table locks in trigger processing"
Take a InnoDB table lock only if user has explicitly requested a table
lock. Added some additional comments to store_lock() and external_lock().
#16387: "InnoDB crash when dropping a foreign key <table>_ibfk_0"
Do not mistake TABLENAME_ibfk_0 for auto-generated id.
dict_table_get_highest_foreign_id(): Ignore foreign constraint
identifiers starting with the pattern TABLENAME_ibfk_0.
#16582: "InnoDB: Error in an adaptive hash index pointer to page"
Account for a race condition when dropping the adaptive hash index
for a B-tree page.
btr_search_drop_page_hash_index(): Retry the operation if a hash index
with different parameters was built meanwhile. Add diagnostics for the
case that hash node pointers to the page remain.
btr_search_info_update_hash(), btr_search_info_update_slow():
Document the parameter "info" as in/out.
#16814: "SHOW INNODB STATUS format error in LATEST FOREIGN KEY ERROR
section"
Add a missing newline to the LAST FOREIGN KEY ERROR section in SHOW
INNODB STATUS output.
dict_foreign_error_report(): Always print a newline after invoking
dict_print_info_on_foreign_key_in_create_format().
#16827: "Better InnoDB error message if ibdata files omitted from my.cnf"
#17126: "CHECK TABLE on InnoDB causes a short hang during check of adaptive
hash"
CHECK TABLE blocking other queries, by releasing the btr_search_latch
periodically during the adaptive hash table validation.
#17405: "Valgrind: conditional jump or move depends on unititialised values"
buf_block_init(): Reset magic_n, buf_fix_count and io_fix to avoid
testing uninitialized variables.
