1
0
mirror of https://github.com/MariaDB/server.git synced 2025-07-29 05:21:33 +03:00

Applying InnoDB Plugin 1.0.5 snapshot, part 4

From revision r5703 to r5716

Detailed revision comments:

r5703 | marko | 2009-08-27 02:25:00 -0500 (Thu, 27 Aug 2009) | 41 lines
branches/zip: Replace the constant 3/8 ratio that controls the LRU_old
size with the settable global variable innodb_old_blocks_pct. The
minimum and maximum values are 5 and 95 per cent, respectively. The
default is 100*3/8, in line with the old behavior.

ut_time_ms(): New utility function, to return the current time in
milliseconds. TODO: Is there a more efficient timestamp function, such
as rdtsc divided by a power of two?

buf_LRU_old_threshold_ms: New variable, corresponding to
innodb_old_blocks_time. The value 0 is the default behaviour: no
timeout before making blocks 'new'.

bpage->accessed, bpage->LRU_position, buf_pool->ulint_clock: Remove.

bpage->access_time: New field, replacing bpage->accessed. Protected by
buf_pool_mutex instead of bpage->mutex. Updated when a page is created
or accessed the first time in the buffer pool.

buf_LRU_old_ratio, innobase_old_blocks_pct: New variables,
corresponding to innodb_old_blocks_pct

buf_LRU_old_ratio_update(), innobase_old_blocks_pct_update(): Update
functions for buf_LRU_old_ratio, innobase_old_blocks_pct.

buf_page_peek_if_too_old(): Compare ut_time_ms() to bpage->access_time
if buf_LRU_old_threshold_ms && bpage->old.  Else observe
buf_LRU_old_ratio and bpage->freed_page_clock.

buf_pool_t: Add n_pages_made_young, n_pages_not_made_young,
n_pages_made_young_old, n_pages_not_made_young, for statistics.

buf_print(): Display buf_pool->n_pages_made_young,
buf_pool->n_pages_not_made_young.  This function is only for crash
diagnostics.

buf_print_io(): Display buf_pool->LRU_old_len and quantities derived
from buf_pool->n_pages_made_young, buf_pool->n_pages_not_made_young.
This function is invoked by SHOW ENGINE INNODB STATUS.

rb://129 approved by Heikki Tuuri.  This addresses Bug #45015.
r5704 | marko | 2009-08-27 03:31:17 -0500 (Thu, 27 Aug 2009) | 32 lines
branches/zip: Fix a critical bug in fast index creation that could
corrupt the created indexes.

row_merge(): Make "half" an in/out parameter. Determine the offset of
half the output file. Copy the last blocks record-by-record instead of
block-by-block, so that the records can be counted. Check that the
input and output have matching n_rec.

row_merge_sort(): Do not assume that two blocks of size N are merged
into a block of size 2*N. The output block can be shorter than the
input if the last page of each input block is almost empty. Use an
accurate termination condition, based on the "half" computed by
row_merge().

row_merge_read(), row_merge_write(), row_merge_blocks(): Add debug output.

merge_file_t, row_merge_file_create(): Add n_rec, the number of records
in the merge file.

row_merge_read_clustered_index(): Update n_rec.

row_merge_blocks(): Update and check n_rec.

row_merge_blocks_copy(): New function, for copying the last blocks in
row_merge().  Update and check n_rec.

This bug was discovered with a user-supplied test case that creates an
index where the initial temporary file is 249 one-megabyte blocks and
the merged files become smaller. In the test, possible merge record
sizes are 10, 18, and 26 bytes.

rb://150 approved by Sunny Bains.  This addresses Issue #320.
r5705 | marko | 2009-08-27 06:56:24 -0500 (Thu, 27 Aug 2009) | 11 lines
branches/zip: dict_index_find_cols(): On column name lookup failure,
return DB_CORRUPTION (HA_ERR_CRASHED) instead of abnormally
terminating the server.  Also, disable the previously added diagnostic
output to the error log, because mysql-test-run does not like extra
output in the error log.  (Bug #44571)

dict_index_add_to_cache(): Handle errors from dict_index_find_cols().

mysql-test/innodb_bug44571.test: A test case for triggering the bug.

rb://135 approved by Sunny Bains.
r5706 | inaam | 2009-08-27 11:00:27 -0500 (Thu, 27 Aug 2009) | 20 lines
branches/zip rb://147

Done away with following two status variables:

innodb_buffer_pool_read_ahead_rnd
innodb_buffer_pool_read_ahead_seq

Introduced two new status variables:
innodb_buffer_pool_read_ahead = number of pages read as part of
readahead since server startup
innodb_buffer_pool_read_ahead_evicted = number of pages that are read
in as readahead but were evicted before ever being accessed since
server startup i.e.: a measure of how badly our readahead is
performing

SHOW INNODB STATUS will show two extra numbers in buffer pool section:
pages read ahead/sec and pages evicted without access/sec

Approved by: Marko

r5707 | inaam | 2009-08-27 11:20:35 -0500 (Thu, 27 Aug 2009) | 6 lines
branches/zip

Remove unused macros as we erased the random readahead code in r5703.
Also fixed some comments.


r5708 | inaam | 2009-08-27 17:43:32 -0500 (Thu, 27 Aug 2009) | 4 lines
branches/zip

Remove redundant TRUE : FALSE from the return statement

r5709 | inaam | 2009-08-28 01:22:46 -0500 (Fri, 28 Aug 2009) | 5 lines
branches/zip rb://152

Disable display of deprecated parameter innodb_file_io_threads in
'show variables'.

r5714 | marko | 2009-08-31 01:10:10 -0500 (Mon, 31 Aug 2009) | 5 lines
branches/zip: buf_chunk_not_freed(): Do not acquire block->mutex unless
block->page.state == BUF_BLOCK_FILE_PAGE.  Check that block->page.state
makes sense.

Approved by Sunny Bains over the IM.
r5716 | vasil | 2009-08-31 02:47:49 -0500 (Mon, 31 Aug 2009) | 9 lines
branches/zip:

Fix Bug#46718 InnoDB plugin incompatible with gcc 4.1 (at least: on PPC): "Undefined symbol"

by implementing our own check in plug.in instead of using the result from
the check from MySQL because it is insufficient.

Approved by:	Marko (rb://154)
This commit is contained in:
Satya B
2009-10-08 16:58:37 +05:30
parent 76c1ae946f
commit abfdf76bbf
24 changed files with 814 additions and 602 deletions

View File

@ -837,16 +837,35 @@ buf_chunk_not_freed(
block = chunk->blocks;
for (i = chunk->size; i--; block++) {
mutex_enter(&block->mutex);
if (buf_block_get_state(block) == BUF_BLOCK_FILE_PAGE
&& !buf_flush_ready_for_replace(&block->page)) {
ibool ready;
switch (buf_block_get_state(block)) {
case BUF_BLOCK_ZIP_FREE:
case BUF_BLOCK_ZIP_PAGE:
case BUF_BLOCK_ZIP_DIRTY:
/* The uncompressed buffer pool should never
contain compressed block descriptors. */
ut_error;
break;
case BUF_BLOCK_NOT_USED:
case BUF_BLOCK_READY_FOR_USE:
case BUF_BLOCK_MEMORY:
case BUF_BLOCK_REMOVE_HASH:
/* Skip blocks that are not being used for
file pages. */
break;
case BUF_BLOCK_FILE_PAGE:
mutex_enter(&block->mutex);
ready = buf_flush_ready_for_replace(&block->page);
mutex_exit(&block->mutex);
return(block);
}
mutex_exit(&block->mutex);
if (!ready) {
return(block);
}
break;
}
}
return(NULL);
@ -966,8 +985,6 @@ buf_pool_init(void)
buf_pool->no_flush[i] = os_event_create(NULL);
}
buf_pool->ulint_clock = 1;
/* 3. Initialize LRU fields
--------------------------- */
/* All fields are initialized by mem_zalloc(). */
@ -1470,31 +1487,6 @@ buf_pool_resize(void)
buf_pool_page_hash_rebuild();
}
/********************************************************************//**
Moves the block to the start of the LRU list if there is a danger
that the block would drift out of the buffer pool. */
UNIV_INLINE
void
buf_block_make_young(
/*=================*/
buf_page_t* bpage) /*!< in: block to make younger */
{
ut_ad(!buf_pool_mutex_own());
/* Note that we read freed_page_clock's without holding any mutex:
this is allowed since the result is used only in heuristics */
if (buf_page_peek_if_too_old(bpage)) {
buf_pool_mutex_enter();
/* There has been freeing activity in the LRU list:
best to move to the head of the LRU list */
buf_LRU_make_block_young(bpage);
buf_pool_mutex_exit();
}
}
/********************************************************************//**
Moves a page to the start of the buffer pool LRU list. This high-level
function can be used to prevent an important page from from slipping out of
@ -1649,7 +1641,7 @@ buf_page_get_zip(
#ifndef UNIV_LOG_DEBUG
ut_ad(!ibuf_inside());
#endif
buf_pool->n_page_gets++;
buf_pool->stat.n_page_gets++;
for (;;) {
buf_pool_mutex_enter();
@ -1713,14 +1705,16 @@ err_exit:
got_block:
must_read = buf_page_get_io_fix(bpage) == BUF_IO_READ;
if (buf_page_peek_if_too_old(bpage)) {
buf_LRU_make_block_young(bpage);
}
buf_page_set_accessed(bpage);
buf_pool_mutex_exit();
buf_page_set_accessed(bpage, TRUE);
mutex_exit(block_mutex);
buf_block_make_young(bpage);
#ifdef UNIV_DEBUG_FILE_ACCESSES
ut_a(!bpage->file_page_was_freed);
#endif
@ -2000,7 +1994,7 @@ buf_page_get_gen(
mtr_t* mtr) /*!< in: mini-transaction */
{
buf_block_t* block;
ibool accessed;
unsigned access_time;
ulint fix_type;
ibool must_read;
@ -2016,7 +2010,7 @@ buf_page_get_gen(
#ifndef UNIV_LOG_DEBUG
ut_ad(!ibuf_inside() || ibuf_page(space, zip_size, offset, NULL));
#endif
buf_pool->n_page_gets++;
buf_pool->stat.n_page_gets++;
loop:
block = guess;
buf_pool_mutex_enter();
@ -2243,17 +2237,20 @@ wait_until_unfixed:
UNIV_MEM_ASSERT_RW(&block->page, sizeof block->page);
buf_block_buf_fix_inc(block, file, line);
buf_pool_mutex_exit();
/* Check if this is the first access to the page */
accessed = buf_page_is_accessed(&block->page);
buf_page_set_accessed(&block->page, TRUE);
mutex_exit(&block->mutex);
buf_block_make_young(&block->page);
/* Check if this is the first access to the page */
access_time = buf_page_is_accessed(&block->page);
if (buf_page_peek_if_too_old(&block->page)) {
buf_LRU_make_block_young(&block->page);
}
buf_page_set_accessed(&block->page);
buf_pool_mutex_exit();
#ifdef UNIV_DEBUG_FILE_ACCESSES
ut_a(!block->page.file_page_was_freed);
@ -2306,7 +2303,7 @@ wait_until_unfixed:
mtr_memo_push(mtr, block, fix_type);
if (!accessed) {
if (!access_time) {
/* In the case of a first access, try to apply linear
read-ahead */
@ -2336,7 +2333,7 @@ buf_page_optimistic_get_func(
ulint line, /*!< in: line where called */
mtr_t* mtr) /*!< in: mini-transaction */
{
ibool accessed;
unsigned access_time;
ibool success;
ulint fix_type;
@ -2353,14 +2350,21 @@ buf_page_optimistic_get_func(
}
buf_block_buf_fix_inc(block, file, line);
accessed = buf_page_is_accessed(&block->page);
buf_page_set_accessed(&block->page, TRUE);
mutex_exit(&block->mutex);
buf_block_make_young(&block->page);
buf_pool_mutex_enter();
/* Check if this is the first access to the page */
access_time = buf_page_is_accessed(&block->page);
if (buf_page_peek_if_too_old(&block->page)) {
buf_LRU_make_block_young(&block->page);
}
buf_page_set_accessed(&block->page);
buf_pool_mutex_exit();
ut_ad(!ibuf_inside()
|| ibuf_page(buf_block_get_space(block),
@ -2412,7 +2416,7 @@ buf_page_optimistic_get_func(
#ifdef UNIV_DEBUG_FILE_ACCESSES
ut_a(block->page.file_page_was_freed == FALSE);
#endif
if (UNIV_UNLIKELY(!accessed)) {
if (UNIV_UNLIKELY(!access_time)) {
/* In the case of a first access, try to apply linear
read-ahead */
@ -2425,7 +2429,7 @@ buf_page_optimistic_get_func(
ut_a(ibuf_count_get(buf_block_get_space(block),
buf_block_get_page_no(block)) == 0);
#endif
buf_pool->n_page_gets++;
buf_pool->stat.n_page_gets++;
return(TRUE);
}
@ -2473,10 +2477,16 @@ buf_page_get_known_nowait(
mutex_exit(&block->mutex);
if (mode == BUF_MAKE_YOUNG) {
buf_block_make_young(&block->page);
buf_pool_mutex_enter();
if (mode == BUF_MAKE_YOUNG && buf_page_peek_if_too_old(&block->page)) {
buf_LRU_make_block_young(&block->page);
}
buf_page_set_accessed(&block->page);
buf_pool_mutex_exit();
ut_ad(!ibuf_inside() || (mode == BUF_KEEP_OLD));
if (rw_latch == RW_S_LATCH) {
@ -2513,7 +2523,7 @@ buf_page_get_known_nowait(
|| (ibuf_count_get(buf_block_get_space(block),
buf_block_get_page_no(block)) == 0));
#endif
buf_pool->n_page_gets++;
buf_pool->stat.n_page_gets++;
return(TRUE);
}
@ -2589,7 +2599,7 @@ buf_page_try_get_func(
#endif /* UNIV_DEBUG_FILE_ACCESSES */
buf_block_dbg_add_level(block, SYNC_NO_ORDER_CHECK);
buf_pool->n_page_gets++;
buf_pool->stat.n_page_gets++;
#ifdef UNIV_IBUF_COUNT_DEBUG
ut_a(ibuf_count_get(buf_block_get_space(block),
@ -2608,10 +2618,10 @@ buf_page_init_low(
buf_page_t* bpage) /*!< in: block to init */
{
bpage->flush_type = BUF_FLUSH_LRU;
bpage->accessed = FALSE;
bpage->io_fix = BUF_IO_NONE;
bpage->buf_fix_count = 0;
bpage->freed_page_clock = 0;
bpage->access_time = 0;
bpage->newest_modification = 0;
bpage->oldest_modification = 0;
HASH_INVALIDATE(bpage, hash);
@ -2953,7 +2963,7 @@ buf_page_create(
buf_LRU_add_block(&block->page, FALSE);
buf_block_buf_fix_inc(block, __FILE__, __LINE__);
buf_pool->n_pages_created++;
buf_pool->stat.n_pages_created++;
if (zip_size) {
void* data;
@ -2990,12 +3000,12 @@ buf_page_create(
rw_lock_x_unlock(&block->lock);
}
buf_page_set_accessed(&block->page);
buf_pool_mutex_exit();
mtr_memo_push(mtr, block, MTR_MEMO_BUF_FIX);
buf_page_set_accessed(&block->page, TRUE);
mutex_exit(&block->mutex);
/* Delete possible entries for the page from the insert buffer:
@ -3201,7 +3211,7 @@ corrupt:
ut_ad(buf_pool->n_pend_reads > 0);
buf_pool->n_pend_reads--;
buf_pool->n_pages_read++;
buf_pool->stat.n_pages_read++;
if (uncompressed) {
rw_lock_x_unlock_gen(&((buf_block_t*) bpage)->lock,
@ -3221,7 +3231,7 @@ corrupt:
BUF_IO_WRITE);
}
buf_pool->n_pages_written++;
buf_pool->stat.n_pages_written++;
break;
@ -3528,6 +3538,7 @@ buf_print(void)
"n pending decompressions %lu\n"
"n pending reads %lu\n"
"n pending flush LRU %lu list %lu single page %lu\n"
"pages made young %lu, not young %lu\n"
"pages read %lu, created %lu, written %lu\n",
(ulong) size,
(ulong) UT_LIST_GET_LEN(buf_pool->LRU),
@ -3538,8 +3549,11 @@ buf_print(void)
(ulong) buf_pool->n_flush[BUF_FLUSH_LRU],
(ulong) buf_pool->n_flush[BUF_FLUSH_LIST],
(ulong) buf_pool->n_flush[BUF_FLUSH_SINGLE_PAGE],
(ulong) buf_pool->n_pages_read, buf_pool->n_pages_created,
(ulong) buf_pool->n_pages_written);
(ulong) buf_pool->stat.n_pages_made_young,
(ulong) buf_pool->stat.n_pages_not_made_young,
(ulong) buf_pool->stat.n_pages_read,
(ulong) buf_pool->stat.n_pages_created,
(ulong) buf_pool->stat.n_pages_written);
/* Count the number of blocks belonging to each index in the buffer */
@ -3744,10 +3758,9 @@ buf_print_io(
{
time_t current_time;
double time_elapsed;
ulint size;
ulint n_gets_diff;
ut_ad(buf_pool);
size = buf_pool->curr_size;
buf_pool_mutex_enter();
@ -3755,12 +3768,14 @@ buf_print_io(
"Buffer pool size %lu\n"
"Free buffers %lu\n"
"Database pages %lu\n"
"Old database pages %lu\n"
"Modified db pages %lu\n"
"Pending reads %lu\n"
"Pending writes: LRU %lu, flush list %lu, single page %lu\n",
(ulong) size,
(ulong) buf_pool->curr_size,
(ulong) UT_LIST_GET_LEN(buf_pool->free),
(ulong) UT_LIST_GET_LEN(buf_pool->LRU),
(ulong) buf_pool->LRU_old_len,
(ulong) UT_LIST_GET_LEN(buf_pool->flush_list),
(ulong) buf_pool->n_pend_reads,
(ulong) buf_pool->n_flush[BUF_FLUSH_LRU]
@ -3772,37 +3787,66 @@ buf_print_io(
current_time = time(NULL);
time_elapsed = 0.001 + difftime(current_time,
buf_pool->last_printout_time);
buf_pool->last_printout_time = current_time;
fprintf(file,
"Pages made young %lu, not young %lu\n"
"%.2f youngs/s, %.2f non-youngs/s\n"
"Pages read %lu, created %lu, written %lu\n"
"%.2f reads/s, %.2f creates/s, %.2f writes/s\n",
(ulong) buf_pool->n_pages_read,
(ulong) buf_pool->n_pages_created,
(ulong) buf_pool->n_pages_written,
(buf_pool->n_pages_read - buf_pool->n_pages_read_old)
(ulong) buf_pool->stat.n_pages_made_young,
(ulong) buf_pool->stat.n_pages_not_made_young,
(buf_pool->stat.n_pages_made_young
- buf_pool->old_stat.n_pages_made_young)
/ time_elapsed,
(buf_pool->n_pages_created - buf_pool->n_pages_created_old)
(buf_pool->stat.n_pages_not_made_young
- buf_pool->old_stat.n_pages_not_made_young)
/ time_elapsed,
(buf_pool->n_pages_written - buf_pool->n_pages_written_old)
(ulong) buf_pool->stat.n_pages_read,
(ulong) buf_pool->stat.n_pages_created,
(ulong) buf_pool->stat.n_pages_written,
(buf_pool->stat.n_pages_read
- buf_pool->old_stat.n_pages_read)
/ time_elapsed,
(buf_pool->stat.n_pages_created
- buf_pool->old_stat.n_pages_created)
/ time_elapsed,
(buf_pool->stat.n_pages_written
- buf_pool->old_stat.n_pages_written)
/ time_elapsed);
if (buf_pool->n_page_gets > buf_pool->n_page_gets_old) {
fprintf(file, "Buffer pool hit rate %lu / 1000\n",
n_gets_diff = buf_pool->stat.n_page_gets - buf_pool->old_stat.n_page_gets;
if (n_gets_diff) {
fprintf(file,
"Buffer pool hit rate %lu / 1000,"
" young-making rate %lu / 1000 not %lu / 1000\n",
(ulong)
(1000 - ((1000 * (buf_pool->n_pages_read
- buf_pool->n_pages_read_old))
/ (buf_pool->n_page_gets
- buf_pool->n_page_gets_old))));
(1000 - ((1000 * (buf_pool->stat.n_pages_read
- buf_pool->old_stat.n_pages_read))
/ (buf_pool->stat.n_page_gets
- buf_pool->old_stat.n_page_gets))),
(ulong)
(1000 * (buf_pool->stat.n_pages_made_young
- buf_pool->old_stat.n_pages_made_young)
/ n_gets_diff),
(ulong)
(1000 * (buf_pool->stat.n_pages_not_made_young
- buf_pool->old_stat.n_pages_not_made_young)
/ n_gets_diff));
} else {
fputs("No buffer pool page gets since the last printout\n",
file);
}
buf_pool->n_page_gets_old = buf_pool->n_page_gets;
buf_pool->n_pages_read_old = buf_pool->n_pages_read;
buf_pool->n_pages_created_old = buf_pool->n_pages_created;
buf_pool->n_pages_written_old = buf_pool->n_pages_written;
/* Statistics about read ahead algorithm */
fprintf(file, "Pages read ahead %.2f/s,"
" evicted without access %.2f/s\n",
(buf_pool->stat.n_ra_pages_read
- buf_pool->old_stat.n_ra_pages_read)
/ time_elapsed,
(buf_pool->stat.n_ra_pages_evicted
- buf_pool->old_stat.n_ra_pages_evicted)
/ time_elapsed);
/* Print some values to help us with visualizing what is
happening with LRU eviction. */
@ -3814,6 +3858,7 @@ buf_print_io(
buf_LRU_stat_sum.io, buf_LRU_stat_cur.io,
buf_LRU_stat_sum.unzip, buf_LRU_stat_cur.unzip);
buf_refresh_io_stats();
buf_pool_mutex_exit();
}
@ -3825,10 +3870,7 @@ buf_refresh_io_stats(void)
/*======================*/
{
buf_pool->last_printout_time = time(NULL);
buf_pool->n_page_gets_old = buf_pool->n_page_gets;
buf_pool->n_pages_read_old = buf_pool->n_pages_read;
buf_pool->n_pages_created_old = buf_pool->n_pages_created;
buf_pool->n_pages_written_old = buf_pool->n_pages_written;
buf_pool->old_stat = buf_pool->stat;
}
/*********************************************************************//**