When compressed myisam files are opened, they are always memory mapped
sometimes causing memory swapping problems.
When we mmap the myisam compressed tables of size greater than the memory
available, the kswapd0 process utilization is very high consuming 30-40% of
the cpu. This happens only with linux kernels older than 2.6.9
With newer linux kernels, we don't have this problem of high cpu consumption
and this option may not be required.
The option 'myisam_mmap_size' is added to limit the amount of memory used for
memory mapping of myisam files. This option is not dynamic.
The default value on 32 bit system is 4294967295 bytes and on 64 bit system it
is 18446744073709547520 bytes.
Note: Testcase only tests the option variable. The actual bug has be to
tested manually.
include/my_global.h:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
define SIZE_T_MAX
include/myisam.h:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
declare 'myisam_mmap_size' and 'myisam_mmap_used' variables and the mutex
THR_LOCK_myisam_mmap
myisam/mi_packrec.c:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
add 'myisam_mmap_size' option which limits the memory available to mmap of
myisam files
myisam/mi_static.c:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
declare 'myisam_mmap_size' and 'myisam_mmap_used' variables and the mutex
THR_LOCK_myisam_mmap
myisam/myisamdef.h:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
move MEMMAP_EXTRA_MARGIN to myisam.h so that it can be used in mysqld.cc
mysql-test/r/variables.result:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
Testcase for BUG#37408 to test the myisam_mmap_size option
mysql-test/t/variables.test:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
Testcase for BUG#37408 to test the myisam_mmap_size option
mysys/my_thr_init.c:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
intialize the lock THR_LOCK_myisam_mmap
sql/mysqld.cc:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
add the 'myisam_mmap_size' option
sql/set_var.cc:
Fix for Bug #37408 - Compressed MyISAM files should not require/use mmap()
add the 'myisam_mmap_size' to the SHOW VARIABLES list
returns incorrect results with where
An outer join of a const table (outer) and a normal table
(inner) with GROUP BY on a field from the outer table would
optimize away GROUP BY, and thus trigger the optimization to
do away with a temporary table if grouping was performed on
columns from the const table, hence executing the query with
filesort without temporary table. But this should not be
done if there is a non-indexed access to the inner table,
since filesort does not handle joins. It expects either ref
access, range ditto or table scan. The join condition will
thus not be applied.
Fixed by always forcing execution with temporary table in
the case of ROLLUP with a query involving an outer join. This
is a slightly broader class of queries than need fixing, but
it is hard to ascertain the position of a ROLLUP field wrt
outer join with current query representation.
mysql-test/r/join_outer.result:
Bug#47650: Test result
mysql-test/t/join_outer.test:
Bug#47650: Test case
sql/sql_select.cc:
Bug#47650: Fix
Problem: inserting a record we don't set unused null bits in the
record buffer if no default field values used.
That may lead to wrong live checksum calculation.
Fix: set unused null bits in the record buffer in such cases.
mysql-test/r/myisam.result:
Fix for bug#49465: valgrind warnings and incorrect live checksum...
- test result.
mysql-test/t/myisam.test:
Fix for bug#49465: valgrind warnings and incorrect live checksum...
- test case.
sql/sql_insert.cc:
Fix for bug#49465: valgrind warnings and incorrect live checksum...
- set unused null bits to 1 in the record buffer in case we
don't call restore_record() before a fill_record() call
(when no default values used).
The bug is caused by a race condition between the
INSERT DELAYED thread and the client thread's FLUSH TABLE. The
FLUSH TABLE does not guarantee (as is (wrongly) suggested in the
test case) that the INSERT DELAYED is ever executed. The
execution of the test case will thus not be deterministic.
The fix has been to do a deterministic verification that both
threads are complete by checking the content of the table.
- Fix PBXT test results (PBXT doesn't support MRR or ICP, but we get result
diffs because we've also backported a fix that
- prints out "Using where" when the table has part of WHERE that it has
got from LEFT JOIN's ON expression
- Does a better job at removing equalities that are guaranteed to be true
by use of ref acccess.
This test case tests a circular replication of four hosts.
A--->B--->C--->D--->A
The replicate is slow and needs more time to replicate all data in the circle.
The time it spends to replicate, sometimes, is longer than the time that
wait_condition.inc spends to wait that all data has been replicated. This
cause sporadical failure of this test case.
This patch uses sync_slave_with_master to ensure that all data can be replicated
successfully in the circle.
WL#2474 "Multi Range Read: Change the default MRR implementation to implement new MRR interface"
WL#2475 "Batched range read functions for MyISAM/InnoDb"
"Index condition pushdown for MyISAM/InnoDB"
- Adjust test results (checked)
- Code cleanup.
int join_read_key(JOIN_TAB*)
The eq_ref access method TABLE_REF (accessed through
JOIN_TAB) to save state and to track if this is the
first row it finds or not.
This state was not reset on subquery re-execution
causing an assert.
Fixed by resetting the state before the subquery
re-execution.
NULLable BIGINT and INT columns in comparison
Problem: a consequence of the fix for 43668.
Some Arg_comparator inner initialization missed,
that may lead to unpredictable (wrong) comparison
results.
Fix: always properly initialize Arg_comparator
before its usage.
mysql-test/r/select.result:
Fix for bug#49517: Inconsistent behavior while using
NULLable BIGINT and INT columns in comparison
-test result.
mysql-test/t/select.test:
Fix for bug#49517: Inconsistent behavior while using
NULLable BIGINT and INT columns in comparison
-test case.
sql/item_cmpfunc.cc:
Fix for bug#49517: Inconsistent behavior while using
NULLable BIGINT and INT columns in comparison
- now all Arg_comparator::set_cmp_func() set
Arg_comparator::set_null to ensure its proper initialization
in all cases (by default it's set to TRUE in constructors).
sql/item_cmpfunc.h:
Fix for bug#49517: Inconsistent behavior while using
NULLable BIGINT and INT columns in comparison
- now all Arg_comparator::set_cmp_func() set
Arg_comparator::set_null to ensure its proper initialization
in all cases (by default it's set to TRUE in constructors).
int join_read_key(JOIN_TAB*)
The eq_ref access method TABLE_REF (accessed through
JOIN_TAB) to save state and to track if this is the
first row it finds or not.
This state was not reset on subquery re-execution
causing an assert.
Fixed by resetting the state before the subquery
re-execution.
Problem: add_collation did not check that cs->number is smaller
than the number of elements in the array all_charsets[],
so server could crash when loading an Index.xml file with
a collation ID greater the number of elements
(for example when downgrading from 5.5).
Fix: adding a condition to check that cs->number is not out of valid range.
WL#2474 "Multi Range Read: Change the default MRR implementation to implement new MRR interface"
WL#2475 "Batched range read functions for MyISAM/InnoDb"
"Index condition pushdown for MyISAM/InnoDB"
Igor's fix from sp1r-igor@olga.mysql.com-20080330055902-07614:
There could be observed the following problems:
1. EXPLAIN did not mention pushdown conditions from on expressions in the
'extra' column. As a result if a query had no where conditions pushed
down to a table, but had on conditions pushed to this table the 'extra'
column in the EXPLAIN for the table missed 'using where'.
2. Conditions for ref access were not eliminated from on expressions
though such conditions were eliminated from the where condition.
'LOAD DATA CONCURRENT [LOCAL] INFILE ...' statment only is binlogged as
'LOAD DATA [LOCAL] INFILE ...' in SBR and MBR. As a result, if replication is on,
queries on slaves will be blocked by the replication SQL thread.
This patch write code to write 'CONCURRENT' into the log event if 'CONCURRENT' option
is in the original statement in SBR and MBR.
timestamp primary key
Since TIMESTAMP values are adjusted by the current time zone
settings in both numeric and string contexts, using any
expressions involving TIMESTAMP values as a
(sub)partitioning function leads to undeterministic behavior of
partitioned tables. The effect may vary depending on a storage
engine, it can be either incorrect data being retrieved or
stored, or an assertion failure. The root cause of this is the
fact that the calculated partition ID may differ from a
previously calculated ID for the same data due to timezone
adjustments of the partitioning expression value.
Fixed by disabling any expressions involving TIMESTAMP values
to be used in partitioning functions with the follwing two
exceptions:
1. Creating or altering into a partitioned table that violates
the above rule is not allowed, but opening existing such tables
results in a warning rather than an error so that such tables
could be fixed.
2. UNIX_TIMESTAMP() is the only way to get a
timezone-independent value from a TIMESTAMP column, because it
returns the internal representation (a time_t value) of a
TIMESTAMP argument verbatim. So UNIX_TIMESTAMP(timestamp_column)
is allowed and should be used to fix existing tables if one
wants to use TIMESTAMP columns with partitioning.
mysql-test/r/partition_bug18198.result:
Corrected the error.
mysql-test/r/partition_error.result:
Corrected error texts.
Added test cases for bug #42849.
mysql-test/t/partition_bug18198.test:
Corrected error code.
mysql-test/t/partition_error.test:
Corrected error codes.
Added test cases for bug #42849.
sql/item.h:
Added is_timezone_dependent_processor() to Item.
sql/item_func.h:
Added has_timestamp_args() and the implementation of
is_timezone_dependent_processor() for Item_func.
sql/item_timefunc.h:
Added is_timezone_dependent_processor() to
Item_func_unix_timestamp.
sql/share/errmsg.txt:
Renamed ER_CONST_EXPR_IN_PARTITION_FUNC_ERROR to
ER_WRONG_EXPR_IN_PARTITION_FUNC_ERROR to better reflect the
meaning. Adjusted the error message.
sql/sql_partition.cc:
Modified fix_fields_part_func() to walk through partitioning
expression tree with is_timezone_dependent_processor() and issue
a warning/error if it depends on the timezone settings.
Changed fix_fields_part_func() to a static function since it is
not used anywhere except sql_partition.cc
sql/sql_partition.h:
Removed the unneeded declaration of fix_fields_part_func()
since it is now a static function.
sql/sql_yacc.yy:
ER_CONST_EXPR_IN_PARTITION_FUNC_ERROR ->
ER_WRONG_EXPR_IN_PARTITION_FUNC_ERROR.
with temporary table and partitions
It was possible to create temporary partitioned tables
via create table ... like ... (which is not allowed with
create temporary table). This lead to a new HA_EXTRA flag
(HA_EXTRA_MMAP) was sent to the partitioning handler,
which was caught on an assert in debug builds.
Solution was to check for partitioned tables when
doing create table ... like ... and disallow it.
mysql-test/r/partition_error.result:
Bug#49477: Assertion `0' failed in ha_partition.cc:5530
with temporary table and partitions
Added result
mysql-test/t/partition_error.test:
Bug#49477: Assertion `0' failed in ha_partition.cc:5530
with temporary table and partitions
Added test
sql/sql_table.cc:
Bug#49477: Assertion `0' failed in ha_partition.cc:5530
with temporary table and partitions
Added check to prevent creation of partitioned temporary
tables.
Only copy .par file for partitioned tables.
This fix changes the character set used within the
IBMDB2I handler to hash table names to information
about open tables. Previously, tables with names
that differed only in letter case would hash to the
same data structure. This caused incorrect behavior
or errors when two such tables were in use simultaneously.
mysql-test/suite/ibmdb2i/r/ibmdb2i_bug_49329.result:
Bug#49329 example (and other) engines use wrong collation for open tables hash
Result file for the test case.
mysql-test/suite/ibmdb2i/t/ibmdb2i_bug_49329.test:
Bug#49329 example (and other) engines use wrong collation for open tables hash
Test case for the bug fix.
storage/ibmdb2i/ha_ibmdb2i.cc:
Bug#49329 example (and other) engines use wrong collation for open tables hash
change the character set used within the IBMDB2I
handler to hash table names to information about
open tables.
SPATIAL and FULLTEXT indexes don't support algorithm
selection.
Disabled by creating a special grammar rule for these
in the parser.
Added some encasulation of duplicate parser code.
A few problems were found in the fix for bug 43668:
1) Comparison of the YEAR column with NULL always returned TRUE;
2) Comparison of the YEAR column with constants always returned
unpredictable result;
3) Unnecessary conversion warnings when comparing a non-integer
constant with a NULL value in the YEAR column;
The problems described above have been resolved with an
exception: zero (i.e. invalid) YEAR column value comparison
with 00 or 2000 still fail (it is not a regression and it was
not a regression), so MIN/MAX on YEAR column containing zero
value still fail.
mysql-test/r/type_year.result:
Test case for bug #49480.
mysql-test/t/type_year.test:
Test case for bug #49480.
sql/item_cmpfunc.cc:
- The get_year_value() function has been modified to make its
return value compatible with the get_datetime_value() return
value (i.e. to convert numeric values into the YYYY0000000000
(YYYY-00-00 00:00:00) form.
- The Arg_comparator::set_cmp_func method has been modified to
use the get_year_value function if get_datetime_value() is not
applicable.
From now only 2 cases have a special processing there:
* both comparing items have MYSQL_TYPE_YEAR field type
or
* one item have is MYSQL_TYPE_YEAR and other one is
is_datetime()-compliant.
- New helper function try_year_cmp_func() has been
added for the better code readability to call from
Arg_comparator::set_cmp_func().
- The Arg_comparator::compare_year method has been removed
since get_year_value() is compatible with the old
Arg_comparator::compare_datetime method that doesn't have
problems #1-#3 (see whole patch entry commentary).
sql/item_cmpfunc.h:
- New helper function try_year_cmp_func() has been
added for the better code readability to call from
Arg_comparator::set_cmp_func().
- Unnecessary Arg_comparator::year_as_datetime and
Arg_comparator::compare_year() declarations have been
removed.
mysql-test/include/kill_query.inc:
Error 1034 can be generated when change MyISAM table indexes was interrupted
mysql-test/r/rpl_killed_ddl.result:
table t4 may not exists because the ALTER above was interrupted
mysql-test/t/rpl_killed_ddl.test:
table t4 may not exists because the ALTER above was interrupted
Arg_comparator uses Item_cache objects to store constants being compared when
they're need a type conversion. Because this cache wasn't initialized properly
Arg_comparator might produce wrong comparison result.
The Arg_comparator::cache_converted_constant function now initializes cache
prior to usage.
mysql-test/r/select.result:
Added a test case for he bug#49489.
mysql-test/t/select.test:
Added a test case for he bug#49489.
sql/item_cmpfunc.cc:
Bug#49489: Uninitialized cache led to a wrong result.
The Arg_comparator::cache_converted_constant function now initializes cache
prior to usage.
