This is a pre-requisite patch for:
- MDEV-8433 Make field<'broken-string' use indexes
- MDEV-8625 Bad result set with ignorable characters when using a prefix key
- MDEV-8626 Bad result set with contractions when using a prefix key
Adding a new virtual function MY_CHARSET_HANDLER::copy_abort().
Moving character set specific code into the correspoding implementations
(for simple, multi-byte and mbmaxlen>1 character sets).
The problem was that my_hash_sort didn't properly delete end-space characters properly, so strings that should compare
identically was seen as different strings. (Space was handled correctly, but not NBSP)
This caused duplicate key errors when a heap table was converted to Aria as part of overflow in group by.
Fixed by removing all characters that compares as end space when creating a hash.
Other things:
- Fixed that --sorted_results also works for errors in mysqltest.
- Speed up hash by not comparing strings that has different hash.
- Speed up many my_hash_sort functions by using registers to calculate hash instead of pointers.
This was previously done for some functions, but not for all.
- Made a macro of the hash function, to simplify code and to be able to experiment with new hash functions.
client/mysqltest.cc:
Fixed that --sorted_results also works for error messages.
mysql-test/r/ctype_partitions.result:
New test to ensure that partitions on hash works
mysql-test/suite/multi_source/gtid.result:
Updated result
mysql-test/suite/multi_source/gtid.test:
Test that --sorted_result works for error messages
mysql-test/suite/multi_source/gtid_ignore_duplicates.result:
Updated result
mysql-test/suite/multi_source/gtid_ignore_duplicates.test:
Updated result
mysql-test/suite/multi_source/load_data.result:
Updated result
mysql-test/suite/multi_source/load_data.test:
Updated result
mysql-test/t/ctype_partitions.test:
New test to ensure that partitions on hash works
storage/heap/hp_write.c:
Speed up hash by not comparing strings that has different hash.
storage/maria/ma_check.c:
Extra debug
strings/ctype-bin.c:
Use macro for hash function
strings/ctype-latin1.c:
Use macro for hash function
Use registers to calculate hash (speedup)
strings/ctype-mb.c:
Use macro for hash function
Use registers to calculate hash (speedup)
strings/ctype-simple.c:
Use macro for hash function
Use same variable names as in other my_hash_sort functions.
Update my_hash_sort_simple() to properly remove end space (patch by Bar)
strings/ctype-uca.c:
Ignore duplicated space inside strings and end space in my_hash_sort_uca(). This fixed MDEV-6255
Use macro for hash function
Use registers to calculate hash (speedup)
strings/ctype-ucs2.c:
Use macro for hash function
Use registers to calculate hash (speedup)
strings/ctype-utf8.c:
Use macro for hash function
Use registers to calculate hash (speedup)
strings/strings_def.h:
Made a macro of the hash function, to simplify code and to be able to experiment with new hash functions.
includes:
* remove some remnants of "Bug#14521864: MYSQL 5.1 TO 5.5 BUGS PARTITIONING"
* introduce LOCK_share, now LOCK_ha_data is strictly for engines
* rea_create_table() always creates .par file (even in "frm-only" mode)
* fix a 5.6 bug, temp file leak on dummy ALTER TABLE
- Avoid needless load/stores in my_hash_sort_simple due to possible aliasing
- Avoid expensive Join_plan_state constructor in choose_subquery_plan when no subquery
- Avoid calling update_virtual_fields for every row when no virtual fields.
sql/sql_insert.cc:
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
******
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
small cleanup
******
small cleanup
Added strings_def.h into strings library to be able to have a DBUG_ASSERT() version without _db_flush() call (as strings.a should not depend on dbug.a)
Remove include of m_string.h in all string files (as it's included by string_def.h).
Fixed include order.
Changed "m_ctype.h" -> <m_ctype.h>
include/my_dbug.h:
Flush DBUG log in case of DBUG_ASSERT()
strings/bchange.c:
Include strings_def.h
strings/bcmp.c:
Include strings_def.h
strings/bfill.c:
Include strings_def.h
strings/bmove.c:
Include strings_def.h
strings/bmove512.c:
Include strings_def.h
strings/bmove_upp.c:
Include strings_def.h
strings/conf_to_src.c:
Include strings_def.h
Fixed copyright
strings/ctype-big5.c:
Include strings_def.h
strings/ctype-bin.c:
Include strings_def.h
strings/ctype-cp932.c:
Include strings_def.h
strings/ctype-czech.c:
Include strings_def.h
strings/ctype-euc_kr.c:
Include strings_def.h
strings/ctype-eucjpms.c:
Include strings_def.h
strings/ctype-extra.c:
Include strings_def.h
strings/ctype-gbk.c:
Include strings_def.h
strings/ctype-latin1.c:
Include strings_def.h
strings/ctype-mb.c:
Include strings_def.h
strings/ctype-simple.c:
Include strings_def.h
strings/ctype-sjis.c:
Include strings_def.h
strings/ctype-tis620.c:
Include strings_def.h
strings/ctype-uca.c:
Include strings_def.h
strings/ctype-ucs2.c:
Include strings_def.h
strings/ctype-ujis.c:
Include strings_def.h
strings/ctype-utf8.c:
Include strings_def.h
strings/ctype-win1250ch.c:
Include strings_def.h
strings/ctype.c:
Include strings_def.h
strings/decimal.c:
Include strings_def.h
strings/do_ctype.c:
Include strings_def.h
strings/int2str.c:
Include strings_def.h
strings/is_prefix.c:
Include strings_def.h
strings/llstr.c:
Include strings_def.h
strings/longlong2str.c:
Include strings_def.h
strings/longlong2str_asm.c:
Include strings_def.h
strings/my_strchr.c:
Include strings_def.h
strings/my_strtoll10.c:
Include strings_def.h
strings/my_vsnprintf.c:
Include strings_def.h
strings/r_strinstr.c:
Include strings_def.h
strings/str2int.c:
Include strings_def.h
strings/str_alloc.c:
Include strings_def.h
strings/str_test.c:
Include strings_def.h
Fixed compiler warnings
strings/strappend.c:
Include strings_def.h
strings/strcend.c:
Include strings_def.h
strings/strcont.c:
Include strings_def.h
strings/strend.c:
Include strings_def.h
strings/strfill.c:
Include strings_def.h
strings/strinstr.c:
Include strings_def.h
strings/strmake.c:
Include strings_def.h
strings/strmov.c:
Include strings_def.h
strings/strmov_overlapp.c:
Include strings_def.h
strings/strnlen.c:
Include strings_def.h
strings/strnmov.c:
Include strings_def.h
strings/strstr.c:
Include strings_def.h
strings/strto.c:
Include strings_def.h
strings/strtod.c:
Include strings_def.h
strings/strtol.c:
Include strings_def.h
strings/strtoll.c:
Include strings_def.h
strings/strtoul.c:
Include strings_def.h
strings/strtoull.c:
Include strings_def.h
strings/strxmov.c:
Include strings_def.h
strings/strxnmov.c:
Include strings_def.h
strings/uctypedump.c:
Include strings_def.h
Fixed compiler warnings
Removed double include of m_ctype.h
strings/udiv.c:
Include strings_def.h
strings/xml.c:
Include strings_def.h
Removed compiler warnings
extra/libevent/epoll.c:
Removed compiler warnings
extra/libevent/evbuffer.c:
Removed compiler warnings
extra/libevent/event.c:
Removed compiler warnings
extra/libevent/select.c:
Removed compiler warnings
extra/libevent/signal.c:
Removed compiler warnings
include/m_ctype.h:
Define CHARSET_INFO, MY_CHARSET_HANDLER, MY_COLLATION_HANDLER, MY_UNICASE_INFO, MY_UNI_CTYPE and MY_UNI_IDX as const structures.
Declare that pointers point to const data
include/m_string.h:
Declare that pointers point to const data
include/my_sys.h:
Redefine variables and function prototypes
include/mysql.h:
Declare charset as const
include/mysql.h.pp:
Declare charset as const
include/mysql/plugin.h:
Declare charset as const
include/mysql/plugin.h.pp:
Declare charset as const
mysys/charset-def.c:
Charset can't be of type CHARSET_INFO as they are changed when they are initialized.
mysys/charset.c:
Functions that change CHARSET_INFO must use 'struct charset_info_st'
Add temporary variables to not have to change all_charsets[] (Which now is const)
sql-common/client.c:
Added cast to const
sql/item_cmpfunc.h:
Added cast to avoid compiler error.
sql/sql_class.cc:
Added cast to const
sql/sql_lex.cc:
Added cast to const
storage/maria/ma_ft_boolean_search.c:
Added cast to avoid compiler error.
storage/maria/ma_ft_parser.c:
Added cast to avoid compiler error.
storage/maria/ma_search.c:
Added cast to const
storage/myisam/ft_boolean_search.c:
Added cast to avoid compiler error
storage/myisam/ft_parser.c:
Added cast to avoid compiler error
storage/myisam/mi_search.c:
Added cast to const
storage/pbxt/src/datadic_xt.cc:
Added cast to const
storage/pbxt/src/ha_pbxt.cc:
Added cast to const
Removed compiler warning by changing prototype of XTThreadPtr()
storage/pbxt/src/myxt_xt.h:
Character sets should be const
storage/pbxt/src/xt_defs.h:
Character sets should be const
storage/xtradb/btr/btr0cur.c:
Removed compiler warning
strings/conf_to_src.c:
Added const
Functions that change CHARSET_INFO must use 'struct charset_info_st'
strings/ctype-big5.c:
Made arrays const
strings/ctype-bin.c:
Made arrays const
strings/ctype-cp932.c:
Made arrays const
strings/ctype-czech.c:
Made arrays const
strings/ctype-euc_kr.c:
Made arrays const
strings/ctype-eucjpms.c:
Made arrays const
strings/ctype-extra.c:
Made arrays const
strings/ctype-gb2312.c:
Made arrays const
strings/ctype-gbk.c:
Made arrays const
strings/ctype-latin1.c:
Made arrays const
strings/ctype-mb.c:
Made arrays const
strings/ctype-simple.c:
Made arrays const
strings/ctype-sjis.c:
Made arrays const
strings/ctype-tis620.c:
Made arrays const
strings/ctype-uca.c:
Made arrays const
strings/ctype-ucs2.c:
Made arrays const
strings/ctype-ujis.c:
Made arrays const
strings/ctype-utf8.c:
Made arrays const
strings/ctype-win1250ch.c:
Made arrays const
strings/ctype.c:
Made arrays const
Added cast to const
Functions that change CHARSET_INFO must use 'struct charset_info_st'
strings/int2str.c:
Added cast to const
(From: gkodinov)
Use and int * where possible to scan for trailing space in a
string instead of always iterating char-by-char.
Using the attached benchmark file on a 32 bit Intel Core 2
Duo CPU I've got 43485 ms run with the fix compared to 44373
without it.
Backported to 5.6.0 (next-mr-runtime)
6.0-codebase revid: 2476.1362.1
include/m_string.h:
scan for space through ints
strings/ctype-bin.c:
scan for space through ints
strings/ctype-latin1.c:
scan for space through ints
strings/ctype-mb.c:
scan for space through ints
strings/ctype-simple.c:
scan for space through ints
In MySQL when the mapping for space is changed to something other than
0x20 by defining a different collation, then space is not ignored when
comparing two strings.
This was happening because the function that performs the comparison
of two strings while ignoring ending spaces, was comparing the collation
value of a space with the ascii value of the ' ' character. This should
be changed to do comparison between the collated values.
mysql-test/r/ctype_ldml.result:
Bug#46448 trailing spaces are not ignored when user collation maps space != 0x20
Result file for test case.
mysql-test/std_data/Index.xml:
Bug#46448 trailing spaces are not ignored when user collation maps space != 0x20
Added entry for new test collation in the index file.
mysql-test/std_data/latin1.xml:
Bug#46448 trailing spaces are not ignored when user collation maps space != 0x20
Added support for new collation for test.
mysql-test/t/ctype_ldml.test:
Bug#46448 trailing spaces are not ignored when user collation maps space != 0x20
Added test case to ensure trailing spaces are not ignored.
strings/ctype-simple.c:
Bug#46448 trailing spaces are not ignored when user collation maps space != 0x20
change my_strnncollsp_simple to compare collated values when checking
for trailing spaces. Currently the comparison happens between a collated
value and the ascii value.
- Remove bothersome warning messages. This change focuses on the warnings
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
An integer overflow in number->string conversion caused completely
wrong output of the number LONGLONG_MIN with gcc 4.2.1.
Fixed by eliminating the overflow, using only operations that are
well-defined in ANSI C.
strings/ctype-simple.c:
An integer overflow in number->string conversion caused completely
wrong output of the number LONGLONG_MIN with gcc 4.2.1.
Fixed by eliminating the overflow, using only operations that are
well-defined in ANSI C.
strings/ctype-ucs2.c:
An integer overflow in number->string conversion caused completely
wrong output of the number LONGLONG_MIN with gcc 4.2.1.
Fixed by eliminating the overflow, using only operations that are
well-defined in ANSI C.
strings/int2str.c:
An integer overflow in number->string conversion caused completely
wrong output of the number LONGLONG_MIN with gcc 4.2.1.
Fixed by eliminating the overflow, using only operations that are
well-defined in ANSI C.
strings/longlong2str.c:
An integer overflow in number->string conversion caused completely
wrong output of the number LONGLONG_MIN with gcc 4.2.1.
Fixed by eliminating the overflow, using only operations that are
well-defined in ANSI C.
into polly.(none):/home/kaa/src/maint/bug30453/my51-bug28878
mysql-test/r/insert.result:
Auto merged
mysql-test/t/insert.test:
Auto merged
strings/ctype-simple.c:
Manual merge.
Problem:
my_strntoull10rnd_8bit() handled incorrectly cases when the input
string contains a decimal point and is long enough to overrun the
'unsigned long long' type. The position of the decimal point was not
taken into account which resulted in miscalculated numbers and
truncation to appropriate SQL data type limits.
Solution:
Fix my_strntoull10rnd_8bit() to take the position of a decimal point
into account in such cases.
mysql-test/r/insert.result:
Added a test case for bug #30453.
mysql-test/t/insert.test:
Added a test case for bug #30453.
strings/ctype-simple.c:
In cases when the 'unsigned long long' type is overrun by the input
string and a decimal point has occurred, adjust the 'shift' according to
the position of the decimal point and skip all subsequent digits.
into magare.gmz:/home/kgeorge/mysql/work/B29325-merge-5.1-opt
client/mysql.cc:
Auto merged
include/my_base.h:
Auto merged
mysql-test/r/ctype_collate.result:
Auto merged
mysql-test/r/gis-rtree.result:
Auto merged
mysql-test/t/gis-rtree.test:
Auto merged
storage/myisam/mi_create.c:
Auto merged
strings/ctype-simple.c:
Auto merged
mysys/my_conio.c:
5.0-opt -> 5.1.-opt merge
storage/myisam/sp_key.c:
5.0-opt -> 5.1.-opt merge
with the space character.
When the my_strnncollsp_simple function compares two strings and one is a prefix
of another then this function compares characters in the rest of longer key
with the space character to find whether the longer key is greater or less.
But the sort order of the collation isn't used in this comparison. This may
lead to a wrong comparison result, wrongly created index or wrong order of the
result set of a query with the ORDER BY clause.
Now the my_strnncollsp_simple function uses collation sort order to compare
the characters in the rest of longer key with the space character.
mysql-test/t/ctype_collate.test:
Added a test case for the bug#29461: Sort order of the collation wasn't used when
comparing characters with the space character.
mysql-test/r/ctype_collate.result:
Added a test case for the bug#29461: Sort order of the collation wasn't used when
comparing characters with the space character.
strings/ctype-simple.c:
Bug#29461: Sort order of the collation wasn't used when comparing characters
with the space character.Now the my_strnncollsp_simple function uses collation sort order to compare
the characters in the rest of longer key with the space character.
into olga.mysql.com:/home/igor/dev-opt/mysql-5.1-opt
mysql-test/r/binary.result:
Auto merged
mysql-test/r/ctype_collate.result:
Auto merged
mysql-test/r/ctype_utf8.result:
Auto merged
mysql-test/t/binary.test:
Auto merged
mysql-test/t/create.test:
Auto merged
mysql-test/t/ctype_utf8.test:
Auto merged
sql/field_conv.cc:
Auto merged
sql/item.cc:
Auto merged
sql/item_subselect.cc:
Auto merged
sql/item_sum.cc:
Auto merged
sql/item_sum.h:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql-common/client.c:
Auto merged
sql/sql_select.h:
Auto merged
sql/table.cc:
Auto merged
storage/myisam/mi_open.c:
Auto merged
strings/ctype-simple.c:
Auto merged
mysql-test/r/create.result:
Manual merge.
mysql-test/r/subselect.result:
Manual merge.
mysql-test/r/type_enum.result:
Manual merge.
mysql-test/t/type_enum.test:
Manual merge.
mysql-test/include/mix1.inc:
Manual merge.
mysql-test/r/innodb_mysql.result:
Manual merge.
mysql-test/t/subselect.test:
Manual merge.
sql/sql_parse.cc:
Manual merge.
storage/myisam/mi_key.c:
Manual merge.
