RESULTING MY_WC_T RESULT IS NOT USED
Issue : handler functions my_ismbchar_utf8,
my_well_formed_len_mb for charset utf8
is calling unicode converion function
to validate and to find the character
length. Because of this, instructions
which will convert the utf8 to unicode
are executed for no use.
A similar issue exist with charset utf8mb4
Solution : reorganized the code such that character
validation part of unicode conversion
handler is extracted(duplicated) in to
separate function. Hence
my_ismbchar_utf8, my_well_formed_len_mb
will call the new function which only
validates and return the length of mb(utf8).
A similar fix for charset utf8mb4.
strings/ctype-utf8.c:
New functions has been added for charset utf8 and utf8mb4
to validate and to get the length of the character.
Problem:-
We have created a table with UTF8_BIN collation.
In case, when in our query we have ORDER BY clause over a function
call we are getting result in incorrect order.
Note:the bug is not there in 5.5.
Analysis:
In 5.5, for UTF16_BIN, we have min and max multi-byte length is 2 and 4
respectively.In make_sortkey(),for 2 byte character character we are
assuming that the resultant length will be 2 byte/character. But when we
use my_strnxfrm_unicode_full_bin(), we store sorting weights using 3 bytes
per character.This result in truncated result.
Same thing happen for UTF8MB4, where we have 1 byte min multi-byte and
4 byte max multi-byte.We will accsume resultant data as 1 byte/character,
which result in truncated result.
Solution:-
use strnxfrm(means use of MY_CS_STRNXFRM macro) is used for sort, in
which the resultant length is not dependent on source length.
HANDLE_FATAL_SIGNAL IN STRNLEN
Fixed the following bounds checking problems :
1. in check_if_legal_filename() make sure the null terminated
string is long enough before accessing the bytes in it.
Prevents pottential read-past-buffer-end
2. in my_wc_mb_filename() of the filename charset check
for the end of the destination buffer before sending single
byte characters into it.
Prevents write-past-end-of-buffer (and garbaling stack in
the cases reported here) errors.
Added test cases.
mysql-test/suite/innodb/t/group_commit_crash.test:
remove autoincrement to avoid rbr being used for insert ... select
mysql-test/suite/innodb/t/group_commit_crash_no_optimize_thread.test:
remove autoincrement to avoid rbr being used for insert ... select
mysys/my_addr_resolve.c:
a pointer to a buffer is returned to the caller -> the buffer cannot be on the stack
mysys/stacktrace.c:
my_vsnprintf() is ok here, in 5.5
Introducing new collations:
utf8_general_mysql500_ci and ucs2_general_mysql500_ci,
to reproduce behaviour of utf8_general_ci and ucs2_general_ci
from mysql-5.1.23 (and earlier).
The collations are added to simplify upgrade from mysql-5.1.23 and earlier.
Note: The patch does not make new server start over old data automatically.
Some manual upgrade procedures are assumed.
Paul: please get in touch with me to discuss upgrade procedures
when documenting this bug.
modified:
include/m_ctype.h
mysql-test/r/ctype_utf8.result
mysql-test/t/ctype_utf8.test
mysys/charset-def.c
strings/ctype-ucs2.c
strings/ctype-utf8.c
sql/sql_insert.cc:
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
******
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
small cleanup
******
small cleanup
Added strings_def.h into strings library to be able to have a DBUG_ASSERT() version without _db_flush() call (as strings.a should not depend on dbug.a)
Remove include of m_string.h in all string files (as it's included by string_def.h).
Fixed include order.
Changed "m_ctype.h" -> <m_ctype.h>
include/my_dbug.h:
Flush DBUG log in case of DBUG_ASSERT()
strings/bchange.c:
Include strings_def.h
strings/bcmp.c:
Include strings_def.h
strings/bfill.c:
Include strings_def.h
strings/bmove.c:
Include strings_def.h
strings/bmove512.c:
Include strings_def.h
strings/bmove_upp.c:
Include strings_def.h
strings/conf_to_src.c:
Include strings_def.h
Fixed copyright
strings/ctype-big5.c:
Include strings_def.h
strings/ctype-bin.c:
Include strings_def.h
strings/ctype-cp932.c:
Include strings_def.h
strings/ctype-czech.c:
Include strings_def.h
strings/ctype-euc_kr.c:
Include strings_def.h
strings/ctype-eucjpms.c:
Include strings_def.h
strings/ctype-extra.c:
Include strings_def.h
strings/ctype-gbk.c:
Include strings_def.h
strings/ctype-latin1.c:
Include strings_def.h
strings/ctype-mb.c:
Include strings_def.h
strings/ctype-simple.c:
Include strings_def.h
strings/ctype-sjis.c:
Include strings_def.h
strings/ctype-tis620.c:
Include strings_def.h
strings/ctype-uca.c:
Include strings_def.h
strings/ctype-ucs2.c:
Include strings_def.h
strings/ctype-ujis.c:
Include strings_def.h
strings/ctype-utf8.c:
Include strings_def.h
strings/ctype-win1250ch.c:
Include strings_def.h
strings/ctype.c:
Include strings_def.h
strings/decimal.c:
Include strings_def.h
strings/do_ctype.c:
Include strings_def.h
strings/int2str.c:
Include strings_def.h
strings/is_prefix.c:
Include strings_def.h
strings/llstr.c:
Include strings_def.h
strings/longlong2str.c:
Include strings_def.h
strings/longlong2str_asm.c:
Include strings_def.h
strings/my_strchr.c:
Include strings_def.h
strings/my_strtoll10.c:
Include strings_def.h
strings/my_vsnprintf.c:
Include strings_def.h
strings/r_strinstr.c:
Include strings_def.h
strings/str2int.c:
Include strings_def.h
strings/str_alloc.c:
Include strings_def.h
strings/str_test.c:
Include strings_def.h
Fixed compiler warnings
strings/strappend.c:
Include strings_def.h
strings/strcend.c:
Include strings_def.h
strings/strcont.c:
Include strings_def.h
strings/strend.c:
Include strings_def.h
strings/strfill.c:
Include strings_def.h
strings/strinstr.c:
Include strings_def.h
strings/strmake.c:
Include strings_def.h
strings/strmov.c:
Include strings_def.h
strings/strmov_overlapp.c:
Include strings_def.h
strings/strnlen.c:
Include strings_def.h
strings/strnmov.c:
Include strings_def.h
strings/strstr.c:
Include strings_def.h
strings/strto.c:
Include strings_def.h
strings/strtod.c:
Include strings_def.h
strings/strtol.c:
Include strings_def.h
strings/strtoll.c:
Include strings_def.h
strings/strtoul.c:
Include strings_def.h
strings/strtoull.c:
Include strings_def.h
strings/strxmov.c:
Include strings_def.h
strings/strxnmov.c:
Include strings_def.h
strings/uctypedump.c:
Include strings_def.h
Fixed compiler warnings
Removed double include of m_ctype.h
strings/udiv.c:
Include strings_def.h
strings/xml.c:
Include strings_def.h
Post-fix:
Reverting the "utf16_bin did not sort supplementary characters
between U+D700 and U+E000" part. We'll use code-point order.
Committing on behalf of Alexander Barkov.
Original changeset:
------------------------------------------------------------
revno: 3197
revision-id: alik@sun.com-20100831135426-h5a4s2w6ih1d8q2x
parent: magnus.blaudd@sun.com-20100830120632-u3xzy002mdwueli8
committer: Alexander Nozdrin <alik@sun.com>
branch nick: mysql-5.5-bugfixing
timestamp: Tue 2010-08-31 17:54:26 +0400
message:
Bug#55980 Character sets: supplementary character _bin ordering is wrong
Problem:
- ORDER BY for utf8mb4_bin, utf16_bin and utf32_bin returned
results in a wrong order, because old functions
(supporting only BMP range) were used to handle these collations.
- Additionally, utf16_bin did not sort supplementary characters
between U+D700 and U+E000, as WL#1213 specification specified.
------------------------------------------------------------
Fixed valgrind warnings found from running main.connect under valgrind
sql/sp_head.cc:
Ensure that vcol_info is reset
sql/sql_acl.cc:
Fixed usage of wrong memroot for password
sql/sql_yacc.yy:
Ensure that vcol_info is reset
Disable pbxt for test cases not using pbxt (speeds up test suite)
extra/comp_err.c:
Added cast to get rid of compiler warning
extra/libevent/kqueue.c:
Added cast to get rid of compiler warning
mysql-test/lib/mtr_cases.pm:
Use --skip-pbxt for test cases that doesn't need pbxt
Collect default-storage-engine from suite.opt file (should actually be my.cnf file, but that wasn't easy to do)
mysql-test/suite/pbxt/t/suite.opt:
Added marker for mysql-test-run.pl that this suite require pbxt
mysys/mf_keycache.c:
Use LINT_INIT() to remove compiler warnings
mysys/my_gethostbyname.c:
Remove compiler warnings
sql/handler.cc:
Reset variable that may be used uninitialized
sql/item.h:
Remove compiler warning
sql/mysqld.cc:
Use LINT_INIT() to remove compiler warnings
sql/sql_class.h:
Remove compiler warning
sql/sql_table.cc:
Ensure variable is always set (to remove compiler warning)
sql/sql_view.cc:
Use LINT_INIT() to remove compiler warnings
storage/maria/ma_loghandler.c:
Use LINT_INIT() to remove compiler warnings
storage/myisammrg/ha_myisammrg.cc:
Fixed wrong type to printf
storage/myisammrg/myrg_open.c:
Use LINT_INIT() to remove compiler warnings
storage/xtradb/include/ut0lst.h:
Trivial change of macro to remove compiler warning
strings/ctype-ucs2.c:
Use LINT_INIT() to remove compiler warnings
strings/ctype-utf8.c:
Use LINT_INIT() to remove compiler warnings
support-files/compiler_warnings.supp:
Suppress some not relevant warnings
unittest/mysys/waiting_threads-t.c:
Don't use ftruncate() as this gives warning about ignored return value
Bug#51675 Server crashes on inserting 4 byte char. after ALTER TABLE to 'utf8mb4'
Bug#51676 Server crashes on SELECT, ORDER BY on 'utf8mb4' column
include/m_ctype.h:
Defining MY_CS_REPLACEMENT_CHARACTER
mysql-test/r/ctype_utf8mb4.result:
Adding tests
mysql-test/t/ctype_utf8mb4.test:
Adding tests
strings/ctype-uca.c:
Don't use UCA data for characters higher than 0xFFFF.
strings/ctype-ucs2.c:
Using newly defined MY_CS_REPLACEMENT_CHARACTER
strings/ctype-utf8.c:
Using newly defined MY_CS_REPLACEMENT_CHARACTER
Removing unesed variable "plane".
