Problem description:
When a view is created using function FORMAT and if FORMAT function uses locale
option,definition of view saved into server doesn't contain that locale information,
Ex:
create table test2 (bb decimal (10,2));
insert into test2 values (10.32),(10009.2),(12345678.21);
create view test3 as select format(bb,1,'sk_SK') as cc from test2;
select * from test3;
+--------------+
| cc |
+--------------+
| 10.3 |
| 10,009.2 |
| 12,345,678.2 |
+--------------+
3 rows in set (0.02 sec)
show create view test3
View: test3
Create View: CREATE ALGORITHM=UNDEFINED DEFINER=`root`@`localhost`
SQL SECURITY DEFINER VIEW `test3` AS select format(`test2`.`bb`,1) AS `cc`
from `test2`
character_set_client: latin1
collation_connection: latin1_swedish_ci
1 row in set (0.02 sec)
Problem Analysis:
The function Item_func_format::print() which prints the query string to create
the view does not print the third argument (i.e the locale information). Hence
view is created without locale information.
Problem Solution:
If argument count is more than 2 we now print the third argument onto the query string.
Files changed:
sql/item_strfunc.cc
Function call changes: Item_func_format::print()
mysql-test/t/select.test
Added test case to test the bug
mysql-test/r/select.result
Result of the test case appended here
Amendment to previous patch:
Failure in CONV() should return NULL instead of
empty set.
When compiled on Windows or Solaris the function
Item_func_conv::val_str() doesn't fail on
longlong2str() but finds an earlier exit path
based on the attributes of the arguments.
This exit path returns NULL on failure and as a
consequence the original patch caused different
test results depending on the OS used.
Failure to check the return state of a longlong2str() call
caused a crash. This could happen if a user executed the sql
function CONV() with certain parameters.
The patch fixes the issue by checking that the returned pointer
isn't NULL.
In sql_class.cc, 'row_count', of type 'ha_rows', was used as last argument for
ER_TRUNCATED_WRONG_VALUE_FOR_FIELD which is
"Incorrect %-.32s value: '%-.128s' for column '%.192s' at row %ld".
So 'ha_rows' was used as 'long'.
On SPARC32 Solaris builds, 'long' is 4 bytes and 'ha_rows' is 'longlong' i.e. 8 bytes.
So the printf-like code was reading only the first 4 bytes.
Because the CPU is big-endian, 1LL is 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x01
so the first four bytes yield 0. So the warning message had "row 0" instead of
"row 1" in test outfile_loaddata.test:
-Warning 1366 Incorrect string value: '\xE1\xE2\xF7' for column 'b' at row 1
+Warning 1366 Incorrect string value: '\xE1\xE2\xF7' for column 'b' at row 0
All error-messaging functions which internally invoke some printf-life function
are potential candidate for such mistakes.
One apparently easy way to catch such mistakes is to use
ATTRIBUTE_FORMAT (from my_attribute.h).
But this works only when call site has both:
a) the format as a string literal
b) the types of arguments.
So:
func(ER(ER_BLAH), 10);
will silently not be checked, because ER(ER_BLAH) is not known at
compile time (it is known at run-time, and depends on the chosen
language).
And
func("%s", a va_list argument);
has the same problem, as the *real* type of arguments is not
known at this site at compile time (it's known in some caller).
Moreover,
func(ER(ER_BLAH));
though possibly correct (if ER(ER_BLAH) has no '%' markers), will not
compile (gcc says "error: format not a string literal and no format
arguments").
Consequences:
1) ATTRIBUTE_FORMAT is here added only to functions which in practice
take "string literal" formats: "my_error_reporter" and "print_admin_msg".
2) it cannot be added to the other functions: my_error(),
push_warning_printf(), Table_check_intact::report_error(),
general_log_print().
To do a one-time check of functions listed in (2), the following
"static code analysis" has been done:
1) replace
my_error(ER_xxx, arguments for substitution in format)
with the equivalent
my_printf_error(ER_xxx,ER(ER_xxx), arguments for substitution in
format),
so that we have ER(ER_xxx) and the arguments *in the same call site*
2) add ATTRIBUTE_FORMAT to push_warning_printf(),
Table_check_intact::report_error(), general_log_print()
3) replace ER(xxx) with the hard-coded English text found in
errmsg.txt (like: ER(ER_UNKNOWN_ERROR) is replaced with
"Unknown error"), so that a call site has the format as string literal
4) this way, ATTRIBUTE_FORMAT can effectively do its job
5) compile, fix errors detected by ATTRIBUTE_FORMAT
6) revert steps 1-2-3.
The present patch has no compiler error when submitted again to the
static code analysis above.
It cannot catch all problems though: see Field::set_warning(), in
which a call to push_warning_printf() has a variable error
(thus, not replacable by a string literal); I checked set_warning() calls
by hand though.
See also WL 5883 for one proposal to avoid such bugs from appearing
again in the future.
The issues fixed in the patch are:
a) mismatch in types (like 'int' passed to '%ld')
b) more arguments passed than specified in the format.
This patch resolves mismatches by changing the type/number of arguments,
not by changing error messages of sql/share/errmsg.txt. The latter would be wrong,
per the following old rule: errmsg.txt must be as stable as possible; no insertions
or deletions of messages, no changes of type or number of printf-like format specifiers,
are allowed, as long as the change impacts a message already released in a GA version.
If this rule is not followed:
- Connectors, which use error message numbers, will be confused (by insertions/deletions
of messages)
- using errmsg.sys of MySQL 5.1.n with mysqld of MySQL 5.1.(n+1)
could produce wrong messages or crash; such usage can easily happen if
installing 5.1.(n+1) while /etc/my.cnf still has --language=/path/to/5.1.n/xxx;
or if copying mysqld from 5.1.(n+1) into a 5.1.n installation.
When fixing b), I have verified that the superfluous arguments were not used in the format
in the first 5.1 GA (5.1.30 'bteam@astra04-20081114162938-z8mctjp6st27uobm').
Had they been used, then passing them today, even if the message doesn't use them
anymore, would have been necessary, as explained above.
(SUBSTRING inside a stored function works too slow).
Background:
- THD classes derives from Query_arena, thus inherits the 'state'
attribute and related operations (is_stmt_prepare() & co).
- Although these operations are available in THD, they must not
be used. THD has its own attribute to point to the active
Query_arena -- stmt_arena.
- So, instead of using thd->is_stmt_prepare(),
thd->stmt_arena->is_stmt_prepare() must be used. This was the root
cause of Bug 60025.
This patch enforces the proper way of calling those operations.
is_stmt_prepare() & co are declared as private operations
in THD (thus, they are hidden from being called on THD instance).
The patch tries to minimize changes in 5.5.
Problem: comparison of a DATETIME sp variable and NOW()
led to Illegal mix of collations error when
character_set_connection=utf8.
Introduced by "WL#2649 Number-to-string conversions".
Error happened in Arg_comparator::set_compare_func(),
because the first argument was errouneously converted to utf8,
while the second argument was not.
Fix: separate agg_arg_charsets_for_comparison() into two functions:
- agg_arg_charsets_for_comparison() - for pure comparison,
when we don't need to return any string result and therefore
don't need to convert arguments to @@character_set_connection:
SELECT a = b;
- agg_arg_charsets_for_string_results_with_comparison() - when
we need to return a string result, but we also need to do
comparison internally: SELECT REPLACE(a,b,c)
If all arguments are numbers:
SELECT REPLACE(123,2,3) -> 133
we convert arguments to @@character_set_connection.
@ mysql-test/include/ctype_numconv.inc
@ mysql-test/r/ctype_binary.result
@ mysql-test/r/ctype_cp1251.result
@ mysql-test/r/ctype_latin1.result
@ mysql-test/r/ctype_ucs.result
@ mysql-test/r/ctype_utf8.result
Adding tests
@ sql/item.cc
@ sql/item.h
@ sql/item_func.cc
@ sql/item_func.h
@ sql/item_strfunc.cc
Introducing and using new function
agg_item_charsets_for_string_result_with_comparison() and
its Item_func wrapper agg_arg_charsets_for_string_result_with_comparison().
Part 2. Function QUOTE() was not multi-byte safe.
@ mysql-test/r/ctype_ucs.result
@ mysql-test/t/ctype_ucs.test
Adding tests
@ sql/item_strfunc.cc
Fixing Item_func_quote::val_str to be multi-byte safe.
@ sql/item_strfunc.h
Multiple size needed for quote characters to mbmaxlen
Problem: wrong character set pointer was passed to my_strtoll10_mb2,
which led to DBUG_ASSERT failure in some cases.
@ mysql-test/r/func_encrypt_ucs2.result
@ mysql-test/t/func_encrypt_ucs2.test
@ mysql-test/r/ctype_ucs.result
@ mysql-test/t/ctype_ucs.test
Adding tests
@ sql/item_func.cc
"cs" initialization was wrong (res does not necessarily point to &str_value)
@ sql/item_strfunc.cc
Item_func_dec_encrypt::val_str() and Item_func_des_descrypt::val_str()
did not set character set for tmp_value (the returned value),
so the old value, which was previously copied from args[1]->val_str(),
was incorrectly returned with tmp_value.
Introduced by the fix for bug#44766.
Problem: it's not correct to use args[0]->str_value as a buffer,
because args[0] may need this buffer for its own purposes.
Fix: adding a new class member tmp_value to use as return value.
@ mysql-test/r/ctype_many.result
@ mysql-test/t/ctype_many.test
Adding tests
@ sql/item_strfunc.cc
Changing code into traditional style:
use "str" as a buffer for the argument and tmp_value for the result value.
@ sql/item_strfunc.h
Adding tmp_value
other crashes
Some string manipulating SQL functions use a shared string object intended to
contain an immutable empty string. This object was used by the SQL function
SUBSTRING_INDEX() to return an empty string when one argument was of the wrong
datatype. If the string object was then modified by the sql function INSERT(),
undefined behavior ensued.
Fixed by instead modifying the string object representing the function's
result value whenever string manipulating SQL functions return an empty
string.
Relevant code has also been documented.
Problem: DATE_ADD() is a hybrid function and can return
DATE, DATETIME or VARCHAR data type depending on arguments.
In case of VARCHAR data type, DATE_ADD() reported "binary" character set,
which was wrong.
Fix: make DATE_ADD() return @character_set_connection in VARCHAR context.
@ mysql-test/include/ctype_numconv.inc
Adding tests
@ mysql-test/r/ctype_binary.result
Adding tests
@ mysql-test/r/ctype_cp1251.result
Adding tests
@ mysql-test/r/ctype_latin1.result
Adding tests
@ mysql-test/r/ctype_ucs.result
Adding tests
@ mysql-test/r/ctype_utf8.result
Adding tests
@ sql/item_strfunc.cc
- Moving code from Item_str_ascii_func::val_str() to
Item_str_func::val_str_from_val_str_ascii(), as
this code needs to be shared by Item_date_add_interval.
- Adding str2 parameter to be used as a buffer, instead of
using private ascii_buf member.
@ sql/item_strfunc.h
- Moving code from Item_str_ascii_func::val_str() to
Item_str_func::val_str_from_val_str_ascii()
- Removing "String *val_str_convert_from_ascii(String *str, String *ascii_buf)"
prototype as it was neither used nor declared.
@ sql/item_timefunc.h
- Overwriting parent's charset_for_protocol() method,
becase we need to behave differenlty in VARCHAR and DATE/DATETYPE context.
- Adding ascii_buf for conversion.
- Adding val_str_ascii() prototype.
- Adding val_str() which uses newly added
Item_str_func::val_str_from_val_str_ascii(),
passing ascii_buf as a conversion buffer.
Problem: LIKE over an indexed column optimized away good results,
because my_like_range_utf32/utf16 returned wrong ranges for contractions.
Contraction related code was missing in my_like_range_utf32/utf16,
but did exist in my_like_range_ucs2/utf8.
It was forgotten in utf32/utf16 versions (during mysql-6.0 push/revert mess).
Fix:
The patch removes individual functions my_like_range_ucs2,
my_like_range_utf16, my_like_range_utf32 and introduces a single function
my_like_range_generic() instead. The new function handles contractions
correctly. It can handle any character set with cs->min_sort_char and
cs->max_sort_char represented in Unicode code points.
added:
@ mysql-test/include/ctype_czech.inc
@ mysql-test/include/ctype_like_ignorable.inc
@ mysql-test/r/ctype_like_range.result
@ mysql-test/t/ctype_like_range.test
Adding tests
modified:
@ include/m_ctype.h
- Adding helper functions for contractions.
- Prototypes: removing ucs2,utf16,utf32 functions, adding generic function.
@ mysql-test/r/ctype_uca.result
@ mysql-test/r/ctype_utf16_uca.result
@ mysql-test/r/ctype_utf32_uca.result
@ mysql-test/t/ctype_uca.test
@ mysql-test/t/ctype_utf16_uca.test
@ mysql-test/t/ctype_utf32_uca.test
- Adding tests.
@ strings/ctype-mb.c
- Pad function did not put the last character.
- Implementing my_like_range_generic() - an universal replacement
for three separate functions
my_like_range_ucs2(), my_like_range_utf16() and my_like_range_utf32(),
with correct contraction handling.
@ strings/ctype-ucs2.c
- my_fill_mb2 did not put the high byte, as previously
it was used to put only characters in ASCII range.
Now it puts high byte as well
(needed to pupulate cs->max_sort_char correctly).
- Adding DBUG_ASSERT()
- Removing character set specific functions:
my_like_range_ucs2(), my_like_range_utf16() and my_like_range_utf32().
- Using my_like_range_generic() instead of the old functions.
@ strings/ctype-uca.c
- Using generic function instead of the old character set specific ones.
@ sql/item_create.cc
@ sql/item_strfunc.cc
@ sql/item_strfunc.h
- Adding SQL functions LIKE_RANGE_MIN and LIKE_RANGE_MAX,
available only in debug build to make sure like_range()
works correctly for all character sets and collations.
Bug#57820 extractvalue crashes
Problem: ExtractValue and Replace crashed in some cases
due to invalid handling of empty and NULL arguments.
Per file comments:
@mysql-test/r/ctype_ujis.result
@mysql-test/r/xml.result
@mysql-test/t/ctype_ujis.test
@mysql-test/t/xml.test
Adding tests
@sql/item_strfunc.cc
Make sure Item_func_replace::val_str safely handles empty strings.
@sql/item_xmlfunc.cc
set null_value if nodeset_func returned NULL,
which is possible when the second argument is an
unset user variable.
Problem: if multibyte and binary string arguments passed to
RPAD(), LPAD() or INSERT() functions, they might return
wrong results or even lead to a server crash due to missed
character set convertion.
Fix: perform the convertion if necessary.
Problems:
- dot character was always printed as decimal point
instead of localized decimal point for short
numbers without thousands
- Item_func_format::val_str always returned values in ASCII
format,
regargless of @@character_set_connection, which in case of utf32
led to crash in debug build, or to incorrect values in release build.
Fix:
- Adding a piece of code to replace dot character to
localized decimal point in short numbers.
- Changing parent class for Item_func_format to
Item_str_ascii_func, because its val_str() implementation is heavily ASCII oriented.
Problem: a few functions did not calculate their max_length correctly.
This is an after-fix for WL#2649 Number-to-string conversions".
Fix: changing the buggy functions to calculate max_length
using fix_char_length() introduced in WL#2649,
instead of setting max_length directly
mysql-test/include/ctype_numconv.inc
Adding new tests
mysql-test/r/ctype_binary.result
Adding new tests
mysql-test/r/ctype_cp1251.result
Adding new tests
mysql-test/r/ctype_latin1.result
Adding new tests
mysql-test/r/ctype_ucs.result
Adding new tests
mysql-test/r/ctype_utf8.result
Adding new tests
mysql-test/t/ctype_utf8.test
Including ctype_numconv
sql/item.h
- Introducing new method fix_char_length_ulonglong(),
for the cases when length is potentially greater
than UINT_MAX32. This method removes a few
instances of duplicate code, e.g. in item_strfunc.cc.
- Setting collation in Item_copy properly. This change
fixes wrong metadata on client side in some cases, when
"binary" instead of the real character set was reported.
sql/item_cmpfunc.cc
- Using fix_char_length() and max_char_length() methods,
instead of direct access to max_length, to calculate
item length properly.
- Moving count_only_length() in COALESCE after
agg_arg_charsets_for_string_result(). The old
order was incorrect and led to wrong length
calucation in case of multi-byte character sets.
sql/item_func.cc
Fixing that count_only_length() didn't work
properly for multi-byte character sets.
Using fix_char_length() and max_char_length()
instead of direct access to max_length.
sql/item_strfunc.cc
- Using fix_char_length(), fix_char_length_ulonglong(),
max_char_length() instead of direct access to max_length.
- Removing wierd condition: "if (collation.collation->mbmaxlen > 0)",
which is never FALSE.
Problem: Item_func_hex::val_str() returned data in ASCII format,
which did not match collation.collation pointing to my_charset_utf32_general_ci.
Fix: changing parent class of Item_func_hex to Item_str_ascii_func,
as val_str() implementation is heavily ASCII-oriented.
mysql-test/r/ctype_utf32.result
mysql-test/t/ctype_utf32.test
Adding test case
sql/item_strfunc.cc
sql/item_strfunc.h
- Changing parent class to Item_str_ascii_func
- Clean-up in Item_func_hex::fix_length_and_dec()
Using fix_char_length() instead of setting max_length directly.
Problem: Item_str_ascii_func::val_str() did not set
charset of the returned value properly.
mysql-test/include/ctype_numconv.inc
mysql-test/r/ctype_binary.result
mysql-test/r/ctype_cp1251.result
mysql-test/r/ctype_latin1.result
mysql-test/r/ctype_ucs.result
- Adding tests
sql/item_strfunc.cc
- Adding initialization of charset
Problem: sha2() reported its result as BINARY
Fix:
- Inheriting Item_func_sha2 from Item_str_ascii_func
- Setting max_length via fix_length_and_charset()
instead of direct assignment.
- Adding tests
Conflicts:
Text conflict in mysql-test/r/explain.result
Text conflict in mysql-test/t/explain.test
Text conflict in sql/net_serv.cc
Text conflict in sql/sp_head.cc
Text conflict in sql/sql_priv.h
Iterative patch improvement. Previously committed patch
caused wrong result on Windows. The previous patch also
broke secure_file_priv for symlinks since not all file
paths which must be compared against this variable are
normalized using the same norm.
The server variable opt_secure_file_priv wasn't
normalized properly and caused the operations
LOAD DATA INFILE .. INTO TABLE ..
and
SELECT load_file(..)
to do different interpretations of the
--secure-file-priv option.
The patch moves code to the server initialization
routines so that the path always is normalized
once and only once.
It was also intended that setting the option
to an empty string should be equal to
lifting all previously set restrictions. This
is also fixed by this patch.
The bug was a side effect of WL#5030 (fix header files) and
WL#5161 (CMake).
The problem was that CMake-generated config.h (and my_config.h
as a copy of it) had a header guard. GNU autotools-generated
[my_]config.h did not. During WL#5030 the order of header files
was changed, so the following started to happen (using GNU autotools,
in embedded server):
- my_config.h included, defining HAVE_OPENSSL
- my_global.h included, un-defining HAVE_OPENSSL
- zlib.h included, including config.h,
defining HAVE_OPENSSL again.
The fix is to check HAVE_OPENSSL in conjuction with EMBEDDED_LIBRARY.
More common fix would be to define a macros as HAVE_OPENSSL && !EMBEDDED_LIBRARY
and use it instead of HAVE_OPENSSL.
The bug was a side effect of WL#5030 (fix header files) and
WL#5161 (CMake).
The problem was that CMake-generated config.h (and my_config.h
as a copy of it) had a header guard. GNU autotools-generated
[my_]config.h did not. During WL#5030 the order of header files
was changed, so the following started to happen (using GNU autotools,
in embedded server):
- my_config.h included, defining HAVE_OPENSSL
- my_global.h included, un-defining HAVE_OPENSSL
- zlib.h included, including config.h,
defining HAVE_OPENSSL again.
The fix is to change the order of header file, moving zlib.h
to the top of the header list. More proper fix would be to wrap
unguarded auto-generated [my_]config.h by guarded non-generated
header file.
ChangeSet@1.2703, 2007-12-07 09:35:28-05:00, cmiller@zippy.cornsilk.net +40 -0
Bug#13174: SHA2 function
Patch contributed from Bill Karwin, paper unnumbered CLA in Seattle
Implement SHA2 functions.
Chad added code to make it work with YaSSL. Also, he removed the
(probable) bug of embedded server never using SSL-dependent
functions. (libmysqld/Makefile.am didn't read ANY autoconf defs.)
Function specification:
SHA2( string cleartext, integer hash_length )
-> string hash, or NULL
where hash_length is one of 224, 256, 384, or 512. If either is
NULL or a length is unsupported, then the result is NULL. The
resulting string is always the length of the hash_length parameter
or is NULL.
Include the canonical hash examples from the NIST in the test
results.
---
Polish and address concerns of reviewers.
Procedure, while DECIMAL works
Selecting of the CONCAT(...<SP variable>...) result into
a user variable may return wrong data.
Item_func_concat::val_str contains a number of memory
allocation-saving tricks. One of them concatenates
strings inplace inserting the value of one string
at the beginning of the other string. However,
this trick didn't care about strings those points
to the same data buffer: this is possible when
a CONCAT() parameter is a stored procedure variable -
Item_sp_variable::val_str() uses the intermediate
Item_sp_variable::str_value field, where it may
store a reference to an external buffer.
The Item_func_concat::val_str function has been
modified to take into account val_str functions
(such as Item_sp_variable::val_str) that return
a pointer to an internal Item member variable
that may reference to a buffer provided.
