This merges the patch series that fixes a lot of missing validation
checks for read_log_event(). The merge includes a few added fixes that
are needed in new code introduced in 13.1.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
This merges the patch series that fixes a lot of missing validation
checks for read_log_event(). The merge includes a few added fixes that
are needed in new code introduced in 12.3.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
This SQL script failed:
SET NAMES latin1 COLLATE latin1_swedish_ci;
CREATE OR REPLACE VIEW v1 AS SELECT 'Y' AS c1;
SET NAMES big5 COLLATE big5_chinese_ci;
SELECT * FROM v1 WHERE c1='y';
with the following error:
ERROR 1267 (HY000): Illegal mix of collations
(latin1_swedish_ci,COERCIBLE) and (big5_chinese_ci,COERCIBLE)
for operation '='
Note, latin1_swedish_ci and big5_chinese_ci are used here as examples.
The error also happened with different collation combinations.
Fix main idea:
If two collations have equal comparison rules (known as "tailoring")
on a given character repertoire,
like latin1_swedish_ci and big5_chinese_ci on ASCII letters,
then the "Illegal mix of collations" error can be avoided in
a comparison operator. We can choose any of the sides as the operation
effective collation - the result will be equal.
This optimization is not applied when at least one side has an explicit
COLLATE clause. Two explicit COLLATE clauses in one comparison are
already illegal when the character sets are the same, so for
consistency this stays illegal when the character sets differ too.
Most important details:
- Splitting enum_repertoire_t into smaller subsets,
for better repertoire granularity.
A variable holding a repertoire value can now have multiple
MY_REPERTOIRE_XXX flags set.
This patch implements detecting tailoring equality
on these repertoires:
* MY_REPERTOIRE_ASCII_DIGITS - [0..9]
* MY_REPERTOIRE_ASCII_MINUS_DOT_DIGITS - [-.0..9], e.g. '-10.5'.
It consists of the new bits MY_REPERTOIRE_ASCII_MINUS and
MY_REPERTOIRE_ASCII_DOT. They were a part of
MY_REPERTOIRE_ASCII_NOT_IDENT before. As a result, the numeric values
of MY_REPERTOIRE_ASCII and MY_REPERTOIRE_UNICODE30 changed
(223 and 255).
* MY_REPERTOIRE_ASCII_ALNUM - [A..Z,a..z,0..9].
* MY_REPERTOIRE_ASCII_IDENT - ALNUM + underscore
* MY_REPERTOIRE_ASCII - the entire range U+0000..U+007F
- Adding a new virtual function "tailoring" in my_collation_handler_st.
It returns the tailoring on the given repertoire for the given
collation.
If cs1->coll->tailoring(cs1, some_repertoire) returns {0,0},
it means the illegal mix optimization cannot be used
for this collation on the given repertoire.
If these calls:
tr1= cs1->coll->tailoring(cs1, some_repertoire);
tr2= cs2->coll->tailoring(cs2, some_repertoire);
return both non-NULL results and tr1.str==tr2.str,
then these collations are equal on the given repertoire
and are mutually replaceable for a comparison operator,
so "Illegal mix of collations" can be avoided.
- Tailoring strings are shared constants my_tailoring_str_*
(strings/strings_def.h). Tailorings are compared by the string
pointer, so tailoring() implementations return strings only from
these constants. The strings do not depend on PAD/NOPAD.
- my_string_repertoire() for ucs2, utf16, utf32 and NONASCII 8bit
character sets no longer stops at the first bad byte sequence with
the repertoire found so far, which could be MY_REPERTOIRE_NONE
(compatible with everything). It returns MY_REPERTOIRE_EXTENDED
for such strings. A unit test was added into unittest/strings.
- Adding CHARSET_INFO::is_ascii_superset(). It tells if a character set
can store all ASCII characters (it is false for swe7).
It is used in Item_func_conv_charset to calculate "safe", and in
left_is_algorithmically_simpler() to prefer other character sets
to those where ASCII strings cannot be converted safely.
- DBUG_ASSERTs were added into mysys/charset.c, to check that
the "tailoring" method is set in collation handlers when a collation
is added: compiled-in, from ctype-extra.c, or loaded from Index.xml.
- Adding a new function my_collations_equal_on_repertoire(cs1, cs2,
repertoire) in strings/. It tells if two collations are equal on
the repertoire, taking tailoring() results and PAD/NOPAD and the UCA
version into account. Callers do not compare tailoring() results
directly, so the way tailorings are compared (currently by the pointer
to the shared constant string) is private to strings/ and can be
extended later, e.g. for UCA collations on MY_REPERTOIRE_UNICODE30.
- Adding a new method DTCollation::aggregate_by_tailoring().
Collations with different MY_CS_NOPAD are never compatible.
UCA collations of different versions are not compatible on
the repertoire with ASCII punctuation, as the UCA version
affects its order (e.g. UCA-6.2.0 moved GRAVE ACCENT
and CIRCUMFLEX ACCENT after PERCENT SIGN).
- DTCollation::aggregate() now makes the repertoire of the result cover
the repertoires of both sides in all successful branches.
Before, set(dt) replaced the repertoire with the repertoire of the
winner side only, so the result could have a too narrow repertoire,
e.g. IF(1, 1.5, HEX(255)) had MY_REPERTOIRE_ASCII_ALNUM although the
value '1.5' has a punctuation character. With the more granular
repertoires this could wrongly allow mixing collations which are
equal on ALNUM but different on punctuation.
- Adding a new flag MY_COLL_ALLOW_BY_TAILORING.
It indicates to DTCollation::aggregate() that the illegal
mix optimization by repertoire can be used in the given context.
MY_COLL_CMP_CONV now includes MY_COLL_ALLOW_BY_TAILORING.
Note, only comparison operators pass this flag.
Functions returning a string result do not pass this flag,
because in operations like CONCAT(a,b) we still need to evaluate
precisely the collation of the result - we cannot just choose
a collation of one of the sides (even if they are compatible
on the given repertoire). This also applies to ExtractValue()
and UpdateXML(), which now aggregate their arguments without
this flag.
- As in my_repertoire_t the value MY_REPERTOIRE_ASCII is now a set of
bits rather than a single bit, the way to detect "is only ASCII"
repertoires has changed in the code.
For example:
// repertoire *IS* ascii
if (repertoire == MY_REPERTOIRE_ASCII)
has changed in multiple places in the code to
// repertoire *HAS* only ascii characters
if (my_repertoire_is_subset_of(repertoire, MY_REPERTOIRE_ASCII))
The new function my_repertoire_is_subset_of() in m_ctype.h
is used for this purpose in C code. In C++ code
DTCollation::repertoire_is_subset_of() is used, e.g.:
if (collation.repertoire_is_subset_of(MY_REPERTOIRE_ASCII))
- Repertoire of some expressions was adjusted to the new meaning:
* Item_null now has MY_REPERTOIRE_NONE (was ASCII).
* MY_LOCALE::repertoire() now returns MY_REPERTOIRE_UNICODE30
(was EXTENDED).
* Lex_string_with_metadata_st::repertoire(cs) now scans the string
contents to detect the actual repertoire.
* HEX() now has MY_REPERTOIRE_ASCII_ALNUM.
* DATE_FORMAT() decides on MY_REPERTOIRE_EXTENDED using the locale
which is actually used: the explicit third argument, or
@@lc_time_names. It was always @@lc_time_names before.
A non-constant locale argument is assumed to be non-ASCII.
* QUOTE, MAKE_SET, EXPORT_SET, LPAD, RPAD, GROUP_CONCAT, JSON_ARRAY,
JSON_OBJECT and JSON_OBJECTAGG add the repertoire of the extra
characters they put into the result (quotes, separators, padding,
brackets).
* LOWER() and UPPER() add the letters of both cases to the repertoire,
because the result can contain letters of the opposite case.
In Turkish collations they also add MY_REPERTOIRE_EXTENDED, as an
ASCII letter can be converted to a non-ASCII one (I -> dotless i).
Unicode collations are detected by the new member
casefold_info_st::can_convert_to_non_ascii_on_casefolding,
simple 8bit collations by to_lower['I'] and to_upper['i'].
- Lex_string_with_metadata_st::repertoire(cs) now scans a string literal
for any byte oriented character set (mbminlen==1), not only for
ASCII based ones. It was assumed to have the full repertoire for swe7,
which is not ASCII based. The scan is done by Unicode code points, so
swe7 strings with digits and letters get a narrow repertoire, while
e.g. 0x5B (LATIN CAPITAL LETTER A WITH DIAERESIS in swe7) is non-ASCII.
Therefore swe7_swedish_ci can be mixed with other collations now,
in the same way as other collations with MY_CS_IDENT_CASEUP_CI.
The test func_time was adjusted: a swe7 literal ' ull' with ASCII
characters is now safe to convert, the error is still tested for '[ull'.
- cp1250_czech_cs sorts -, . and 0..9 in the code point order, so it gets
the MINUS_DOT_DIGITS tailoring. latin2_czech_cs gets only DIGITS,
as it sorts the minus and the dot differently. Both were verified
with ctype_digits_verify.
- The tis620 collations now set CHARSET_INFO::tab_to_uni (was NULL),
so my_charset_is_ascii_based() is true for tis620.
It was false for tis620, although tis620 is ASCII compatible.
Therefore tis620 string literals were always considered to have
a non-ASCII repertoire, and the repertoire optimizations did not work
for them. Now ASCII-only tis620 literals are detected as ASCII.
This changes the result of subselect_extra_no_semijoin
from an error to success.
- tis620_thai_ci and tis620_thai_nopad_ci fold letters to lower case
and then compare by the code point on the entire ASCII range
(the Thai specific rules affect only non-ASCII characters).
A new function my_tailoring_ascii_casedn_ci() returns tailorings
for such collations on ALNUM, IDENT and ASCII. For example, the
underscore sorts before the letters, unlike in the caseup tailorings.
- New flags were added for CHARSET_INFO::state
* MY_CS_ASCII_CASEUP_CI - for simple 8bit case insensitive collations.
It means that this collation has no irregularities on the ASCII
repertoire, and converts lower case letters to upper case ones,
so the underscore sorts after the letters (with an upper to lower
case conversion it would sort before the letters).
* MY_CS_IDENT_CASEUP_CI - for simple 8bit case insensitive collations.
It means that this collation has no irregularities on the IDENT
repertoire (but can have irregularities say on punctuation).
* MY_CS_ASCII_STD_UCA - for UCA collations.
It means that a UCA collation does not reorder ASCII characters.
* MY_CS_DIGITS_STD - for any collation. It means that the ASCII digits
0..9 are sorted in the code point order and are not tailored, and,
for PAD collations, the space sorts before the digits. Strings of
digits are compared in the same way in all such collations, even if
they have irregularities on letters (latin7, cp866, czech, danish,
turkish UCA, etc.), or differ by case sensitivity. It is detected
from sort_order for 8bit collations, and from the tailoring rules
for UCA collations. Collations with MY_CS_DIGITS_STD still must have
the same PAD/NOPAD attribute.
* MY_CS_ASCII_MINUS_DOT_DIGITS - for any collation. Means
MY_CS_DIGITS_STD, and additionally the minus sorts before the dot,
which sorts before the digits (and the space sorts before the minus
in PAD collations). Strings like '-10.5' are compared in the same
way in all such collations. It is detected from sort_order for simple
8bit collations (my_coll_init_simple()), and from the tailoring rules
for UCA collations. For the multi-byte collations big5_chinese_ci,
gbk_chinese_ci, gb2312_chinese_ci and sjis_japanese_ci it is declared
by my_tailoring_ident_caseup_ci_generic(). The flags MY_CS_DIGITS_STD
and MY_CS_ASCII_MINUS_DOT_DIGITS are stored in ctype-extra.c for
compiled collations (conf_to_src.c prints them), and the detection is
skipped for a collation which already has any of the repertoire flags
(MY_CS_REPERTOIRE_FLAGS).
- strings/conf_to_src.c was modified to detect and print
MY_CS_ASCII_CASEUP_CI and MY_CS_IDENT_CASEUP_CI flags.
- strings/ctype-extra.c was regenerated with new flags.
- 8bit collations are not given MY_CS_ASCII_CASEUP_CI and
MY_CS_IDENT_CASEUP_CI if they are PAD and the space does not sort
before the digits. A shorter string is padded with spaces for
comparison, so the position of the space matters even for strings
without spaces ("ab" is compared to "abc" as "ab " to "abc").
A test collation latin1_space_after_letters_ci was added to
mysql-test/std_data/ldml/ for this. ctype-extra.c does not change.
- The initializer of my_collation_cs_handler in ctype-utf8.c
(utf8mb3_general_cs, under HAVE_UTF8_GENERAL_CS) was fixed to match
my_collation_handler_st: the missing get_id and get_collation_name
members were added, so that eq_collation was not initialized in their
place, and the new member "tailoring" is now set to my_tailoring_none.
- Results of the existing tests func_str, func_test, ps, view
and subselect_sj changed from "Illegal mix of collations" to success.
In these tests ASCII-only strings were compared using collations
with equal tailoring on ASCII, e.g. latin1_swedish_ci with
latin2_general_ci (func_str, ps), koi8r_general_ci with
latin1_swedish_ci (func_test), latin1_general_ci with
latin1_swedish_ci on letters and digits (view), cp932_japanese_ci
with latin1_swedish_ci (subselect_sj).
Such comparisons are now allowed.
Cases with collations that are still incompatible on ASCII,
e.g. latin7_general_ci, were added to the tests and still fail.
- Adding a number of MTR tests in
plugin/func_test/mysql-test/func_test/.
They display a tailoring by collation name and repertoire as
returned by:
cs->coll->tailoring(cs, some_repertoire)
A dynamically linked plugin function collation_tailoring() was added
for the purpose of these tests. Tailorings do not depend on PAD/NOPAD
and on the UCA version, so the function shows "[nopad]" for NOPAD
collations, and "[version X.Y.Z]" for UCA collations on the ASCII
repertoire. The version is formatted by a new class UCAVersion,
which uses a new method CharBuffer::append_uint8().
- Adding the test plugin/func_test/mysql-test/func_test/
ctype_caseup_ci_verify.test. It checks the order of ASCII characters
for all collations declared "caseup CI" on the ASCII or IDENT range.
- Adding the test plugin/func_test/mysql-test/func_test/
collation_tailoring_args.test. It checks the arguments of the plugin
function collation_tailoring(): the repertoire passed by name or by
number, and the NULL results for unknown names.
- Adding the test plugin/func_test/mysql-test/func_test/ctype_ldml.test.
It displays the tailorings of collations loaded at runtime from
Index.xml: simple 8bit collations (flags detected from the weights)
and Unicode collations (LDML rules which do or do not reorder ASCII).
- Adding a number of MTR tests mysql-test/main/ctype_xxx_tailoring.test
They display two-dimensional charts showing which collations
are compatible on which repertoires.
Tests for DATE_FORMAT() with an explicit locale were added to
ctype_big5_tailoring.test, ctype_cp932_tailoring.test and
ctype_latin5_tailoring.test, which also has the chart for latin5.
- Adding a number of MTR tests in the form of the originally
reported script for various collations:
SELECT Insert_priv FROM mysql.user WHERE Insert_priv='...';
The repeated blocks of queries are shared through
mysql-test/include/
ctype_tailoring_0{1,2,3}_mysql_user_Insert_priv*.inc.
- Adding tests to ctype_cp932.test checking that ExtractValue() and
UpdateXML() still raise "Illegal mix of collations".
Problem: SUM(DISTINCT b'1100') returns 9 instead of 12. The hybrid
hex/bit literal handler uses the default DOUBLE temporary field, so
its byte length becomes a DOUBLE(M,0) display width and clips the
numeric value.
Fix: Override numeric DISTINCT field construction for hybrid hex and
bit literals to allocate an unsigned Field_longlong directly. Preserve
argument-based field selection for other types so DISTINCT retains the
input expression's precision and scale, including decimal AVG inputs.
Extend main.sum_distinct to cover bit and hex literals, zero, byte and
unsigned 64-bit boundaries, duplicates, AVG, COUNT and decimal scale.
Recreated-by: Georgi "Joro" Kodinov <joro@mariadb.org>
A 0xHHHH / b'..' literal keeps its hex_hybrid type through COALESCE/IF/CASE/...,
but was evaluated, sized and stored as a string, so it behaved differently from
a bare literal (10+COALESCE(0x31) gave 11, not 59).
Type_handler_hex_hybrid now implements the numeric side to match the bare
literal: val_int/val_real/val_decimal read bytes as a hybrid, Item_decimal_precision
returns the integer precision, Item_save_in_field stores like the bare literal.
The traditional type merge keeps the hybrid through a typeless NULL (as for BIT),
and Item_hybrid_func_fix_attributes attributes it binary/unsigned.
This patch to mysqldump is only merged partially:
commit 666ff1db36
Author: Monty <monty@mariadb.org>
Date: Fri May 15 16:39:35 2026 +0300
MDEV-39519 Change mariadb-dump to use SHOW REPLICA STATUS instead of SHOW SLAVE STATUS
The functionality for --dump-slave cannot be merged as the underlying
functionality was removed in this commit:
commit 21c940d599
Author: ParadoxV5 <paradox.ver5@gmail.com>
Date: Thu Dec 4 21:22:20 2025 -0700
MDEV-37146 mariadb-dump: Replace SHOW SLAVE STATUS usage with information_schema.SLAVE_STATUS
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
When setup_copy_fields() needs to copy a non-aggregate function value,
it doesn't construct an Item_copy directly. Instead, it calls
Type_handler::create_item_copy, which is a kind of factory. The base
Type_handler::create_item_copy returns Item_copy_string. Some type
handlers override it, like timestamp and fixed binary. However, the
numeric type handlers (e.g., float, double, int, decimal) did not, so
they fell through to that base and got Item_copy_string.
A SELECT that aggregates will copy each non aggregate function value
into a temporary table through an Item_copy object, whose concrete
type is chosen by the create_item_copy method on the value's type
handler. For numeric types that method returned Item_copy_string,
which stores the value as text. A FLOAT keeps only FLT_DIG
significant digits as text, too few to reproduce its 24 bit mantissa,
so the copied value differed from the original. With one row per
group, CAST(c1 AS FLOAT) - MIN(CAST(c1 AS FLOAT)) returned a large
number instead of zero.
Add Item_copy_real with Item_copy_float and Item_copy_double variants
that keep the value as a double, the same way Item_cache_real does, and
let the float and double type handlers create them. This mirrors the
existing copy items for timestamp and fixed binary types.
New optimizations were introduced that substitute virtual column
expressions (abbr. vcol expr) with index virtual columns (abbr. vcol
field) in ORDER BY and GROUP BY.
In this patch we introduce checks that the type of a vcol field is the
supertype to its vcol expr, and if not, do not proceed the
optimization. This ensures that the substitution is safe and does not
for example lose information due to truncation.
For simplicity, the version of the check implemented is strict with
100% precision, meaning that there are instances where vcol field is a
supertype to vcol expr, but the check returns false.
Type not covered in tests: Type_handler_null
Thanks to Jarir Khan <kjarir23@gmail.com> for initial patches
addressing this issue.
Type_handler_int_result::Item_decimal_precision assumes that
max_length includes space for a sign, but Item_int literals don't
include such a space. When aggregate_attributes_int propagates these
into functions via a Item_type_holder, the formula computes
precision-1 (or 0 for 1-digit cases), producing the incorrect
calculation with AVG.
This fix changes aggregate_attributes_int to follow the convention the
Type_handler expects, that max_length = max digit-count + (signed ? 1
: 0). This matches the TODO comment that begins with "TODO: rewrite
aggregate_attributes_int()... " which suggested this idea.
Adding support for the strict cursor data types:
Example 1a:
TYPE rec0_t IS RECORD (a INT, VARCHAR(10));
TYPE cur0_t IS REF CURSOR RETURN rec0_t;
Example 1b:
TYPE rec0_t IS RECORD (a t1.a%TYPE, b t1.b%TYPE);
TYPE cur0_t IS REF CURSOR RETURN rec0_t;
Example 1c:
TYPE rec0_t IS RECORD (a INT, VARCHAR(10));
r0 rec0_t;
TYPE cur0_t IS REF CURSOR RETURN r0%TYPE;
Example 1d:
TYPE rec0_t IS RECORD (a t1.a%TYPE, b t1.b%TYPE);
r0 rec0_t;
TYPE cur0_t IS REF CURSOR RETURN r0%TYPE;
Example2a:
TYPE cur0_t IS REF CURSOR RETURN t1%ROWTYPE; -- t1 is a table
Example 2b:
r0 t1%ROWTYPE;
TYPE cur0_t IS REF CURSOR RETURN r0%TYPE;
Example3a:
CURSOR cursor_sample IS SELECT a,b FROM t1;
TYPE cur0_t IS REF CURSOR RETURN cursor_sample%ROWTYPE;
Example3b:
CURSOR cursor_sample IS SELECT a,b FROM t1;
r0 cursor_sample%ROWTYPE;
TYPE cur0_t IS REF CURSOR RETURN r0%TYPE;
If a cursor variable is declared with a RETURN clause then:
1. At OPEN type the data type of the SELECT list row is compared
for compatibility with the cursor RETURN data type.
The SELECT list row must be assignable to the RETURN type row.
If case if assignability is not meet, an error is raised
Assignability means:
- The arity of the SELECT list must be equal to the arity
of the RETURN clause
- Every n-th field of the SELECT list must be assignable to the
n-th field of the RETURN Clause
2. At FETCH time, the data is fetched in two steps:
a. On the first step the data is fetched into a virtual table
with the row type described in the RETURN clause
b. On the second step the data is copied from the virtual table
to the target fetch list. Data type conversion can happen
on this step.
Change details:
Adding new methods:
- sp_cursor::check_assignability_to
- Virtual_tmp_table::check_assignability_from
- Virtual_tmp_table::sp_set_from_select_list
- Virtual_tmp_table::sp_save_in_vtable
- Virtual_tmp_table::sp_save_in_target_list
- LEX::check_ref_cursor_components
- LEX::make_sp_instr_copy_struct_for_last_context_variables
- LEX::declare_type_ref_cursor
- sp_cursor::Select_fetch_into_spvars::send_data_with_return_type
Adding new members:
- sp_instr_copen_by_ref::m_cursor_name
- Select_fetch_into_spvars::m_return_type
- Select_materialize::m_cursor_name
- Select_materialize::m_return_type
Adding new virtual methods:
- Item::resolve_spvar_cursor_rowtype
- Type_handler::Spvar_definition_resolve_type_refs
- Server_side_cursor::check_assignability_to
- Overriding Select_materialize::prepare to raise an error when the cursor
returned data type is not compatible with the RETURN clause
Making these methods virtual:
- Field::check_assignability_from
Adding new classes:
- sp_type_def_ref
- RowTypeBuffer
Adding new constructors to:
- Spvar_definition
Adding new helper methods (e.g. to reuse the code)
- Field::store_field_maybe_null
- ChanBuffer::append_ulonglong
- sp_pcontext::set_type_for_last_context_variables
Minor changes:
- Making TABLE::export_structure const
- Overriding Item_splocal::type_extra_attributes. It was forgotten in earlier changes.
Adding new error messages
- ER_CANNOT_CAST_ON_IDENT1_ASSIGNMENT_FOR_OPERATION
- ER_CANNOT_CAST_ON_IDENT2_ASSIGNMENT_FOR_OPERATION
Some data types:
- RECORD
- assoc arrays
- SYS_REFCURSOR
erroneously accepted non-relevant attributes:
- length
- scale
- character set / collation
- REF_SYSTEM_ID
Fixing to raise an error.
Details:
- Overriding get_column_attributes() in Type_handler_row,
Type_handler_sys_refcursor, Type_handler_assoc_array.
- Moving a piece of the old code into a new method
Type_handler::check_data_type_attributes(), to reuse it
for testing attributes for both plugin types and TYPE-def types.
- Reusing check_data_type_attributes() in methods
* LEX::set_field_type_udt_or_typedef()
* LEX::set_field_type_udt()
- Adding opt_binary into the rule field_type_all_with_typedefs,
into the grammar branch for the plugin types, as XMLTYPE needs it.
The method Type_handler::Column_definition_fix_attributes() was an
addition for Type_handler::Column_definition_set_attributes().
The code in virtual implementations of Column_definition_fix_attributes()
has now been joined to corresponding virtual implementations of
Column_definition_set_attributes().
After this change everything is done in Column_definition_set_attributes().
This makes the code clearer. Previous implementation was rather confusing.
For example, for temporal data types:
(1) After Column_definition_set_attributes():
- Column_definition::length meant the fractional precision
- Column_definition::decimals was set to 0
(2) After Column_definition_fix_attributes():
- Column_definition::length means the full character length
- Column_definition::decimals means the fractional precision
Now everything gets set to (2) right in Column_definition_set_attributes().
The first key point of this commit is changing Type_extra_attributes in the way
that the pointer to a generic attribute is now "Type_generic_attributes*" -
a pointer to a new class with vitrual method type_handler().
Before this change the pointer to a generic attribute was
"void *m_attr_const_void_ptr", which was error prone, as allowed
to pass a pointer to a wrong structure without control.
Another key point is that the SET/ENUM data type related structures
are now attributed by the changed class Type_typelib_attributes,
which now looks as follows:
class Type_typelib_attributes: public Sql_alloc,
public Type_generic_attributes,
public TYPELIB
(with virtual method type_handler()), instead of being attributed by
TYPELIB directly.
Details:
- Using Column_definition's method typelib() where the member m_typelib
was used directly. The related lines change anyway, because the member
is renamed m_typelib_attr.
- Adding a new class Type_generic_attributes.
Deriving the following classes from it:
* Type_typelib_attributes
* sp_type_def
- sp_type_def does not derive from Type_handler_hybrid_field_type any more,
because the method type_handler() is now implemented virtually.
- Removing Type_extra_attributes::m_attr_const_void_ptr and
adding Type_extra_attributes::m_attr_const_generic_attributes_ptr instead -
a pointer to the new class Type_generic_attributes.
Renaming methods to set and read it according to the new data type name.
- Type_typelib_attributes now derives from TYPELIB instead of having
a pointer to TYPELIB.
- Adding a new class Type_typelib_ptr_attributes. It's a replacement for
the old implementation of Type_typelib_attributes.
Instead of deriving from Type_typelib_attributes, Field_enum now derives
from Type_typelib_ptr_attributes. The latter can store/read itself into/from
Type_extra_attributes. It's a bridge between the new Type_typelib_attributes
and Type_extra_attributes.
- Changing parameter data type in a few methods in Field_enum from
a pointer to TYPELIB to a pointer to Type_typelib_attributes.
- Removing typelib related methods from Type_extra_attributes.
Moving this functionality into Type_typelib_ptr_attributes and
Column_definition_attributes.
This turns Type_extra_attributes into a data structure independent
from any data type specific structures/methods.
- Adding methods:
Column_definition_attributes::typelib_attr() - Column_definition derives it
Column_definition_attributes::typelib() - Column_definition derives it
Type_typelib_ptr_attributes::typelib_attr() - Field_enum derives it
Type_typelib_ptr_attributes::typelib() - Field_enum derives it
- Renaming the member Create_field::save_interval into
Create_field::save_typelib_attr.
Changing its data type from a pointer to TYPELIB into a pointer
to Type_typelib_attributes pointer.
- Removing the method Type_typelib_attributes::store().
Adding Type_typelib_ptr_attributes::save_in_type_extra_attributes() instead.
The new method name makes the code more readable.
Year::year_precision(): Make static, so that VYear::VYear
and VYear_op::VYear_op() will avoid invoking this function with
an unnecessary "this" pointer to an uninitialized object.
Fixing the return type of Type_handler::Item_save_in_value() from
bool to void.
Adding a new method st_value::is_null().
This makes the code less confusing.
PARTITION BY [LINEAR] KEY ALGORITHM={MYSQL51|MYSQL55|BASE31|CRC32C|XXH32|XXH3}
- The MYSQL5X algorithms are the existing algorithms, with MYSQL55
being the default
- The BASE31 algorithm uses a base-31 representation of the bytes, see
Modular hashing in https://algs4.cs.princeton.edu/34hash/. It serves
as a simple baseline that distributes better than the old mysql5x
algorithms
- CRC32C uses my_crc32c.
- XXH32 and XXH3 are xxhash algorithms - xxhash.h copied from latest
release (0.8.3) of https://github.com/Cyan4973/xxHash
For performance (esp. xxh) we use one-shot hash functions in binary
hash_sort, and streaming hash function otherwise for byte-by-byte
hashing. XXH is the only stateful hash function. The other hash
algorithms are stateless and homomorphic, so streaming and one-shot
functions are identical.
Also updated the columnstore hash due to the change of MY_HASH_ADD
signature.
The following testing commands were run at an earlier version of the
patch with the following patch that changes the default algorithm from
MYSQL55 to CRC32C, and XXH32 (changing the patch accordingly)
mtr --suite main --do-test=.*partition
mtr --suite parts
modified sql/ha_partition.cc
@@ -10336,6 +10336,8 @@ uint32 ha_partition::calculate_key_hash_value(Field **field_array)
switch ((*field_array)->table->part_info->key_algorithm)
{
case partition_info::KEY_ALGORITHM_NONE:
+ hasher.set_algorithm(HASH_ALGORITHM_CRC32C);
+ break;
case partition_info::KEY_ALGORITHM_55:
/* Hasher default to mysql55 */
break;
modified sql/partition_info.cc
@@ -2328,7 +2328,7 @@ bool partition_info::fix_parser_data(THD *thd)
if ((thd_sql_command(thd) == SQLCOM_CREATE_TABLE ||
thd_sql_command(thd) == SQLCOM_ALTER_TABLE) &&
key_algorithm == KEY_ALGORITHM_NONE)
- key_algorithm= KEY_ALGORITHM_55;
+ key_algorithm= PARTITION_INFO_DEFAULT_ALGORITHM;
}
DBUG_RETURN(FALSE);
}
@@ -2344,7 +2344,7 @@ bool partition_info::fix_parser_data(THD *thd)
if ((thd_sql_command(thd) == SQLCOM_CREATE_TABLE ||
thd_sql_command(thd) == SQLCOM_ALTER_TABLE) &&
key_algorithm == KEY_ALGORITHM_NONE)
- key_algorithm= KEY_ALGORITHM_55;
+ key_algorithm= PARTITION_INFO_DEFAULT_ALGORITHM;
}
defined_max_value= FALSE; // in case it already set (CREATE TABLE LIKE)
do
modified sql/partition_info.h
@@ -446,6 +446,8 @@ class partition_info : public DDL_LOG_STATE, public Sql_alloc
int gen_part_type(THD *thd, String *str) const;
};
+#define PARTITION_INFO_DEFAULT_ALGORITHM partition_info::KEY_ALGORITHM_CRC32C
+
void part_type_error(THD *thd, partition_info *work_part_info,
const char *part_type, partition_info *tab_part_info);
modified sql/sql_partition.cc
@@ -2471,7 +2471,7 @@ static int add_key_with_algorithm(String *str, const partition_info *part_info)
err+= str->append(STRING_WITH_LEN("KEY "));
if (part_info->key_algorithm != partition_info::KEY_ALGORITHM_NONE &&
- part_info->key_algorithm != partition_info::KEY_ALGORITHM_55)
+ part_info->key_algorithm != PARTITION_INFO_DEFAULT_ALGORITHM)
{
err+= str->append(STRING_WITH_LEN("ALGORITHM = "));
switch (part_info->key_algorithm)
@@ -2479,6 +2479,9 @@ static int add_key_with_algorithm(String *str, const partition_info *part_info)
case partition_info::KEY_ALGORITHM_51:
err+= str->append(STRING_WITH_LEN("MYSQL51"));
break;
+ case partition_info::KEY_ALGORITHM_55:
+ err+= str->append(STRING_WITH_LEN("MYSQL55"));
+ break;
case partition_info::KEY_ALGORITHM_BASE31:
err+= str->append(STRING_WITH_LEN("BASE31"));
break;
Item_singerow_subselect::fix_length_and_dec() incorrectly calculated
its Item::max_length when the underlying expression was Item_int.
The reason of the problem:
Item_int has an optimized max_length to make CONCAT(1) create a VARCHAR(1)
column rather than a VARCHAR(2) column. Its max_length does not include one
extra character for the sign in case the value is positive but the value
is not marked as Item::unsigned==true.
So copying max_length from the underlying Item_int (with value==9)
in cases like this:
SELECT CONCAT((SELECT 9 FROM t0));
was not correct.
Implementing a new virtual method
Type_handler::Item_type_std_attributes_generic(const Item *item)
- The default implementation just copies attributes from "item" as is.
- In case of Type_handler_int_result it evaluates max_length
using item->decimal_precision() rather than item->max_length.
This works correctly for both "optimized" items like Item_int and
non-"optimized" Items whose max_length includes +1 for the sign
in case of signed expressions.
all params are initialized to use Type_handler_null.
NULL params do not have a value to read from the packet buffer,
so their Item_param_set_param_func() should do nothing.
Type_handler_null inherits from Type_handler_string_result,
that is, Type_handler_null *must* override parent's
Item_param_set_param_func().
This patch implements an Oracle style function to_number()
with the following signatures:
- to_number(number_or_string_subject)
- to_number(string_subject, string_format)
The function is implemented in sql/item_numconvfunc.cc.
The function returns the DOUBLE data type for
all signatures and input data types.
The format parser understands the following components:
- Digits: 0, 9
- Hex digits: X
- Group separators: comma (,) and G
- Decimal delimiers: period (.) and D
- Approximate number signature: EEEE
- Currency/numeric flags: $ and B
- Currency signatures: C, L, U
- Sign signatures: S, MI, PR
- Special format signatures: V, TM, TM9, TME
- Format flag: FM
Note, the parser was implemented assuming that we'll also have the
oppostite function to_char() soon for numeric input.
So it parser all known components.
However, the function to_number() does not support:
- Formats V, TM, TM9, TME. to_number() returns NULL if the format
string has these components.
These componens are supported only by to_char() in Oracle.
Features not inclided into this patch:
- The ON CONVERSION ERROR clause
- The third parameter (nlsparam)
- Internationalized components: G, D, C, L, U.
These features will be implemented later, under terms of MDEV-36978.
Notable changes in the other files:
- item_func.h: adding Item_handled_func::Handler_double
- simple_parser.h: adding a number of *CONTAINER* templates
They help to save on duplicate code when creating classes
suitable for passing into parsing templates such as OPT, OR2C, OR3C, etc
- simple_parser.h: Adding parsing templates OR4C and OR5C
- simple_parser.h: Moving the template "OPT" towars the beginning of the file
Rule parsing templates TOKEN, TokenChoice, AND2, OR2C, OR3C, OR4C, OR5C, LIST
now provide a sub-class Opt, to parse its optional rule.
- simple_parser.h: Adding "explicit" to all "operator bool" definitions
- Renaming Parser_templates::TOKEN to Parser_templates::TokenParser
- Adding "explicit" to all "operator bool()" templates/classes,
to avoid hidden implicit conversion (to int, void*, etc).
- Renaming the LIST template parameter ELEMENT to ELEMENT_PARSER,
to make it clearer what it is for.
- Renaming the OPT template parameter RULE to RULE_PARSER,
to make it clearer what it is for.
