1
0
mirror of https://github.com/MariaDB/server.git synced 2026-10-23 18:49:29 +03:00
Commit Graph

431 Commits

Author SHA1 Message Date
Marko Mäkelä
7ba156d52e Merge 11.4 into 11.8 2026-10-16 09:25:49 +03:00
Sergei Golubchik
50dd408f81 Merge branch '10.11' into 11.4 2026-10-15 20:25:34 +02:00
Alexander Barkov
8c4943d208 MDEV-41246 "Illegal mix of collations" on the mysql.user view
This SQL script failed:

SET NAMES latin1 COLLATE latin1_swedish_ci;
CREATE OR REPLACE VIEW v1 AS SELECT 'Y' AS c1;
SET NAMES big5 COLLATE big5_chinese_ci;
SELECT * FROM v1 WHERE c1='y';

with the following error:

ERROR 1267 (HY000): Illegal mix of collations
(latin1_swedish_ci,COERCIBLE) and (big5_chinese_ci,COERCIBLE)
for operation '='

Note, latin1_swedish_ci and big5_chinese_ci are used here as examples.
The error also happened with different collation combinations.

Fix main idea:

If two collations have equal comparison rules (known as "tailoring")
on a given character repertoire,
like latin1_swedish_ci and big5_chinese_ci on ASCII letters,
then the "Illegal mix of collations" error can be avoided in
a comparison operator. We can choose any of the sides as the operation
effective collation - the result will be equal.

This optimization is not applied when at least one side has an explicit
COLLATE clause. Two explicit COLLATE clauses in one comparison are
already illegal when the character sets are the same, so for
consistency this stays illegal when the character sets differ too.

Most important details:

- Splitting enum_repertoire_t into smaller subsets,
  for better repertoire granularity.
  A variable holding a repertoire value can now have multiple
  MY_REPERTOIRE_XXX flags set.

  This patch implements detecting tailoring equality
  on these repertoires:
  * MY_REPERTOIRE_ASCII_DIGITS - [0..9]
  * MY_REPERTOIRE_ASCII_MINUS_DOT_DIGITS - [-.0..9], e.g. '-10.5'.
    It consists of the new bits MY_REPERTOIRE_ASCII_MINUS and
    MY_REPERTOIRE_ASCII_DOT. They were a part of
    MY_REPERTOIRE_ASCII_NOT_IDENT before. As a result, the numeric values
    of MY_REPERTOIRE_ASCII and MY_REPERTOIRE_UNICODE30 changed
    (223 and 255).
  * MY_REPERTOIRE_ASCII_ALNUM - [A..Z,a..z,0..9].
  * MY_REPERTOIRE_ASCII_IDENT - ALNUM + underscore
  * MY_REPERTOIRE_ASCII       - the entire range U+0000..U+007F

- Adding a new virtual function "tailoring" in my_collation_handler_st.

  It returns the tailoring on the given repertoire for the given
  collation.

  If cs1->coll->tailoring(cs1, some_repertoire) returns {0,0},
  it means the illegal mix optimization cannot be used
  for this collation on the given repertoire.

  If these calls:
     tr1= cs1->coll->tailoring(cs1, some_repertoire);
     tr2= cs2->coll->tailoring(cs2, some_repertoire);
  return both non-NULL results and tr1.str==tr2.str,
  then these collations are equal on the given repertoire
  and are mutually replaceable for a comparison operator,
  so "Illegal mix of collations" can be avoided.

- Tailoring strings are shared constants my_tailoring_str_*
  (strings/strings_def.h). Tailorings are compared by the string
  pointer, so tailoring() implementations return strings only from
  these constants. The strings do not depend on PAD/NOPAD.

- my_string_repertoire() for ucs2, utf16, utf32 and NONASCII 8bit
  character sets no longer stops at the first bad byte sequence with
  the repertoire found so far, which could be MY_REPERTOIRE_NONE
  (compatible with everything). It returns MY_REPERTOIRE_EXTENDED
  for such strings. A unit test was added into unittest/strings.

- Adding CHARSET_INFO::is_ascii_superset(). It tells if a character set
  can store all ASCII characters (it is false for swe7).
  It is used in Item_func_conv_charset to calculate "safe", and in
  left_is_algorithmically_simpler() to prefer other character sets
  to those where ASCII strings cannot be converted safely.

- DBUG_ASSERTs were added into mysys/charset.c, to check that
  the "tailoring" method is set in collation handlers when a collation
  is added: compiled-in, from ctype-extra.c, or loaded from Index.xml.

- Adding a new function my_collations_equal_on_repertoire(cs1, cs2,
  repertoire) in strings/. It tells if two collations are equal on
  the repertoire, taking tailoring() results and PAD/NOPAD and the UCA
  version into account. Callers do not compare tailoring() results
  directly, so the way tailorings are compared (currently by the pointer
  to the shared constant string) is private to strings/ and can be
  extended later, e.g. for UCA collations on MY_REPERTOIRE_UNICODE30.

- Adding a new method DTCollation::aggregate_by_tailoring().
  Collations with different MY_CS_NOPAD are never compatible.
  UCA collations of different versions are not compatible on
  the repertoire with ASCII punctuation, as the UCA version
  affects its order (e.g. UCA-6.2.0 moved GRAVE ACCENT
  and CIRCUMFLEX ACCENT after PERCENT SIGN).

- DTCollation::aggregate() now makes the repertoire of the result cover
  the repertoires of both sides in all successful branches.
  Before, set(dt) replaced the repertoire with the repertoire of the
  winner side only, so the result could have a too narrow repertoire,
  e.g. IF(1, 1.5, HEX(255)) had MY_REPERTOIRE_ASCII_ALNUM although the
  value '1.5' has a punctuation character. With the more granular
  repertoires this could wrongly allow mixing collations which are
  equal on ALNUM but different on punctuation.

- Adding a new flag MY_COLL_ALLOW_BY_TAILORING.
  It indicates to DTCollation::aggregate() that the illegal
  mix optimization by repertoire can be used in the given context.

  MY_COLL_CMP_CONV now includes MY_COLL_ALLOW_BY_TAILORING.
  Note, only comparison operators pass this flag.
  Functions returning a string result do not pass this flag,
  because in operations like CONCAT(a,b) we still need to evaluate
  precisely the collation of the result - we cannot just choose
  a collation of one of the sides (even if they are compatible
  on the given repertoire). This also applies to ExtractValue()
  and UpdateXML(), which now aggregate their arguments without
  this flag.

- As in my_repertoire_t the value MY_REPERTOIRE_ASCII is now a set of
  bits rather than a single bit, the way to detect "is only ASCII"
  repertoires has changed in the code.

  For example:
    // repertoire *IS* ascii
    if (repertoire == MY_REPERTOIRE_ASCII)

  has changed in multiple places in the code to

    // repertoire *HAS* only ascii characters
    if (my_repertoire_is_subset_of(repertoire, MY_REPERTOIRE_ASCII))

  The new function my_repertoire_is_subset_of() in m_ctype.h
  is used for this purpose in C code. In C++ code
  DTCollation::repertoire_is_subset_of() is used, e.g.:

    if (collation.repertoire_is_subset_of(MY_REPERTOIRE_ASCII))

- Repertoire of some expressions was adjusted to the new meaning:
  * Item_null now has MY_REPERTOIRE_NONE (was ASCII).
  * MY_LOCALE::repertoire() now returns MY_REPERTOIRE_UNICODE30
    (was EXTENDED).
  * Lex_string_with_metadata_st::repertoire(cs) now scans the string
    contents to detect the actual repertoire.
  * HEX() now has MY_REPERTOIRE_ASCII_ALNUM.
  * DATE_FORMAT() decides on MY_REPERTOIRE_EXTENDED using the locale
    which is actually used: the explicit third argument, or
    @@lc_time_names. It was always @@lc_time_names before.
    A non-constant locale argument is assumed to be non-ASCII.
  * QUOTE, MAKE_SET, EXPORT_SET, LPAD, RPAD, GROUP_CONCAT, JSON_ARRAY,
    JSON_OBJECT and JSON_OBJECTAGG add the repertoire of the extra
    characters they put into the result (quotes, separators, padding,
    brackets).
  * LOWER() and UPPER() add the letters of both cases to the repertoire,
    because the result can contain letters of the opposite case.
    In Turkish collations they also add MY_REPERTOIRE_EXTENDED, as an
    ASCII letter can be converted to a non-ASCII one (I -> dotless i).
    Unicode collations are detected by the new member
    casefold_info_st::can_convert_to_non_ascii_on_casefolding,
    simple 8bit collations by to_lower['I'] and to_upper['i'].

- Lex_string_with_metadata_st::repertoire(cs) now scans a string literal
  for any byte oriented character set (mbminlen==1), not only for
  ASCII based ones. It was assumed to have the full repertoire for swe7,
  which is not ASCII based. The scan is done by Unicode code points, so
  swe7 strings with digits and letters get a narrow repertoire, while
  e.g. 0x5B (LATIN CAPITAL LETTER A WITH DIAERESIS in swe7) is non-ASCII.
  Therefore swe7_swedish_ci can be mixed with other collations now,
  in the same way as other collations with MY_CS_IDENT_CASEUP_CI.
  The test func_time was adjusted: a swe7 literal ' ull' with ASCII
  characters is now safe to convert, the error is still tested for '[ull'.

- cp1250_czech_cs sorts -, . and 0..9 in the code point order, so it gets
  the MINUS_DOT_DIGITS tailoring. latin2_czech_cs gets only DIGITS,
  as it sorts the minus and the dot differently. Both were verified
  with ctype_digits_verify.

- The tis620 collations now set CHARSET_INFO::tab_to_uni (was NULL),
  so my_charset_is_ascii_based() is true for tis620.
  It was false for tis620, although tis620 is ASCII compatible.
  Therefore tis620 string literals were always considered to have
  a non-ASCII repertoire, and the repertoire optimizations did not work
  for them. Now ASCII-only tis620 literals are detected as ASCII.
  This changes the result of subselect_extra_no_semijoin
  from an error to success.

- tis620_thai_ci and tis620_thai_nopad_ci fold letters to lower case
  and then compare by the code point on the entire ASCII range
  (the Thai specific rules affect only non-ASCII characters).
  A new function my_tailoring_ascii_casedn_ci() returns tailorings
  for such collations on ALNUM, IDENT and ASCII. For example, the
  underscore sorts before the letters, unlike in the caseup tailorings.

- New flags were added for CHARSET_INFO::state
  * MY_CS_ASCII_CASEUP_CI - for simple 8bit case insensitive collations.
    It means that this collation has no irregularities on the ASCII
    repertoire, and converts lower case letters to upper case ones,
    so the underscore sorts after the letters (with an upper to lower
    case conversion it would sort before the letters).

  * MY_CS_IDENT_CASEUP_CI - for simple 8bit case insensitive collations.
    It means that this collation has no irregularities on the IDENT
    repertoire (but can have irregularities say on punctuation).

  * MY_CS_ASCII_STD_UCA - for UCA collations.
    It means that a UCA collation does not reorder ASCII characters.

  * MY_CS_DIGITS_STD - for any collation. It means that the ASCII digits
    0..9 are sorted in the code point order and are not tailored, and,
    for PAD collations, the space sorts before the digits. Strings of
    digits are compared in the same way in all such collations, even if
    they have irregularities on letters (latin7, cp866, czech, danish,
    turkish UCA, etc.), or differ by case sensitivity. It is detected
    from sort_order for 8bit collations, and from the tailoring rules
    for UCA collations. Collations with MY_CS_DIGITS_STD still must have
    the same PAD/NOPAD attribute.

  * MY_CS_ASCII_MINUS_DOT_DIGITS - for any collation. Means
    MY_CS_DIGITS_STD, and additionally the minus sorts before the dot,
    which sorts before the digits (and the space sorts before the minus
    in PAD collations). Strings like '-10.5' are compared in the same
    way in all such collations. It is detected from sort_order for simple
    8bit collations (my_coll_init_simple()), and from the tailoring rules
    for UCA collations. For the multi-byte collations big5_chinese_ci,
    gbk_chinese_ci, gb2312_chinese_ci and sjis_japanese_ci it is declared
    by my_tailoring_ident_caseup_ci_generic(). The flags MY_CS_DIGITS_STD
    and MY_CS_ASCII_MINUS_DOT_DIGITS are stored in ctype-extra.c for
    compiled collations (conf_to_src.c prints them), and the detection is
    skipped for a collation which already has any of the repertoire flags
    (MY_CS_REPERTOIRE_FLAGS).

- strings/conf_to_src.c was modified to detect and print
  MY_CS_ASCII_CASEUP_CI and MY_CS_IDENT_CASEUP_CI flags.

- strings/ctype-extra.c was regenerated with new flags.

- 8bit collations are not given MY_CS_ASCII_CASEUP_CI and
  MY_CS_IDENT_CASEUP_CI if they are PAD and the space does not sort
  before the digits. A shorter string is padded with spaces for
  comparison, so the position of the space matters even for strings
  without spaces ("ab" is compared to "abc" as "ab " to "abc").
  A test collation latin1_space_after_letters_ci was added to
  mysql-test/std_data/ldml/ for this. ctype-extra.c does not change.

- The initializer of my_collation_cs_handler in ctype-utf8.c
  (utf8mb3_general_cs, under HAVE_UTF8_GENERAL_CS) was fixed to match
  my_collation_handler_st: the missing get_id and get_collation_name
  members were added, so that eq_collation was not initialized in their
  place, and the new member "tailoring" is now set to my_tailoring_none.

- Results of the existing tests func_str, func_test, ps, view
  and subselect_sj changed from "Illegal mix of collations" to success.
  In these tests ASCII-only strings were compared using collations
  with equal tailoring on ASCII, e.g. latin1_swedish_ci with
  latin2_general_ci (func_str, ps), koi8r_general_ci with
  latin1_swedish_ci (func_test), latin1_general_ci with
  latin1_swedish_ci on letters and digits (view), cp932_japanese_ci
  with latin1_swedish_ci (subselect_sj).
  Such comparisons are now allowed.
  Cases with collations that are still incompatible on ASCII,
  e.g. latin7_general_ci, were added to the tests and still fail.

- Adding a number of MTR tests in
  plugin/func_test/mysql-test/func_test/.
  They display a tailoring by collation name and repertoire as
  returned by:
    cs->coll->tailoring(cs, some_repertoire)
  A dynamically linked plugin function collation_tailoring() was added
  for the purpose of these tests. Tailorings do not depend on PAD/NOPAD
  and on the UCA version, so the function shows "[nopad]" for NOPAD
  collations, and "[version X.Y.Z]" for UCA collations on the ASCII
  repertoire. The version is formatted by a new class UCAVersion,
  which uses a new method CharBuffer::append_uint8().

- Adding the test plugin/func_test/mysql-test/func_test/
  ctype_caseup_ci_verify.test. It checks the order of ASCII characters
  for all collations declared "caseup CI" on the ASCII or IDENT range.

- Adding the test plugin/func_test/mysql-test/func_test/
  collation_tailoring_args.test. It checks the arguments of the plugin
  function collation_tailoring(): the repertoire passed by name or by
  number, and the NULL results for unknown names.

- Adding the test plugin/func_test/mysql-test/func_test/ctype_ldml.test.
  It displays the tailorings of collations loaded at runtime from
  Index.xml: simple 8bit collations (flags detected from the weights)
  and Unicode collations (LDML rules which do or do not reorder ASCII).

- Adding a number of MTR tests mysql-test/main/ctype_xxx_tailoring.test
  They display two-dimensional charts showing which collations
  are compatible on which repertoires.
  Tests for DATE_FORMAT() with an explicit locale were added to
  ctype_big5_tailoring.test, ctype_cp932_tailoring.test and
  ctype_latin5_tailoring.test, which also has the chart for latin5.

- Adding a number of MTR tests in the form of the originally
  reported script for various collations:

    SELECT Insert_priv FROM mysql.user WHERE Insert_priv='...';

  The repeated blocks of queries are shared through
  mysql-test/include/
  ctype_tailoring_0{1,2,3}_mysql_user_Insert_priv*.inc.

- Adding tests to ctype_cp932.test checking that ExtractValue() and
  UpdateXML() still raise "Illegal mix of collations".
2026-10-15 15:39:04 +04:00
DerZc
46ea84b186 MDEV-39194: Fix DISTINCT bit and hex aggregation
Problem: SUM(DISTINCT b'1100') returns 9 instead of 12. The hybrid
hex/bit literal handler uses the default DOUBLE temporary field, so
its byte length becomes a DOUBLE(M,0) display width and clips the
numeric value.

Fix: Override numeric DISTINCT field construction for hybrid hex and
bit literals to allocate an unsigned Field_longlong directly. Preserve
argument-based field selection for other types so DISTINCT retains the
input expression's precision and scale, including decimal AVG inputs.
Extend main.sum_distinct to cover bit and hex literals, zero, byte and
unsigned 64-bit boundaries, duplicates, AVG, COUNT and decimal scale.

Recreated-by: Georgi "Joro" Kodinov <joro@mariadb.org>
2026-10-13 19:39:19 +03:00
forkfun
eaca80bf32 Merge branch '11.4' into '11.8' 2026-07-29 12:27:46 +02:00
forkfun
08eeb6a417 Merge branch '10.11' into '11.4' 2026-07-28 13:14:38 +02:00
forkfun
a6c9db7cc2 MDEV-39380 Assertion `arg2_int >= 0' failed in Item_func_additive_op::result_precision
A 0xHHHH / b'..' literal keeps its hex_hybrid type through COALESCE/IF/CASE/...,
but was evaluated, sized and stored as a string, so it behaved differently from
a bare literal (10+COALESCE(0x31) gave 11, not 59).

Type_handler_hex_hybrid now implements the numeric side to match the bare
literal: val_int/val_real/val_decimal read bytes as a hybrid, Item_decimal_precision
returns the integer precision, Item_save_in_field stores like the bare literal.
The traditional type merge keeps the hybrid through a typeless NULL (as for BIT),
and Item_hybrid_func_fix_attributes attributes it binary/unsigned.
2026-07-23 13:38:02 +02:00
Kristian Nielsen
420e28ce28 Merge 11.4 to 11.8
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2026-06-15 15:15:10 +02:00
Kristian Nielsen
85ec122c15 Merge 10.11 to 11.4
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2026-06-15 12:50:45 +02:00
Dave Gosselin
96ca4d3d5a MDEV-38158: Incorrect query result
When setup_copy_fields() needs to copy a non-aggregate function value,
it doesn't construct an Item_copy directly.  Instead, it calls
Type_handler::create_item_copy, which is a kind of factory.  The base
Type_handler::create_item_copy returns Item_copy_string.  Some type
handlers override it, like timestamp and fixed binary.  However, the
numeric type handlers (e.g., float, double, int, decimal) did not, so
they fell through to that base and got Item_copy_string.

A SELECT that aggregates will copy each non aggregate function value
into a temporary table through an Item_copy object, whose concrete
type is chosen by the create_item_copy method on the value's type
handler.  For numeric types that method returned Item_copy_string,
which stores the value as text.  A FLOAT keeps only FLT_DIG
significant digits as text, too few to reproduce its 24 bit mantissa,
so the copied value differed from the original.  With one row per
group, CAST(c1 AS FLOAT) - MIN(CAST(c1 AS FLOAT)) returned a large
number instead of zero.

Add Item_copy_real with Item_copy_float and Item_copy_double variants
that keep the value as a double, the same way Item_cache_real does, and
let the float and double type handlers create them.  This mirrors the
existing copy items for timestamp and fixed binary types.
2026-06-11 12:44:40 -04:00
Oleksandr Byelkin
9cb70f0220 Merge branch '11.4' into 11.8 2026-05-08 08:11:03 +02:00
Oleksandr Byelkin
7598e4e95d Merge branch '10.11' into 11.4 2026-05-08 08:00:01 +02:00
Dave Gosselin
aeb477b71a MDEV-23278: Incorrect Calculation with AVG() Function
Type_handler_int_result::Item_decimal_precision assumes that
max_length includes space for a sign, but Item_int literals don't
include such a space.  When aggregate_attributes_int propagates these
into functions via a Item_type_holder, the formula computes
precision-1 (or 0 for 1-digit cases), producing the incorrect
calculation with AVG.

This fix changes aggregate_attributes_int to follow the convention the
Type_handler expects, that max_length = max digit-count + (signed ? 1
: 0).  This matches the TODO comment that begins with "TODO: rewrite
aggregate_attributes_int()... " which suggested this idea.
2026-05-01 14:12:13 -04:00
Marko Mäkelä
1a144f713a Merge 11.4 into 11.8 2026-04-01 15:03:59 +03:00
Marko Mäkelä
7c5cac0d51 Merge 10.11 into 11.4 2026-04-01 10:49:05 +03:00
Marko Mäkelä
29be734aaf Merge 10.6 into 10.11 2026-03-31 11:48:18 +03:00
Marko Mäkelä
cb32aa9aa8 MDEV-16926 fixup: GCC 16 -Wmaybe-uninitialized
Year::year_precision(): Make static, so that VYear::VYear
and VYear_op::VYear_op() will avoid invoking this function with
an unnecessary "this" pointer to an uninitialized object.
2026-03-31 11:13:39 +03:00
Sergei Golubchik
9bfea48ce1 Merge branch '11.4' into 11.8 2026-01-31 14:08:54 +01:00
Sergei Golubchik
40f7084661 Merge branch '10.11' into 11.4 2026-01-28 21:52:18 +01:00
Alexander Barkov
a84b8ec5b1 MDEV-38626 Unexpected Data too long error on subselect as a multiplication argument
Item_singerow_subselect::fix_length_and_dec() incorrectly calculated
its Item::max_length when the underlying expression was Item_int.

The reason of the problem:
Item_int has an optimized max_length to make CONCAT(1) create a VARCHAR(1)
column rather than a VARCHAR(2) column. Its max_length does not include one
extra character for the sign in case the value is positive but the value
is not marked as Item::unsigned==true.

So copying max_length from the underlying Item_int (with value==9)
in cases like this:
  SELECT CONCAT((SELECT 9 FROM t0));
was not correct.

Implementing a new virtual method
  Type_handler::Item_type_std_attributes_generic(const Item *item)

- The default implementation just copies attributes from "item" as is.

- In case of Type_handler_int_result it evaluates max_length
  using item->decimal_precision() rather than item->max_length.
  This works correctly for both "optimized" items like Item_int and
  non-"optimized" Items whose max_length includes +1 for the sign
  in case of signed expressions.
2026-01-22 12:47:52 +04:00
Sergei Golubchik
c1fcedf443 Merge branch '11.4' into 11.8 2026-01-05 20:34:44 +01:00
Marko Mäkelä
678ff03ee7 Merge 10.11 into 11.4 2026-01-02 11:53:09 +02:00
Marko Mäkelä
8fab129382 Merge 10.6 into 10.11 2026-01-02 11:06:05 +02:00
Sergei Golubchik
0cd55ca5a5 MDEV-38242 crash when first COM_STMT_EXECUTE doesn't send types
all params are initialized to use Type_handler_null.
NULL params do not have a value to read from the packet buffer,
so their Item_param_set_param_func() should do nothing.

Type_handler_null inherits from Type_handler_string_result,
that is, Type_handler_null *must* override parent's
Item_param_set_param_func().
2025-12-26 16:42:34 +01:00
Aleksey Midenkov
ff33f49d9a Merge 11.4 into 11.8 2025-09-29 18:25:09 +03:00
Oleksandr Byelkin
15b1426c3a Merge branch '10.11' into bb-11.4-release 2025-09-15 16:17:33 +02:00
Monty
882f6fa3aa Fixed typos
- Removed duplicate words, like "the the" and "to to"
- Removed duplicate lines (one double sort line found in mysql.cc)
- Fixed some typos found while searching for duplicate words.

Command used to find duplicate words:
egrep -rI "\s([a-zA-Z]+)\s+\1\s" | grep -v param

Thanks to Artjoms Rimdjonoks for the command and pointing out the
spelling errors.
2025-09-04 18:08:39 +03:00
Marko Mäkelä
257f4b30ef Merge 10.11 into 11.4 2025-09-03 10:32:56 +03:00
Marko Mäkelä
b489d5f813 Merge 10.6 into 10.11 2025-08-26 14:24:31 +03:00
Marko Mäkelä
6bf1ed55e6 MDEV-37477: decimal_digits_t trips GCC -Wconversion
In commit fa7d4abf16
when the 16-bit type alias decimal_digits_t was introduced,
some incompatible use of the 32-bit uint data type was overlooked.

Only for converting Item_result_field::max_length (uint), we will use
explicit conversion.
2025-08-22 11:22:12 +03:00
Alexander Barkov
5a8e6230d7 MDEV-34189 Unexpected error on WHERE inet6col
normalize_cond() translated `WHERE col` into `WHERE col<>0`

But the opetator "not equal to 0" does not necessarily exists
for all data types.

For example, the query:

  SELECT * FROM t1 WHERE inet6col;

was translated to:

  SELECT * FROM t1 WHERE inet6col<>0;

which further failed with this error:

  ERROR : Illegal parameter data types inet6 and bigint for operation '<>'

This patch changes the translation from `col<>0` to `col IS TRUE`.
So now
  SELECT * FROM t1 WHERE inet6col;
gets translated to:
  SELECT * FROM t1 WHERE inet6col IS TRUE;

Details:
1. Implementing methods:
   - Field_longstr::val_bool()
   - Field_string::val_bool()
   - Item::val_int_from_val_str()
   If the input contains bad data,
   these methods raise a better error message:
     Truncated incorrect BOOLEAN value
   Before the change, the error was:
     Truncated incorrect DOUBLE value

2. Fixing normalize_cond() to generate Item_func_istrue/Item_func_isfalse
   instances instead of Item_func_ne/Item_func_eq

3. Making Item_func_truth sargable, so it uses the range optimizer.
   Implementing the following methods:
   - get_mm_tree(), get_mm_leaf(), add_key_fields() in Item_func_truth.
   - get_func_mm_tree(), for all Item_func_truth descendants.

4. Implementing the method negated_item() for all Item_func_truth
   descendants, so the negated item has a chance to be sargable:
   For example,
     WHERE NOT col IS NOT FALSE    -- this notation is not sargable
   is now translated to:
     WHERE col IS FALSE            -- this notation is sargable
2025-01-29 09:08:19 +04:00
Marko Mäkelä
33907f9ec6 Merge 11.4 into 11.7 2024-12-02 17:51:17 +02:00
Marko Mäkelä
2719cc4925 Merge 10.11 into 11.4 2024-12-02 11:35:34 +02:00
Marko Mäkelä
3d23adb766 Merge 10.6 into 10.11 2024-11-29 13:43:17 +02:00
Marko Mäkelä
7d4077cc11 Merge 10.5 into 10.6 2024-11-29 12:37:46 +02:00
ParadoxV5
cf2d49ddcf Extract some of #3360 fixes to 10.5.x
That PR uncovered countless issues on `my_snprintf` uses.
This commit backports a squashed subset of their fixes.
2024-11-21 22:43:56 +11:00
Oleksandr Byelkin
b12ff287ec Merge branch '11.6' into 11.7 2024-11-10 19:22:21 +01:00
Oleksandr Byelkin
9e1fb104a3 Merge tag '11.4' into 11.6
MariaDB 11.4.4 release
2024-11-08 07:17:00 +01:00
Sergei Golubchik
78119d1ae5 MDEV-33410 VECTOR data type 2024-11-05 14:00:51 -08:00
Sergei Golubchik
d6add9a03d initial support for vector indexes
MDEV-33407 Parser support for vector indexes

The syntax is

  create table t1 (... vector index (v) ...);

limitation:
* v is a binary string and NOT NULL
* only one vector index per table
* temporary tables are not supported

MDEV-33404 Engine-independent indexes: subtable method

added support for so-called "high level indexes", they are not visible
to the storage engine, implemented on the sql level. For every such
an index in a table, say, t1, the server implicitly creates a second
table named, like, t1#i#05 (where "05" is the index number in t1).
This table has a fixed structure, no frm, not accessible directly,
doesn't go into the table cache, needs no MDLs.

MDEV-33406 basic optimizer support for k-NN searches

for a query like SELECT ... ORDER BY func() optimizer will use
item_func->part_of_sortkey() to decide what keys can be used
to resolve ORDER BY.
2024-11-05 14:00:48 -08:00
Sergei Golubchik
32e6f8ff2e cleanup: remove unconditional #ifdef's 2024-11-05 14:00:47 -08:00
Oleksandr Byelkin
c770bce898 Merge branch '11.2' into 11.4 2024-10-30 15:11:17 +01:00
Oleksandr Byelkin
69d033d165 Merge branch '10.11' into 11.2 2024-10-29 16:42:46 +01:00
Oleksandr Byelkin
3d0fb15028 Merge branch '10.6' into 10.11 2024-10-29 15:24:38 +01:00
Alexander Barkov
a88c71b294 MDEV-35041 Simple comparison causes "Illegal mix of collations" even with default server settings
The task "MDEV-25829 Change default Unicode collation to uca1400_ai_ci"
previously changed collation derivation for string user variables
from DERIVATION_EXPLICIT to DERIVATION_COERCIBLE, to resolve illegal
collation mix conflicts between table columns and user variables
when they have different collations.

However, DERIVATION_COERCIBLE was a wrong choice because it caused
conflicts between string literals and user variables when they have
different collations.

Adding a new collation derivation level DERIVATION_USERVAR.
This makes the collation of a user variable:
- weaker than a table column (like it was intended by MDEV-25829)
- but stronger than a literal (like it was in pre-MDEV-25829)

Cleanup in sql_type.h:
  Removing the line "- BINARY(expr)" from the before-DERIVATION_CAST
  comment, as it was on a wrong place. It's also listed on the correct
  place before DERIVATION_IMPLICIT.
2024-10-28 16:30:49 +04:00
Alexander Barkov
dfaf7e2eb4 MDEV-15751 CURRENT_TIMESTAMP should return a TIMESTAMP [WITH TIME ZONE?]
Changing the return type of the following functions:
  - CURRENT_TIMESTAMP, CURRENT_TIMESTAMP(), NOW()
  - SYSDATE()
  - FROM_UNIXTIME()
from DATETIME to TIMESTAMP.

Note, the old function NOW() returning DATETIME is still available
as LOCALTIMESTAMP or LOCALTIMESTAMP(), e.g.:

  SELECT
    LOCALTIMESTAMP,     -- DATETIME
    CURRENT_TIMESTAMP;  -- TIMESTAMP

The change in the functions return data type fixes some problems
that occurred near a DST change:

- Problem #1

INSERT INTO t1 (timestamp_field) VALUES (CURRENT_TIMESTAMP);
INSERT INTO t1 (timestamp_field) VALUES (COALESCE(CURRENT_TIMESTAMP));

could result into two different values inserted.

- Problem #2

INSERT INTO t1 (timestamp_field) VALUES (FROM_UNIXTIME(1288477526));
INSERT INTO t1 (timestamp_field) VALUES (FROM_UNIXTIME(1288477526+3600));

could result into two equal TIMESTAMP values near a DST change.

Additional changes:

- FROM_UNIXTIME(0) now returns SQL NULL instead of '1970-01-01 00:00:00'
  (assuming time_zone='+00:00')

- UNIX_TIMESTAMP('1970-01-01 00:00:00') now returns SQL NULL instead of 0
  (assuming time_zone='+00:00'

These additional changes are needed for consistency with TIMESTAMP fields,
which cannot store '1970-01-01 00:00:00 +00:00'
2024-10-19 22:48:23 +02:00
Alexander Barkov
a931da82fa MDEV-34123 CONCAT Function Returns Unexpected Empty Set in Query
Search conditions were evaluated using val_int(), which was wrong.
Fixing the code to use val_bool() instead.

Details:
- Adding a new item_base_t::IS_COND flag which marks Items used
  as <search condition> in WHERE, HAVING, JOIN ON, CASE WHEN clauses.
  The flag is at the parse time.
  These expressions must be evaluated using val_bool() rather than val_int().

  Note, the optimizer creates more Items which are used as search conditions.
  Most of these items are not marked with IS_COND yet. This is OK for now,
  but eventually these Items can also be fixed to have the flag.

- Adding a method Item::is_cond() which tests if the Item has the IS_COND flag.

- Implementing Item_cache_bool. It evaluates the cached expression using
  val_bool() rather than val_int().
  Overriding Type_handler_bool::Item_get_cache() to create Item_cache_bool.

- Implementing Item::save_bool_in_field(). It uses val_bool() rather than
  val_int() to evaluate the expression.

- Implementing Type_handler_bool::Item_save_in_field()
  using Item::save_bool_in_field().

- Fixing all Item_bool_func descendants to implement a virtual val_bool()
  rather than a virtual val_int().

- To find places where val_int() should be fixed to val_bool(), a few
  DBUG_ASSERT(!is_cond()) where added into val_int() implementations
  of selected (most frequent) classes:

  Item_field
  Item_str_func
  Item_datefunc
  Item_timefunc
  Item_datetimefunc
  Item_cache_bool
  Item_bool_func
  Item_func_hybrid_field_type
  Item_basic_constant descendants

- Fixing all places where DBUG_ASSERT() happened during an "mtr" run
  to use val_bool() instead of val_int().
2024-10-08 11:58:46 +02:00
Oleksandr Byelkin
342fa29615 Merge branch '11.4' into 11.5 2024-08-21 11:52:54 +02:00
Oleksandr Byelkin
eb70e0d6e2 Merge branch '11.2' into 11.4 2024-08-21 09:30:54 +02:00
Oleksandr Byelkin
6197e6abc4 Merge branch '10.11' into 11.2 2024-08-21 07:58:46 +02:00