1
0
mirror of https://github.com/MariaDB/server.git synced 2026-10-23 18:49:29 +03:00
Commit Graph

386 Commits

Author SHA1 Message Date
Marko Mäkelä
7ba156d52e Merge 11.4 into 11.8 2026-10-16 09:25:49 +03:00
Sergei Golubchik
50dd408f81 Merge branch '10.11' into 11.4 2026-10-15 20:25:34 +02:00
Alexander Barkov
8c4943d208 MDEV-41246 "Illegal mix of collations" on the mysql.user view
This SQL script failed:

SET NAMES latin1 COLLATE latin1_swedish_ci;
CREATE OR REPLACE VIEW v1 AS SELECT 'Y' AS c1;
SET NAMES big5 COLLATE big5_chinese_ci;
SELECT * FROM v1 WHERE c1='y';

with the following error:

ERROR 1267 (HY000): Illegal mix of collations
(latin1_swedish_ci,COERCIBLE) and (big5_chinese_ci,COERCIBLE)
for operation '='

Note, latin1_swedish_ci and big5_chinese_ci are used here as examples.
The error also happened with different collation combinations.

Fix main idea:

If two collations have equal comparison rules (known as "tailoring")
on a given character repertoire,
like latin1_swedish_ci and big5_chinese_ci on ASCII letters,
then the "Illegal mix of collations" error can be avoided in
a comparison operator. We can choose any of the sides as the operation
effective collation - the result will be equal.

This optimization is not applied when at least one side has an explicit
COLLATE clause. Two explicit COLLATE clauses in one comparison are
already illegal when the character sets are the same, so for
consistency this stays illegal when the character sets differ too.

Most important details:

- Splitting enum_repertoire_t into smaller subsets,
  for better repertoire granularity.
  A variable holding a repertoire value can now have multiple
  MY_REPERTOIRE_XXX flags set.

  This patch implements detecting tailoring equality
  on these repertoires:
  * MY_REPERTOIRE_ASCII_DIGITS - [0..9]
  * MY_REPERTOIRE_ASCII_MINUS_DOT_DIGITS - [-.0..9], e.g. '-10.5'.
    It consists of the new bits MY_REPERTOIRE_ASCII_MINUS and
    MY_REPERTOIRE_ASCII_DOT. They were a part of
    MY_REPERTOIRE_ASCII_NOT_IDENT before. As a result, the numeric values
    of MY_REPERTOIRE_ASCII and MY_REPERTOIRE_UNICODE30 changed
    (223 and 255).
  * MY_REPERTOIRE_ASCII_ALNUM - [A..Z,a..z,0..9].
  * MY_REPERTOIRE_ASCII_IDENT - ALNUM + underscore
  * MY_REPERTOIRE_ASCII       - the entire range U+0000..U+007F

- Adding a new virtual function "tailoring" in my_collation_handler_st.

  It returns the tailoring on the given repertoire for the given
  collation.

  If cs1->coll->tailoring(cs1, some_repertoire) returns {0,0},
  it means the illegal mix optimization cannot be used
  for this collation on the given repertoire.

  If these calls:
     tr1= cs1->coll->tailoring(cs1, some_repertoire);
     tr2= cs2->coll->tailoring(cs2, some_repertoire);
  return both non-NULL results and tr1.str==tr2.str,
  then these collations are equal on the given repertoire
  and are mutually replaceable for a comparison operator,
  so "Illegal mix of collations" can be avoided.

- Tailoring strings are shared constants my_tailoring_str_*
  (strings/strings_def.h). Tailorings are compared by the string
  pointer, so tailoring() implementations return strings only from
  these constants. The strings do not depend on PAD/NOPAD.

- my_string_repertoire() for ucs2, utf16, utf32 and NONASCII 8bit
  character sets no longer stops at the first bad byte sequence with
  the repertoire found so far, which could be MY_REPERTOIRE_NONE
  (compatible with everything). It returns MY_REPERTOIRE_EXTENDED
  for such strings. A unit test was added into unittest/strings.

- Adding CHARSET_INFO::is_ascii_superset(). It tells if a character set
  can store all ASCII characters (it is false for swe7).
  It is used in Item_func_conv_charset to calculate "safe", and in
  left_is_algorithmically_simpler() to prefer other character sets
  to those where ASCII strings cannot be converted safely.

- DBUG_ASSERTs were added into mysys/charset.c, to check that
  the "tailoring" method is set in collation handlers when a collation
  is added: compiled-in, from ctype-extra.c, or loaded from Index.xml.

- Adding a new function my_collations_equal_on_repertoire(cs1, cs2,
  repertoire) in strings/. It tells if two collations are equal on
  the repertoire, taking tailoring() results and PAD/NOPAD and the UCA
  version into account. Callers do not compare tailoring() results
  directly, so the way tailorings are compared (currently by the pointer
  to the shared constant string) is private to strings/ and can be
  extended later, e.g. for UCA collations on MY_REPERTOIRE_UNICODE30.

- Adding a new method DTCollation::aggregate_by_tailoring().
  Collations with different MY_CS_NOPAD are never compatible.
  UCA collations of different versions are not compatible on
  the repertoire with ASCII punctuation, as the UCA version
  affects its order (e.g. UCA-6.2.0 moved GRAVE ACCENT
  and CIRCUMFLEX ACCENT after PERCENT SIGN).

- DTCollation::aggregate() now makes the repertoire of the result cover
  the repertoires of both sides in all successful branches.
  Before, set(dt) replaced the repertoire with the repertoire of the
  winner side only, so the result could have a too narrow repertoire,
  e.g. IF(1, 1.5, HEX(255)) had MY_REPERTOIRE_ASCII_ALNUM although the
  value '1.5' has a punctuation character. With the more granular
  repertoires this could wrongly allow mixing collations which are
  equal on ALNUM but different on punctuation.

- Adding a new flag MY_COLL_ALLOW_BY_TAILORING.
  It indicates to DTCollation::aggregate() that the illegal
  mix optimization by repertoire can be used in the given context.

  MY_COLL_CMP_CONV now includes MY_COLL_ALLOW_BY_TAILORING.
  Note, only comparison operators pass this flag.
  Functions returning a string result do not pass this flag,
  because in operations like CONCAT(a,b) we still need to evaluate
  precisely the collation of the result - we cannot just choose
  a collation of one of the sides (even if they are compatible
  on the given repertoire). This also applies to ExtractValue()
  and UpdateXML(), which now aggregate their arguments without
  this flag.

- As in my_repertoire_t the value MY_REPERTOIRE_ASCII is now a set of
  bits rather than a single bit, the way to detect "is only ASCII"
  repertoires has changed in the code.

  For example:
    // repertoire *IS* ascii
    if (repertoire == MY_REPERTOIRE_ASCII)

  has changed in multiple places in the code to

    // repertoire *HAS* only ascii characters
    if (my_repertoire_is_subset_of(repertoire, MY_REPERTOIRE_ASCII))

  The new function my_repertoire_is_subset_of() in m_ctype.h
  is used for this purpose in C code. In C++ code
  DTCollation::repertoire_is_subset_of() is used, e.g.:

    if (collation.repertoire_is_subset_of(MY_REPERTOIRE_ASCII))

- Repertoire of some expressions was adjusted to the new meaning:
  * Item_null now has MY_REPERTOIRE_NONE (was ASCII).
  * MY_LOCALE::repertoire() now returns MY_REPERTOIRE_UNICODE30
    (was EXTENDED).
  * Lex_string_with_metadata_st::repertoire(cs) now scans the string
    contents to detect the actual repertoire.
  * HEX() now has MY_REPERTOIRE_ASCII_ALNUM.
  * DATE_FORMAT() decides on MY_REPERTOIRE_EXTENDED using the locale
    which is actually used: the explicit third argument, or
    @@lc_time_names. It was always @@lc_time_names before.
    A non-constant locale argument is assumed to be non-ASCII.
  * QUOTE, MAKE_SET, EXPORT_SET, LPAD, RPAD, GROUP_CONCAT, JSON_ARRAY,
    JSON_OBJECT and JSON_OBJECTAGG add the repertoire of the extra
    characters they put into the result (quotes, separators, padding,
    brackets).
  * LOWER() and UPPER() add the letters of both cases to the repertoire,
    because the result can contain letters of the opposite case.
    In Turkish collations they also add MY_REPERTOIRE_EXTENDED, as an
    ASCII letter can be converted to a non-ASCII one (I -> dotless i).
    Unicode collations are detected by the new member
    casefold_info_st::can_convert_to_non_ascii_on_casefolding,
    simple 8bit collations by to_lower['I'] and to_upper['i'].

- Lex_string_with_metadata_st::repertoire(cs) now scans a string literal
  for any byte oriented character set (mbminlen==1), not only for
  ASCII based ones. It was assumed to have the full repertoire for swe7,
  which is not ASCII based. The scan is done by Unicode code points, so
  swe7 strings with digits and letters get a narrow repertoire, while
  e.g. 0x5B (LATIN CAPITAL LETTER A WITH DIAERESIS in swe7) is non-ASCII.
  Therefore swe7_swedish_ci can be mixed with other collations now,
  in the same way as other collations with MY_CS_IDENT_CASEUP_CI.
  The test func_time was adjusted: a swe7 literal ' ull' with ASCII
  characters is now safe to convert, the error is still tested for '[ull'.

- cp1250_czech_cs sorts -, . and 0..9 in the code point order, so it gets
  the MINUS_DOT_DIGITS tailoring. latin2_czech_cs gets only DIGITS,
  as it sorts the minus and the dot differently. Both were verified
  with ctype_digits_verify.

- The tis620 collations now set CHARSET_INFO::tab_to_uni (was NULL),
  so my_charset_is_ascii_based() is true for tis620.
  It was false for tis620, although tis620 is ASCII compatible.
  Therefore tis620 string literals were always considered to have
  a non-ASCII repertoire, and the repertoire optimizations did not work
  for them. Now ASCII-only tis620 literals are detected as ASCII.
  This changes the result of subselect_extra_no_semijoin
  from an error to success.

- tis620_thai_ci and tis620_thai_nopad_ci fold letters to lower case
  and then compare by the code point on the entire ASCII range
  (the Thai specific rules affect only non-ASCII characters).
  A new function my_tailoring_ascii_casedn_ci() returns tailorings
  for such collations on ALNUM, IDENT and ASCII. For example, the
  underscore sorts before the letters, unlike in the caseup tailorings.

- New flags were added for CHARSET_INFO::state
  * MY_CS_ASCII_CASEUP_CI - for simple 8bit case insensitive collations.
    It means that this collation has no irregularities on the ASCII
    repertoire, and converts lower case letters to upper case ones,
    so the underscore sorts after the letters (with an upper to lower
    case conversion it would sort before the letters).

  * MY_CS_IDENT_CASEUP_CI - for simple 8bit case insensitive collations.
    It means that this collation has no irregularities on the IDENT
    repertoire (but can have irregularities say on punctuation).

  * MY_CS_ASCII_STD_UCA - for UCA collations.
    It means that a UCA collation does not reorder ASCII characters.

  * MY_CS_DIGITS_STD - for any collation. It means that the ASCII digits
    0..9 are sorted in the code point order and are not tailored, and,
    for PAD collations, the space sorts before the digits. Strings of
    digits are compared in the same way in all such collations, even if
    they have irregularities on letters (latin7, cp866, czech, danish,
    turkish UCA, etc.), or differ by case sensitivity. It is detected
    from sort_order for 8bit collations, and from the tailoring rules
    for UCA collations. Collations with MY_CS_DIGITS_STD still must have
    the same PAD/NOPAD attribute.

  * MY_CS_ASCII_MINUS_DOT_DIGITS - for any collation. Means
    MY_CS_DIGITS_STD, and additionally the minus sorts before the dot,
    which sorts before the digits (and the space sorts before the minus
    in PAD collations). Strings like '-10.5' are compared in the same
    way in all such collations. It is detected from sort_order for simple
    8bit collations (my_coll_init_simple()), and from the tailoring rules
    for UCA collations. For the multi-byte collations big5_chinese_ci,
    gbk_chinese_ci, gb2312_chinese_ci and sjis_japanese_ci it is declared
    by my_tailoring_ident_caseup_ci_generic(). The flags MY_CS_DIGITS_STD
    and MY_CS_ASCII_MINUS_DOT_DIGITS are stored in ctype-extra.c for
    compiled collations (conf_to_src.c prints them), and the detection is
    skipped for a collation which already has any of the repertoire flags
    (MY_CS_REPERTOIRE_FLAGS).

- strings/conf_to_src.c was modified to detect and print
  MY_CS_ASCII_CASEUP_CI and MY_CS_IDENT_CASEUP_CI flags.

- strings/ctype-extra.c was regenerated with new flags.

- 8bit collations are not given MY_CS_ASCII_CASEUP_CI and
  MY_CS_IDENT_CASEUP_CI if they are PAD and the space does not sort
  before the digits. A shorter string is padded with spaces for
  comparison, so the position of the space matters even for strings
  without spaces ("ab" is compared to "abc" as "ab " to "abc").
  A test collation latin1_space_after_letters_ci was added to
  mysql-test/std_data/ldml/ for this. ctype-extra.c does not change.

- The initializer of my_collation_cs_handler in ctype-utf8.c
  (utf8mb3_general_cs, under HAVE_UTF8_GENERAL_CS) was fixed to match
  my_collation_handler_st: the missing get_id and get_collation_name
  members were added, so that eq_collation was not initialized in their
  place, and the new member "tailoring" is now set to my_tailoring_none.

- Results of the existing tests func_str, func_test, ps, view
  and subselect_sj changed from "Illegal mix of collations" to success.
  In these tests ASCII-only strings were compared using collations
  with equal tailoring on ASCII, e.g. latin1_swedish_ci with
  latin2_general_ci (func_str, ps), koi8r_general_ci with
  latin1_swedish_ci (func_test), latin1_general_ci with
  latin1_swedish_ci on letters and digits (view), cp932_japanese_ci
  with latin1_swedish_ci (subselect_sj).
  Such comparisons are now allowed.
  Cases with collations that are still incompatible on ASCII,
  e.g. latin7_general_ci, were added to the tests and still fail.

- Adding a number of MTR tests in
  plugin/func_test/mysql-test/func_test/.
  They display a tailoring by collation name and repertoire as
  returned by:
    cs->coll->tailoring(cs, some_repertoire)
  A dynamically linked plugin function collation_tailoring() was added
  for the purpose of these tests. Tailorings do not depend on PAD/NOPAD
  and on the UCA version, so the function shows "[nopad]" for NOPAD
  collations, and "[version X.Y.Z]" for UCA collations on the ASCII
  repertoire. The version is formatted by a new class UCAVersion,
  which uses a new method CharBuffer::append_uint8().

- Adding the test plugin/func_test/mysql-test/func_test/
  ctype_caseup_ci_verify.test. It checks the order of ASCII characters
  for all collations declared "caseup CI" on the ASCII or IDENT range.

- Adding the test plugin/func_test/mysql-test/func_test/
  collation_tailoring_args.test. It checks the arguments of the plugin
  function collation_tailoring(): the repertoire passed by name or by
  number, and the NULL results for unknown names.

- Adding the test plugin/func_test/mysql-test/func_test/ctype_ldml.test.
  It displays the tailorings of collations loaded at runtime from
  Index.xml: simple 8bit collations (flags detected from the weights)
  and Unicode collations (LDML rules which do or do not reorder ASCII).

- Adding a number of MTR tests mysql-test/main/ctype_xxx_tailoring.test
  They display two-dimensional charts showing which collations
  are compatible on which repertoires.
  Tests for DATE_FORMAT() with an explicit locale were added to
  ctype_big5_tailoring.test, ctype_cp932_tailoring.test and
  ctype_latin5_tailoring.test, which also has the chart for latin5.

- Adding a number of MTR tests in the form of the originally
  reported script for various collations:

    SELECT Insert_priv FROM mysql.user WHERE Insert_priv='...';

  The repeated blocks of queries are shared through
  mysql-test/include/
  ctype_tailoring_0{1,2,3}_mysql_user_Insert_priv*.inc.

- Adding tests to ctype_cp932.test checking that ExtractValue() and
  UpdateXML() still raise "Illegal mix of collations".
2026-10-15 15:39:04 +04:00
DerZc
6e62fd0112 MDEV-41067: Fix JSON_OVERLAPS for nested arrays
Problem: JSON_OVERLAPS treats unequal nested arrays as equal when only
one parser reaches the array end. Requiring both ends also exposes a
cleanup issue: scanning ']' moves stack_p to the parent, so skipping
the current level consumes parent elements. This misses the common [2]
in [[1],[2]] and [[],[2]], and can skip unequal recursive elements.

Fix: Require both parsers to reach JST_ARRAY_END. Remember their array
levels before comparing and skip to those saved levels after completed
comparisons and mismatches, preserving parent siblings. Explain why the
saved levels are needed. Add regression coverage for both argument
orders, unequal lengths, empty arrays, recursive comparisons, type
mismatches, array order and a later common object member.
2026-10-15 14:09:24 +03:00
Rucha Deodhar
c7f549e26b MDEV-39049: Memory corruption & crash in check_key_in_list upon using
JSON_KEYS after modifying character set name/collation

Analysis:
Buffer overflow crashes and empty key duplication in check_key_in_list.

Fix:
Checking result buffer validity prevents segmentation faults on empty
strings and malformed inputs while preserving correct key matching behavior.
2026-09-25 11:04:46 -04:00
Rucha Deodhar
0d10b89fbf MDEV-39049: Memory corruption & crash in check_key_in_list upon using
JSON_KEYS after modifying character set name/collation

Analysis:
Since the length of string is 0, accessing out of boundry memory,
leads to crash

Fix:
If string length is empty, return success.
2026-09-25 11:04:46 -04:00
Oleksandr Byelkin
3b8db45403 Merge branch '11.4' into 11.8 2026-08-19 11:28:01 +02:00
Oleksandr Byelkin
170b1d7073 Merge branch '10.11' into 11.4 2026-08-19 08:40:05 +02:00
Daniel Black
c38073eb6c MDEV-40572 SIGSEGV in JSON_OVERLAPS()
on a truncated nested JSON object with different character sets/collations.

The value->s.c_str and js->s.c_str became invalid once the
function returned which caused the crash.
2026-08-11 08:28:45 +10:00
Sergei Golubchik
2894e90b65 MDEV-40165 post-fix
never use item->null_value before this item is evaluated

followup for 14c16e02b2
2026-08-01 19:44:19 +02:00
Daniel Black
1dab253482 MDEV-32331: JSON path functions with no charset on path crash server
Across a range of JSON functions taking a path argument
there are SQL expressions that dont' have a character
set. If these expressions don't have a character set
fall back to the character set of the argument of the
json function that represent the document being operated
on. If this doesn't have a character set fall back to
my_charset_utf8mb4_bin.

This covers the 11.4 JSON_KEY_VALUE function also as it
reuses the Json_path_extractor::extract method.

Add nonnull and warn_unused_result to the json path
functions to facilitate compiler and UBSAN catching of the
problem early.

As null values of s_p are incompatible with report_path_error,
jump directly to a null return which is consistent with
the defination of the JSON sql funciton.
2026-08-01 09:14:49 +10:00
forkfun
eaca80bf32 Merge branch '11.4' into '11.8' 2026-07-29 12:27:46 +02:00
forkfun
08eeb6a417 Merge branch '10.11' into '11.4' 2026-07-28 13:14:38 +02:00
Daniel Black
8370daf768 MDEV-40414 Server crash in Charset::charset upon JSON operations
JSON_EQUALS didn't check that ::val_str() of its arguments
where not-null before attempting to test their equality.

JSON_OVERLAPS also ensure that ::val_json() isn't null.
2026-07-25 09:46:08 +10:00
forkfun
ad7f29afb3 Merge branch 10.6 into 10.11 2026-07-23 11:12:27 +02:00
Sergei Golubchik
59141cbd2c MDEV-39817 JSON_ARRAYAGG() returns garbage because of off-by-one error
don't truncate to a position somewhere between
[old_length, max_length-1] if old_length == max_length.
Simply set the length to old_length in this case.

Also, max_length cannot be zero.
2026-07-22 13:22:02 +02:00
Daniel Black
36b0c525ef MDEV-39742 11.4 JSON functions not interuptable
The JSON functions added in 11.4 where not interuptable
with a KILL QUERY or exceeding the max_statement_time.

* JSON_ARRAY_INTERSECT
* JSON_OBJECT_TO_ARRAY
* JSON_SCHEMA_VALID

This behaviour is corrected by setting the killed_ptr
of the json_engine_t structure.

JSON_KEY_VALUE was added in 11.4, however as it doesn't call
json_next_value(), its processing doesn't check the killed pointer
of the json engine. So its quick anyway.

JSON_OBJECT_FILTER_KEYS, has some structural problem. These
will be addressed in MDEV-39941 and the KILL QUERY/max_statement_time
will be implemented then.

Reviewed by: Rucha Deodhar
2026-07-16 11:22:45 +10:00
Daniel Black
413ff6aed4 json_key_value - simplify error handling
Same result, just consolidating the implementation.
2026-07-16 11:22:45 +10:00
Daniel Black
0a31f7aeba MDEV-40175: JSON_VALID (postfix)
a90779098b changed the
interface to the json_valid to take a json_engine_t
so that it could be interupted and so that warnings
could become visible.

On merge from that 10.11 to 11.4 it was discovered
that the ColumnStore engine uses this interface.

Restored the original json_valid function exactly
how ColumnStore expects this and added renamed
the function with json_engine_t to be json_valid_engine.

This follows the same convention as in the commit
756fc6fd9a that was
for json_normalize, that ColumnStore also used.
2026-07-16 09:02:45 +10:00
Daniel Black
78f8a49ed2 MDEV-40175: JSON_VALID (postfix)
a90779098b changed the
interface to the json_valid to take a json_engine_t
so that it could be interupted and so that warnings
could become visible.

On merge from that 10.11 to 11.4 it was discovered
that the ColumnStore engine uses this interface.

Restored the original json_valid function exactly
how ColumnStore expects this and added renamed
the function with json_engine_t to be json_valid_engine.

This follows the same convention as in the commit
756fc6fd9a that was
for json_normalize, that ColumnStore also used.
2026-07-16 09:01:16 +10:00
Daniel Black
e94ae88e1a MDEV-40165: json_normalize (fix)
14c16e02b2 changed the
interface for json_normalize to take a json_engine_t so that
there could be some error checking.

Unfortunately ColumnStore depended on this interface.

Restored original interface of json_normalize() and
added a json_normalize_engine() that take as engine as the
argument.
2026-07-16 09:01:16 +10:00
Daniel Black
78c575806f MDEV-28404 json killed_ptr cleanup
Since 24ee5fd6bf the result of the killed_ptr
is part of the je->s.error after json_scan_next.
2026-07-15 16:21:59 +10:00
Daniel Black
03d70be617 merge - json_normalize changes 2026-07-15 16:21:59 +10:00
Daniel Black
756fc6fd9a MDEV-40165: json_normalize (fix)
14c16e02b2 changed the
interface for json_normalize to take a json_engine_t so that
there could be some error checking.

Unfortunately ColumnStore depended on this interface.

Restored original interface of json_normalize() and
added a json_normalize_engine() that take as engine as the
argument.
2026-07-15 16:21:59 +10:00
Daniel Black
1e972a8d70 Merge 10.11 into 11.4 2026-07-14 12:49:36 +10:00
Daniel Black
a90779098b MDEV-40175: JSON_VALID don't respect KILL QUERY
Or max_statement_time limit.

Change the json_valid to take a json_engine_t argument.
Adjust the Item_json_valid to have a json_engine_t and
to report an error.

Note this means that invalid json syntax now has a
note on any json error. Raising this to a warning can
cause check constraints of JSON_VALID() to result
in ER_JSON_SYNTAX rather than ER_CONSTRAINT_FAILED
(valided in main.type_json test).

Consequential changes in debug build to json acl
testing validation and unit test having a dummy json_engine_t.

json_normalize also uses the same engine that was
passed as its argument to test json_valid. Because
the character set may have changed, if there's an error
adjust the je->s.c_str pointer. Its only the offset that
is currently used in the error messages anyway so there
may be offset errors depending on the original character
set.
2026-07-14 11:59:00 +10:00
Daniel Black
14c16e02b2 MDEV-40165: JSON_EQUAL/JSON_NORMALIZE dont respect KILL QUERY
Or max_statement_time limit.

JSON_EQUAL/JSON_NORMALIZE also didn't report warnings
on invalid JSON inputs.

Change the json_normalize function to take a json_engine_t
as a argument and implement the sql/item_jsonfunc.cc calls
to have a json_engine_t.

To the JSON_EQUALS and JSON_NORMALIZE, add error handling
to produce an error when there was one. compare_nested_object,
part of JSON_OVERLAPS, uses a new json_engine and
copies the error/position to the returning function.

json_normalize expects the incoming je structure to have
a valid, or nullptr, killed_ptr, as json_start will reset this.
2026-07-14 11:59:00 +10:00
Daniel Black
ca87d09917 MDEV-40165 10.11 JSON functions don't respect KILL QUERY
Or max_statement_time limit.

Implement interuptability of:
* JSON_OVERLAPS
* JSON_LENGTH
* JSON_DEPTH

As json_report_error now processes the killed there's
no need of thd->check_killed(). Removed this from a few
locations.

Verified that the path after every json_scan_next, the
location of checking the killed_ptr, reports errors
correctly.
2026-07-14 11:59:00 +10:00
Marko Mäkelä
761c2333ca Merge 11.4 into 11.8 2026-06-26 13:46:45 +03:00
Marko Mäkelä
3ce999df6b Merge 10.11 into 11.4 2026-06-26 13:45:38 +03:00
Daniel Black
f249b0196d Merge 10.6 into 10.11 2026-06-26 17:56:20 +10:00
Daniel Black
24ee5fd6bf MDEV-28404 JSON functions don't respect KILL QUERY
Or max_statement_time limit or MAX_DEPTH.

Correct interuptability and depth checking of:
* JSON_CONTAINS
* JSON_CONTAINS_PATH
* JSON_EXISTS
* JSON_EXTRACT
* JSON_KEYS
* ST_GeomFromGeoJSON

Populate in the json library the killed as a specific
error message. As the json library has a killed_ptr
is approprate it handles it.

As json_report_error now processes the killed there's
no need of thd->check_killed().

Verified that the path after every json_scan_next, the
location of checking the killed_ptr, reports errors
correctly.
2026-06-26 07:32:21 +03:00
Daniel Black
0ff2fda7b3 MDEV-30518: JSON functions cannot be killed on big-endian (backport)
Backport to 10.6 without JSON_OVERLAPS

The json_engine killed_ptr was of type uchar however the enum
in the server is dependant on the architecture. On big-endian
architectures like IBM Z, retreiving a uchar* retreived the
unmodifed part of the THD->kill enum location and was always 0.

As C++11 allows enums to inherit a type, used uint32_t as the base
class of killed_state type in the server and used uint32_t in the
json library.

reload_acl_and_cache required an expression change to avoid
the compile error:
sql/sql_reload.cc:472:24: error: enumerated and non-enumerated
type in conditional expression [-Werror=enum-conversion]
  472 |  return result || (thd ? thd->killed : 0);

Added the kill_ptr assignment the following functions can be
killed:
* Item_func_json_depth::val_int
* Item_func_json_type::val_str
* Item_func_json_length::val_int

Item_func_json_array_append::val_str, check_killed() only applied
to json_error: label and not return_null.

Item_func_json_format::val_str(), corresponding to the SQL,
json_compact, json_detailed, json_loose - add debug instrumentation
for the func_json_notembedded test.

MDEV-26726 (fcd345de48) added a json_pause_execution sync point
however the test case used debug_max_statement_time, which doesn't
exist. Fix the sync point name in the test.
2026-06-26 07:32:21 +03:00
Rucha Deodhar
29417cfe37 MDEV-32726: Fix failing test for freebsd for json
(10.6 backport)

Json test about max statement time fails with freebsd because on some
architectures the test might execute faster and the statement may not fail.

To simulate failure regardless of architecture, introduce a wait of seconds
longer than the max_statement_time.
2026-06-26 07:32:21 +03:00
forkfun
867d16cea7 Merge branch '11.4' into '11.8' 2026-06-23 12:52:39 +02:00
forkfun
bbd558c386 Merge branch '10.11' into '11.4' 2026-06-22 12:26:09 +02:00
Oleksandr Byelkin
0d59c434fa Merge branch '10.6' into 10.11 2026-06-21 15:32:20 +02:00
Kristian Nielsen
420e28ce28 Merge 11.4 to 11.8
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2026-06-15 15:15:10 +02:00
Kristian Nielsen
85ec122c15 Merge 10.11 to 11.4
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2026-06-15 12:50:45 +02:00
Vladislav Vaintroub
080a7318ac Merge 11.4 to 11.8 2026-06-01 13:59:20 +02:00
Daniel Black
478d6f5561 MDEV-37864 json_array_intersect result has incorrect length
MDEV-37864 added a length setting to the function
Item_func_json_array_intersect::prepare_json_and_create_hash
however some paths of Item_func_json_array_intersect::fix_length_and_dec
never reach this function.

Leave the length calculation the same but place in the
::fix_length_and_dec function.

Collation is based on the arg[0], but potentially after they
are swapped.

Corrects cursor protocol for the test case added in MDEV-36808.
2026-05-28 10:18:18 +10:00
Sergei Golubchik
fe3039eef7 MDEV-39673 group_concat ignores max_allowed_packet
GROUP_CONCAT is limited by group_concat_max_len,
but also, as a string function it must respect max_allowed_packet.

Let's introduce THD::gconcat_max_len() helper to simplify checks.

Also:
* make max group_concat_max_len value the same as max max_allowed_packet
* use the same MY_MIN((ulonglong) ..., UINT_MAX32) in
  Item_func_json_objectagg as in Item_func_group_concat
* use overflow-safe type for lengths in Item_func_quote
  (MAX_MAX_ALLOWED_PACKET is 1G so uint cannot overflow yet, but it's
  a fragile assumption)
2026-05-20 22:49:21 +02:00
Marko Mäkelä
b89cb5d5c7 Merge mariadb-10.11.17 into 10.11 2026-05-19 08:12:14 +03:00
Daniel Black
17ac875377 MDEV-30518: JSON functions cannot be killed on big-endian (fix)
Test func_json_notembedded became unreliable after
509e40e706 where JSON_ARRAY_APPEND
would return NULL rather than:
ERROR 70100: Query execution was interrupted (max_statement_time exceeded)

Restore the original fragment.
2026-05-15 17:53:25 +10:00
Daniel Black
509e40e706 MDEV-30518: JSON functions cannot be killed on big-endian
The json_engine killed_ptr was of type uchar however the enum
in the server is dependant on the architecture. On big-endian
architectures like IBM Z, retreiving a uchar* retreived the
unmodifed part of the THD->kill enum location and was always 0.

As C++11 allows enums to inherit a type, used uint32_t as the base
class of killed_state type in the server and used uint32_t in the
json library.

reload_acl_and_cache required an expression change to avoid
the compile error:
sql/sql_reload.cc:472:24: error: enumerated and non-enumerated
type in conditional expression [-Werror=enum-conversion]
  472 |  return result || (thd ? thd->killed : 0);

Added the kill_ptr assignment the following functions can be
killed:
* Item_func_json_depth::val_int
* Item_func_json_type::val_str
* Item_func_json_length::val_int

Item_func_json_array_append::val_str, check_killed() only applied
to json_error: label and not return_null.

Item_func_json_overlaps::val_bool ensure check_killed() is called.

Item_func_json_format::val_str(), corresponding to the SQL,
json_compact, json_detailed, json_loose - add debug instrumentation
for the func_json_notembedded test.

MDEV-26726 (fcd345de48) added a json_pause_execution sync point
however the test case used debug_max_statement_time, which doesn't
exist. Fix the sync point name in the test.
2026-05-14 08:44:01 +10:00
Oleksandr Byelkin
9cb70f0220 Merge branch '11.4' into 11.8 2026-05-08 08:11:03 +02:00
Oleksandr Byelkin
7598e4e95d Merge branch '10.11' into 11.4 2026-05-08 08:00:01 +02:00
Oleksandr Byelkin
6828ff7390 Merge branch '10.6' into 10.11 2026-05-08 07:38:22 +02:00
Rucha Deodhar
902d05a22a MDEV-39213: json range syntax crash
Analysis:
When json is being parsed, the step decreases without a out-of-bound check
resulting in failure.
Fix:
Before decreasing the step, check if it will result into out of bound.
2026-05-07 14:39:35 -04:00
Daniel Black
97642a1353 MDEV-26814: UBSAN: offset to nullptr in JSON_ARRAY_INSERT
SELECT JSON_ARRAY_INSERT (0,NULL,1); triggered a UBSAN error.
Specification of JSON_ARRAY_INSERT should return NULL if any arguments
are null.

SQL NULL, aka Item_null::val_str will return a nullptr so check this and
then return a NULL value.
2026-05-05 09:04:12 +10:00