MyCTX used an EVP_CIPHER_CTX in a stack buffer instead of allocating it
with EVP_CIPHER_CTX_new(). OpenSSL 4.0 rejects a context that was not
created that way for ciphers that use an IV: EVP_CipherInit_ex() fails
with "invalid iv length", so AES_ENCRYPT() and KDF() in CBC/CTR/GCM
modes return NULL. ECB has no IV and still works, which is why only the
non-ECB modes broke.
Allocate the context with EVP_CIPHER_CTX_new()/EVP_CIPHER_CTX_free(), and
on an init error free and null it, so a later update()/finish() from a
caller that ignored the error is a safe no-op, not a double free. Remove
check_openssl_compatibility() and the EVP_CIPHER_CTX_SIZE and
EVP_CIPHER_CTX_init macros.
my_md5() used the same stack-buffer-sized-by-EVP_MD_CTX_SIZE hack, whose
only guard was check_openssl_compatibility(). MD5_CTX is a concrete struct
in both OpenSSL and WolfSSL, so use it directly with MD5_Init/Update/Final
on a caller-owned context, like my_sha.inl does for SHA*_CTX; this removes
the EVP_MD_CTX hack and EVP_MD_CTX_SIZE. The exception is OpenSSL 1.0.x,
which aborts on the low-level MD5 API in FIPS mode (MDEV-7788): there, and
only there, keep the EVP path with the non-FIPS flag. 1.1.0 removed the
FIPS Object Module and 3.0+ low-level MD5 bypasses the provider, so the
concrete MD5_CTX is correct on every other version. SHA is unaffected, it
already used concrete SHA*_CTX structs.
Verified against OpenSSL 4.0.1: the mysys aes-t test fails on the
CBC/CTR/GCM cases with the stack buffer and passes with
EVP_CIPHER_CTX_new(). The 1.0.x MD5 path verified against a FIPS build of
OpenSSL 1.0.2u: the EVP path computes MD5, the low-level path aborts.
No visible performance degradation: the extra allocation costs
~12 ns/call (WolfSSL) and ~30 ns (OpenSSL) on Windows at a 30-byte
payload, nothing at 16 KB, and nothing on Linux/glibc; sysbench OLTP
over encrypted tables and redo log is unchanged.
Tests main.tlsv13 and main.ssl_crl: OpenSSL 4.0.3 reports TLS alerts as
"tls alert ..." (was "ssl/tls alert ..."), accept both.
Assisted-by: Claude:claude-opus-4-8
This SQL script failed:
SET NAMES latin1 COLLATE latin1_swedish_ci;
CREATE OR REPLACE VIEW v1 AS SELECT 'Y' AS c1;
SET NAMES big5 COLLATE big5_chinese_ci;
SELECT * FROM v1 WHERE c1='y';
with the following error:
ERROR 1267 (HY000): Illegal mix of collations
(latin1_swedish_ci,COERCIBLE) and (big5_chinese_ci,COERCIBLE)
for operation '='
Note, latin1_swedish_ci and big5_chinese_ci are used here as examples.
The error also happened with different collation combinations.
Fix main idea:
If two collations have equal comparison rules (known as "tailoring")
on a given character repertoire,
like latin1_swedish_ci and big5_chinese_ci on ASCII letters,
then the "Illegal mix of collations" error can be avoided in
a comparison operator. We can choose any of the sides as the operation
effective collation - the result will be equal.
This optimization is not applied when at least one side has an explicit
COLLATE clause. Two explicit COLLATE clauses in one comparison are
already illegal when the character sets are the same, so for
consistency this stays illegal when the character sets differ too.
Most important details:
- Splitting enum_repertoire_t into smaller subsets,
for better repertoire granularity.
A variable holding a repertoire value can now have multiple
MY_REPERTOIRE_XXX flags set.
This patch implements detecting tailoring equality
on these repertoires:
* MY_REPERTOIRE_ASCII_DIGITS - [0..9]
* MY_REPERTOIRE_ASCII_MINUS_DOT_DIGITS - [-.0..9], e.g. '-10.5'.
It consists of the new bits MY_REPERTOIRE_ASCII_MINUS and
MY_REPERTOIRE_ASCII_DOT. They were a part of
MY_REPERTOIRE_ASCII_NOT_IDENT before. As a result, the numeric values
of MY_REPERTOIRE_ASCII and MY_REPERTOIRE_UNICODE30 changed
(223 and 255).
* MY_REPERTOIRE_ASCII_ALNUM - [A..Z,a..z,0..9].
* MY_REPERTOIRE_ASCII_IDENT - ALNUM + underscore
* MY_REPERTOIRE_ASCII - the entire range U+0000..U+007F
- Adding a new virtual function "tailoring" in my_collation_handler_st.
It returns the tailoring on the given repertoire for the given
collation.
If cs1->coll->tailoring(cs1, some_repertoire) returns {0,0},
it means the illegal mix optimization cannot be used
for this collation on the given repertoire.
If these calls:
tr1= cs1->coll->tailoring(cs1, some_repertoire);
tr2= cs2->coll->tailoring(cs2, some_repertoire);
return both non-NULL results and tr1.str==tr2.str,
then these collations are equal on the given repertoire
and are mutually replaceable for a comparison operator,
so "Illegal mix of collations" can be avoided.
- Tailoring strings are shared constants my_tailoring_str_*
(strings/strings_def.h). Tailorings are compared by the string
pointer, so tailoring() implementations return strings only from
these constants. The strings do not depend on PAD/NOPAD.
- my_string_repertoire() for ucs2, utf16, utf32 and NONASCII 8bit
character sets no longer stops at the first bad byte sequence with
the repertoire found so far, which could be MY_REPERTOIRE_NONE
(compatible with everything). It returns MY_REPERTOIRE_EXTENDED
for such strings. A unit test was added into unittest/strings.
- Adding CHARSET_INFO::is_ascii_superset(). It tells if a character set
can store all ASCII characters (it is false for swe7).
It is used in Item_func_conv_charset to calculate "safe", and in
left_is_algorithmically_simpler() to prefer other character sets
to those where ASCII strings cannot be converted safely.
- DBUG_ASSERTs were added into mysys/charset.c, to check that
the "tailoring" method is set in collation handlers when a collation
is added: compiled-in, from ctype-extra.c, or loaded from Index.xml.
- Adding a new function my_collations_equal_on_repertoire(cs1, cs2,
repertoire) in strings/. It tells if two collations are equal on
the repertoire, taking tailoring() results and PAD/NOPAD and the UCA
version into account. Callers do not compare tailoring() results
directly, so the way tailorings are compared (currently by the pointer
to the shared constant string) is private to strings/ and can be
extended later, e.g. for UCA collations on MY_REPERTOIRE_UNICODE30.
- Adding a new method DTCollation::aggregate_by_tailoring().
Collations with different MY_CS_NOPAD are never compatible.
UCA collations of different versions are not compatible on
the repertoire with ASCII punctuation, as the UCA version
affects its order (e.g. UCA-6.2.0 moved GRAVE ACCENT
and CIRCUMFLEX ACCENT after PERCENT SIGN).
- DTCollation::aggregate() now makes the repertoire of the result cover
the repertoires of both sides in all successful branches.
Before, set(dt) replaced the repertoire with the repertoire of the
winner side only, so the result could have a too narrow repertoire,
e.g. IF(1, 1.5, HEX(255)) had MY_REPERTOIRE_ASCII_ALNUM although the
value '1.5' has a punctuation character. With the more granular
repertoires this could wrongly allow mixing collations which are
equal on ALNUM but different on punctuation.
- Adding a new flag MY_COLL_ALLOW_BY_TAILORING.
It indicates to DTCollation::aggregate() that the illegal
mix optimization by repertoire can be used in the given context.
MY_COLL_CMP_CONV now includes MY_COLL_ALLOW_BY_TAILORING.
Note, only comparison operators pass this flag.
Functions returning a string result do not pass this flag,
because in operations like CONCAT(a,b) we still need to evaluate
precisely the collation of the result - we cannot just choose
a collation of one of the sides (even if they are compatible
on the given repertoire). This also applies to ExtractValue()
and UpdateXML(), which now aggregate their arguments without
this flag.
- As in my_repertoire_t the value MY_REPERTOIRE_ASCII is now a set of
bits rather than a single bit, the way to detect "is only ASCII"
repertoires has changed in the code.
For example:
// repertoire *IS* ascii
if (repertoire == MY_REPERTOIRE_ASCII)
has changed in multiple places in the code to
// repertoire *HAS* only ascii characters
if (my_repertoire_is_subset_of(repertoire, MY_REPERTOIRE_ASCII))
The new function my_repertoire_is_subset_of() in m_ctype.h
is used for this purpose in C code. In C++ code
DTCollation::repertoire_is_subset_of() is used, e.g.:
if (collation.repertoire_is_subset_of(MY_REPERTOIRE_ASCII))
- Repertoire of some expressions was adjusted to the new meaning:
* Item_null now has MY_REPERTOIRE_NONE (was ASCII).
* MY_LOCALE::repertoire() now returns MY_REPERTOIRE_UNICODE30
(was EXTENDED).
* Lex_string_with_metadata_st::repertoire(cs) now scans the string
contents to detect the actual repertoire.
* HEX() now has MY_REPERTOIRE_ASCII_ALNUM.
* DATE_FORMAT() decides on MY_REPERTOIRE_EXTENDED using the locale
which is actually used: the explicit third argument, or
@@lc_time_names. It was always @@lc_time_names before.
A non-constant locale argument is assumed to be non-ASCII.
* QUOTE, MAKE_SET, EXPORT_SET, LPAD, RPAD, GROUP_CONCAT, JSON_ARRAY,
JSON_OBJECT and JSON_OBJECTAGG add the repertoire of the extra
characters they put into the result (quotes, separators, padding,
brackets).
* LOWER() and UPPER() add the letters of both cases to the repertoire,
because the result can contain letters of the opposite case.
In Turkish collations they also add MY_REPERTOIRE_EXTENDED, as an
ASCII letter can be converted to a non-ASCII one (I -> dotless i).
Unicode collations are detected by the new member
casefold_info_st::can_convert_to_non_ascii_on_casefolding,
simple 8bit collations by to_lower['I'] and to_upper['i'].
- Lex_string_with_metadata_st::repertoire(cs) now scans a string literal
for any byte oriented character set (mbminlen==1), not only for
ASCII based ones. It was assumed to have the full repertoire for swe7,
which is not ASCII based. The scan is done by Unicode code points, so
swe7 strings with digits and letters get a narrow repertoire, while
e.g. 0x5B (LATIN CAPITAL LETTER A WITH DIAERESIS in swe7) is non-ASCII.
Therefore swe7_swedish_ci can be mixed with other collations now,
in the same way as other collations with MY_CS_IDENT_CASEUP_CI.
The test func_time was adjusted: a swe7 literal ' ull' with ASCII
characters is now safe to convert, the error is still tested for '[ull'.
- cp1250_czech_cs sorts -, . and 0..9 in the code point order, so it gets
the MINUS_DOT_DIGITS tailoring. latin2_czech_cs gets only DIGITS,
as it sorts the minus and the dot differently. Both were verified
with ctype_digits_verify.
- The tis620 collations now set CHARSET_INFO::tab_to_uni (was NULL),
so my_charset_is_ascii_based() is true for tis620.
It was false for tis620, although tis620 is ASCII compatible.
Therefore tis620 string literals were always considered to have
a non-ASCII repertoire, and the repertoire optimizations did not work
for them. Now ASCII-only tis620 literals are detected as ASCII.
This changes the result of subselect_extra_no_semijoin
from an error to success.
- tis620_thai_ci and tis620_thai_nopad_ci fold letters to lower case
and then compare by the code point on the entire ASCII range
(the Thai specific rules affect only non-ASCII characters).
A new function my_tailoring_ascii_casedn_ci() returns tailorings
for such collations on ALNUM, IDENT and ASCII. For example, the
underscore sorts before the letters, unlike in the caseup tailorings.
- New flags were added for CHARSET_INFO::state
* MY_CS_ASCII_CASEUP_CI - for simple 8bit case insensitive collations.
It means that this collation has no irregularities on the ASCII
repertoire, and converts lower case letters to upper case ones,
so the underscore sorts after the letters (with an upper to lower
case conversion it would sort before the letters).
* MY_CS_IDENT_CASEUP_CI - for simple 8bit case insensitive collations.
It means that this collation has no irregularities on the IDENT
repertoire (but can have irregularities say on punctuation).
* MY_CS_ASCII_STD_UCA - for UCA collations.
It means that a UCA collation does not reorder ASCII characters.
* MY_CS_DIGITS_STD - for any collation. It means that the ASCII digits
0..9 are sorted in the code point order and are not tailored, and,
for PAD collations, the space sorts before the digits. Strings of
digits are compared in the same way in all such collations, even if
they have irregularities on letters (latin7, cp866, czech, danish,
turkish UCA, etc.), or differ by case sensitivity. It is detected
from sort_order for 8bit collations, and from the tailoring rules
for UCA collations. Collations with MY_CS_DIGITS_STD still must have
the same PAD/NOPAD attribute.
* MY_CS_ASCII_MINUS_DOT_DIGITS - for any collation. Means
MY_CS_DIGITS_STD, and additionally the minus sorts before the dot,
which sorts before the digits (and the space sorts before the minus
in PAD collations). Strings like '-10.5' are compared in the same
way in all such collations. It is detected from sort_order for simple
8bit collations (my_coll_init_simple()), and from the tailoring rules
for UCA collations. For the multi-byte collations big5_chinese_ci,
gbk_chinese_ci, gb2312_chinese_ci and sjis_japanese_ci it is declared
by my_tailoring_ident_caseup_ci_generic(). The flags MY_CS_DIGITS_STD
and MY_CS_ASCII_MINUS_DOT_DIGITS are stored in ctype-extra.c for
compiled collations (conf_to_src.c prints them), and the detection is
skipped for a collation which already has any of the repertoire flags
(MY_CS_REPERTOIRE_FLAGS).
- strings/conf_to_src.c was modified to detect and print
MY_CS_ASCII_CASEUP_CI and MY_CS_IDENT_CASEUP_CI flags.
- strings/ctype-extra.c was regenerated with new flags.
- 8bit collations are not given MY_CS_ASCII_CASEUP_CI and
MY_CS_IDENT_CASEUP_CI if they are PAD and the space does not sort
before the digits. A shorter string is padded with spaces for
comparison, so the position of the space matters even for strings
without spaces ("ab" is compared to "abc" as "ab " to "abc").
A test collation latin1_space_after_letters_ci was added to
mysql-test/std_data/ldml/ for this. ctype-extra.c does not change.
- The initializer of my_collation_cs_handler in ctype-utf8.c
(utf8mb3_general_cs, under HAVE_UTF8_GENERAL_CS) was fixed to match
my_collation_handler_st: the missing get_id and get_collation_name
members were added, so that eq_collation was not initialized in their
place, and the new member "tailoring" is now set to my_tailoring_none.
- Results of the existing tests func_str, func_test, ps, view
and subselect_sj changed from "Illegal mix of collations" to success.
In these tests ASCII-only strings were compared using collations
with equal tailoring on ASCII, e.g. latin1_swedish_ci with
latin2_general_ci (func_str, ps), koi8r_general_ci with
latin1_swedish_ci (func_test), latin1_general_ci with
latin1_swedish_ci on letters and digits (view), cp932_japanese_ci
with latin1_swedish_ci (subselect_sj).
Such comparisons are now allowed.
Cases with collations that are still incompatible on ASCII,
e.g. latin7_general_ci, were added to the tests and still fail.
- Adding a number of MTR tests in
plugin/func_test/mysql-test/func_test/.
They display a tailoring by collation name and repertoire as
returned by:
cs->coll->tailoring(cs, some_repertoire)
A dynamically linked plugin function collation_tailoring() was added
for the purpose of these tests. Tailorings do not depend on PAD/NOPAD
and on the UCA version, so the function shows "[nopad]" for NOPAD
collations, and "[version X.Y.Z]" for UCA collations on the ASCII
repertoire. The version is formatted by a new class UCAVersion,
which uses a new method CharBuffer::append_uint8().
- Adding the test plugin/func_test/mysql-test/func_test/
ctype_caseup_ci_verify.test. It checks the order of ASCII characters
for all collations declared "caseup CI" on the ASCII or IDENT range.
- Adding the test plugin/func_test/mysql-test/func_test/
collation_tailoring_args.test. It checks the arguments of the plugin
function collation_tailoring(): the repertoire passed by name or by
number, and the NULL results for unknown names.
- Adding the test plugin/func_test/mysql-test/func_test/ctype_ldml.test.
It displays the tailorings of collations loaded at runtime from
Index.xml: simple 8bit collations (flags detected from the weights)
and Unicode collations (LDML rules which do or do not reorder ASCII).
- Adding a number of MTR tests mysql-test/main/ctype_xxx_tailoring.test
They display two-dimensional charts showing which collations
are compatible on which repertoires.
Tests for DATE_FORMAT() with an explicit locale were added to
ctype_big5_tailoring.test, ctype_cp932_tailoring.test and
ctype_latin5_tailoring.test, which also has the chart for latin5.
- Adding a number of MTR tests in the form of the originally
reported script for various collations:
SELECT Insert_priv FROM mysql.user WHERE Insert_priv='...';
The repeated blocks of queries are shared through
mysql-test/include/
ctype_tailoring_0{1,2,3}_mysql_user_Insert_priv*.inc.
- Adding tests to ctype_cp932.test checking that ExtractValue() and
UpdateXML() still raise "Illegal mix of collations".
When a select containing 32 ranges is made on a table containing a
compound key with 32 parts, the range optimizer can run off the end
of a stack variable, invalidly overwriting subsequent stack variables.
In the struct st_sel_arg_range_seq, we have an array
RANGE_SEQ_ENTRY stack[MAX_REF_PARTS];
MAX_REF_PARTS is 32.
check_quick_select
/ sel_arg_range_seq_init
initialises stack[0] as NOT a key part
/ sel_arg_range_seq_next
iterates through the key parts, adding key part n to stack[n+1]
key part #32 gets referenced by step_down_to(), setting seq->i off the
end of the array.
Fix: RANGE_SEQ_ENTRY stack[MAX_REF_PARTS+1];
The above change exposed an issue with key length calculation on
MS Windows. Calling make_prev_keypart_map(32) caused the resultant
bitmap to be calculated as (1UL << 32) - 1. Using the MSVC compiler
this resulted in an empty key length calculation during
handler::index_read_map, causing an assertion in ha_innobase::index_read().
As we only need 32 bits to represent our key map, we change the type thus
-typedef ulong key_part_map;
+typedef uint32 key_part_map;
We correct make_keypart_map() and make_prev_keypart_map() to call our
overflow safe my_set_bits(). We also correct bka_range_seq_next()
and bkah_range_seq_next() to use make_prev_keypart_map().
We also add some DBUG_ASSERTS in key_part_map processing elsewhere,
exposing some issues in our BNLH implementation. We cap the number of
keyuse parts here, altering the explain output of 2 of our tests.
Numerous places needed bit shift operations altered to use
make*keymap_part and various format strings needed to be corrected.
This works on Linux and on Windows, with rpm/deb/tar.gz/zip
installations.
For rpm/deb it just works, for tar.gz/zip there is no
standard location, so one needs to configure plugin with
-DCMAKE_PREFIX_PATH=/pah/to/mariadb/basedir
after that, `cmake --install .` works too, installing in the same
basedir.
`cmake --build . --target package` works, creating rpm/deb/targz/zip
depending on whether it's Linux or Windows and whether -DRPM or -DDEB
was specified.
* create and install mariadb-plugin-config.cmake
* for now it only supports one plugin per project, error out
if there are many
* deb: move all headers that plugins need to libmariadb-dev,
together with libmysqlservices.a. At least until we'll
create mariadb-plugin-dev. Nobody should need huge
libmariadbd-dev to develop a plugin
* rpm: all in MariaDB-devel already, no changes here
* install wsrep headers too, THD layout depends on WITH_WSREP
* show DBUG_OFF, ENABLED_DEBUG_SYNC, and SAFE_MUTEX to plugins, same
reason (it doesn't happen automatically as they're not in my_config.h)
* but don't install config.h - high chance of name conflict with other
projects and it's an exact copy of my_config.h anyway.
* adjust plugin.cmake to work for external plugins
* move server-internal part of it to top-level CMakeLists.txt
* remove double-defined macros from unireg.h (the guard doesn't help
if unireg.h is included first)
* package plugin metadata as yaml in .tar.gz/.zip
ColumnStore, until fixed, needs a backward-compatibility workaround
Defer the host-privileged/host-blocked check for a PROXY-header-derived
address until after the client's SSL handshake completes, instead of
sending it immediately in clear text.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
my_large_malloc() attempted large pages whenever --large-pages was
enabled, silently rounding the size up and reporting it back via an
in/out parameter. ut_malloc_dontdump() never passed that adjusted
size on to its own callers (the InnoDB redo log buffer and
recv_sys_t::tmp_buf), so freeing later used the original, smaller
size, causing the reported "faux memory leak".
Only the buffer pool and the MyISAM/Aria key caches are documented
to benefit from large pages. Everything else that ended up calling
my_large_malloc() only wanted its "do not dump to core" property and
picked up large pages as an undocumented side effect; those buffers
are also small and sequentially accessed, so they would have gained
little from large pages anyway.
Add MY_TRY_LARGE_PAGES: my_large_malloc() and my_large_virtual_alloc()
now only attempt large pages when a caller passes this flag, instead
of always trying whenever the global option is set. Only the buffer
pool and the key caches pass it. The redo log buffer, tmp_buf, and
row0log.cc's crypt buffers no longer request large pages at all,
which removes the size-rounding bug for them without touching that
code.
my_large_virtual_alloc()'s fallback (no usable large page size) must
also return read-write memory right away, like the Windows large-pages
fallback already does, since my_virtual_mem_commit() is a no-op for
MY_TRY_LARGE_PAGES. my_large_pages_flag is now set once, in
my_init_large_pages(), and never changed thereafter, on any platform.
Both my_virtual_mem_commit() and my_virtual_mem_decommit() are now
no-ops, aside from accounting, whenever large pages are requested.
Also fix a broken mtr suppression regex in main.large_pages that
would fail the test on Windows.
Across a range of JSON functions taking a path argument
there are SQL expressions that dont' have a character
set. If these expressions don't have a character set
fall back to the character set of the argument of the
json function that represent the document being operated
on. If this doesn't have a character set fall back to
my_charset_utf8mb4_bin.
This covers the 11.4 JSON_KEY_VALUE function also as it
reuses the Json_path_extractor::extract method.
Add nonnull and warn_unused_result to the json path
functions to facilitate compiler and UBSAN catching of the
problem early.
As null values of s_p are incompatible with report_path_error,
jump directly to a null return which is consistent with
the defination of the JSON sql funciton.
Protect all READ_ONLY sysvars from run-time changes.
Put them into a separate section and use my_virtual_mem_protect()
to make this section read-only before the server starts accepting
connections.
Verify that they're all protected in the sys_var constructor.
One exception: opt_noacl (--skip-grant-tables) can be changed
from 1 to 0 on FLUSH PRIVILEGES. Let's briefly drop the
protection for this 1->0 change. It can be needed only once
in a server lifetime and only if it was started with --skip-grant-tables
On shutdown the protection is removed, making variables writable
again because shutdown resets some of them during the cleanup
Assisted-By: Claude:claude-4.8-opus
* it's called ROOT_FLAG_VMEM
* it does not imply any protection, ROOT_FLAG_MPROTECT was a misnomer
* it means that the memory is allocated using my_virtual_mem_commit,
not my_malloc, so not on heap
* memory allocated this way can be protected with my_virtual_mem_protect()
* thus root using ROOT_FLAG_VMEM can be protected with protect_root()
Assisted-By: Claude:claude-5-sonnet
make my_virtual_mem_reserve() work on Windows and on Linux
(falling back to my_large_virtual_alloc() - that's what InnoDB
did instead). This makes the usage pattern portable without #ifdef:
- reserve a memory range
- commit (allocate) memory from a reserved range
- decommit memory, keeping it reserved
- release reserved memory back to OS
invalid pointer pairs are when the length/memory of one string are
intermixed with another.
For comp_err, the end null pointer was compared to soffset within
my_strtoll10. As we didn't need the end position a NULL arg option
was compatble.
For uca-dump, Address Sanitizer raises invalid pointer pair because
argv options (opt) don't have an obvious correlation of having a start
at opt_X.length, even though the lstrncmp makes this true.
The DBUG_ASSERTS of strmov (added MDEV-11752) where incompatible with
pointer-compare. Replaced strmov with static inline version in
m_string.h using memmove that allows overlaps, and being inline allows
the uneeded parts of the implementation to be optimized away.
* restore result=NULL after free in mysqlshow.c (to prevent double-free)
* remove DBUG_EVALUATE_IF again
* remove useless 'stop slave' from a test
* restore charset encoding in mysqltest.{test,result}
* correct error for unparseable BINLOG data is ER_SYNTAX_ERROR now
* restore old libfmt on rhel-7, gcc 4.8.5 cannot compile anything newer
With proxy protocol thd_set_peer_addr() runs twice (proxy host, then the
real client from the proxy header). Connect errors are accounted against
the real client, but check_connection() incorrectly uses condition on the
proxy host's count, rather than real client's address.
Fix: reset both the proxy host, and real client's connect errors on
successful connection.
Added tests for incomplete handshake, and reset behavior,
under proxy protocol, for both real client errors, and proxy host errors.
a90779098b changed the
interface to the json_valid to take a json_engine_t
so that it could be interupted and so that warnings
could become visible.
On merge from that 10.11 to 11.4 it was discovered
that the ColumnStore engine uses this interface.
Restored the original json_valid function exactly
how ColumnStore expects this and added renamed
the function with json_engine_t to be json_valid_engine.
This follows the same convention as in the commit
756fc6fd9a that was
for json_normalize, that ColumnStore also used.
a90779098b changed the
interface to the json_valid to take a json_engine_t
so that it could be interupted and so that warnings
could become visible.
On merge from that 10.11 to 11.4 it was discovered
that the ColumnStore engine uses this interface.
Restored the original json_valid function exactly
how ColumnStore expects this and added renamed
the function with json_engine_t to be json_valid_engine.
This follows the same convention as in the commit
756fc6fd9a that was
for json_normalize, that ColumnStore also used.
14c16e02b2 changed the
interface for json_normalize to take a json_engine_t so that
there could be some error checking.
Unfortunately ColumnStore depended on this interface.
Restored original interface of json_normalize() and
added a json_normalize_engine() that take as engine as the
argument.
14c16e02b2 changed the
interface for json_normalize to take a json_engine_t so that
there could be some error checking.
Unfortunately ColumnStore depended on this interface.
Restored original interface of json_normalize() and
added a json_normalize_engine() that take as engine as the
argument.
Or max_statement_time limit.
Change the json_valid to take a json_engine_t argument.
Adjust the Item_json_valid to have a json_engine_t and
to report an error.
Note this means that invalid json syntax now has a
note on any json error. Raising this to a warning can
cause check constraints of JSON_VALID() to result
in ER_JSON_SYNTAX rather than ER_CONSTRAINT_FAILED
(valided in main.type_json test).
Consequential changes in debug build to json acl
testing validation and unit test having a dummy json_engine_t.
json_normalize also uses the same engine that was
passed as its argument to test json_valid. Because
the character set may have changed, if there's an error
adjust the je->s.c_str pointer. Its only the offset that
is currently used in the error messages anyway so there
may be offset errors depending on the original character
set.
Or max_statement_time limit.
JSON_EQUAL/JSON_NORMALIZE also didn't report warnings
on invalid JSON inputs.
Change the json_normalize function to take a json_engine_t
as a argument and implement the sql/item_jsonfunc.cc calls
to have a json_engine_t.
To the JSON_EQUALS and JSON_NORMALIZE, add error handling
to produce an error when there was one. compare_nested_object,
part of JSON_OVERLAPS, uses a new json_engine and
copies the error/position to the returning function.
json_normalize expects the incoming je structure to have
a valid, or nullptr, killed_ptr, as json_start will reset this.
Or max_statement_time limit or MAX_DEPTH.
Correct interuptability and depth checking of:
* JSON_CONTAINS
* JSON_CONTAINS_PATH
* JSON_EXISTS
* JSON_EXTRACT
* JSON_KEYS
* ST_GeomFromGeoJSON
Populate in the json library the killed as a specific
error message. As the json library has a killed_ptr
is approprate it handles it.
As json_report_error now processes the killed there's
no need of thd->check_killed().
Verified that the path after every json_scan_next, the
location of checking the killed_ptr, reports errors
correctly.
Backport to 10.6 without JSON_OVERLAPS
The json_engine killed_ptr was of type uchar however the enum
in the server is dependant on the architecture. On big-endian
architectures like IBM Z, retreiving a uchar* retreived the
unmodifed part of the THD->kill enum location and was always 0.
As C++11 allows enums to inherit a type, used uint32_t as the base
class of killed_state type in the server and used uint32_t in the
json library.
reload_acl_and_cache required an expression change to avoid
the compile error:
sql/sql_reload.cc:472:24: error: enumerated and non-enumerated
type in conditional expression [-Werror=enum-conversion]
472 | return result || (thd ? thd->killed : 0);
Added the kill_ptr assignment the following functions can be
killed:
* Item_func_json_depth::val_int
* Item_func_json_type::val_str
* Item_func_json_length::val_int
Item_func_json_array_append::val_str, check_killed() only applied
to json_error: label and not return_null.
Item_func_json_format::val_str(), corresponding to the SQL,
json_compact, json_detailed, json_loose - add debug instrumentation
for the func_json_notembedded test.
MDEV-26726 (fcd345de48) added a json_pause_execution sync point
however the test case used debug_max_statement_time, which doesn't
exist. Fix the sync point name in the test.
Clang upstream bug https://github.com/llvm/llvm-project/issues/173210
shows that a "int x=x" construct will in the intermediate representation
have code to read from x. With this generated sanitizer=undefined and
static analyzers will all see the uninitialized read and write.
Because clang has a stronger implementation of following paths to
uninitialized variables, "#define UNINIT_VAR(x) x" is the which
is what our release binaries use is the ideal path for this compiler.
Clang based compilers with error during compilation if any
uninitialized behaviour is detected at compile time because of
0c80ddb519.
Corrects MDEV-36542 - 6fd57f478f.
When calling buf_flush_ahead() with furious=false, the function
updates the async target buf_flush_async_lsn and, if necessary notifies
the page cleaner thread to start flushing more eagerly, to avoid
a long stall later when checkpoint will be required.
When many threads call buf_flush_ahead() with furious=false, if
the update of buf_flush_async_lsn is gated inside
buf_pool.flush_list_mutex, this can cause significant contention,
just to deliver the new buf_flush_async_lsn value and a potential
notification to the page cleaner thread.
This commit enables, for the non furious case, lock-free update
of buf_flush_async_lsn and lock-free check of the page cleaner idle
flag, to avoid the contention on buf_pool.flush_list_mutex.
The lock is acquired by the CAS-loop winner thread and only
after the page cleaner is observed idle.
The cleaner clears buf_flush_async_lsn via CAS, so a concurrent
lock-free bump is preserved across the clear.
Since the reads are not protected by the mutex, there is a chance that
the decision to not signal the page cleaner thread is wrong.
In the worst case though, under sustained redo-log pressure,
eventually a wakeup signal will be delivered to the page cleaner thread
anyway, so the risk of a long stall is limited.
Similar reasoning applies to the possibly wrong decision taken by the
page cleaner thread to go to idle: eventually it will be woken up.
scripts/fill_help_tables.sql from
https://github.com/mariadb-corporation/mariadb-docs/ as of
3fb208fcdfed06beb52ef9e305017de2387ff012
Other URL changes are taken from the HTTP redirection URL
from the old KB.
Some test cases where impacted:
* ps_missed_cmds{,_bin_prot} - used a topic_id of 0 that is
now used, used MAX_INT instead.
* range_notembedded used the help data as a test case. As its not
a stable table data, replaces with static data. Test case valid
by ensuring the ANALYZE EXPLAIN and optimizer trace had the same
structure. Also valided the MDEV-24739/MDEV-25069 bug fixes from
where the test case originaled, when reverted, exhibited the
previous behavour before the bug fixes.
The json_engine killed_ptr was of type uchar however the enum
in the server is dependant on the architecture. On big-endian
architectures like IBM Z, retreiving a uchar* retreived the
unmodifed part of the THD->kill enum location and was always 0.
As C++11 allows enums to inherit a type, used uint32_t as the base
class of killed_state type in the server and used uint32_t in the
json library.
reload_acl_and_cache required an expression change to avoid
the compile error:
sql/sql_reload.cc:472:24: error: enumerated and non-enumerated
type in conditional expression [-Werror=enum-conversion]
472 | return result || (thd ? thd->killed : 0);
Added the kill_ptr assignment the following functions can be
killed:
* Item_func_json_depth::val_int
* Item_func_json_type::val_str
* Item_func_json_length::val_int
Item_func_json_array_append::val_str, check_killed() only applied
to json_error: label and not return_null.
Item_func_json_overlaps::val_bool ensure check_killed() is called.
Item_func_json_format::val_str(), corresponding to the SQL,
json_compact, json_detailed, json_loose - add debug instrumentation
for the func_json_notembedded test.
MDEV-26726 (fcd345de48) added a json_pause_execution sync point
however the test case used debug_max_statement_time, which doesn't
exist. Fix the sync point name in the test.
