database/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-08-08 11:22:35 +03:00
Code Activity

MDEV-36213 Doubled memory usage (11.4.4 <-> 11.4.5)

Browse Source 
Fixing the code adding MySQL _0900_ collations as _uca1400_ aliases
not to perform deep initialization of the corresponding _uca1400_
collations.

Only basic initialization is now performed which allows to watch
these collations (both _0900_ and _uca1400_) in queries to
INFORMATION_SCHEMA tables COLLATIONS and
COLLATION_CHARACTER_SET_APPLICABILITY,
as well as in SHOW COLLATION statements.

Deep initialization is now performed only when a collation
(either the _0900_ alias or the corresponding  _uca1400_ collation)
is used for the very first time after the server startup.

Refactoring was done to maintain the code easier:
- most of the _uca1400_ code was moved from ctype-uca.c
  to a new file ctype-uca1400.c
- most of the _0900_ code was moved from type-uca.c
  to a new file ctype-uca0900.c

Change details:

- The original function add_alias_for_collation() added by the patch for
   "MDEV-20912 Add support for utf8mb4_0900_* collations in MariaDB Server"
  was removed from mysys/charset.c, as it had two two problems:

  a. it forced deep initialization of the _uca1400_ collations
     when adding _0900_ aliases for them at the server startup
     (the main reported problem)

  b. the collation initialization code in add_alias_for_collation()
     was related more to collations rather than to memory management,
     so /strings should be a better place for it than /mysys.

  The code from add_alias_for_collation() was split into separate functions.
  Cyclic dependency was removed. `#include <my_sys.h>` was removed
  from /strings/ctype-uca.c. Collations are now added using a callback
  function MY_CHARSET_LOADED::add_collation, like it is done for
  user collations defined in Index.xml. The code in /mysys sets
  MY_CHARSET_LOADED::add_collation to add_compiled_collation().

- The function compare_collations() was removed.
  A new virtual function was added into my_collation_handler_st instead:

    my_bool (*eq_collation)(CHARSET_INFO *self, CHARSET_INFO *other);

  because it is the collation handler who knows how to detect equal
  collations by comparing only some of CHARSET_INFO members without
  their deep initialization.

  Three implementations were added:
  - my_ci_eq_collation_uca() for UCA collations, it compares
    _0900_ collations as equal to their corresponding _uca1400_ collations.
  - my_ci_eq_collation_utf8mb4_bin(), it compares
    utf8mb4_nopad_bin and utf8mb4_0900_bin as equal.
  - my_ci_eq_collation_generic() - the default implementation,
    which compares all collations as not equal.

  A C++ wrapper CHARSET_INFO::eq_collations() was added.
  The code in /sql was changes to use the wrapper instead of
  the former calls for the removed function compare_collations().

- A part of add_alias_for_collation() was moved into a new function
  my_ci_alloc(). It allocates a memory for a new charset_info_st
  instance together with the collation name and the comment using a single
  MY_CHARSET_LOADER::once_alloc call, which points to my_once_alloc()
  in the server.

- A part of add_alias_for_collation() was moved into a new function
  my_ci_make_comment_for_alias(). It makes an "Alias for xxx" string,
  e.g. "Alias for utf8mb4_uca1400_swedish_ai_ci" in case of
  utf8mb4_sv_0900_ai_ci.

- A part of the code in create_tailoring() was moved to
  a new function my_uca1400_collation_get_initialized_shared_uca(),
  to reuse the code between _uca1400_ and _0900_ collations.

- A new function my_collation_id_is_mysql_uca0900() was added
  in addition to my_collation_id_is_mysql_uca1400().

- Functions to build collation names were added:
   my_uca0900_collation_build_name()
   my_uca1400_collation_build_name()

- A shared function function was added:

  my_bool
  my_uca1400_collation_alloc_and_init(MY_CHARSET_LOADER *loader,
                                      LEX_CSTRING name,
                                      LEX_CSTRING comment,
                                      const uca_collation_def_param_t *param,
                                      uint id)

  It's reused to add _uca1400_ and _0900_ collations, with basic
  initialization (without deep initialization).

- The function add_compiled_collation() changed its return type from
  void to int, to make it compatible with MY_CHARSET_LOADER::add_collation.

- Functions mysql_uca0900_collation_definition_add(),
  mysql_uca0900_utf8mb4_collation_definitions_add(),
  mysql_utf8mb4_0900_bin_add() were added into ctype-uca0900.c.
  They get MY_CHARSET_LOADER as a parameter.

- Functions my_uca1400_collation_definition_add(),
  my_uca1400_collation_definitions_add() were moved from
  charset-def.c to strings/ctype-uca1400.c.
  The latter now accepts MY_CHARSET_LOADER as the first parameter
  instead of initializing a MY_CHARSET_LOADER inside.

- init_compiled_charsets() now initializes a MY_CHARSET_LOADER
  variable and passes it to all functions adding collations:
  - mysql_utf8mb4_0900_collation_definitions_add()
  - mysql_uca0900_utf8mb4_collation_definitions_add()
  - mysql_utf8mb4_0900_bin_add()

- A new structure was added into ctype-uca.h:

  typedef struct uca_collation_def_param
  {
    my_cs_encoding_t cs_id;
    uint tailoring_id;
    uint nopad_flags;
    uint level_flags;
  } uca_collation_def_param_t;

  It simplifies reusing the code for _uca1400_ and _0900_ collations.

- The definition of MY_UCA1400_COLLATION_DEFINITION was
  moved from ctype-uca.c to ctype-uca1400.h, to reuse
  the code for _uca1400_ and _0900_ collations.

- The definitions of "MY_UCA_INFO my_uca_v1400" and
  "MY_UCA_INFO my_uca1400_info_tailored[][]" were moved from
  ctype-uca.c to ctype-uca1400.c.

- The definitions/declarations of:
  - mysql_0900_collation_start,
  - struct mysql_0900_to_mariadb_1400_mapping
  - mysql_0900_to_mariadb_1400_mapping
  - mysql_utf8mb4_0900_collation_definitions_add()
  were moved from ctype-uca.c to ctype-uca0900.c

- Functions
  my_uca1400_make_builtin_collation_id()
  my_uca1400_collation_definition_init()
  my_uca1400_collation_id_uca400_compat()
  my_ci_get_collation_name_uca1400_context()
  were moved from ctype-uca.c to ctype-uca1400.c and ctype-uca1400.h

- A part of my_uca1400_collation_definition_init()
  was moved into my_uca0520_builtin_collation_by_id(),
  to make functions smaller.
This commit is contained in:
Alexander Barkov
2025-03-22 12:45:13 +04:00
parent 0dad1458e7
commit 10c063f9f0
38 changed files with 1498 additions and 646 deletions
Download Patch File Download Diff File Expand all files Collapse all files

114
mysys/charset.c
View File

@@ -597,7 +597,7 @@ CHARSET_INFO *default_charset_info = &my_charset_latin1;
All related character sets should share same cname
*/
void add_compiled_collation(struct charset_info_st *cs)
int add_compiled_collation(struct charset_info_st *cs)
{
DBUG_ASSERT(cs->number < array_elements(all_charsets));
all_charsets[cs->number]= cs;
@@ -613,6 +613,7 @@ void add_compiled_collation(struct charset_info_st *cs)
DBUG_ASSERT(org->cs_name.length == strlen(cs->cs_name.str));
#endif
}
return 0;
}
@@ -640,69 +641,6 @@ void add_compiled_extra_collation(struct charset_info_st *cs)
}
/*
Add an alias for a collation with an unique id
Used to add MySQL utf8mb4_0900 collations to MariaDB as an alias for the
corresponding utf8mb4_1400 collation
*/
my_bool add_alias_for_collation(LEX_CSTRING *collation_name, uint org_id,
LEX_CSTRING *alias, uint alias_id)
{
char *coll_name, *comment;
struct charset_info_st *new_ci;
CHARSET_INFO *org;
MY_CHARSET_LOADER loader;
char comment_buff[64+15];
size_t comment_length;
DBUG_ASSERT(all_charsets[org_id]);
if (!(org= all_charsets[org_id]))
return 1;
DBUG_ASSERT(!my_strcasecmp(&my_charset_latin1, org->coll_name.str,
collation_name->str));
#ifdef DEBUG_PRINT_ALIAS
fprintf(stderr, "alias: %s collation: %s org_id: %u\n",
alias->str, collation_name->str, org_id);
#endif
/*
We have to init the character set to ensure it is not changed after we copy
it.
*/
my_charset_loader_init_mysys(&loader);
if (my_ci_init_charset((struct charset_info_st*) org, &loader) ||
my_ci_init_collation((struct charset_info_st*) org, &loader) ||
(org->m_ctype &&
init_state_maps((struct charset_info_st*) org)))
return 1;
((struct charset_info_st*) org)->state|= MY_CS_READY;
comment_length= strxnmov(comment_buff, sizeof(comment_buff)-1,
"Alias for ", collation_name->str,
NullS) - comment_buff;
if (!(new_ci= ((struct charset_info_st*)
my_once_alloc(sizeof(CHARSET_INFO) +
alias->length + comment_length + 2,
MYF(MY_WME)))))
return 1;
coll_name= (char*) (new_ci+1);
comment= coll_name + alias->length +1;
memcpy((void*) new_ci, org, sizeof(CHARSET_INFO));
(new_ci->coll_name.str)= coll_name;
memcpy(coll_name, alias->str, alias->length+1);
memcpy(comment, comment_buff, comment_length+1);
new_ci->coll_name.length= alias->length;
new_ci->comment= comment;
new_ci->number= alias_id;
all_charsets[alias_id]= new_ci;
return 0;
}
static my_pthread_once_t charsets_initialized= MY_PTHREAD_ONCE_INIT;
static my_pthread_once_t charsets_template= MY_PTHREAD_ONCE_INIT;
@@ -722,54 +660,6 @@ my_bool my_collation_is_known_id(uint id)
}
/*
Compare if two collations are identical.
They are identical if all slots are identical except collation name and
number. Note that alias collations are made by memcpy(), which means that
also the also padding in the structures are identical.
Note that this code assumes knowledge of the CHARSET_INFO structure.
Especially the place of number, cs_name, coll_name and tailoring.
Other option would have been to add a new member 'alias_collation'
into CHARSET_INFO where all identical collations would point to,
but that would have changed the CHARSET_INFO structure which would
have required a lot more changes.
@return 0 Identical
@return 1 Different
*/
my_bool compare_collations(CHARSET_INFO *cs1, CHARSET_INFO *cs2)
{
size_t length;
if (cs1 == cs2)
return 0;
/* Quick check to detect different collation */
if (cs1->cset != cs2->cset || cs1->coll != cs2->coll ||
cs1->uca != cs2->uca)
goto diff;
/* We don't compare character set number */
if (cs1->primary_number != cs2->primary_number)
goto diff;
if (cs1->binary_number != cs2->binary_number)
goto diff;
if (cs1->state != cs2->state)
goto diff;
/* Compare everything after comment_name */
length= sizeof(CHARSET_INFO) - (((char*) &cs1->tailoring) - (char*) cs1);
if (!memcmp(&cs1->tailoring, &cs2->tailoring, length))
return 0;
diff:
return 1;
}
/*
Collation use statistics functions do not lock
counters to avoid mutex contention. This can lose