database/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-08-08 11:22:35 +03:00
Code Activity

MDEV-27266 Improve UCA collation performance for utf8mb3 and utf8mb4

Browse Source 
Adding two levels of optimization:

1. For every bytes pair [00..FF][00..FF] which:
  a. consists of two ASCII characters or makes a well-formed two-byte character
  b. whose total weight string fits into 4 weights
     (concatenated weight string in case of two ASCII characters,
     or a single weight string in case of a two-byte character)
  c. whose weight is context independent (i.e. does not depend on contractions
     or previous context pairs)
  store weights in a separate array of MY_UCA_2BYTES_ITEM,
  so during scanner_next() we can scan two bytes at a time.
  Byte pairs that do not match the conditions a-c are marked in this array
  as not applicable for optimization and scanned as before.

2. For every byte pair which is applicable for optimization in #1,
   and which produces only one or two weights, store
   weights in one more array of MY_UCA_WEIGHT2. So in the beginning
   of strnncoll*() we can skip equal prefixes using an even more efficient
   loop. This loop consumes two bytes at a time. The loop scans while the
   two bytes on both sides produce weight strings of equal length
   (i.e. one weight on both sides, or two weight on both sides).
   This allows to compare efficiently:
   - Context independent sequences consisting of two ASCII characters
   - Context independent 2-byte characters
   - Contractions consisting of two ASCII characters, e.g. Czech "ch".
   - Some tricky cases: "ss" vs "SHARP S"
     ("ss" produces two weights, 0xC39F also produces two weights)
This commit is contained in:
Alexander Barkov
2022-02-25 13:54:59 +04:00
committed by Oleksandr Byelkin
parent a0858b2cff
commit d8f172c11c
5 changed files with 660 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
unittest/strings/strings-t.c
View File

@@ -1341,7 +1341,7 @@ strnncollsp_char_one(CHARSET_INFO *cs, const STRNNCOLLSP_CHAR_PARAM *p)
str2hex(ahex, sizeof(ahex), p->a.str, p->a.length);
str2hex(bhex, sizeof(bhex), p->b.str, p->b.length);
diag("%-25s %-12s %-12s %3d %7d %7d%s",
cs->cs_name.str, ahex, bhex, (int) p->nchars, p->res, res,
cs->coll_name.str, ahex, bhex, (int) p->nchars, p->res, res,
eqres(res, p->res) ? "" : " FAILED");
if (!eqres(res, p->res))
{