The query cache protected all operations with one exclusive lock
(m_cache_lock_status, guarded by structure_guard_mutex). This
serialized every lookup, even though a lookup mainly reads the
query cache structures.
Added a new global variable, query_cache_use_rw_lock (default OFF).
When set, send_result_to_client() takes a shared (read) lock instead
of an exclusive one, which allows lookups to run concurrently. Having
it as an option makes it possible to benchmark both alternatives with
the same binary and then decide which one to keep.
Implementation:
- try_lock() has a new 'read_lock' argument. The new inline function
try_read_lock() sets it from query_cache_use_rw_lock.
- Readers are counted in m_readers and leave m_cache_lock_status as
UNLOCKED. Writers wait until m_readers is 0 and are counted in
m_waiting_writers. Waiting writers have priority over new readers,
which ensures that invalidations are not starved by cache hits.
- Readers wait on the new COND_cache_read_lock. wake_up_waiters()
wakes one waiting writer, or all waiting readers if there is none.
- unlock() is split into unlock() and unlock_internal(). The kind of
lock to release is deduced from m_cache_lock_status, so none of the
unlock() callers had to be changed.
- send_result_to_client() updates the query list and the statistics
under structure_guard_mutex when it only holds a read lock. With a
write lock this is done as before, without taking the mutex.
- If an engine requests invalidation during a lookup done with a read
lock, the table key is copied and the invalidation is done after
the lock is released, as invalidation requires a write lock.
- Added the DEBUG_SYNC point "in_query_cache_hit" used by the new test.
With query_cache_use_rw_lock=0 the code works as before.
Did a simple sysbench run with 64 threads and 200000 simple select queries,
all served from query cache. query_cache_use_rw_lock=1 gave a 2.25x
speedup. The difference to an unmodied MariaDB version for the same
test is 2.3 x faster.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- Added query_cache_type == ALWAYS_OFF to turn of query cache permanently
for new open tables. Without this option storage engines has to call
query cache for all update querys and invalidate used tables that has
a small overhead. When running benchmarks without query cache, this
is the option to use!
- Updated MyISAM, Aria, MYISAM_MRG and InnoDB to support ALWAYS_OFF.
- Setting query_cache_size in a config file will not automatically enable
the query cache.
- Added query_cache_available() that functions can check if query cache
can be used or if it is permanently disabled.
- Moved things around in Query_cache::store_query() to get more work done
outside of query cache mutex.
- Same for Query_cache::send_result_to_client()
Co-author: Thirunarayanan Balathandayuthapani <thiru@mariadb.com>
- ÌnnoDB changes reviewed and improved
- Do checks inline before calling query_cache.send_result_to_client()
Things tried that failed:
- I tried removing calling lex_start() and reset_for_next_command() if
query is cached, but this is would cause duplicated, hard to maintain
code and would require double resets of things if query cache was
not used. I have documented in sql_parse.cc what would need to be
reset for this approach to work.
Add SQL_CACHE=0|1 option for tables to disable/enable query caches if
table is part of the query.
SQL_CACHE option can be disable for a table by using SQL_CACHE=DEFAULT
Also provide a workaround for the problem described at:
MDEV-6631 "No cache directive(SQL_CACHE) query still wait for
query cache lock" on query_cache_type=DEMAND mode"
by adding DEMAND_STRICT mode.
The new mode is needed as modifying how DEMAND works would made the
change backward incompatible.
query_cache_type has now following options (0-2 are same as before)
OFF (0) = Query cache off
ON (1) = Query cache on, all queries are cached except queries marked
with SQL_NO_CACHE
DEMAND (2) = Query cache on, only queries marked with SQL_CACHE are cached
DEMAND_STRICT
(3) = Query cache on, only queries starting with SELECT SQL_CACHE
are cached. This avoids a query cache lock+lookup for every
query.
TABLES (4) = Query cache on, only cache queries marked with SQL_CACHE or
where all used tables are using the create option SQL_CACHE=1
Note that if any of the tables used in the query has the create option
SQL_CACHE=0 or is a system table, temporary table or other non
cacheable table, then the query will not be cached.
SHOW CREATE TABLE prints SQL_CACHE=0|1 when the option is set. For tables
where the option can never apply (temporary tables, tables in the mysql
database, and engines that do not support the query cache) it is printed
as /* SQL_CACHE=1 */.
co-author: Seunguck Lee
- Took some changes related to MDEV-6631 from his patch.
Partial commit of the greater MDEV-34348 scope.
MDEV-34348: MariaDB is violating clang-16 -Wcast-function-type-strict
Change the type of my_hash_get_key to:
1) Return const
2) Change the context parameter to be const void*
Also fix casting in hash adjacent areas.
Reviewed By:
============
Marko Mäkelä <marko.makela@mariadb.com>
- Changing the global function ok_for_lower_case_names()
into a method in class Lex_ident_fs.
- Changing a few functions/methods to get the database name
as a "const LEX_CSTRING" instead of a "const char *".
All these functions/methods use ok_for_lower_case_names()
inside. This change helps to avoid new strlen() calls, and also
removes a few old strlen() calls.
This patch is the result of running
run-clang-tidy -fix -header-filter=.* -checks='-*,modernize-use-equals-default' .
Code style changes have been done on top. The result of this change
leads to the following improvements:
1. Binary size reduction.
* For a -DBUILD_CONFIG=mysql_release build, the binary size is reduced by
~400kb.
* A raw -DCMAKE_BUILD_TYPE=Release reduces the binary size by ~1.4kb.
2. Compiler can better understand the intent of the code, thus it leads
to more optimization possibilities. Additionally it enabled detecting
unused variables that had an empty default constructor but not marked
so explicitly.
Particular change required following this patch in sql/opt_range.cc
result_keys, an unused template class Bitmap now correctly issues
unused variable warnings.
Setting Bitmap template class constructor to default allows the compiler
to identify that there are no side-effects when instantiating the class.
Previously the compiler could not issue the warning as it assumed Bitmap
class (being a template) would not be performing a NO-OP for its default
constructor. This prevented the "unused variable warning".
The problem happened because the the new client capability flag
CLIENT_EXTENDED_METADATA was not put into the cache entry key.
So results cached by a new client were sent to the old client (and vica versa)
with a mis-matching metadata, which made the client abort the connection on
an unexpected result set metadata packet format.
The problem was caused by the patch for:
MDEV-17832 Protocol: extensions for Pluggable types and JSON, GEOMETRY
which forgot to adjust the query cache code.
Fix:
- Adding a new member Query_cache_query_flags::client_extended_metadata,
so only clients with equal CLIENT_EXTENDED_METADATA flag values can
reuse results.
- Adding a new column CLIENT_EXTENDED_METADATA into
INFORMATION_SCHEMA.QUERY_CACHE_INFO (privided by the qc_info plugin).
Stop linking plugins to the server executable on Windows.
Instead, extract whole server functionality into a large DLL, called
server.dll. Link both plugins, and small server "stub" exe to it.
This eliminates plugin dependency on the name of the server executable.
It also reduces the size of the packages (since tiny mysqld.exe
and mariadbd.exe are now both linked to one big DLL)
Also, simplify the functionality of exporing all symbols from selected
static libraries. Rely on WINDOWS_EXPORT_ALL_SYMBOLS, rather than old
self-backed solution.
fix compile error
replace GetProcAddress(GetModuleHandle(NULL), "variable_name")
for server exported data with actual variable names.
Runtime loading was never required,was error prone
, since symbols could be missing at runtime, and now it actually failed,
because we do not export symbols from executable anymore, but from a shared
library
This did require a MYSQL_PLUGIN_IMPORT decoration for the plugin,
but made the code more straightforward, and avoids missing symbols at
runtime (as mentioned before).
The audit plugin is still doing some dynamic loading, as it aims to work
cross-version. Now it won't work cross-version on Windows, as it already
uses some symbols that are *not* dynamically loaded, e.g fn_format
and those symbols now exported from server.dll , when earlier they were
exported by mysqld.exe
Windows, fixes for storage engine plugin loading
after various rebranding stuff
Create server.dll containing functionality of the whole server
make mariadbd.exe/mysqld.exe a stub that is only calling mysqld_main()
fix build
Handle string length as size_t, consistently (almost always:))
Change function prototypes to accept size_t, where in the past
ulong or uint were used. change local/member variables to size_t
when appropriate.
This fix excludes rocksdb, spider,spider, sphinx and connect for now.
This only occurs non-DEBUG builds. Its main purpose is to
save space in the core dump from elements not usually useful.
Signed-off-by: Daniel Black <daniel@linux.vnet.ibm.com>
Added hit_count field to Query_cache_query and methods to get and increment it.
The counter is incremented when query results are read from query cache.
Benefits of this patch:
- Removed a lot of calls to strlen(), especially for field_string
- Strings generated by parser are now const strings, less chance of
accidently changing a string
- Removed a lot of calls with LEX_STRING as parameter (changed to pointer)
- More uniform code
- Item::name_length was not kept up to date. Now fixed
- Several bugs found and fixed (Access to null pointers,
access of freed memory, wrong arguments to printf like functions)
- Removed a lot of casts from (const char*) to (char*)
Changes:
- This caused some ABI changes
- lex_string_set now uses LEX_CSTRING
- Some fucntions are now taking const char* instead of char*
- Create_field::change and after changed to LEX_CSTRING
- handler::connect_string, comment and engine_name() changed to LEX_CSTRING
- Checked printf() related calls to find bugs. Found and fixed several
errors in old code.
- A lot of changes from LEX_STRING to LEX_CSTRING, especially related to
parsing and events.
- Some changes from LEX_STRING and LEX_STRING & to LEX_CSTRING*
- Some changes for char* to const char*
- Added printf argument checking for my_snprintf()
- Introduced null_clex_str, star_clex_string, temp_lex_str to simplify
code
- Added item_empty_name and item_used_name to be able to distingush between
items that was given an empty name and items that was not given a name
This is used in sql_yacc.yy to know when to give an item a name.
- select table_name."*' is not anymore same as table_name.*
- removed not used function Item::rename()
- Added comparision of item->name_length before some calls to
my_strcasecmp() to speed up comparison
- Moved Item_sp_variable::make_field() from item.h to item.cc
- Some minimal code changes to avoid copying to const char *
- Fixed wrong error message in wsrep_mysql_parse()
- Fixed wrong code in find_field_in_natural_join() where real_item() was
set when it shouldn't
- ER_ERROR_ON_RENAME was used with extra arguments.
- Removed some (wrong) ER_OUTOFMEMORY, as alloc_root will already
give the error.
TODO:
- Check possible unsafe casts in plugin/auth_examples/qa_auth_interface.c
- Change code to not modify LEX_CSTRING for database name
(as part of lower_case_table_names)
includes:
* remove some remnants of "Bug#14521864: MYSQL 5.1 TO 5.5 BUGS PARTITIONING"
* introduce LOCK_share, now LOCK_ha_data is strictly for engines
* rea_create_table() always creates .par file (even in "frm-only" mode)
* fix a 5.6 bug, temp file leak on dummy ALTER TABLE
Merged all ddl_logging code.
Merged sql_partition.cc
innodb_mysql_lock2.test and partition_cache.test now works.
Changed interface to strconvert() to make it easier to use with not \0 terminated strings.
sql/sql_partition.cc:
Full merge with 5.6
sql/sql_table.cc:
Merged all ddl_logging code
sql/strfunc.cc:
Added from_length argument to strconvert() to make it possible to use without end terminated strings.
sql/strfunc.h:
Added from_length argument to strconvert() to make it possible to use without end terminated strings.
Now partition engine adds underlying tables to the QC and ask underlying tables engine permittion to cache the query and return result of the query.
Incorrect QC cleanup in case of table registration failure fixe.
Unified interface for myisammrg & partitioned engnes for QC.
