database/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-08-08 11:22:35 +03:00
Code Activity

MDEV-35750 Change MEM_ROOT allocation sizes to reduse calls to malloc() and avoid memory fragmentation

Browse Source 
This commit updates default memory allocations size used with MEM_ROOT
objects to minimize the number of calls to malloc().

Changes:
- Updated MEM_ROOT block sizes in sql_const.h
- Updated MALLOC_OVERHEAD to also take into account the extra memory
  allocated by my_malloc()
- Updated init_alloc_root() to only take MALLOC_OVERHEAD into account as
  buffer size, not MALLOC_OVERHEAD + sizeof(USED_MEM).
- Reset mem_root->first_block_usage if and only if first block was used.
- Increase MEM_ROOT buffers sized used by my_load_defaults, plugin_init,
  Create_tmp_table, allocate_table_share, TABLE and TABLE_SHARE.
  This decreases number of malloc calls during queries.
- Use a small buffer for THD->main_mem_root in THD::THD. This avoids
  multiple malloc() call for new connections.

I tried the above changes on a complex select query with 12 tables.
The following shows the number of extra allocations that where used
to increase the size of the MEM_ROOT buffers.

Original code:
- Connection to MariaDB:   9 allocations
- First query run:       146 allocations
- Second query run:       24 allocations

Max memory allocated for thd when using with heap table:  61,262,408
Max memory allocated for thd when using Aria tmp table:      419,464

After changes:
Connection to MariaDB:     0 allocations
- First run:              25 allocations
- Second run:              7 allocations

Max memory allocated for thd when using with heap table:  61,347,424
Max memory allocated for thd when using Aria table:          529,168

The new code uses slightly more memory, but avoids memory fragmentation
and is slightly faster thanks to much fewer calls to malloc().

Reviewed-by: Sergei Golubchik <serg@mariadb.org>
This commit is contained in:
Monty
2024-11-22 14:23:57 +02:00
parent f297623345
commit e600f9aebb
20 changed files with 98 additions and 76 deletions
Download Patch File Download Diff File Expand all files Collapse all files

21
sql/sql_const.h
View File

@@ -175,21 +175,26 @@
#define MYSQLD_NET_RETRY_COUNT 10 ///< Abort read after this many int.
#endif
#define QUERY_ALLOC_BLOCK_SIZE 16384
#define QUERY_ALLOC_PREALLOC_SIZE 24576
/*
Allocations with MEM_ROOT. We should try to keep these as powers of 2
and not higher than 32768 to get full benefit of allocators like
tcmalloc that will for these use a local heap without locks.
*/
#define QUERY_ALLOC_BLOCK_SIZE 32768
#define QUERY_ALLOC_PREALLOC_SIZE 32768 /* 65536 could be better */
#define TRANS_ALLOC_BLOCK_SIZE 8192
#define TRANS_ALLOC_PREALLOC_SIZE 4096
#define RANGE_ALLOC_BLOCK_SIZE 4096
#define ACL_ALLOC_BLOCK_SIZE 1024
#define UDF_ALLOC_BLOCK_SIZE 1024
#define TABLE_ALLOC_BLOCK_SIZE 1024
#define TABLE_PREALLOC_BLOCK_SIZE 8192
#define TABLE_ALLOC_BLOCK_SIZE 4096
#define WARN_ALLOC_BLOCK_SIZE 2048
#define WARN_ALLOC_PREALLOC_SIZE 1024
/*
Note that if we are using 32K or less, then TCmalloc will use a local
heap without locks!
*/
#define SHOW_ALLOC_BLOCK_SIZE (32768-MALLOC_OVERHEAD)
#define TMP_TABLE_BLOCK_SIZE 16384
#define TMP_TABLE_PREALLOC_SIZE 32768
#define SHOW_ALLOC_BLOCK_SIZE 32768
/*
The following parameters is to decide when to use an extra cache to