database/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-07-30 16:24:05 +03:00
Code Activity

Fix all warnings given by UBSAN

Browse Source 
The easiest way to compile and test the server with UBSAN is to run:
./BUILD/compile-pentium64-ubsan
and then run mysql-test-run.
After this commit, one should be able to run this without any UBSAN
warnings. There is still a few compiler warnings that should be fixed
at some point, but these do not expose any real bugs.

The 'special' cases where we disable, suppress or circumvent UBSAN are:
- ref10 source (as here we intentionally do some shifts that UBSAN
  complains about.
- x86 version of optimized int#korr() methods. UBSAN do not like unaligned
  memory access of integers.  Fixed by using byte_order_generic.h when
  compiling with UBSAN
- We use smaller thread stack with ASAN and UBSAN, which forced me to
  disable a few tests that prints the thread stack size.
- Verifying class types does not work for shared libraries. I added
  suppression in mysql-test-run.pl for this case.
- Added '#ifdef WITH_UBSAN' when using integer arithmetic where it is
  safe to have overflows (two cases, in item_func.cc).

Things fixed:
- Don't left shift signed values
  (byte_order_generic.h, mysqltest.c, item_sum.cc and many more)
- Don't assign not non existing values to enum variables.
- Ensure that bool and enum values are properly initialized in
  constructors.  This was needed as UBSAN checks that these types has
  correct values when one copies an object.
  (gcalc_tools.h, ha_partition.cc, item_sum.cc, partition_element.h ...)
- Ensure we do not called handler functions on unallocated objects or
  deleted objects.
  (events.cc, sql_acl.cc).
- Fixed bugs in Item_sp::Item_sp() where we did not call constructor
  on Query_arena object.
- Fixed several cast of objects to an incompatible class!
  (Item.cc, Item_buff.cc, item_timefunc.cc, opt_subselect.cc, sql_acl.cc,
   sql_select.cc ...)
- Ensure we do not do integer arithmetic that causes over or underflows.
  This includes also ++ and -- of integers.
  (Item_func.cc, Item_strfunc.cc, item_timefunc.cc, sql_base.cc ...)
- Added JSON_VALUE_UNITIALIZED to json_value_types and ensure that
  value_type is initialized to this instead of to -1, which is not a valid
  enum value for json_value_types.
- Ensure we do not call memcpy() when second argument could be null.
- Fixed that Item_func_str::make_empty_result() creates an empty string
  instead of a null string (safer as it ensures we do not do arithmetic
  on null strings).

Other things:

- Changed struct st_position to an OBJECT and added an initialization
  function to it to ensure that we do not copy or use uninitialized
  members. The change to a class was also motived that we used "struct
  st_position" and POSITION randomly trough the code which was
  confusing.
- Notably big rewrite in sql_acl.cc to avoid using deleted objects.
- Changed in sql_partition to use '^' instead of '-'. This is safe as
  the operator is either 0 or 0x8000000000000000ULL.
- Added check for select_nr < INT_MAX in JOIN::build_explain() to
  avoid bug when get_select() could return NULL.
- Reordered elements in POSITION for better alignment.
- Changed sql_test.cc::print_plan() to use pointers instead of objects.
- Fixed bug in find_set() where could could execute '1 << -1'.
- Added variable have_sanitizer, used by mtr.  (This variable was before
  only in 10.5 and up).  It can now have one of two values:
  ASAN or UBSAN.
- Moved ~Archive_share() from ha_archive.cc to ha_archive.h and marked
  it virtual. This was an effort to get UBSAN to work with loaded storage
  engines. I kept the change as the new place is better.
- Added in CONNECT engine COLBLK::SetName(), to get around a wrong cast
  in tabutil.cpp.
- Added HAVE_REPLICATION around usage of rgi_slave, to get embedded
  server to compile with UBSAN. (Patch from Marko).
- Added #ifdef for powerpc64 to avoid a bug in old gcc versions related
  to integer arithmetic.

Changes that should not be needed but had to be done to suppress warnings
from UBSAN:

- Added static_cast<<uint16_t>> around shift to get rid of a LOT of
  compiler warnings when using UBSAN.
- Had to change some '/' of 2 base integers to shift to get rid of
  some compile time warnings.

Reviewed by:
- Json changes: Alexey Botchkov
- Charset changes in ctype-uca.c: Alexander Barkov
- InnoDB changes & Embedded server: Marko Mäkelä
- sql_acl.cc changes: Vicențiu Ciorbaru
- build_explain() changes: Sergey Petrunia
This commit is contained in:
Monty
2021-04-18 15:29:13 +03:00
parent eb4123eefc
commit 031f11717d
73 changed files with 533 additions and 296 deletions
Download Patch File Download Diff File Expand all files Collapse all files

107
sql/sql_select.h
View File

@ -698,8 +698,6 @@ end_write_group(JOIN *join, JOIN_TAB *join_tab __attribute__((unused)),
bool end_of_records);
struct st_position;
class Semi_join_strategy_picker
{
public:
@ -710,7 +708,7 @@ public:
Update internal state after another table has been added to the join
prefix
*/
virtual void set_from_prev(struct st_position *prev) = 0;
virtual void set_from_prev(POSITION *prev) = 0;
virtual bool check_qep(JOIN *join,
uint idx,
@ -720,7 +718,7 @@ public:
double *read_time,
table_map *handled_fanout,
sj_strategy_enum *strategy,
struct st_position *loose_scan_pos) = 0;
POSITION *loose_scan_pos) = 0;
virtual void mark_used() = 0;
@ -751,7 +749,7 @@ public:
first_dupsweedout_table= MAX_TABLES;
is_used= FALSE;
}
void set_from_prev(struct st_position *prev);
void set_from_prev(POSITION *prev);
bool check_qep(JOIN *join,
uint idx,
@ -761,7 +759,7 @@ public:
double *read_time,
table_map *handled_fanout,
sj_strategy_enum *stratey,
struct st_position *loose_scan_pos);
POSITION *loose_scan_pos);
void mark_used() { is_used= TRUE; }
friend void fix_semijoin_strategies_for_picked_join_order(JOIN *join);
@ -797,7 +795,7 @@ public:
is_used= FALSE;
}
void set_from_prev(struct st_position *prev);
void set_from_prev(POSITION *prev);
bool check_qep(JOIN *join,
uint idx,
table_map remaining_tables,
@ -806,7 +804,7 @@ public:
double *read_time,
table_map *handled_fanout,
sj_strategy_enum *strategy,
struct st_position *loose_scan_pos);
POSITION *loose_scan_pos);
void mark_used() { is_used= TRUE; }
friend void fix_semijoin_strategies_for_picked_join_order(JOIN *join);
@ -815,6 +813,7 @@ public:
class LooseScan_picker : public Semi_join_strategy_picker
{
public:
/* The first (i.e. driving) table we're doing loose scan for */
uint first_loosescan_table;
/*
@ -833,14 +832,13 @@ class LooseScan_picker : public Semi_join_strategy_picker
uint loosescan_parts; /* Number of keyparts to be kept distinct */
bool is_used;
public:
void set_empty()
{
first_loosescan_table= MAX_TABLES;
is_used= FALSE;
}
void set_from_prev(struct st_position *prev);
void set_from_prev(POSITION *prev);
bool check_qep(JOIN *join,
uint idx,
table_map remaining_tables,
@ -849,19 +847,19 @@ public:
double *read_time,
table_map *handled_fanout,
sj_strategy_enum *strategy,
struct st_position *loose_scan_pos);
POSITION *loose_scan_pos);
void mark_used() { is_used= TRUE; }
friend class Loose_scan_opt;
friend void best_access_path(JOIN *join,
JOIN_TAB *s,
table_map remaining_tables,
const struct st_position *join_positions,
const POSITION *join_positions,
uint idx,
bool disable_jbuf,
double record_count,
struct st_position *pos,
struct st_position *loose_scan_pos);
POSITION *pos,
POSITION *loose_scan_pos);
friend bool get_best_combination(JOIN *join);
friend int setup_semijoin_loosescan(JOIN *join);
friend void fix_semijoin_strategies_for_picked_join_order(JOIN *join);
@ -888,7 +886,7 @@ public:
sjm_scan_last_inner= 0;
is_used= FALSE;
}
void set_from_prev(struct st_position *prev);
void set_from_prev(POSITION *prev);
bool check_qep(JOIN *join,
uint idx,
table_map remaining_tables,
@ -897,7 +895,7 @@ public:
double *read_time,
table_map *handled_fanout,
sj_strategy_enum *strategy,
struct st_position *loose_scan_pos);
POSITION *loose_scan_pos);
void mark_used() { is_used= TRUE; }
friend void fix_semijoin_strategies_for_picked_join_order(JOIN *join);
@ -912,8 +910,9 @@ class Rowid_filter;
Information about a position of table within a join order. Used in join
optimization.
*/
typedef struct st_position
class POSITION
{
public:
/* The table that's put into join order */
JOIN_TAB *table;
@ -925,7 +924,7 @@ typedef struct st_position
double records_read;
/* The selectivity of the pushed down conditions */
double cond_selectivity;
double cond_selectivity;
/*
Cost accessing the table in course of the entire complete join execution,
@ -934,8 +933,6 @@ typedef struct st_position
*/
double read_time;
/* Cumulative cost and record count for the join prefix */
Cost_estimate prefix_cost;
double prefix_record_count;
/*
@ -944,35 +941,14 @@ typedef struct st_position
*/
KEYUSE *key;
/* Info on splitting plan used at this position */
SplM_plan_info *spl_plan;
/* Cost info for the range filter used at this position */
Range_rowid_filter_cost_info *range_rowid_filter_info;
/* If ref-based access is used: bitmap of tables this table depends on */
table_map ref_depend_map;
/*
TRUE <=> join buffering will be used. At the moment this is based on
*very* imprecise guesses made in best_access_path().
*/
bool use_join_buffer;
/*
Current optimization state: Semi-join strategy to be used for this
and preceding join tables.
Join optimizer sets this for the *last* join_tab in the
duplicate-generating range. That is, in order to interpret this field,
one needs to traverse join->[best_]positions array from right to left.
When you see a join table with sj_strategy!= SJ_OPT_NONE, some other
field (depending on the strategy) tells how many preceding positions
this applies to. The values of covered_preceding_positions->sj_strategy
must be ignored.
*/
enum sj_strategy_enum sj_strategy;
/*
Valid only after fix_semijoin_strategies_for_picked_join_order() call:
if sj_strategy!=SJ_OPT_NONE, this is the number of subsequent tables that
are covered by the specified semi-join strategy
*/
uint n_sj_tables;
/*
Bitmap of semi-join inner tables that are in the join prefix and for
@ -982,19 +958,43 @@ typedef struct st_position
table_map dups_producing_tables;
table_map inner_tables_handled_with_other_sjs;
Duplicate_weedout_picker dups_weedout_picker;
Firstmatch_picker firstmatch_picker;
LooseScan_picker loosescan_picker;
Sj_materialization_picker sjmat_picker;
/* Info on splitting plan used at this position */
SplM_plan_info *spl_plan;
/* Cumulative cost and record count for the join prefix */
Cost_estimate prefix_cost;
/* Cost info for the range filter used at this position */
Range_rowid_filter_cost_info *range_rowid_filter_info;
/*
Current optimization state: Semi-join strategy to be used for this
and preceding join tables.
} POSITION;
Join optimizer sets this for the *last* join_tab in the
duplicate-generating range. That is, in order to interpret this field,
one needs to traverse join->[best_]positions array from right to left.
When you see a join table with sj_strategy!= SJ_OPT_NONE, some other
field (depending on the strategy) tells how many preceding positions
this applies to. The values of covered_preceding_positions->sj_strategy
must be ignored.
*/
enum sj_strategy_enum sj_strategy;
/*
Valid only after fix_semijoin_strategies_for_picked_join_order() call:
if sj_strategy!=SJ_OPT_NONE, this is the number of subsequent tables that
are covered by the specified semi-join strategy
*/
uint n_sj_tables;
/*
TRUE <=> join buffering will be used. At the moment this is based on
*very* imprecise guesses made in best_access_path().
*/
bool use_join_buffer;
POSITION();
};
typedef Bounds_checked_array<Item_null_result*> Item_null_array;
@ -1590,6 +1590,7 @@ public:
fields_list= fields_arg;
non_agg_fields.empty();
bzero((char*) &keyuse,sizeof(keyuse));
having_value= Item::COND_UNDEF;
tmp_table_param.init();
tmp_table_param.end_write_records= HA_POS_ERROR;
rollup.state= ROLLUP::STATE_NONE;