In sql_class.cc, 'row_count', of type 'ha_rows', was used as last argument for
ER_TRUNCATED_WRONG_VALUE_FOR_FIELD which is
"Incorrect %-.32s value: '%-.128s' for column '%.192s' at row %ld".
So 'ha_rows' was used as 'long'.
On SPARC32 Solaris builds, 'long' is 4 bytes and 'ha_rows' is 'longlong' i.e. 8 bytes.
So the printf-like code was reading only the first 4 bytes.
Because the CPU is big-endian, 1LL is 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x01
so the first four bytes yield 0. So the warning message had "row 0" instead of
"row 1" in test outfile_loaddata.test:
-Warning 1366 Incorrect string value: '\xE1\xE2\xF7' for column 'b' at row 1
+Warning 1366 Incorrect string value: '\xE1\xE2\xF7' for column 'b' at row 0
All error-messaging functions which internally invoke some printf-life function
are potential candidate for such mistakes.
One apparently easy way to catch such mistakes is to use
ATTRIBUTE_FORMAT (from my_attribute.h).
But this works only when call site has both:
a) the format as a string literal
b) the types of arguments.
So:
func(ER(ER_BLAH), 10);
will silently not be checked, because ER(ER_BLAH) is not known at
compile time (it is known at run-time, and depends on the chosen
language).
And
func("%s", a va_list argument);
has the same problem, as the *real* type of arguments is not
known at this site at compile time (it's known in some caller).
Moreover,
func(ER(ER_BLAH));
though possibly correct (if ER(ER_BLAH) has no '%' markers), will not
compile (gcc says "error: format not a string literal and no format
arguments").
Consequences:
1) ATTRIBUTE_FORMAT is here added only to functions which in practice
take "string literal" formats: "my_error_reporter" and "print_admin_msg".
2) it cannot be added to the other functions: my_error(),
push_warning_printf(), Table_check_intact::report_error(),
general_log_print().
To do a one-time check of functions listed in (2), the following
"static code analysis" has been done:
1) replace
my_error(ER_xxx, arguments for substitution in format)
with the equivalent
my_printf_error(ER_xxx,ER(ER_xxx), arguments for substitution in
format),
so that we have ER(ER_xxx) and the arguments *in the same call site*
2) add ATTRIBUTE_FORMAT to push_warning_printf(),
Table_check_intact::report_error(), general_log_print()
3) replace ER(xxx) with the hard-coded English text found in
errmsg.txt (like: ER(ER_UNKNOWN_ERROR) is replaced with
"Unknown error"), so that a call site has the format as string literal
4) this way, ATTRIBUTE_FORMAT can effectively do its job
5) compile, fix errors detected by ATTRIBUTE_FORMAT
6) revert steps 1-2-3.
The present patch has no compiler error when submitted again to the
static code analysis above.
It cannot catch all problems though: see Field::set_warning(), in
which a call to push_warning_printf() has a variable error
(thus, not replacable by a string literal); I checked set_warning() calls
by hand though.
See also WL 5883 for one proposal to avoid such bugs from appearing
again in the future.
The issues fixed in the patch are:
a) mismatch in types (like 'int' passed to '%ld')
b) more arguments passed than specified in the format.
This patch resolves mismatches by changing the type/number of arguments,
not by changing error messages of sql/share/errmsg.txt. The latter would be wrong,
per the following old rule: errmsg.txt must be as stable as possible; no insertions
or deletions of messages, no changes of type or number of printf-like format specifiers,
are allowed, as long as the change impacts a message already released in a GA version.
If this rule is not followed:
- Connectors, which use error message numbers, will be confused (by insertions/deletions
of messages)
- using errmsg.sys of MySQL 5.1.n with mysqld of MySQL 5.1.(n+1)
could produce wrong messages or crash; such usage can easily happen if
installing 5.1.(n+1) while /etc/my.cnf still has --language=/path/to/5.1.n/xxx;
or if copying mysqld from 5.1.(n+1) into a 5.1.n installation.
When fixing b), I have verified that the superfluous arguments were not used in the format
in the first 5.1 GA (5.1.30 'bteam@astra04-20081114162938-z8mctjp6st27uobm').
Had they been used, then passing them today, even if the message doesn't use them
anymore, would have been necessary, as explained above.
SECONDARY INDEX IN INNODB
The patches for Bug#11751388 and Bug#11784056 enabled concurrent
reads while creating secondary indexes in InnoDB. However, they
introduced a regression. This regression occured if ALTER TABLE
failed after the index had been added, for example during the
lock upgrade needed to update .FRM. If this happened, InnoDB
and the server got out of sync with regards to which indexes
actually existed. Therefore the patch for Bug#11815600 again
disabled concurrent reads.
This patch re-enables concurrent reads. The original regression
is fixed by splitting the ADD INDEX operation into two parts.
First the new index is created but not made active. This is
done while concurrent reads are allowed. The second part of
the operation makes the index active (or reverts the change).
This is done after lock upgrade, which prevents the original
regression.
In order to implement this change, the patch changes the storage
API for in-place index creation. handler::add_index() is split
into two functions, handler_add_index() and
handler::final_add_index(). The former for creating indexes without
making them visible and the latter for commiting (i.e. making
visible) new indexes or reverting the changes.
Large parts of this patch were written by Marko Mäkelä.
Test case added to innodb_mysql_lock.test.
The partitioning engine checked the auto_increment column even if it was not to be written,
triggering a DBUG_ASSERT.
Fixed by checking if table->write_set for that column was set.
The partitioning engine checked the auto_increment column even if it was not to be written,
triggering a DBUG_ASSERT.
Fixed by checking if table->write_set for that column was set.
Partitions can have different ref_length (position data length).
Removed DBUG_ASSERT which crashed debug builds when using
MAX_ROWS on some partitions.
Partitions can have different ref_length (position data length).
Removed DBUG_ASSERT which crashed debug builds when using
MAX_ROWS on some partitions.
Update for previous patch according to reviewers comments.
Updated the constructors for ha_partitions to use the common
init_handler_variables functions
Added use of defines for size and offset to get better readability for the code that reads
and writes the .par file. Also refactored the get_from_handler_file function.
Update for previous patch according to reviewers comments.
Updated the constructors for ha_partitions to use the common
init_handler_variables functions
Added use of defines for size and offset to get better readability for the code that reads
and writes the .par file. Also refactored the get_from_handler_file function.
When executing row-ordered-retrieval index merge,
the handler was cloned, but it used the wrong
memory root, so instead of allocating memory
on the thread/query's mem_root, it used the table's
mem_root, resulting in non released memory in the
table object, and was not freed until the table was
closed.
Solution was to ensure that memory used during cloning
of a handler was allocated from the correct memory root.
This was implemented by fixing handler::clone() to also
take a name argument, so it can be used with partitioning.
And in ha_partition only allocate the ha_partition's ref, and
call the original ha_partition partitions clone() and set at cloned
partitions.
Fix of .bzrignore on Windows with VS 2010
When executing row-ordered-retrieval index merge,
the handler was cloned, but it used the wrong
memory root, so instead of allocating memory
on the thread/query's mem_root, it used the table's
mem_root, resulting in non released memory in the
table object, and was not freed until the table was
closed.
Solution was to ensure that memory used during cloning
of a handler was allocated from the correct memory root.
This was implemented by fixing handler::clone() to also
take a name argument, so it can be used with partitioning.
And in ha_partition only allocate the ha_partition's ref, and
call the original ha_partition partitions clone() and set at cloned
partitions.
Fix of .bzrignore on Windows with VS 2010
Regression from bug#11766232.
m_last_part could be set beyond the last partition.
Fixed by only setting it if within the limit.
Also added check in print_error.
Regression from bug#11766232.
m_last_part could be set beyond the last partition.
Fixed by only setting it if within the limit.
Also added check in print_error.
Changed some String.ptr() -> String.c_ptr() for String that are not guaranteed to end with \0
Removed some c_ptr() usage from parameters to functions that takes ptr & length
Use preallocate buffers to avoid calling malloc() for most operations.
sql/event_db_repository.cc:
alias is now a String
sql/event_scheduler.cc:
c_ptr -> c_ptr_safe() to avoid warnings from valgrind.
sql/events.cc:
c_ptr -> c_ptr_safe() to avoid warnings from valgrind.
c_ptr -> ptr() as function takes ptr & length
sql/field.cc:
alias is now a String
sql/field.h:
alias is now a String
sql/ha_partition.cc:
alias is now a String
sql/handler.cc:
alias is now a String
ptr() -> c_ptr() as string is not guaranteed to be \0 terminated
sql/item.cc:
Store error parameter in separarte buffer to ensure correct error message
sql/item_func.cc:
ptr() -> c_ptr_safe() as string is not guaranteed to be \0 terminated
sql/item_sum.h:
Use my_strtod() instead of my_atof() to not have to make string \0 terminated
sql/lock.cc:
alias is now a String
sql/log.cc:
c_ptr() -> ptr() as function takes ptr & length
sql/log_event.cc:
c_ptr_quick() -> ptr() as we only want to get the pointer to String buffer
sql/opt_range.cc:
ptr() -> c_ptr() as string is not guaranteed to be \0 terminated
sql/opt_table_elimination.cc:
alias is now a String
sql/set_var.cc:
ptr() -> c_ptr() as string is not guaranteed to be \0 terminated
c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
c_ptr() -> ptr() as function takes ptr & length
Simplify some code.
sql/sp.cc:
c_ptr() -> ptr() as function takes ptr & length
sql/sp_rcontext.cc:
alias is now a String
sql/sql_base.cc:
alias is now a String.
Here we win a realloc() for most alias usage.
sql/sql_class.cc:
Use size descriptor for printf() to avoid accessing bytes outside of buffer
sql/sql_insert.cc:
Change allocation of TABLE as it's now contains a String
_ptr() -> ptr() as function takes ptr & length
sql/sql_load.cc:
Use preallocate buffers to avoid calling malloc() for most operations.
sql/sql_parse.cc:
Use c_ptr_safe() to ensure string is \0 terminated.
sql/sql_plugin.cc:
c_ptr_quick() -> ptr() as function takes ptr & length
sql/sql_select.cc:
alias is now a String
sql/sql_show.cc:
alias is now a String
sql/sql_string.h:
Added move() function to change who owns the string (owner does the free)
sql/sql_table.cc:
alias is now a String
c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
sql/sql_test.cc:
c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
alias is now a String
sql/sql_trigger.cc:
c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
Use field->init() to setup pointers to alias.
sql/sql_update.cc:
alias is now a String
sql/sql_view.cc:
ptr() -> c_ptr_safe() as string is not guaranteed to be \0 terminated
sql/sql_yacc.yy:
r() -> c_ptr() as string is not guaranteed to be \0 terminated
sql/table.cc:
alias is now a String
sql/table.h:
alias is now a String
storage/federatedx/ha_federatedx.cc:
Remove extra 1 byte alloc that is automaticly done by strmake()
Ensure that error message ends with \0
storage/maria/ha_maria.cc:
alias is now a String
storage/myisam/ha_myisam.cc:
alias is now a String
- Fixed some issues with partitions and connection_string, which also fixed lp:716890 "Pre- and post-recovery crash in Aria"
- Fixed wrong assert in Aria
Now need to merge with latest xtradb before pushing
sql/ha_partition.cc:
Ensure that m_ordered_rec_buffer is not freed before close.
sql/mysqld.cc:
Changed to use opt_stack_trace instead of opt_pstack.
Removed references to pstack
sql/partition_element.h:
Ensure that connect_string is initialized
storage/maria/ma_key_recover.c:
Fixed wrong assert
Fixed that connection string is returned for partitioned federated tables.
mysql-test/r/partition_federated.result:
Fixed error message
mysql-test/suite/federated/federated_partition.result:
Added test to show that connection string is returned in 'show create'.
sql/ha_partition.cc:
Fixed a set of bugs introduced by the last federated patch:
- We can't allocate m_ordered_rec_buffer in memroot as it has to survive call to clear_handler_file()
sql/partition_element.h:
Ensure that connect_string is properly initialized.
(This caused crashed in partition tests)
sql/sql_partition.cc:
Print CONNECTION option for federated partitioned tables
- Fixed Partition engine to store CONNECTION string for partitions.
Removed HA_NO_PARTITION flag from FederatedX.
Added test 'federated_partition' to suite.
- lp:#585688 - maridb crashes in federatedx code
FederatedX handler instances, created on one thread and used on
another thread (via table cache) when "show table status" is executed
crashed because txn member was not initialized for current thread.
Added test 'federated_bug_585688' to suite.
Author for the patch is Antony Curtis
mysql-test/suite/federated/federated_bug_585688.result:
Test for lp:585688
mysql-test/suite/federated/federated_bug_585688.test:
Test for lp:585688
mysql-test/suite/federated/federated_partition-slave.opt:
Test for partition support in federatedx
mysql-test/suite/federated/federated_partition.result:
Test for partition support in federatedx
mysql-test/suite/federated/federated_partition.test:
Test for partition support in federatedx
mysql-test/t/partition_federated.test:
Updated error message
sql/ha_partition.cc:
Added support for connection strings to partitions for federatedx
sql/ha_partition.h:
Added support for connection strings to partitions for federatedx
sql/partition_element.h:
Added support for connection strings to partitions for federatedx
sql/sql_yacc.yy:
Added support for connection strings to partitions for federatedx
storage/federatedx/ha_federatedx.cc:
Added support for partitions.
FederatedX handler instances, created on one thread and used on another thread (via table cache) when "show table status"
is executed crashed because txn member was not initialized for current thread.
Regression introduced in bug#52455. Problem was that the
fixed function did not set the last used partition variable, resulting
in wrong partition used when storing the position of the newly
retrieved row.
Fixed by setting the last used partition in ha_partition::index_read_idx_map.
Regression introduced in bug#52455. Problem was that the
fixed function did not set the last used partition variable, resulting
in wrong partition used when storing the position of the newly
retrieved row.
Fixed by setting the last used partition in ha_partition::index_read_idx_map.
that implement add_index
The problem was that ALTER TABLE blocked reads on an InnoDB table
while adding a secondary index, even if this was not needed. It is
only needed for the final step where the .frm file is updated.
The reason queries were blocked, was that ALTER TABLE upgraded the
metadata lock from MDL_SHARED_NO_WRITE (which blocks writes) to
MDL_EXCLUSIVE (which blocks all accesses) before index creation.
The way the server handles index creation, is that storage engines
publish their capabilities to the server and the server determines
which of the following three ways this can be handled: 1) build a
new version of the table; 2) change the existing table but with
exclusive metadata lock; 3) change the existing table but without
metadata lock upgrade.
For InnoDB and secondary index creation, option 3) should have been
selected. However this failed for two reasons. First, InnoDB did
not publish this capability properly.
Second, the ALTER TABLE code failed to made proper use of the
information supplied by the storage engine. A variable
need_lock_for_indexes was set accordingly, but was not later used.
This patch fixes this problem by only doing metadata lock upgrade
before index creation/deletion if this variable has been set.
This patch also changes some of the related terminology used
in the code. Specifically the use of "fast" and "online" with
respect to ALTER TABLE. "Fast" was used to indicate that an
ALTER TABLE operation could be done without involving a
temporary table. "Fast" has been renamed "in-place" to more
accurately describe the behavior.
"Online" meant that the operation could be done without taking
a table lock. However, in the current implementation writes
are always prohibited during ALTER TABLE and an exclusive
metadata lock is held while updating the .frm, so ALTER TABLE
is not completely online. This patch replaces "online" with
"in-place", with additional comments indicating if concurrent
reads are allowed during index creation/deletion or not.
An important part of this update of terminology is renaming
of the handler flags used by handlers to indicate if index
creation/deletion can be done in-place and if concurrent reads
are allowed. For example, the HA_ONLINE_ADD_INDEX_NO_WRITES
flag has been renamed to HA_INPLACE_ADD_INDEX_NO_READ_WRITE,
while HA_ONLINE_ADD_INDEX is now HA_INPLACE_ADD_INDEX_NO_WRITE.
Note that this is a rename to clarify current behavior, the
flag values have not changed and no flags have been removed or
added.
Test case added to innodb_mysql_sync.test.
