into weblab.(none):/home/marcsql/TREE/mysql-5.0-19194
sql/sp_head.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_yacc.yy:
Auto merged
on altered table" and Bug#19733 "Repeated alter, or repeated
create/drop, fails"
mysql-test/r/ps.result:
Post-merge fixes: update results with new tests.
mysql-test/r/sp.result:
Post-merge fixes: update results.
mysql-test/t/ps.test:
Add more test cases for Bug#4968 and related.
mysql-test/t/sp.test:
A post-merge fix: add more testcases for Bug#4968 and related.
sql/sql_insert.cc:
Post-merge fixes: update comments, fix errors of the manual merge.
sql/sql_lex.cc:
Fix a manual merge error.
sql/sql_parse.cc:
Fix a few errors of the manual merge, style.
sql/sql_table.cc:
Post-merge fixes, fix a few errors of the manual merge, fix style.
sql/sql_yacc.yy:
A post-merge fix.
into govinda.patg.net:/home/patg/mysql-build/mysql-5.1-arch-wl3031-merge
mysql-test/lib/init_db.sql:
Auto merged
mysql-test/r/1st.result:
Auto merged
mysql-test/r/connect.result:
Auto merged
mysql-test/r/information_schema.result:
Auto merged
mysql-test/r/mysql.result:
Auto merged
mysql-test/r/mysqlcheck.result:
Auto merged
mysql-test/r/show_check.result:
Auto merged
mysql-test/r/system_mysql_db.result:
Auto merged
scripts/mysql_create_system_tables.sh:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/share/errmsg.txt:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
storage/federated/ha_federated.cc:
Auto merged
mysql-test/r/sp.result:
WL# 3031
More fun with error codes. These error codes vary in the result files because new error codes have been added
mysql-test/r/sp_gis.result:
WL# 3031
More fun with error codes. These error codes vary in the result files because new error codes have been added
into mysql.com:/home/bk/MERGE/mysql-5.1-merge
client/mysqlbinlog.cc:
Auto merged
client/mysqldump.c:
Auto merged
config/ac-macros/ha_ndbcluster.m4:
Auto merged
configure.in:
Auto merged
include/my_global.h:
Auto merged
mysql-test/r/mysqldump.result:
Auto merged
mysql-test/r/rpl_timezone.result:
Auto merged
mysql-test/t/disabled.def:
Auto merged
sql/handler.cc:
Auto merged
sql/item_create.cc:
Auto merged
sql/item_timefunc.cc:
Auto merged
sql/log.cc:
Auto merged
sql/log.h:
Auto merged
sql/log_event.h:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/rpl_injector.h:
Auto merged
sql/set_var.cc:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sql_acl.cc:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_view.cc:
Auto merged
sql/table.cc:
Auto merged
mysql-test/t/mysqldump.test:
Manual merge
sql/log_event.cc:
manual merge
Bug#4968 "Stored procedure crash if cursor opened on altered table"
Bug#19733 "Repeated alter, or repeated create/drop, fails"
Bug#19182 "CREATE TABLE bar (m INT) SELECT n FROM foo; doesn't work from
stored procedure."
Bug#6895 "Prepared Statements: ALTER TABLE DROP COLUMN does nothing"
Bug#22060 "ALTER TABLE x AUTO_INCREMENT=y in SP crashes server"
Test cases for bugs 4968, 19733, 6895 will be added in 5.0.
Re-execution of CREATE DATABASE, CREATE TABLE and ALTER TABLE
statements in stored routines or as prepared statements caused
incorrect results (and crashes in versions prior to 5.0.25).
In 5.1 the problem occured only for CREATE DATABASE, CREATE TABLE
SELECT and CREATE TABLE with INDEX/DATA DIRECTOY options).
The problem of bugs 4968, 19733, 19282 and 6895 was that functions
mysql_prepare_table, mysql_create_table and mysql_alter_table were not
re-execution friendly: during their operation they used to modify contents
of LEX (members create_info, alter_info, key_list, create_list),
thus making the LEX unusable for the next execution.
In particular, these functions removed processed columns and keys from
create_list, key_list and drop_list. Search the code in sql_table.cc
for drop_it.remove() and similar patterns to find evidence.
The fix is to supply to these functions a usable copy of each of the
above structures at every re-execution of an SQL statement.
To simplify memory management, LEX::key_list and LEX::create_list
were added to LEX::alter_info, a fresh copy of which is created for
every execution.
The problem of crashing bug 22060 stemmed from the fact that the above
metnioned functions were not only modifying HA_CREATE_INFO structure in
LEX, but also were changing it to point to areas in volatile memory of
the execution memory root.
The patch solves this problem by creating and using an on-stack
copy of HA_CREATE_INFO (note that code in 5.1 already creates and
uses a copy of this structure in mysql_create_table()/alter_table(),
but this approach didn't work well for CREATE TABLE SELECT statement).
mysql-test/r/ps.result:
Update test results (Bug#19182, Bug#22060)
mysql-test/t/ps.test:
Add a test case for Bug#19182, Bug#22060 (4.1-only parts)
sql/mysql_priv.h:
LEX::key_list and LEX::create_list were moved to LEX::alter_info.
Update declarations to use LEX::alter_info instead of these two
members.
sql/sql_class.h:
Replace pair<columns, keys> with an instance of Alter_info in
select_create constructor. We create a new copy of Alter_info
each time we re-execute SELECT .. CREATE prepared statement.
sql/sql_insert.cc:
Adjust to a new signature of create_table_from_items.
sql/sql_lex.cc:
Implement Alter_info::Alter_info that would make a "deep" copy
of all definition lists (keys, columns).
sql/sql_lex.h:
Move key_list and create_list to class Alter_info. Implement
Alter_info::Alter_info that can be used with PS and SP.
sql/sql_list.h:
Implement a copy constructor of class List that makes a deep copy
of all list nodes.
sql/sql_parse.cc:
Adjust to new signatures of mysql_create_table, mysql_alter_table,
select_create. Functions mysql_create_index and mysql_drop_index has
become identical after initialization of alter_info was moved to the
parser, and were merged. Flag enable_slow_log was not updated for
SQLCOM_DROP_INDEX, which is a bug. Just like CREATE INDEX, DROP INDEX
is currently done via complete table rebuild and is rightfully a slow
administrative statement.
sql/sql_show.cc:
Adjust mysqld_show_create_db to a new signature.
sql/sql_table.cc:
Adjust mysql_alter_table, mysql_recreate_table, mysql_create_table,
mysql_prepare_table to new signatures.
sql/sql_yacc.yy:
LEX::key_list and LEX::create_list moved to class Alter_info
Bug#4968 "Stored procedure crash if cursor opened on altered table"
Bug#19733 "Repeated alter, or repeated create/drop, fails"
Bug#19182 "CREATE TABLE bar (m INT) SELECT n FROM foo; doesn't work from
stored procedure."
Bug#6895 "Prepared Statements: ALTER TABLE DROP COLUMN does nothing"
Bug#22060 "ALTER TABLE x AUTO_INCREMENT=y in SP crashes server"
Test cases for bugs 4968, 19733, 6895 will be added in 5.0.
Re-execution of CREATE DATABASE, CREATE TABLE and ALTER TABLE
statements in stored routines or as prepared statements caused
incorrect results (and crashes in versions prior to 5.0.25).
In 5.1 the problem occured only for CREATE DATABASE, CREATE TABLE
SELECT and CREATE TABLE with INDEX/DATA DIRECTOY options).
The problem of bugs 4968, 19733, 19282 and 6895 was that functions
mysql_prepare_table, mysql_create_table and mysql_alter_table were not
re-execution friendly: during their operation they used to modify contents
of LEX (members create_info, alter_info, key_list, create_list),
thus making the LEX unusable for the next execution.
In particular, these functions removed processed columns and keys from
create_list, key_list and drop_list. Search the code in sql_table.cc
for drop_it.remove() and similar patterns to find evidence.
The fix is to supply to these functions a usable copy of each of the
above structures at every re-execution of an SQL statement.
To simplify memory management, LEX::key_list and LEX::create_list
were added to LEX::alter_info, a fresh copy of which is created for
every execution.
The problem of crashing bug 22060 stemmed from the fact that the above
metnioned functions were not only modifying HA_CREATE_INFO structure in
LEX, but also were changing it to point to areas in volatile memory of
the execution memory root.
The patch solves this problem by creating and using an on-stack
copy of HA_CREATE_INFO (note that code in 5.1 already creates and
uses a copy of this structure in mysql_create_table()/alter_table(),
but this approach didn't work well for CREATE TABLE SELECT statement).
into neptunus.(none):/home/msvensson/mysql/mysql-5.1-maint
BitKeeper/etc/collapsed:
auto-union
BitKeeper/etc/ignore:
auto-union
Makefile.am:
Auto merged
client/mysql.cc:
Auto merged
client/mysqltest.c:
Auto merged
configure.in:
Auto merged
extra/yassl/taocrypt/include/algebra.hpp:
Auto merged
include/Makefile.am:
Auto merged
include/my_sys.h:
Auto merged
mysql-test/lib/mtr_io.pl:
Auto merged
mysql-test/lib/mtr_process.pl:
Auto merged
mysql-test/mysql-test-run.pl:
Auto merged
mysql-test/r/information_schema.result:
Auto merged
mysql-test/r/parser.result:
Auto merged
mysql-test/r/ps.result:
Auto merged
mysql-test/r/view_grant.result:
Auto merged
mysql-test/t/information_schema.test:
Auto merged
mysql-test/t/parser.test:
Auto merged
mysql-test/t/ps.test:
Auto merged
mysql-test/t/sp.test:
Auto merged
mysql-test/t/system_mysql_db_fix30020.test:
Auto merged
mysql-test/t/udf.test:
Auto merged
mysql-test/t/view_grant.test:
Auto merged
scripts/mysql_fix_privilege_tables.sql:
Auto merged
sql/ha_ndbcluster.cc:
Auto merged
sql/handler.cc:
Auto merged
sql/item.cc:
Auto merged
sql/item_create.cc:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_func.h:
Auto merged
sql/item_timefunc.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/sp.cc:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_handler.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql-common/my_time.c:
Auto merged
sql/sql_yacc.yy:
Auto merged
storage/myisam/myisampack.c:
Auto merged
mysql-test/r/sp.result:
Manual merge
mysql-test/r/udf.result:
Manual merge
mysql-test/t/events_bugs.test:
Manual merge
sql/share/errmsg.txt:
Manual merge
support-files/mysql.spec.sh:
Manual merge
Before this fix, a call to a User Defined Function (UDF) could,
under some circumstances, be interpreted as a call to a Stored function
instead. This occurred if a native function was invoked in the parameters
for the UDF, as in "select my_udf(abs(x))".
The root cause of this defect is the introduction, by the fix for Bug 21809,
of st_select_lex::udf_list, and it's usage in the parser in sql_yacc.yy
in the rule function_call_generic (in 5.1).
While the fix itself for Bug 21809 is correct in 5.0, the code change
merged into the 5.1 release created the issue, because the calls in 5.1 to :
- lex->current_select->udf_list.push_front(udf)
- lex->current_select->udf_list.pop()
are not balanced in case of native functions, causing the udf_list,
which is really a stack, to be out of sync with the internal stack
maintained by the bison parser.
Instead of moving the call to udf_list.pop(), which would have fixed the
symptom, this patch goes further and removes the need for udf_list.
This is motivated by two reasons:
a) Maintaining a stack in the MySQL code in sync with the stack maintained
internally in sql_yacc.cc (not .yy) is extremely dependent of the
implementation of yacc/bison, and extremely difficult to maintain.
It's also totally dependent of the structure of the grammar, and has a risk
to break with regression defects each time the grammar itself is changed.
b) The previous code did report construct like "foo(expr AS name)" as
syntax errors (ER_PARSER_ERROR), which is incorrect, and misleading.
The syntax is perfectly valid, as this expression is valid when "foo" is
a UDF. Whether this syntax is legal or not depends of the semantic of "foo".
With this change:
a) There is only one stack (in bison), and no List<udf_func> to maintain.
b) "foo(expr AS name)", when used incorrectly, is reported as semantic error:
- ER_WRONG_PARAMETERS_TO_NATIVE_FCT (for native functions)
- ER_WRONG_PARAMETERS_TO_STORED_FCT (for stored functions)
This is achieved by the changes implemented in item_create.cc
mysql-test/r/parser.result:
New tests
mysql-test/r/udf.result:
New tests
mysql-test/t/parser.test:
New tests
mysql-test/t/udf.test:
New tests
sql/item_create.cc:
Semantic checks for named parameters, as in "foo(expr AS name)".
sql/share/errmsg.txt:
New error message
sql/sql_lex.cc:
Remove usage of udf_list.
sql/sql_lex.h:
Remove usage of udf_list.
sql/sql_yacc.yy:
Remove usage of udf_list.
Before this fix, a call to a User Defined Function (UDF) could,
under some circumstances, be interpreted as a call to a Stored function
instead. This occurred if a native function was invoked in the parameters
for the UDF, as in "select my_udf(abs(x))".
The root cause of this defect is the introduction, by the fix for Bug 21809,
of st_select_lex::udf_list, and it's usage in the parser in sql_yacc.yy
in the rule function_call_generic (in 5.1).
While the fix itself for Bug 21809 is correct in 5.0, the code change
merged into the 5.1 release created the issue, because the calls in 5.1 to :
- lex->current_select->udf_list.push_front(udf)
- lex->current_select->udf_list.pop()
are not balanced in case of native functions, causing the udf_list,
which is really a stack, to be out of sync with the internal stack
maintained by the bison parser.
Instead of moving the call to udf_list.pop(), which would have fixed the
symptom, this patch goes further and removes the need for udf_list.
This is motivated by two reasons:
a) Maintaining a stack in the MySQL code in sync with the stack maintained
internally in sql_yacc.cc (not .yy) is extremely dependent of the
implementation of yacc/bison, and extremely difficult to maintain.
It's also totally dependent of the structure of the grammar, and has a risk
to break with regression defects each time the grammar itself is changed.
b) The previous code did report construct like "foo(expr AS name)" as
syntax errors (ER_PARSER_ERROR), which is incorrect, and misleading.
The syntax is perfectly valid, as this expression is valid when "foo" is
a UDF. Whether this syntax is legal or not depends of the semantic of "foo".
With this change:
a) There is only one stack (in bison), and no List<udf_func> to maintain.
b) "foo(expr AS name)", when used incorrectly, is reported as semantic error:
- ER_WRONG_PARAMETERS_TO_NATIVE_FCT (for native functions)
- ER_WRONG_PARAMETERS_TO_STORED_FCT (for stored functions)
This is achieved by the changes implemented in item_create.cc
Backport of functionality in private 5.2 tree.
Added new language to parser, new mysql.servers table and associated code
to be used by the federated storage engine to allow central connection information
per WL entry.
libmysqld/Makefile.am:
WL# 3031
Added sql_servers.cc to libmysqld Makefile.am
mysql-test/lib/init_db.sql:
WL# 3031
Added mysql.servers creation to init_db.sql, which is in turn called by mysql-test-run.pl
scripts/mysql_create_system_tables.sh:
WL# 3031
Added mysql.servers table creation to script called by non-perl mysql-test-run
sql/Makefile.am:
WL# 3031
Added sql_servers source and headers to Makefile.am to be built
sql/lex.h:
WL# 3031
Added tokens needed for SERVERS scheme
sql/mysql_priv.h:
WL #3031
Added sql_servers.h to mysql_priv.h so mysqld.cc and other code can link with sql_servers code
sql/mysqld.cc:
WL# 3031
Added servers_free and servers_init to mysqld
sql/sql_lex.cc:
WL# 3031
Added lex->server_options parameter initialisation
sql/sql_lex.h:
WL #3031
Added SQLCOM commands needed for sql_server actions
sql/sql_parse.cc:
WL# 3031
Added switch actions for new SQLCOM sql_server actions
sql/sql_yacc.yy:
WL #3031
Added tokens needed for sql_servers, CREATE/DROP/ALTER server functionality
sql/share/errmsg.txt:
WL# 3031
Added two new errors to errormsg.sys
storage/federated/ha_federated.cc:
WL #3031
Modified federated storage engine to use new connection scheme (servers
table) if connection string only specifies connection name vs. complete
URL
storage/federated/ha_federated.h:
WL# 3031
Added new share members needed for connection scheme
sql/sql_servers.cc:
WL #3031
sql_servers.cc - all methods required for manipulating mysql.servers table
to work with federated new connection scheme (CREATE/ALTER/DROP SERVER ...)
sql/sql_servers.h:
WL #3031
New header file for sql_servers functionality
mysql-test/r/federated_server.result:
WL #3031
New test results for testing new connection scheme
sql/sql_yacc.yy.bak:
WL #3031
Added tokens and definitions required for new CREATE/ALTER/DROP SERVER
for sql_servers functionality. See WL for details.
mysql-test/t/federated_server.test:
WL #3031
New test for testing CREATE/ALTER/DROP SERVER, as well as testing that
federated works with both tradition connection scheme as well as new
connection scheme.
Backport of functionality in private 5.2 tree.
Added new language to parser, new mysql.servers table and associated code
to be used by the federated storage engine to allow central connection information
per WL entry.
into bodhi.local:/opt/local/work/mysql-5.1-runtime-merge
libmysql/libmysql.c:
Auto merged
libmysqld/lib_sql.cc:
Auto merged
mysql-test/r/udf.result:
Auto merged
mysql-test/t/udf.test:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_func.h:
Auto merged
sql/item_sum.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_yacc.yy:
Auto merged
into bodhi.local:/opt/local/work/mysql-5.1-runtime-merge
libmysql/libmysql.c:
Auto merged
mysql-test/r/ps.result:
Auto merged
mysql-test/t/func_str.test:
Auto merged
mysql-test/t/ps.test:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_strfunc.cc:
Auto merged
sql/item_sum.cc:
Auto merged
sql/mysqld.cc:
Auto merged
sql/set_var.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_yacc.yy:
Auto merged
server-tools/instance-manager/mysql_connection.cc:
Rollback Monty's removals of explicit casts
tests/mysql_client_test.c:
SCCS merged
into dl145s.mysql.com:/data0/bk/team_tree_merge/MERGE/mysql-5.1-opt
client/mysqltest.c:
Auto merged
libmysql/libmysql.c:
Auto merged
mysql-test/r/information_schema.result:
Auto merged
mysql-test/t/disabled.def:
Auto merged
mysql-test/t/information_schema.test:
Auto merged
mysql-test/t/subselect.test:
Auto merged
mysql-test/t/type_newdecimal.test:
Auto merged
sql/field.cc:
Auto merged
sql/field.h:
Auto merged
sql/filesort.cc:
Auto merged
sql/handler.cc:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_subselect.cc:
Auto merged
sql/item_sum.cc:
Auto merged
sql/item_timefunc.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/opt_range.cc:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
sql-common/client.c:
Auto merged
sql-common/my_time.c:
Auto merged
sql/table.cc:
Auto merged
strings/decimal.c:
Auto merged
into mysql.com:/home/my/mysql-5.1
BUILD/SETUP.sh:
Auto merged
mysql-test/t/create.test:
Auto merged
sql/item_timefunc.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/table.cc:
Auto merged
tests/mysql_client_test.c:
Auto merged
sql/log_event.cc:
Manual merge
sql/sql_binlog.cc:
Manual merge
Added missing DBUG_RETURN statements (in mysqldump.c)
Added missing enums
Fixed a lot of wrong DBUG_PRINT() statements, some of which could cause crashes
Removed usage of %lld and %p in printf strings as these are not portable or produces different results on different systems.
limitation)
Note to the reviewer
====================
Warning: reviewing this patch is somewhat involved.
Due to the nature of several issues all affecting the same area,
fixing separately each issue is not practical, since each fix can not be
implemented and tested independently.
In particular, the issues with
- rule recursion
- nested case statements
- forward jump resolution (backpatch list)
are tightly coupled (see below).
Definitions
===========
The expression
CASE expr
WHEN expr THEN expr
WHEN expr THEN expr
...
END
is a "Simple Case Expression".
The expression
CASE
WHEN expr THEN expr
WHEN expr THEN expr
...
END
is a "Searched Case Expression".
The statement
CASE expr
WHEN expr THEN stmts
WHEN expr THEN stmts
...
END CASE
is a "Simple Case Statement".
The statement
CASE
WHEN expr THEN stmts
WHEN expr THEN stmts
...
END CASE
is a "Searched Case Statement".
A "Left Recursive" rule is like
list:
element
| list element
;
A "Right Recursive" rule is like
list:
element
| element list
;
Left and right recursion produces the same language, the difference only
affects the *order* in which the text is parsed.
In a descendant parser (usually written manually), right recursion works
very well, and is typically implemented with a while loop.
In an ascendant parser (yacc/bison) left recursion works very well,
and is implemented naturally by the parser stack.
In both cases, using the wrong type or recursion is very bad and should be
avoided, as it causes technical issues with the parser implementation.
Before this change
==================
The "Simple Case Expression" and "Searched Case Expression" were both
implemented by the "when_list" and "when_list2" rules, which are left
recursive (ok).
These rules, however, used lex->when_list instead of using the parser stack,
which is more complex that necessary, and potentially dangerous because
of other rules using THD::reset_lex.
The "Simple Case Statement" and "Searched Case Statements" were implemented
by the "sp_case", "sp_whens" and in part by "sp_proc_stmt" rules.
Both cases were right recursive (bad).
The grammar involved was convoluted, and is assumed to be the results of
tweaks to get the code generation to work, but is not what someone would
naturally write.
In addition, using a common rule for both "Simple" and "Searched" case
statements was implemented with sp_head::m_flags |= IN_SIMPLE_CASE,
which is a flag and not a stack, and therefore does not take into account
*nested* case statements. This leads to incorrect generated code, and either
a server crash or an incorrect result.
With regards to the backpatch mechanism, a *different* backpatch list was
created for each jump from "WHEN expr THEN stmt" to "END CASE", which
relied on the grammar to be right recursive.
This is a mis-use of the backpatch list, since this list can resolve
multiple references to the same target at once.
The optimizer algorithm used to detect dead code in the "assembly" SQL
instructions, implemented by sp_head::opt_mark(uint ip), was recursive
in some cases (a conditional jump pointing forward to another conditional
jump).
In case of specially crafted code, like
- a long list of "IF expr THEN stmt END IF"
- a long CASE statement
this would actually cause a server crash with a stack overflow.
In general, having a stack that grows proportionally with user data (the
SQL code given by the client in a CREATE PROCEDURE) is to be avoided.
In debug builds only, creating a SP / SF / Trigger which had a significant
amount of code would spend --literally-- several minutes in sp_head::create,
because of the debug code involved with DBUG_PRINT("info", ("Code %s ...
There are several issues with this code:
- in a CASE with 5 000 WHEN, there are 15 000 instructions generated,
which create a sting representation of the code which is 500 000 bytes
long,
- using a String instead of an io stream causes performances to degrade
to a total server freeze, as time is spent doing realloc of a buffer
always too short,
- Printing a 500 000 long string in the debug log is too verbose,
- Generating this string even when DBUG_PRINT is off is useless,
- Having code that potentially can affect the server behavior, used with
#ifdef / #endif is useful in some cases, but is also a bad practice.
After this change
=================
"Case Expressions" (both simple and searched) have been simplified to
not use LEX::when_list, which has been removed.
Considering all the issues affecting case statements, the grammar for these
has been totally re written.
The existing actions, used to generate "assembly" sp_inst* code, have been
preserved but moved in the new grammar, with the following changes:
a) Bison rules are no longer shared between "Simple" and "Searched" case
statements, because a stack instead of a flag is required to handle them.
Nested statements are handled naturally by the parser stack, which by
definition uses the correct rule in the correct context.
Nested statements of the opposite type (simple vs searched) works correctly.
The flag sp_head::IN_SIMPLE_CASE is no longer used.
This is a step towards resolution of WL#2999, which correctly identified
that temporary parsing flags do not belong to sp_head.
The code in the action is shared by mean of the case_stmt_action_xxx()
helpers.
b) The backpatch mechanism, used to resolve forward jumps in the generated
code, has been changed to:
- create a label for the instruction following 'END CASE',
- register each jump at the end of a "WHEN expr THEN stmt" in a *unique*
backpatch list associated with the 'END CASE' label
- resolve all the forward jumps for this label at once.
In addition, the code involving backpatch has been commented, so that a
reader can now understand by reading matching "Registering" and "Resolving"
comments how the forward jumps are resolved and what target they resolve to,
as this is far from evident when reading the code alone.
The implementation of sp_head::opt_mark() has been revised to avoid
recursive calls from jump instructions, and instead add the jump location
to the list of paths to explore during the flow analysis of the instruction
graph, with a call to sp_head::add_mark_lead().
In addition, the flow analysis will stop if an instruction has already
been marked as reachable, which the previous code failed to do in the
recursive case.
sp_head::opt_mark() is now private, to prevent new calls to this method from
being introduced.
The debug code present in sp_head::create() has been removed.
Considering that SHOW PROCEDURE CODE is also available in debug builds,
and can be used anytime regardless of the trace level, as opposed to
"CREATE PROCEDURE" time and only if the trace was on,
removing the code actually makes debugging easier (usable trace).
Tests have been written to cover the parser overflow (big CASE),
and to cover nested CASE statements.
mysql-test/r/sp-code.result:
Test cases for nested CASE statements.
mysql-test/t/sp-code.test:
Test cases for nested CASE statements.
sql/sp_head.cc:
Re factored opt_mark() to avoid recursion, clean up.
sql/sp_head.h:
Re factored opt_mark() to avoid recursion, clean up.
sql/sql_lex.cc:
Removed when_list.
sql/sql_lex.h:
Removed when_list.
sql/sql_yacc.yy:
Minor clean up for case expressions,
Major re write for case statements (Bug#19194).
mysql-test/r/sp_stress_case.result:
New test for massive CASE statements.
mysql-test/t/sp_stress_case.sh:
New test for massive CASE statements.
mysql-test/t/sp_stress_case.test:
New test for massive CASE statements.
limitation)
Note to the reviewer
====================
Warning: reviewing this patch is somewhat involved.
Due to the nature of several issues all affecting the same area,
fixing separately each issue is not practical, since each fix can not be
implemented and tested independently.
In particular, the issues with
- rule recursion
- nested case statements
- forward jump resolution (backpatch list)
are tightly coupled (see below).
Definitions
===========
The expression
CASE expr
WHEN expr THEN expr
WHEN expr THEN expr
...
END
is a "Simple Case Expression".
The expression
CASE
WHEN expr THEN expr
WHEN expr THEN expr
...
END
is a "Searched Case Expression".
The statement
CASE expr
WHEN expr THEN stmts
WHEN expr THEN stmts
...
END CASE
is a "Simple Case Statement".
The statement
CASE
WHEN expr THEN stmts
WHEN expr THEN stmts
...
END CASE
is a "Searched Case Statement".
A "Left Recursive" rule is like
list:
element
| list element
;
A "Right Recursive" rule is like
list:
element
| element list
;
Left and right recursion produces the same language, the difference only
affects the *order* in which the text is parsed.
In a descendant parser (usually written manually), right recursion works
very well, and is typically implemented with a while loop.
In an ascendant parser (yacc/bison) left recursion works very well,
and is implemented naturally by the parser stack.
In both cases, using the wrong type or recursion is very bad and should be
avoided, as it causes technical issues with the parser implementation.
Before this change
==================
The "Simple Case Expression" and "Searched Case Expression" were both
implemented by the "when_list" and "when_list2" rules, which are left
recursive (ok).
These rules, however, used lex->when_list instead of using the parser stack,
which is more complex that necessary, and potentially dangerous because
of other rules using THD::reset_lex.
The "Simple Case Statement" and "Searched Case Statements" were implemented
by the "sp_case", "sp_whens" and in part by "sp_proc_stmt" rules.
Both cases were right recursive (bad).
The grammar involved was convoluted, and is assumed to be the results of
tweaks to get the code generation to work, but is not what someone would
naturally write.
In addition, using a common rule for both "Simple" and "Searched" case
statements was implemented with sp_head::m_flags |= IN_SIMPLE_CASE,
which is a flag and not a stack, and therefore does not take into account
*nested* case statements. This leads to incorrect generated code, and either
a server crash or an incorrect result.
With regards to the backpatch mechanism, a *different* backpatch list was
created for each jump from "WHEN expr THEN stmt" to "END CASE", which
relied on the grammar to be right recursive.
This is a mis-use of the backpatch list, since this list can resolve
multiple references to the same target at once.
The optimizer algorithm used to detect dead code in the "assembly" SQL
instructions, implemented by sp_head::opt_mark(uint ip), was recursive
in some cases (a conditional jump pointing forward to another conditional
jump).
In case of specially crafted code, like
- a long list of "IF expr THEN stmt END IF"
- a long CASE statement
this would actually cause a server crash with a stack overflow.
In general, having a stack that grows proportionally with user data (the
SQL code given by the client in a CREATE PROCEDURE) is to be avoided.
In debug builds only, creating a SP / SF / Trigger which had a significant
amount of code would spend --literally-- several minutes in sp_head::create,
because of the debug code involved with DBUG_PRINT("info", ("Code %s ...
There are several issues with this code:
- in a CASE with 5 000 WHEN, there are 15 000 instructions generated,
which create a sting representation of the code which is 500 000 bytes
long,
- using a String instead of an io stream causes performances to degrade
to a total server freeze, as time is spent doing realloc of a buffer
always too short,
- Printing a 500 000 long string in the debug log is too verbose,
- Generating this string even when DBUG_PRINT is off is useless,
- Having code that potentially can affect the server behavior, used with
#ifdef / #endif is useful in some cases, but is also a bad practice.
After this change
=================
"Case Expressions" (both simple and searched) have been simplified to
not use LEX::when_list, which has been removed.
Considering all the issues affecting case statements, the grammar for these
has been totally re written.
The existing actions, used to generate "assembly" sp_inst* code, have been
preserved but moved in the new grammar, with the following changes:
a) Bison rules are no longer shared between "Simple" and "Searched" case
statements, because a stack instead of a flag is required to handle them.
Nested statements are handled naturally by the parser stack, which by
definition uses the correct rule in the correct context.
Nested statements of the opposite type (simple vs searched) works correctly.
The flag sp_head::IN_SIMPLE_CASE is no longer used.
This is a step towards resolution of WL#2999, which correctly identified
that temporary parsing flags do not belong to sp_head.
The code in the action is shared by mean of the case_stmt_action_xxx()
helpers.
b) The backpatch mechanism, used to resolve forward jumps in the generated
code, has been changed to:
- create a label for the instruction following 'END CASE',
- register each jump at the end of a "WHEN expr THEN stmt" in a *unique*
backpatch list associated with the 'END CASE' label
- resolve all the forward jumps for this label at once.
In addition, the code involving backpatch has been commented, so that a
reader can now understand by reading matching "Registering" and "Resolving"
comments how the forward jumps are resolved and what target they resolve to,
as this is far from evident when reading the code alone.
The implementation of sp_head::opt_mark() has been revised to avoid
recursive calls from jump instructions, and instead add the jump location
to the list of paths to explore during the flow analysis of the instruction
graph, with a call to sp_head::add_mark_lead().
In addition, the flow analysis will stop if an instruction has already
been marked as reachable, which the previous code failed to do in the
recursive case.
sp_head::opt_mark() is now private, to prevent new calls to this method from
being introduced.
The debug code present in sp_head::create() has been removed.
Considering that SHOW PROCEDURE CODE is also available in debug builds,
and can be used anytime regardless of the trace level, as opposed to
"CREATE PROCEDURE" time and only if the trace was on,
removing the code actually makes debugging easier (usable trace).
Tests have been written to cover the parser overflow (big CASE),
and to cover nested CASE statements.
into mysql.com:/home/hf/work/mysql-5.1-mrg
BitKeeper/deleted/.del-bdb-deadlock.test:
Auto merged
Makefile.am:
Auto merged
include/mysql.h:
Auto merged
libmysqld/lib_sql.cc:
Auto merged
mysql-test/include/deadlock.inc:
Auto merged
mysql-test/r/order_by.result:
Auto merged
mysql-test/r/subselect.result:
Auto merged
mysql-test/r/type_newdecimal.result:
Auto merged
mysql-test/t/disabled.def:
Auto merged
mysql-test/t/flush.test:
Auto merged
mysql-test/t/flush_block_commit.test:
Auto merged
mysql-test/t/innodb-lock.test:
Auto merged
mysql-test/t/lock_multi.test:
Auto merged
mysql-test/t/rename.test:
Auto merged
mysql-test/t/show_check.test:
Auto merged
mysql-test/t/status.test:
Auto merged
mysql-test/t/subselect.test:
Auto merged
mysql-test/t/type_newdecimal.test:
Auto merged
sql/field.cc:
Auto merged
sql/field.h:
Auto merged
sql/filesort.cc:
Auto merged
sql/handler.cc:
Auto merged
sql/item.cc:
Auto merged
sql/item.h:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_func.h:
Auto merged
sql/item_subselect.cc:
Auto merged
sql/item_subselect.h:
Auto merged
sql/item_sum.cc:
Auto merged
sql/item_sum.h:
Auto merged
sql/item_timefunc.cc:
Auto merged
sql/item_timefunc.h:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_show.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql-common/my_time.c:
Auto merged
sql/table.cc:
Auto merged
client/mysqltest.c:
merging
include/my_time.h:
merging
libmysql/libmysql.c:
merging
mysql-test/t/order_by.test:
merging
sql/sql_yacc.yy:
merging
into mysql.com:/home/hf/work/mysql-5.0-0mrg
BitKeeper/deleted/.del-mysql_client.test:
Auto merged
include/mysql.h:
Auto merged
libmysqld/lib_sql.cc:
Auto merged
mysql-test/r/order_by.result:
Auto merged
mysql-test/r/query_cache.result:
Auto merged
mysql-test/r/subselect.result:
Auto merged
mysql-test/r/type_newdecimal.result:
Auto merged
mysql-test/t/flush.test:
Auto merged
mysql-test/t/flush_block_commit.test:
Auto merged
mysql-test/t/innodb-deadlock.test:
Auto merged
mysql-test/t/innodb-lock.test:
Auto merged
mysql-test/t/lock_multi.test:
Auto merged
mysql-test/t/mysql.test:
Auto merged
mysql-test/t/query_cache.test:
Auto merged
mysql-test/t/rename.test:
Auto merged
mysql-test/t/show_check.test:
Auto merged
mysql-test/t/status.test:
Auto merged
mysql-test/t/subselect.test:
Auto merged
mysql-test/t/type_newdecimal.test:
Auto merged
sql/field.cc:
Auto merged
sql/field.h:
Auto merged
sql/handler.cc:
Auto merged
sql/item.cc:
Auto merged
sql/item.h:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_func.h:
Auto merged
sql/item_subselect.cc:
Auto merged
sql/item_subselect.h:
Auto merged
sql/item_sum.cc:
Auto merged
sql/item_sum.h:
Auto merged
sql/item_timefunc.cc:
Auto merged
sql/item_timefunc.h:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/protocol.cc:
Auto merged
sql/protocol.h:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_show.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
sql/table.cc:
Auto merged
sql-common/client.c:
Auto merged
sql-common/my_time.c:
Auto merged
Makefile.am:
merging
client/mysqltest.c:
merging
include/my_time.h:
merging
libmysql/libmysql.c:
merging
mysql-test/t/order_by.test:
merging
into ua141d10.elisa.omakaista.fi:/home/my/bk/mysql-5.1-lex-string
BUILD/SETUP.sh:
Auto merged
mysql-test/r/create.result:
Auto merged
mysql-test/t/create.test:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_db.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
sql/table.cc:
Auto merged
tests/mysql_client_test.c:
Auto merged
sql/item_timefunc.cc:
Manual merge.
sql/mysqld.cc:
Manual merge.
sql/mysqld.cc.rej:
manual merge
Bug#21025 (misleading error message when creating functions named 'x', or 'y')
Bug#22619 (Spaces considered harmful)
This change contains a fix to report warnings or errors, and multiple tests
cases.
Before this fix, name collisions between:
- Native functions
- User Defined Functions
- Stored Functions
were not systematically reported, leading to confusing behavior.
I) Native / User Defined Function
Before this fix, is was possible to create a UDF named "foo", with the same
name as a native function "foo", but it was impossible to invoke the UDF,
since the syntax "foo()" always refer to the native function.
After this fix, creating a UDF fails with an error if there is a name
collision with a native function.
II) Native / Stored Function
Before this fix, is was possible to create a SF named "db.foo", with the same
name as a native function "foo", but this was confusing since the syntax
"foo()" would refer to the native function. To refer to the Stored Function,
the user had to use the "db.foo()" syntax.
After this fix, creating a Stored Function reports a warning if there is a
name collision with a native function.
III) User Defined Function / Stored Function
Before this fix, creating a User Defined Function "foo" and a Stored Function
"db.foo" are mutually exclusive operations. Whenever the second function is
created, an error is reported. However, the test suite did not cover this
behavior.
After this fix, the behavior is unchanged, and is now covered by test cases.
Note that the code change in this patch depends on the fix for Bug 21114.
mysql-test/r/sp.result:
New test cases.
mysql-test/r/udf.result:
New test cases.
mysql-test/t/sp.test:
New test cases.
mysql-test/t/udf.test:
New test cases.
sql/sql_lex.cc:
Name collisions with native functions.
sql/sql_lex.h:
Name collisions with native functions.
sql/sql_yacc.yy:
Name collisions with native functions.
sql/share/errmsg.txt:
Name collisions with native functions.
mysql-test/r/sp_gis.result:
New test cases.
mysql-test/t/sp_gis.test:
New test cases.
Bug#21025 (misleading error message when creating functions named 'x', or 'y')
Bug#22619 (Spaces considered harmful)
This change contains a fix to report warnings or errors, and multiple tests
cases.
Before this fix, name collisions between:
- Native functions
- User Defined Functions
- Stored Functions
were not systematically reported, leading to confusing behavior.
I) Native / User Defined Function
Before this fix, is was possible to create a UDF named "foo", with the same
name as a native function "foo", but it was impossible to invoke the UDF,
since the syntax "foo()" always refer to the native function.
After this fix, creating a UDF fails with an error if there is a name
collision with a native function.
II) Native / Stored Function
Before this fix, is was possible to create a SF named "db.foo", with the same
name as a native function "foo", but this was confusing since the syntax
"foo()" would refer to the native function. To refer to the Stored Function,
the user had to use the "db.foo()" syntax.
After this fix, creating a Stored Function reports a warning if there is a
name collision with a native function.
III) User Defined Function / Stored Function
Before this fix, creating a User Defined Function "foo" and a Stored Function
"db.foo" are mutually exclusive operations. Whenever the second function is
created, an error is reported. However, the test suite did not cover this
behavior.
After this fix, the behavior is unchanged, and is now covered by test cases.
Note that the code change in this patch depends on the fix for Bug 21114.
into mysql.com:/home/hf/work/mysql-5.1.clean
include/my_time.h:
Auto merged
mysql-test/r/func_group.result:
Auto merged
mysql-test/r/gis-rtree.result:
Auto merged
mysql-test/r/order_by.result:
Auto merged
mysql-test/r/subselect.result:
Auto merged
mysql-test/r/type_datetime.result:
Auto merged
mysql-test/r/udf.result:
Auto merged
mysql-test/t/func_group.test:
Auto merged
mysql-test/t/gis-rtree.test:
Auto merged
mysql-test/t/type_datetime.test:
Auto merged
mysql-test/t/udf.test:
Auto merged
sql/field.cc:
Auto merged
sql/field.h:
Auto merged
sql/filesort.cc:
Auto merged
sql/item.h:
Auto merged
sql/item_cmpfunc.cc:
Auto merged
sql/item_cmpfunc.h:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_func.h:
Auto merged
sql/item_subselect.h:
Auto merged
sql/item_sum.cc:
Auto merged
sql/item_sum.h:
Auto merged
sql/item_timefunc.cc:
Auto merged
sql/my_decimal.cc:
Auto merged
sql/my_decimal.h:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/records.cc:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_show.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
sql-common/my_time.c:
Auto merged
sql/table.cc:
Auto merged
sql/table.h:
Auto merged
mysql-test/r/type_newdecimal.result:
SCCS merged
mysql-test/t/type_newdecimal.test:
merging
sql/item.cc:
merging
sql/item_subselect.cc:
SCCS merged
sql/item_timefunc.h:
merging
sql/sql_lex.cc:
SCCS merged
sql/sql_select.h:
merging
into example.com:/work/mysql-5.1-runtime-bug22830-new
sql/sql_lex.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_parse.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
Events: crash with procedure which alters events with function
Post-review CS
This fix also changes the handling of KILL command combined with
subquery. It changes the error message given back to "not supported",
from parse error. The error for CREATE|ALTER EVENT has also been changed
to generate "not supported yet" instead of parse error.
In case of a SP call, the error is "not supported yet". This change
cleans the parser from code which should not belong to there. Still
LEX::expr_allows_subselect is existant because it simplifies the handling
of SQLCOM_HA_READ which forbids subselects.
mysql-test/r/events_bugs.result:
update resut
mysql-test/r/events_grant.result:
update result
mysql-test/r/kill.result:
the error message has been changed for KILL
mysql-test/t/events_bugs.test:
Update old tests with the new emitted error
Add a test case for
BUG#22830 Events: crash with procedure which alters events with function
mysql-test/t/events_grant.test:
add ORDER BY clause to keep the result deterministic.
mysql-test/t/kill.test:
use name of the error, and change the error
from parse error, to not supported
sql/sql_lex.cc:
Add an auxiliary function that checks whether SP and/or
tables are used in the statement. This function is helpful for
statements that cannot handle subqueries ans SP calls. Adding out
of the parser cleans the latter of handling of special cases and
letting it do its job of parsing.
sql/sql_lex.h:
helper function to check whether a table or SP was used
sql/sql_parse.cc:
Use LEX::table_or_sp_used() for SQLCOM_CREATE_EVENT, SQLCOM_ALTER_EVENT
and SQLCOM_KILL. SQLCOM_DROP event does not use `expr` rule and thus a check is
not needed.
sql/sql_yacc.yy:
Remove usage of LEX::expr_allows_subselect for CREATE|ALTER EVENT
and KILL. There is only one left occurence - SQLCOM_HAREAD, but it
adds one table to the list of tables
Events: crash with procedure which alters events with function
Post-review CS
This fix also changes the handling of KILL command combined with
subquery. It changes the error message given back to "not supported",
from parse error. The error for CREATE|ALTER EVENT has also been changed
to generate "not supported yet" instead of parse error.
In case of a SP call, the error is "not supported yet". This change
cleans the parser from code which should not belong to there. Still
LEX::expr_allows_subselect is existant because it simplifies the handling
of SQLCOM_HA_READ which forbids subselects.
