The patch replaces the use of the POSIX I/O interfaces in mysys on Windows with
the Win32 API calls (CreateFile, WriteFile, etc). The Windows HANDLE for the open
file is stored in the my_file_info struct, along with a flag for append mode
(because the Windows API does not support opening files in append mode in all cases)
The default max open files has been increased to 16384 and can be increased further
by setting --max-open-files=<value> during the server start.
Noteworthy benefit of this patch is that it removes limits from the table_cache size -
allowing for more simultaneus users
Noted that there was no memory leak, just a lot of used partitioned tables.
Fixed old bug: 'show status' now shows memory usage when compiled with safemalloc.
Added option --flush to mysqlcheck.c to run a 'flush tables' between each check to keep down memory usage.
Changed '--safemalloc' options to mysqld so that one can use --safemalloc and --skip-safemalloc.
Now skip-safemalloc is default (ie, we only do checking of memory overrun during free()) to speed up tests.
client/client_priv.h:
Added OPT_FLUSH_TABLES
client/mysqlcheck.c:
Added option --flush to mysqlcheck.c to run a 'flush tables' between each check to keep down memory usage.
mysql-test/mysql-test-run.pl:
Always run tests with --loose-skip-safemysqld for higher speed
sql/mysqld.cc:
Changed '--safemalloc' options so that one can use --safemalloc and --skip-safemalloc.
Now skip-safemalloc is default (ie, we only do checking of memory overrun during free()) to speed up tests
sql/sql_parse.cc:
Fixed old bug: 'show status' now shows memory usage when compiled with safemalloc.
storage/archive/archive_reader.c:
Changed all malloc() calls to use my_malloc()/my_free()
Added checks of malloc() calls.
storage/archive/ha_archive.cc:
Detect failure if init_archive_reader() and return errno. This fixed assert crash in my_seek().
Changed all malloc() calls to use my_malloc()/my_free()
LEAK WITH PARTITIONED ARCHIVE TABLES
CHECK TABLE against archive table, when file descriptors
are exhausted, caused server crash.
Archive didn't handle errors when opening data file for
CHECK TABLE.
mysql-test/r/archive_debug.result:
A test case for BUG#12402794.
mysql-test/t/archive_debug.test:
A test case for BUG#12402794.
storage/archive/azio.c:
A test case for BUG#12402794.
storage/archive/ha_archive.cc:
Handle init_archive_reader() failure.
- Fix active lock in freed memory in ha_archive (share mutex was not released prior to free())
- Do not attempt vio_fastsend or vio_keepalive on named pipes and shared memory.
Manual merge from mysql-5.1 into mysql-5.5.
Conflicts
=========
Text conflict in mysql-test/suite/rpl/t/rpl_row_until.test
Text conflict in sql/handler.h
Text conflict in storage/archive/ha_archive.cc
ordered data from archive tables
Archive was using wrong memory address to check if field
is NULL (after filesort, when reading record again).
mysql-test/r/archive.result:
A test case for BUG#11764339.
mysql-test/t/archive.test:
A test case for BUG#11764339.
storage/archive/ha_archive.cc:
Null bytes are restored to "record" buffer, which may
or may not be equal to record buffer for field. Check
null bits in "record" buffer, instead of Field::null_ptr.
storage/archive/ha_archive.cc:
Fixed bug that caused archive.test to fail with: Warning: Found locks from different threads for lock 't2' in 'write' at 'start of release lock'...
The reason was that the TL_WRITE_DELAYED lock was changed to TL_WRITE_ALLOW_WRITE which conflicted with how the lock was used in sql_insert.cc (in particular with thr_upgrade_write_delay_lock())
archive_discover
Fixed buffer underrun in cleanup_dirname().
Also fixed that original (unencoded) database and table
names were used to discover archive tables.
mysql-test/r/archive.result:
A test case for BUG#58205.
mysql-test/t/archive.test:
A test case for BUG#58205.
mysys/mf_pack.c:
Fixed buffer underrun in cleanup_dirname(), when
it gets path like "a/../" (relative path, where
first directory is to be cut off).
storage/archive/ha_archive.cc:
Handler discover method gets database and table
names as is. It must use build_table_filename()
to get name similar to what it gets on create()
and open().
Bug#54678: InnoDB, TRUNCATE, ALTER, I_S SELECT, crash or deadlock
- Incompatible change: truncate no longer resorts to a row by
row delete if the storage engine does not support the truncate
method. Consequently, the count of affected rows does not, in
any case, reflect the actual number of rows.
- Incompatible change: it is no longer possible to truncate a
table that participates as a parent in a foreign key constraint,
unless it is a self-referencing constraint (both parent and child
are in the same table). To work around this incompatible change
and still be able to truncate such tables, disable foreign checks
with SET foreign_key_checks=0 before truncate. Alternatively, if
foreign key checks are necessary, please use a DELETE statement
without a WHERE condition.
Problem description:
The problem was that for storage engines that do not support
truncate table via a external drop and recreate, such as InnoDB
which implements truncate via a internal drop and recreate, the
delete_all_rows method could be invoked with a shared metadata
lock, causing problems if the engine needed exclusive access
to some internal metadata. This problem originated with the
fact that there is no truncate specific handler method, which
ended up leading to a abuse of the delete_all_rows method that
is primarily used for delete operations without a condition.
Solution:
The solution is to introduce a truncate handler method that is
invoked when the engine does not support truncation via a table
drop and recreate. This method is invoked under a exclusive
metadata lock, so that there is only a single instance of the
table when the method is invoked.
Also, the method is not invoked and a error is thrown if
the table is a parent in a non-self-referencing foreign key
relationship. This was necessary to avoid inconsistency as
some integrity checks are bypassed. This is inline with the
fact that truncate is primarily a DDL operation that was
designed to quickly remove all data from a table.
mysql-test/suite/innodb/t/innodb-truncate.test:
Add test cases for truncate and foreign key checks.
Also test that InnoDB resets auto-increment on truncate.
mysql-test/suite/innodb/t/innodb.test:
FK is not necessary, test is related to auto-increment.
Update error number, truncate is no longer invoked if
table is parent in a FK relationship.
mysql-test/suite/innodb/t/innodb_mysql.test:
Update error number, truncate is no longer invoked if
table is parent in a FK relationship.
Use delete instead of truncate, test is used to check
the interaction of FKs, triggers and delete.
mysql-test/suite/parts/inc/partition_check.inc:
Fix typo.
mysql-test/suite/sys_vars/t/foreign_key_checks_func.test:
Update error number, truncate is no longer invoked if
table is parent in a FK relationship.
mysql-test/t/mdl_sync.test:
Modify test case to reflect and ensure that truncate takes
a exclusive metadata lock.
mysql-test/t/trigger-trans.test:
Update error number, truncate is no longer invoked if
table is parent in a FK relationship.
sql/ha_partition.cc:
Reorganize the various truncate methods. delete_all_rows is now
passed directly to the underlying engines, so as truncate. The
code responsible for truncating individual partitions is moved
to ha_partition::truncate_partition, which is invoked when a
ALTER TABLE t1 TRUNCATE PARTITION p statement is executed.
Since the partition truncate no longer can be invoked via
delete, the bitmap operations are not necessary anymore. The
explicit reset of the auto-increment value is also removed
as the underlying engines are now responsible for reseting
the value.
sql/handler.cc:
Wire up the handler truncate method.
sql/handler.h:
Introduce and document the truncate handler method. It assumes
certain use cases of delete_all_rows.
Add method to retrieve the list of foreign keys referencing a
table. Method is used to avoid truncating tables that are
parent in a foreign key relationship.
sql/share/errmsg-utf8.txt:
Add error message for truncate and FK.
sql/sql_lex.h:
Introduce a flag so that the partition engine can detect when
a partition is being truncated. Used to give a special error.
sql/sql_parse.cc:
Function mysql_truncate_table no longer exists.
sql/sql_partition_admin.cc:
Implement the TRUNCATE PARTITION statement.
sql/sql_truncate.cc:
Change the truncate table implementation to use the new truncate
handler method and to not rely on row-by-row delete anymore.
The truncate handler method is always invoked with a exclusive
metadata lock. Also, it is no longer possible to truncate a
table that is parent in some non-self-referencing foreign key.
storage/archive/ha_archive.cc:
Rename method as the description indicates that in the future
this could be a truncate operation.
storage/blackhole/ha_blackhole.cc:
Implement truncate as no operation for the blackhole engine in
order to remain compatible with older releases.
storage/federated/ha_federated.cc:
Introduce truncate method that invokes delete_all_rows.
This is required to support partition truncate as this
form of truncate does not implement the drop and recreate
protocol.
storage/heap/ha_heap.cc:
Introduce truncate method that invokes delete_all_rows.
This is required to support partition truncate as this
form of truncate does not implement the drop and recreate
protocol.
storage/ibmdb2i/ha_ibmdb2i.cc:
Introduce truncate method that invokes delete_all_rows.
This is required to support partition truncate as this
form of truncate does not implement the drop and recreate
protocol.
storage/innobase/handler/ha_innodb.cc:
Rename delete_all_rows to truncate. InnoDB now does truncate
under a exclusive metadata lock.
Introduce and reorganize methods used to retrieve the list
of foreign keys referenced by a or referencing a table.
storage/myisammrg/ha_myisammrg.cc:
Introduce truncate method that invokes delete_all_rows.
This is required in order to remain compatible with earlier
releases where truncate would resort to a row-by-row delete.
Made long file names from previous patch shorter
mysql-test/r/archive.result:
Added testing of repair (for upgrade) of 5.0 tables.
mysql-test/std_data/archive_5_0.ARM:
Archive table created in MySQL 5.0
mysql-test/std_data/archive_5_0.ARZ:
Archive table created in MySQL 5.0
mysql-test/std_data/archive_5_0.frm:
Archive table created in MySQL 5.0
mysql-test/std_data/long_table_name.MYD:
Made long file names shorter
mysql-test/std_data/long_table_name.MYI:
Made long file names shorter
mysql-test/std_data/long_table_name.frm:
Made long file names shorter
mysql-test/t/archive.test:
Added testing of repair (for upgrade) of 5.0 tables.
sql/sql_table.cc:
Allow recreate to open crashed tables.
sql/table.cc:
Fix error message if storage engine doesn't exists.
storage/archive/azio.c:
Reset status values in case archive is of old versions
storage/archive/ha_archive.cc:
Fix to allow one to open old versions of table during repair
Reset status variables for old version tables
If the the table is of old version, force upgrade with ALTER TABLE when doing repair
storage/archive/ha_archive.h:
Added variables to detect old versions
The problem was that the optimize method of the ARCHIVE storage
engine was not preserving the FRM embedded in the ARZ file when
rewriting the ARZ file for optimization. The ARCHIVE engine stores
the FRM in the ARZ file so it can be transferred from machine to
machine without also copying the FRM -- the engine restores the
embedded FRM during discovery.
The solution is to copy over the FRM when rewriting the ARZ file.
In addition, some initial error checking is performed to ensure
garbage is not copied over.
mysql-test/t/archive.test:
Add test case for Bug#45377.
storage/archive/azio.c:
Add error checking to ensure that the I/O operations are
successful.
storage/archive/ha_archive.cc:
Copy over the embedded FRM.
Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.
Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.
Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost.
The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.
Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.
client/mysqldump.c:
Pass my_free directly as its signature is compatible with the
callback type -- which wasn't the case for free_table_ent.
Interface for maria extensions.
Alternative plugin interface with additional info (maturity and string version).
CMakeLists.txt:
Maria plugin interface used.
config/ac-macros/plugins.m4:
Maria plugin interface used.
configure.in:
Maria plugin interface used.
include/mysql/plugin.h:
Maria plugin interface added.
include/mysql/plugin_auth.h.pp:
Maria plugin interface added.
plugin/auth/auth_socket.c:
Maria plugin interface added.
plugin/auth/dialog.c:
Maria plugin interface added.
plugin/daemon_example/daemon_example.cc:
Maria plugin interface added.
plugin/fulltext/plugin_example.c:
Maria plugin interface added.
sql/ha_ndbcluster.cc:
Maria plugin interface added.
sql/ha_partition.cc:
Maria plugin interface added.
sql/log.cc:
Maria plugin interface added.
sql/sql_acl.cc:
Maria plugin interface added.
sql/sql_builtin.cc.in:
Maria plugin interface used.
sql/sql_plugin.cc:
Maria plugin interface added.
sql/sql_plugin.h:
Maria plugin interface used.
sql/sql_show.cc:
Maria plugin interface added.
storage/archive/ha_archive.cc:
Maria plugin interface added.
storage/blackhole/ha_blackhole.cc:
Maria plugin interface added.
storage/csv/ha_tina.cc:
Maria plugin interface added.
storage/example/ha_example.cc:
Maria plugin interface added.
storage/federated/ha_federated.cc:
Maria plugin interface added.
storage/federatedx/ha_federatedx.cc:
Maria plugin interface added.
storage/heap/ha_heap.cc:
Maria plugin interface added.
storage/ibmdb2i/ha_ibmdb2i.cc:
Maria plugin interface added.
storage/innobase/handler/ha_innodb.cc:
Maria plugin interface added.
storage/innodb_plugin/handler/i_s.cc:
Maria plugin interface added.
storage/maria/ha_maria.cc:
Maria plugin interface added.
storage/myisam/ha_myisam.cc:
Maria plugin interface added.
storage/myisammrg/ha_myisammrg.cc:
Maria plugin interface added.
storage/pbxt/src/ha_pbxt.cc:
Maria plugin interface added.
storage/xtradb/handler/ha_innodb.cc:
Maria plugin interface added.
storage/xtradb/handler/i_s.cc:
Maria plugin interface added.
storage/xtradb/handler/i_s.h:
Maria plugin interface added.
This patch:
- Moves all definitions from the mysql_priv.h file into
header files for the component where the variable is
defined
- Creates header files if the component lacks one
- Eliminates all include directives from mysql_priv.h
- Eliminates all circular include cycles
- Rename time.cc to sql_time.cc
- Rename mysql_priv.h to sql_priv.h
There was no way to repair corrupt ARCHIVE data file,
when unrecoverable data loss is inevitable.
With this fix REPAIR ... EXTENDED attempts to restore
as much rows as possible, ignoring unrecoverable data.
Normal REPAIR is still able to repair meta-data file
only.
mysql-test/r/archive.result:
A test case for BUG#46565.
mysql-test/std_data/bug46565.ARZ:
A test case for BUG#46565.
mysql-test/std_data/bug46565.frm:
A test case for BUG#46565.
mysql-test/t/archive.test:
A test case for BUG#46565.
storage/archive/ha_archive.cc:
Allow unrecoverable data loss when extended repair
is requested.
Server crashes when accessing ARCHIVE table with missing
.ARZ file.
When opening a table, ARCHIVE didn't properly pass through
error code from lower level azopen() to higher level open()
method.
mysql-test/r/archive.result:
A test case for BUG#48757.
mysql-test/t/archive.test:
A test case for BUG#48757.
storage/archive/ha_archive.cc:
Pass through error code from azopen().
- Marked a couple of tests with --big
- Fixed xtradb/handler/ha_innodb.cc to call explain_filename()
storage/xtradb/handler/ha_innodb.cc:
Call explain_filename() to get proper names for partitioned tables
