Remove the too restrictive bugfix for bug#67088.
FIFO can be used for general/slow logs, but lseek() and fsync() on
FIFO fail. And open() needs to be non-blocking, in case the other
end isn't reading.
When the binlog was rotated due to @@max_binlog_size, the values of the
binlog_shapshot_file and binlog_snapshot_position were inconsistent in case of
non-transactional DML. The position was refering to the old file, while the
filename was of the new file after rotation. This patch makes them consistent
by making sure the position is also refering to the new file.
GOES AWAY, MYSQL QUITS WORKING.
Analysis:
-----------------
Issue in this bug and in bug 11907705 is, the socket file or
fifo file is set for general log at command line while starting
the server. But currently, only regular file can be set for the
general log. Instead of reporting any error, the provided files
are opened for writing and continued. Because of this issues
mentioned in the bug reports are seen.
As mentioned, only when any non-regular file is set for general
log at command line while starting the server, these issues are
seen. If general log file is set to non-regular file from CLI
using system variable general_log_file then error is reported.
These issues can also be faced with slow query log file, if it is
set to non-regular file.
Fix:
-----------------
Currently while starting the server if we fail to open log file
then we report an error, disable logging to file and continue.
To fix issue reported code is modified to check whether file
is regular file or not before opening it. If file is not a
regular file then error is logged to error log and logging to
file is disabled.
server initialization
ER() macro was used during server initialization. It refers to
current_thd, which is not available that early.
Print error to error log in "lc-messages" locale.
Avoid duplicate error message during server initialization.
SHOW PROCESSLIST, SHOW BINLOGS
Problem: A deadlock was occurring when 4 threads were
involved in acquiring locks in the following way
Thread 1: Dump thread ( Slave is reconnecting, so on
Master, a new dump thread is trying kill
zombie dump threads. It acquired thread's
LOCK_thd_data and it is about to acquire
mysys_var->current_mutex ( which LOCK_log)
Thread 2: Application thread is executing show binlogs and
acquired LOCK_log and it is about to acquire
LOCK_index.
Thread 3: Application thread is executing Purge binary logs
and acquired LOCK_index and it is about to
acquire LOCK_thread_count.
Thread 4: Application thread is executing show processlist
and acquired LOCK_thread_count and it is
about to acquire zombie dump thread's
LOCK_thd_data.
Deadlock Cycle:
Thread 1 -> Thread 2 -> Thread 3-> Thread 4 ->Thread 1
The same above deadlock was observed even when thread 4 is
executing 'SELECT * FROM information_schema.processlist' command and
acquired LOCK_thread_count and it is about to acquire zombie
dump thread's LOCK_thd_data.
Analysis:
There are four locks involved in the deadlock. LOCK_log,
LOCK_thread_count, LOCK_index and LOCK_thd_data.
LOCK_log, LOCK_thread_count, LOCK_index are global mutexes
where as LOCK_thd_data is local to a thread.
We can divide these four locks in two groups.
Group 1 consists of LOCK_log and LOCK_index and the order
should be LOCK_log followed by LOCK_index.
Group 2 consists of other two mutexes
LOCK_thread_count, LOCK_thd_data and the order should
be LOCK_thread_count followed by LOCK_thd_data.
Unfortunately, there is no specific predefined lock order defined
to follow in the MySQL system when it comes to locks across these
two groups. In the above problematic example,
there is no problem in the way we are acquiring the locks
if you see each thread individually.
But If you combine all 4 threads, they end up in a deadlock.
Fix:
Since everything seems to be fine in the way threads are taking locks,
In this patch We are changing the duration of the locks in Thread 4
to break the deadlock. i.e., before the patch, Thread 4
('show processlist' command) mysqld_list_processes()
function acquires LOCK_thread_count for the complete duration
of the function and it also acquires/releases
each thread's LOCK_thd_data.
LOCK_thread_count is used to protect addition and
deletion of threads in global threads list. While show
process list is looping through all the existing threads,
it will be a problem if a thread is exited but there is no problem
if a new thread is added to the system. Hence a new mutex is
introduced "LOCK_thd_remove" which will protect deletion
of a thread from global threads list. All threads which are
getting exited should acquire LOCK_thd_remove
followed by LOCK_thread_count. (It should take LOCK_thread_count
also because other places of the code still thinks that exit thread
is protected with LOCK_thread_count. In this fix, we are changing
only 'show process list' query logic )
(Eg: unlink_thd logic will be protected with
LOCK_thd_remove).
Logic of mysqld_list_processes(or file_schema_processlist)
will now be protected with 'LOCK_thd_remove' instead of
'LOCK_thread_count'.
Now the new locking order after this patch is:
LOCK_thd_remove -> LOCK_thd_data -> LOCK_log ->
LOCK_index -> LOCK_thread_count
See also MySQL Bug #39750 and similar ones.
Fix my_delete() on Windows, to safely remvoe files on Windows, including files that are opened by another threads in the same process, antiviruses and backup applications. If file to be deleted is also opened by another thread, the file is renamed to unique name prior to deletion - this makes it possible to create file with the same name right after deletion.
With this patch my_delete_allow_opened() becomes obsolete and is replaced with my_delete().
This patch is rework of the patch http://lists.mysql.com/commits/59327 for MySQL bug#39750.
Fix some problems in the TC_LOG_MMAP commit processing, which could
lead to assertions in some cases.
Problems are mostly reproducible in MariaDB 10.0 with asynchroneous
commit checkpoints, but most of the problems were present in earlier
versions also.
QUOTING IN REPLICATION
Problem: Misquoting or unquoted identifiers may lead to
incorrect statements to be logged to the binary log.
Fix: we use specialized functions to append quoted identifiers in
the statements generated by the server.
Problem:
=======
trx_data->empty() assert happens at `binlog_close_connection'
Analysis:
========
trx_data->empty() function checks for no pending events
and the transaction cache to be empty.This function returns
"true" if no pending events are present and cache is empty.
Otherwise it returns false. `binlog_close_connection' call
expects the above function to return true. But if the
return value is false then assert is raised.
This bug was reproducible in a diskfull scenario. In this
disk full scenario try to do an insert operation so that
a new pending event is created and flushing this pending
event fails. Due to this failure the server goes down
and invokes `binlog_close_connection' for clean closure.
Since the pending event still remains the assert is caused.
This assert is caused only in non transactional databases.
Fix:
===
In a disk full scenario when the insertion fails the
transaction is rolled back and `binlog_end_trans`
is called to flush the pending events. But flush operation
fails as the disk is full and the function simply returns
`1' without taking any action to delete the pending event.
This leaves the event to remain till the closure of
connection. `delete pending' statement has been added to
do the required clean up action.
sql/log.cc:
Added "delete pending" statement to clean pending event
When we append data to the binlog file, we use fdatasync() to ensure
the data gets to disk so that crash recovery can work.
Unfortunately there seems to be a bug in ext3/ext4 on linux, so that
fdatasync() does not correctly sync all data when the size of a file
is increased. This causes crash recovery to not work correctly (it
loses transactions from the binlog).
As a work-around, use fsync() for the binlog, not fdatasync(). Since
we are increasing the file size, (correct) fdatasync() will most
likely not be faster than fsync() on any file system, and fsync()
does work correctly on ext3/ext4. This avoids the need to try to
detect if we are running on buggy ext3/ext4.
two tests still fail:
main.innodb_icp and main.range_vs_index_merge_innodb
call records_in_range() with both range ends being open
(which triggers an assert)
