connections. PR 60956.
start_lingering_close_nonblocking() now puts connections in defer_linger_chain
which is emptied by any worker thread (all atomically) after its usual work,
hence any possibly blocking flush and lingering close run outside the listener.
The listener may create a dedicated worker if it fills defer_linger_chain or
while it's not empty, calling push2worker with a NULL cs.
The state machine in process_socket() is slighly modified to be able to enter
with CONN_STATE_LINGER directly w/o clogging_input_filters to interfer.
New abort_socket_nonblocking() allows to reset connections when nonblocking is
required and we can't do much about the connection anymore, nor we want the
system to linger on its own after close().
Many thanks to Stefan Priebe for his heavy testing on many event's changes!
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1802875 13f79535-47bb-0310-9956-ffa450edef68
declarations of the structures and functions to access the completion
contexts reside in a header file.
This no longer holds, as all the necessary functions and structures are
located in the single .c file (child.c).
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1801749 13f79535-47bb-0310-9956-ffa450edef68
represent a queue of the completion contexts.
Starting from r1801655, the "queue" isn't really a queue, as all the
access happens with a LIFO order. So, instead of that, call it a "pool
of completion contexts", adjust names of all relevant variables and
tweak the comments.
This patch changes
- qlock to ctxpool_lock,
- qhead to ctxpool_head, and
- qwait_event to ctxpool_wait_event.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1801748 13f79535-47bb-0310-9956-ffa450edef68
instead of the global variable (shutdown_in_progress).
This change has two purposes. First of all, it makes the listener threads
which are blocked waiting for a completion context exit immediately during
shutdown. Previously, such threads would only check for exit every second.
The second reason for this change is to put the child_main() function in
charge of controlling the listeners life cycle. Previously, such relation
was circumvented by the fact that the listeners were also waiting for the
global child exit_event. With the new separate listener_shutdown_event,
only the child_main() function is responsible for shutting down the
listeners, and I think that this makes the code a bit clearer.
All the original behavior, including the special APLOG_DEBUG diagnostic
message when we fail to acquire a free completion context in 1 second,
is kept unchanged.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1801747 13f79535-47bb-0310-9956-ffa450edef68
reason to choose the LIFO processing order for completion contexts.
It would be better to keep this important information in the code, instead
of just having it in the log message.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1801710 13f79535-47bb-0310-9956-ffa450edef68
and perform shutdown in respond to SERVICE_CONTROL_PRESHUTDOWN.
The pure shutdown notification leaves a small amount of time for the service
to finish (and the allowed amount of time has been shrinking with every new
version of Windows), and handling only it increases the chance of the process
being killed by SCM, instead of gracefully shutting down. Handling the
preshutdown control code extends this period, and increases the chances of
finishing everything properly when the machine is rebooted or shut down.
(See https://msdn.microsoft.com/en-us/library/windows/desktop/ms683241)
Please note that although the preshutdown notifications are available only
starting from Windows Vista, the code is compatible with the previous versions
of Windows, since the SCM ignores unknown SERVICE_ACCEPT codes, and will
still send an ordinary SERVICE_CONTROL_SHUTDOWN under old Windows
versions.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1801659 13f79535-47bb-0310-9956-ffa450edef68
completion contexts, as that may significantly reduce the memory usage.
This simple change can have a noticeable impact on the amount of memory
consumed by the child process in various cases. Every completion context
in the queue has an associated allocator, and every allocator has it's
ap_max_mem_free memory limit which is not given back to the operating
system. Once the queue grows, it cannot shrink back, and every allocator
in each of the queued completion contexts keeps up to its max_free amount
of memory. The queue can only grow when a server has to serve multiple
concurrent connections at once.
With that in mind, consider a case with a server that doesn't encounter many
concurrent connections most of the time, but has occasional spikes when
it has to serve multiple concurrent connections. During such spikes, the
size of the completion context queue grows.
The actual difference between using LIFO and FIFO orders shows up after
such spikes, when the server is back to light load and doesn't see a lot
of concurrency. With FIFO order, every completion context in the queue
will be used in a round-robin manner, thus using *every* available allocator
one by one and ultimately claiming up to (N * ap_max_mem_free memory) from
the OS. With LIFO order, only the completion contexts that are close to
the top of the stack will be used and reused for subsequent connections.
Hence, only a small part of the allocators will be used, and this can
prevent all other allocators from unnecessarily acquiring memory from
the OS (and keeping it), and this reduces the overall memory footprint.
Please note that this change doesn't affect the worst case behavior, as
it's still (N * ap_max_mem_free memory), but tends to behave better in
practice, for the reasons described above.
Another thing worth considering is the new behavior when the OS decides
to swap out pages of the child process, for example, in a close-to-OOM
condition. Handling every new connection after the swap requires the OS
to load the memory pages for the allocator from the completion context that
is used for this connection. With FIFO order, the completion contexts are
used one by one, and this would cause page loads for every new connection.
With LIFO order, there will be almost no swapping, since the same completion
context is going to be reused for subsequent new connections.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1801655 13f79535-47bb-0310-9956-ffa450edef68
are blocked on the I/O completion port during the shutdown.
Prior to r1801635, the shutdown code required to know the amount of blocked
threads, as it has been dispatching the same amount of completion packets.
But this no longer holds, and the only reason why we maintain the
corresponding g_blocked_threads variable is because of this debug
diagnostic message.
Drop it in order to reduce complexity of the quite critical code in the
winnt_get_connection() function and to reduce the amount of global
variables.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1801640 13f79535-47bb-0310-9956-ffa450edef68
This sleep occured in a situation when:
- We don't have a free completion context in the queue
- We can't add one, as doing so would exceed the max_num_completion_contexts
limit (all worker threads are busy)
- We have exceeded a 1 second timeout while waiting for it
In this case, the Sleep() call is unnecessary, as there is no intermittent
failure that can be waited out, but rather than that, it's an ordinary
situation with all workers being busy. Presumably, calling Sleep() here
can be even considered harmful, as it affects the fairness between the
listeners that are blocked waiting for the completion context.
So, instead of calling Sleep() just check for the possible shutdown and
immediately retry acquiring a completion context. If all worker threads
are still busy, the retry will block in the same WaitForSingleObject() call,
which is fine.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1801639 13f79535-47bb-0310-9956-ffa450edef68
thread handles in batches.
Starting from r1801636, there is no difference between ending the wait with
one or multiple remaining threads. This is because we terminate the process
if at least one thread is still active when we hit a timeout.
Therefore, instead of making an effort to evenly distribute and batch the
handles with WaitForMultipleObjects(), we could just start from one end,
and wait for one thread handle at a time.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1801637 13f79535-47bb-0310-9956-ffa450edef68
hits a timeout while waiting for the worker threads to exit.
Using TerminateThread() can have dangerous consequences such as deadlocks —
say, if the the thread is terminated while holding a lock or a heap lock
in the middle of HeapAlloc(), as these locks would not be released.
Or it can corrupt the application state and cause a crash.
(See https://msdn.microsoft.com/en-us/library/windows/desktop/ms686717)
Rework the code to call TerminateProcess() in the described circumstances
and leave the cleanup to the operating system.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1801636 13f79535-47bb-0310-9956-ffa450edef68
when shutting down the worker threads.
Previously, the shutdown code was posting an amount of I/O completion
packets equal to the amount of the threads blocked on the I/O completion
port. Then it would Sleep() until all these threads "acknowledge" the
completion packets by decrementing the global amount of blocked threads.
A better way would be to send the number of IOCP_SHUTDOWN completion
packets equal to the total amount of threads and immediately proceed to
the next step. There is no need to block until the threads actually receive
the completion, as the shutdown process includes a separate step that waits
until the threads exit, and the new approach avoids an unnecessary delay.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1801635 13f79535-47bb-0310-9956-ffa450edef68
would return a proper apr_status_t instead of yielding the result via the
*timeout out variable.
This makes the calling side easier to follow by avoiding an additional
layer of if's.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1801147 13f79535-47bb-0310-9956-ffa450edef68
from the mpm_get_completion_context() function.
Currently, the only possible reasons why mpm_get_completion_context() could
fail are real errors such as being unable to WaitForSingleObject(), allocate
memory or create an event. Retrying under such circumstances doesn't make
sense, and could be as well considered harmful.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1801146 13f79535-47bb-0310-9956-ffa450edef68
filter and use an appropriate enum for it.
This makes the code in winnt_accept() a bit easier to follow. As a minor
side effect, it also fixes a small bug where the "unrecognized AcceptFilter
'%s'" log entry would always contain "none" instead of the actually
unrecognized kind of the accept filter.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1801144 13f79535-47bb-0310-9956-ffa450edef68
Discussion on-list, but any occurance of a platform-specific behavior in this
code path will alter the behavior of the core code and introduce the very
fingerprintable behavior this patch pretended to obscuficate.
Returning 404 for /CON for example may lead to a module such as mod_speling
revealing the existance of a real file named similar to /.conf, which makes
this an unwise behavior.
Further discussion of returning 404 for all CHR files encountered in the
filepath (not URI path), which currently return 403 on all platforms,
belongs on the dev list.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1799965 13f79535-47bb-0310-9956-ffa450edef68
ap_pbase64decode_strict() adds to the functionality of
ap_pbase64decode() in two ways:
- the length of the decoded buffer is returned, allowing embedded NULLs
to be retained by the caller
- the input string is strictly checked for Base64 validity, including
correct zero-padding at the end of the string
(This was originally added to the httpdunit feature/backport branch in
r1796208, then reverted in r1799376, since it's currently intended for
trunk only.)
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1799380 13f79535-47bb-0310-9956-ffa450edef68
In the aftermath of CVE-2017-7668, decouple the business logic ("is NULL
a T_HTTP_CTRL") from the postcondition ("must not go past the end of the
string"). The NULL-byte classification in the TEST_CHAR table may change
in the future.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1799375 13f79535-47bb-0310-9956-ffa450edef68
The boilerplate code for config sections conflicts with TAKE1
because of the trailing stuff to terminate the opening tag.
Change from TAKE1 to RAW_ARGS and call ap_getword_conf()
directly.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1797844 13f79535-47bb-0310-9956-ffa450edef68
It has been reported multiple times that nested
If/ElseIf/Else sections are not evaluated but
silently ignored.
This patch adds a simple recursion to the ap_if_walk
logic in order to allow arbitrary nested configs.
The overhead seems negligible compared to the actual
version of the ap_if_walk, but more expert feedback
is surely needed since this code gets called for every
HTTP request.
Tests are going to be added to t/apache/if_sections.t
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1792589 13f79535-47bb-0310-9956-ffa450edef68
A configtest isn't useful if you're just trying to get compile settings.
Move the settings dump up to just after ap_read_config(), which has
already done the minimum necessary to figure out which MPM is in use.
Even if ap_read_config() failed, still print as many compile settings as
we can. The user will see the error log entry on stderr.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1791975 13f79535-47bb-0310-9956-ffa450edef68
are configured (ListenCoresBucketsRatio > 0), consistently with the single
bucket case (default), thus fixing the leak of the corresponding socket
descriptors on graceful restart.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1789220 13f79535-47bb-0310-9956-ffa450edef68
