Remove support for Request-Range header sent by Navigator 2-3 and
MSIE 3
* Follow up to r1918814: Strings are from configuration and thus trusted
Submitted by: sf, rpluem
Reviewed by: rpluem, covener, jorton
Github: closes#475
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1920981 13f79535-47bb-0310-9956-ffa450edef68
*) mod_http2: sync with module's github.
- on newer HTTPD versions, return connection monitoring
to the event MPM when block on client updates.
2.4.x versions still treat connections in the event
MPM as KeepAlive and purge them on load in the middle
of response processing.
- spelling fixes
- support for yield calls in c2 "network" filter
mpm_event,core: Handle async POLLIN/POLLOUT in CONN_STATE_PROCESS state.
* include/httpd.h:
Rename CONN_STATE_CHECK_REQUEST_LINE_READABLE to CONN_STATE_KEEPALIVE
and CONN_STATE_READ_REQUEST_LINE to CONN_STATE_PROCESS, keeping the
old enums as aliases. Rework comments about each state.
* server/mpm/event/event.c:
Use the new states names.
Let the process_connection hooks return CONN_STATE_PROCESS for mpm_event
to POLLIN or POLLOUT depending on c->cs->sense being CONN_SENSE_WANT_READ
or CONN_SENSE_WANT_WRITE respectively.
Remove (ab)use of CONN_STATE_WRITE_COMPLETION with CONN_SENSE_WANT_READ to
mean poll() for read (and the need for the obscure c->clogging_input_filters
to make it work as expected). This is what CONN_STATE_PROCESS is for now.
Update the comment about the states that can be returned by process_connection
hooks (and their usage).
Use the same queue (process_q renamed from write_completion_q) for polling
connections in both CONN_STATE_PROCESS and CONN_STATE_WRITE_COMPLETION
states since they both use the same (server_rec's) Timeout. This implies
that both states are accounted as "write-completion" in mod_status for now.
* server/mpm/motorz/motorz.c, server/mpm/simple/simple_io.c, modules/http/http_core.c:
Use the new states names (only).
* include/scoreboard.h:
Change comment about process_score->write_completion to note that the
counter refers to CONN_STATE_PROCESS connections returned to the MPM
too.
* modules/http2/h2_c1.c:
Return the c1 connection with the CONN_STATE_PROCESS state rather than
CONN_STATE_WRITE_COMPLETION when waiting for a window update (i.e. ask
the MPM to poll for read directly). This avoids the transition to
CONN_STATE_KEEPALIVE which could kill the connection under high load.
Github: closes#448
Follow up to r1918022: MMN minor bump and checks for the new conn_state_e aliases' usability.
mpm_event: Don't spam with "Stopping process due to MaxConnectionsPerChild"
When MaxConnectionsPerChild is reached there may be some connections to process
still and the listener should stop writing this at every loop. Logging once
is enough.
* server/mpm/event/event.c(check_infinite_requests): Raise conns_this_child
unconditionally.
mpm_event, mod_status: Separate processing and write completion queues.
As a follow up to r1918022 which handled the new CONN_STATE_PROCESS(ing) and
existing CONN_STATE_WRITE_COMPLETION in the same async queue, let's now have
two separates ones which allows more relevant async accounting in mod_status.
Rename CONN_STATE_PROCESS to CONN_STATE_PROCESSING as it's how it will be
called in mod_status.
* include/ap_mmn.h:
MMN minor bump for process_score->processing counter.
* include/httpd.h:
Rename CONN_STATE_PROCESS to CONN_STATE_PROCESSING.
* include/scoreboard.h:
Add process_score->processing field.
* include/httpd.h, modules/http/http_core.c, modules/http2/h2_c1.c,
server/mpm/event/event.c, server/mpm/motorz/motorz.c,
server/mpm/simple/simple_io.c:
Rename CONN_STATE_PROCESS to CONN_STATE_PROCESSING.
* server/mpm/event/event.c:
Restore write_completion_q to handle connections in CONN_STATE_WRITE_COMPLETION.
Use processing_q (renamed from process_q) solely for CONN_STATE_PROCESSING.
Update process_score->processing according to the length of processing_q.
* modules/generators/mod_status.c:
Show the value of process_score->processing in the stats.
Follow up to r1918098 (and r1918022): Push missing changes.
mpm_event,mod_http2: Keep compatibility with CONN_STATE_PROCESSING + OK
Before r1918022, returning OK with CONN_STATE_PROCESSING to mpm_event was
handled like/by CONN_STATE_LINGER "to not break old or third-party modules
which might return OK w/o touching the state and expect lingering close,
like with worker or prefork MPMs".
So we need a new return code to be allowed to apply the new POLLIN/POLLOUT
behaviour for CONN_STATE_PROCESSING, thus revive AGAIN as introduced by
Graham some times ago for a nonblocking WIP (moved to a branch/PR since then).
MPM event will advertise its ability to handle CONN_STATE_PROCESSING + AGAIN
with AP_MPMQ_CAN_AGAIN, and mod_http2 can use that to know how to return to
the MPM as expected. When !AP_MPMQ_CAN_AGAIN modules/mod_http2 can still use
CONN_STATE_WRITE_COMPLETION + CONN_SENSE_WANT_READ + c->clogging_input_filters
which will work in mpm_even-2.4.x still.
* include/ap_mmn.h:
Bump MMN minor for AP_MPMQ_CAN_AGAIN and AGAIN.
* include/ap_mpm.h:
Define AP_MPMQ_CAN_AGAIN.
* include/httpd.h:
Define AGAIN.
* modules/http2/h2.h:
No need for H2_USE_STATE_PROCESSING anymore with AP_MPMQ_CAN_AGAIN.
* modules/http2/h2_c1.c:
For !keepalive case return to the MPM using CONN_STATE_PROCESSING + AGAIN
or CONN_STATE_WRITE_COMPLETION + c->clogging_input_filters depending on
AP_MPMQ_CAN_AGAIN only.
* modules/http2/h2_session.c:
Can return to the MPM for h2_send_flow_blocked() provided it's async only.
* server/mpm/event/event.c:
Rework process_socket()'s CONN_STATE_PROCESSING to handle AGAIN and preserve
compatibility. Have a lingering_close label to goto there faster when
process_lingering_close() is to be called. Improve relevant comments.
mpm_event,mod_http2,mod_status: Follow up to r1918257: CONN_STATE_ASYNC_WAITIO.
Per discussion on PR #449, have a separate state for returning the connection
to the MPM to wait for an IO (namely CONN_STATE_ASYNC_WAITIO), rather than
(ab)using CONN_STATE_PROCESSING.
This removes the need for AGAIN added in r1918257 (for now), and AP_MPMQ_CAN_AGAIN
is renamed to AP_MPMQ_CAN_WAITIO.
This is also the state that mod_status accounts for, so rename ->processing
to ->wait_io in process_score (shows as "wait-io" in mod_status and mod_lua).
mpm_event: Follow up to r1918482: CONN_STATE_ASYNC_WAITIO > CONN_STATE_LINGER.
mpm_event: Follow up to r1918482: CONN_STATE_LINGER* are not the last anymore.
Since CONN_STATE_ASYNC_WAITIO, we cannot check for < or >= CONN_STATE_LINGER
anymore to determine if in an lingering close state, so let's add a new
CONN_STATE_IS_LINGERING_CLOSE() macro for this and use it in mpm_event.
The test for state == CONN_STATE_LINGER in process_lingering_close() is a
bit weak too in order to call ap_start_lingering_close() the first time only,
so have a conn_state->linger_started flag instead.
mod_status: Follow up to r1918482: Bump colspan for the new wait-io colomn
mod_status: "Threads" span three colomns (busy, graceful, idle), not two.
Submitted by: icing, ylavic, ylavic, ylavic, ylavic, ylavic, ylavic, ylavic, ylavic, ylavic, ylavic, ylavic
Reviewed by: ylavic, icing, gbechis
Github: closes#449
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1919548 13f79535-47bb-0310-9956-ffa450edef68
* In the reverse proxy case r->filename might contain a query string if
the nocanon option was used with ProxyPass.
If this is the case cut off the query string as the last parameter in
this query string might end up on an extension we take care about, but
we only want to match against path components not against query
parameters.
* Add CHANGES entry for r1907505 [skip ci]
Reviewed By: rpluem, ylavic, covener
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1908538 13f79535-47bb-0310-9956-ffa450edef68
* Close the connection in case an EOC bucket was seen
* In case we see an EOC bucket and there was an error bucket before, use its
status as status for the request. This should ensure proper status logging
in the access log.
* We need to set r->status on each call after we noticed an EOC as
data bucket generators like ap_die might have changed the status
code. But we know better in this case and insist on the status
code that we have seen in the error bucket.
* Keep track of the number of keepalives we processed on this connection.
* Report a broken backend in case reading the response line failed on the
first request on this connection otherwise we assume we have just run
into a keepalive race and the backend is still healthy.
* Add Changelog for r1899451, r1899454, r1899562, r1899564, r1899584
Submitted by: rpluem
Reviewed by: jim
Github: closes#314
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1900755 13f79535-47bb-0310-9956-ffa450edef68
If ap_discard_request_body() sets AP_CONN_CLOSE by itself it simplifies and
allows to consolidate end_output_stream() and error_output_stream().
Merge r1898683 from trunk.
Submitted by: ylavic, rpluem
Reviewed by: ylavic, rpluem, covener
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1898692 13f79535-47bb-0310-9956-ffa450edef68
*) http: Enforce that fully qualified uri-paths not to be forward-proxied
have an http(s) scheme, and that the ones to be forward proxied have a
hostname, per HTTP specifications.
trunk patch: http://svn.apache.org/r1895914http://svn.apache.org/r1895921
2.4.x patch: https://patch-diff.githubusercontent.com/raw/apache/httpd/pull/286.patch
backport PR: https://github.com/apache/httpd/pull/286
+1: ylavic, minfrin, gbechis
mod_proxy: Detect unix: scheme syntax errors at load time.
* modules/proxy/mod_proxy.c(add_pass, add_member, set_proxy_param,
proxysection):
Check return value of ap_proxy_de_socketfy().
* modules/proxy/proxy_util.c(ap_proxy_get_worker_ex):
Check return value of ap_proxy_de_socketfy().
http: Enforce that fully qualified uri-paths not to be forward-proxied
have an http(s) scheme, and that the ones to be forward proxied have a
hostname, per HTTP specifications.
The early checks avoid failing the request later on and thus save cycles
for those invalid cases.
Submitted by: ylavic
Reviewed by: ylavic, minfrin, gbechis
Closes#286
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1895955 13f79535-47bb-0310-9956-ffa450edef68
*) mod_http: Correctly sent a 100 Continue status code when sending an interim
response as result of an Expect: 100-Continue in the request and not the
current status code of the request. PR 65725 [Ruediger Pluem]
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1895897 13f79535-47bb-0310-9956-ffa450edef68
*) Easy patches: synch 2.4.x and trunk
- mod_auth_basic: Use ap_cstr_casecmp instead of strcasecmp.
- mod_ldap: log and abort locking errors.
- mod_ldap: style fix for r1831165
- mod_ldap: build break fix for r1831165
--> These 3 commits are preparation work for backporting r1856735
- mod_deflate: Avoid hard-coded "%ld" format strings in mod_deflate's logging statements
- mod_deflate: Use apr_uint64_t instead of uint64_t (follow up to r1849590)
- mod_forensic: Follow up to r1856490: missing one mod_log_forensic test_char_table case.
(r1856490 has been backported in r1870261)
- mod_rewrite: Save a few cycles.
- mod_request: Fix a comment (missing '_' in 'keep_body') and some style issues
- core: remove extra whitespace in HTTP_NOT_IMPLEMENTED
trunk patch:
https://svn.apache.org/r1817063https://svn.apache.org/r1831165https://svn.apache.org/r1831171https://svn.apache.org/r1831172https://svn.apache.org/r1849590https://svn.apache.org/r1849630https://svn.apache.org/r1856491https://svn.apache.org/r1883421https://svn.apache.org/r1889327https://svn.apache.org/r1891200
2.4.x patch: svn merge -c 1817063,1831165,1831171,1831172,1849590,1849630,1856491,1883421,1889327,1891200 ^/httpd/httpd/trunk .
+1: jailletc36, covener, jim
icing: I have trouble running the svn merge due to conflicts in CHANGES and
the infamous log tags. Do you have a patch or a PR to use for checking?
covener: on faith
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1892510 13f79535-47bb-0310-9956-ffa450edef68
* modules/http/http_core.c server/protocol.c:
Make sure that conn->keepalive is NOT reset after being set in ap_read_request()
Submitted by: jfclere
Reviewed by: jfclere, covener, ylavic
Backported by: jailletc36
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1888926 13f79535-47bb-0310-9956-ffa450edef68
"[mod_dav_fs etag handling] should really honor the FileETag setting".
- It now does.
- Add "Digest" to FileETag directive, allowing a strong ETag to be
generated using a file digest.
- Add ap_make_etag_ex() and ap_set_etag_fd() to allow full control over
ETag generation.
- Add concept of "binary notes" to request_rec, allowing packed bit flags
to be added to a request.
- First binary note - AP_REQUEST_STRONG_ETAG - allows modules to force
the ETag to a strong ETag to comply with RFC requirements, such as those
mandated by various WebDAV extensions.
Be defensive when the request is missing.
Remove tabs.
Use a dedicated constant for the base64 sha1 length.
Be defensive when calculating the digest. Make sure the offset is initialised
to zero before reading the current offset.
Seek to the start of the file when calculating the ETag.
Add log numbers to util_etag.c.
Add MMAP support to ETag generation.
Use a brigade instead of direct reads, allow APR to handle MMAP.
Bump the logno.
Submitted by: minfrin
Reviewed by: minfrin, jim, ylavic
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1887384 13f79535-47bb-0310-9956-ffa450edef68
* When ap_cstr_casecmp[n] have been introduced, in r1747469, it has
introduced many differences between 2.4.x and trunk.
Direct backporting was not that easy because:
- it was initialy introduced as ap_casecmpstr[n] with several patches
- it touched some files that were not in 2.4.x branch
Later, parts of it have already been backported.
This patch tries to clean all this.
Submitted by: many people
Reviewed by: jailletc36, minfrin, ylavic
Backported by: jailletc36
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1885697 13f79535-47bb-0310-9956-ffa450edef68
Style only
Be more consistent:
- add space between (if|while) and \(
- place of 'break ' statement
Fix cut and paste typo in error message + remove empty lines to be consistent
follow-up to r1656549.
Instead of logging a password (which is not a good practice), clarify the associated message
* Silence compiler warning
Be less tolerant when parsing the credencial for Basic authorization. Only spaces should be accepted after the authorization scheme. \t are also tolerated.
The current code accepts \v and \f as well.
The same behavior is already used in 'ap_get_basic_auth_pw()' which is mostly the same function as 'get_basic_auth()'.
Function used as 'apr_reslist_destructor' when calling 'apr_reslist_create()' should have the following prototype:
apr_status_t (*apr_reslist_destructor)(void *resource, void *params, apr_pool_t *pool);
Submitted by: jailletc36, rpluem, jailletc36, jailletc36
Reviewed by: jailletc36, minfrin, jim
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1841329 13f79535-47bb-0310-9956-ffa450edef68
mod_ratelimit: Don't interfere with "chunked" encoding.
By the time ap_http_header_filter() sends the header brigade and adds the
"CHUNK" filter, we need to garantee that the header went through all the
filters' stack, and more specifically above ap_http_chunk_filter() which
assumes that all it receives is content data.
Since rate_limit_filter() may retain the header brigade, make it run after
ap_http_chunk_filter(), just before AP_FTYPE_CONNECTION filters.
Also, ap_http_header_filter() shouldn't eat the EOS for HEAD/no-body responses.
For instance mod_ratelimit depends on it since r1835168, but any next request
filter may as well to flush and/or bail out approprietely.
This fixes the regression introduced in 2.4.34 (r1835168).
PR 62568.
Submitted by: ylavic
Reviewed by: covener, ylavic, jim
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1839497 13f79535-47bb-0310-9956-ffa450edef68
* modules/http/http_request.c (ap_process_request_after_handler,
ap_process_request): Cache and retrieve the brigade structure used
to send EOR and FLUSH between requests in c->pool userdata, to avoid
allocating a brigade structure per-request out of c->pool.
Submitted by: rpluem, jorton
Submitted by: jorton
Reviewed by: jorton, covener, jim
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1833841 13f79535-47bb-0310-9956-ffa450edef68
Fix timeout logging in ap_process_request().
We can't use 'r' after ap_process_request_after_handler(), the core output
filter might have cleaned up its deferred bucket brigade on error, including
the EOR bucket.
Reported by: steffenal
ClosesSpiderLabs/ModSecurity#1542
Follow up to r1826556: CHANGES entry.
Submitted by: ylavic
Reviewed by: ylavic, covener, rjung
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1826899 13f79535-47bb-0310-9956-ffa450edef68
core, mod_rewrite: introduce the 'redirect-keeps-vary' note
to allow proper Vary header insertion when
dealing with a RewriteRule in a directory
context.
This change is an attempt to fix a long standing problem,
brought up while working on PR 58231. Our documentation clearly
states the following:
"If a HTTP header is used in a condition this header is added
to the Vary header of the response in case the condition
evaluates to true for the request."
This is currently not true for RewriteCond/Rules working in
a directory context, since when an internal redirect happens
all the outstanding response headers get dropped.
There might be a better solution so I am looking forward to
hear more opinions and comments. My goal for a delicate change
like this one would be to affect the least amount of configurations
possible, without triggering unwanted side effects.
If the solution is good for everybody tests will be written
in the suite asap.
Submitted by: elukey
Reviewed by: elukey, icing, ylavic
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1815100 13f79535-47bb-0310-9956-ffa450edef68
Since r1753257, "HEAD" method is registered into the registry hash with the M_GET ID.
(r1757672 in 2.4.x)
We iterate over all the values of the registery, so there is no need anymore to have a special case for "HEAD" in 'make_allow()'. It has its own entry now.
With the current code, we have "HEAD" 3 times in the Allow Header field.
This is because we find M_GET 2 times in the registry hash. The first one gives "GET" and "HEAD" (as the special handling), and the second "HEAD" and "HEAD" (as the special handling).
BTW, use APR_ARRAY_PUSH instead of hand coding it, in oder to have the code more readable.
PR 61207
Add CHANGE entry
Submitted by: jailletc36
Reviewed by: jailletc36, rpluem, jim
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1799890 13f79535-47bb-0310-9956-ffa450edef68
mod_mime can read one byte past the end of a buffer when sending a
malicious Content-Type response header.
Merge 1797550 from trunk:
mod_mime: fix quoted pair scanning
Submitted By: ylavic
Reviewed By: covener, ylavic, jim
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1797653 13f79535-47bb-0310-9956-ffa450edef68
http: allow folding in check_headers(), still compliant with RFC 7230 (3.2.4).
http: follow up to r1777460.
We MUST unfold outgoing HTTP headers in any case, "message/http" is for
inner content.
Submitted by: ylavic
Reviewed by: ylavic, covener, wrowe
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1778052 13f79535-47bb-0310-9956-ffa450edef68
http_header_filter: on check_headers() failure, use AP_FILTER_ERROR and EOC
semantics to respectively warn the caller and cleanly terminate the connection
afterwards.
Suggested by: rpluem
Submitted by: ylavic
Reviewed/backported by: jim
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1775828 13f79535-47bb-0310-9956-ffa450edef68
change error handling for bad resp headers
- avoid looping between ap_die and the http filter
- remove the header that failed the check
- keep calling apr_table_do until our fn stops matching
This is still not great. We get the original body, a 500 status
code and status line.
(r1773285 + fix for first return from check_headers)
Follow up to r1773293.
When check_headers() fails, clear anything (headers and body) from original/errorneous
response before returning 500.
Follow up to r1773761: don't check_headers() more than once.
Follow up to r1773761: don't recurse on internal redirects.
Follow up to r1773761: don't recurse on ap_send_error_response() either.
Follow up to r1773761: we need to check both ap_send_error_response() and internal redirect recursions.
Follow up to r1773761: improved recursion detection.
Submitted by: covener, ylavic, ylavic, ylavic, ylavic, ylavic, ylavic
Reviewed/backported by: jim
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1773995 13f79535-47bb-0310-9956-ffa450edef68
After eliminating unusual whitespace in Unsafe mode (e.g. \f \v), we are left
with the same behavior in both of these cases. Simplify. Noted by rpluem.
Submitted by: wrowe
Reviewed/backported by: jim
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1773803 13f79535-47bb-0310-9956-ffa450edef68
Drop C-L header and message-body from HTTP 204 responses.
The C-L header can be set in a fcgi/cgi backend or in other
filters like ap_content_length_filter (with the value of 0),
meanwhile the message-body can be returned incorrectly
by any backend. The idea is to remove unnecessary bytes
from a HTTP 204 response.
PR 51350
Submitted by: elukey
Reviewed/backported by: jim
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1773801 13f79535-47bb-0310-9956-ffa450edef68
