* server/util.c (ap_size_parameter_parse): Allow an optional
case-insensitive 'b', 'k', 'm' or 'g' suffix, bounding the
multiplication by the range of apr_off_t.
* include/httpd.h: Update doxygen accordingly.
* test/unit/util.c (size_parameter_parse_handles_suffixes): New test.
* test/modules/core/test_012_config_off_t.py: Update for suffix
support.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GitHub: PR #851
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1939295 13f79535-47bb-0310-9956-ffa450edef68
Add ap_size_parameter_parse(), a helper to parse a configuration value
as an apr_off_t, which fails on trailing characters and on overflow.
The range of the value remains up to the directive.
Use it for configuration directives which parsed such values in
different ways, some of them accepting trailing characters or doing no
checks at all: LimitRequestBody (unchanged behavior), ScriptLogLength,
ScriptLogBuffer, DavQuota, BufferSize, AuthFormSize, and the
mod_cache_disk and mod_cache_socache size and time directives.
GitHub: closes#829
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1939129 13f79535-47bb-0310-9956-ffa450edef68
"unused" - this field has been present in request_rec since r86043
(702dd42de, August 2000) but any use of it was removed in r86210
(5d50a3e27, September 2000) which predates any stable 2.0 release.
Done WITHOUT an MMN major bump and we could probably get away with
doing the same in 2.4.x as well (where adding fields to request_rec
breaks mod_wsgi).
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1938284 13f79535-47bb-0310-9956-ffa450edef68
NUL-terminated string in place, returning a pointer to the new
terminating NUL.
* include/httpd.h: Declare ap_cstr_stripws().
* server/util.c (ap_cstr_stripws): Implement it, handling the
empty-string case safely.
(ap_getline): Use it; adjust the returned length and memmove size
accordingly.
* modules/http/byterange_filter.c, modules/mappers/mod_negotiation.c,
modules/metadata/mod_cern_meta.c, modules/proxy/mod_proxy_hcheck.c,
modules/proxy/mod_proxy_http.c, modules/proxy/mod_proxy_uwsgi.c,
server/util_script.c: Replace open-coded trailing-whitespace
stripping loops with ap_cstr_stripws(). This avoids the undefined
behaviour of forming &value[strlen(value)-1], which points before
the start of the string when the value is empty.
* modules/metadata/mod_cern_meta.c, server/util_script.c: Use
apr_pstrmemdup() with the returned end pointer rather than
apr_pstrdup().
* include/ap_mmn.h: Bump MMN minor.
* test/unit/util.c: Add ap_cstr_stripws() unit tests.
* test/modules/proxy/test_03_response.py,
test/modules/proxy/test_05_uwsgi.py: Add tests for empty backend
response header values.
Submitted by: Robert McConnell <robert mcc0nnell.org>, jorton
GitHub: closes#749
Assisted-By: Claude Opus 4.8 <noreply@anthropic.com>
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1938194 13f79535-47bb-0310-9956-ffa450edef68
* include/httpd.h: Declare ap_memeq_timingsafe(),
ap_streq_timingsafe(), ap_strneq_timingsafe().
* server/util.c: Implement, wrapping apr_*_timingsafe() if
APR >= 1.8, with a fallback to copied-in versions.
* modules/aaa/mod_auth_digest.c: Replace apr_crypto_equals()
with ap_memeq_timingsafe(). Remove apr_crypto.h include.
* modules/session/mod_session_crypto.c: Replace local
ap_crypto_equals() with ap_memeq_timingsafe(). Remove
the local implementation and macro alias.
Github: closes#638
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1933389 13f79535-47bb-0310-9956-ffa450edef68
%{m} prints the timestamp in millisecond-resolution.
* include/util_time.h:
Define new AP_CTIME_OPTION_MSEC option for printing time in milliseconds
format.
* server/util_time.c (ap_recent_ctime_ex):
Handle AP_CTIME_OPTION_MSEC to print time in a millisecond format.
* server/log.c (log_ctime):
Recognize the m time option in both fast-path and composite %{...}t formats.
Submitted by: Luboš Uhliarik <luhliari redhat.com>
Github: closes#597
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1931452 13f79535-47bb-0310-9956-ffa450edef68
* include/ap_listen.h:
- Add prototype for include/ap_listen.heraccept
- Wire in new directive ListenTCPDeferAccept
* include/mpm_common.h:
Define the previous static value as default value via DEFAULT_TCP_DEFER_ACCEPT
* server/listen.c:
- Add static int ap_listentcpdeferaccept
- ap_apply_accept_filter: Use value of ap_listenbacklog for setting TCP_DEFER_ACCEPT
- ap_listen_pre_config: Set default value
- Add ap_set_listentcpdeferaccept
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1927885 13f79535-47bb-0310-9956-ffa450edef68
Multipath TCP (MPTCP), standardized in RFC8684 [1],
is a TCP extension that enables a TCP connection to
use different paths.
Multipath TCP has been used for several use cases.
On smartphones, MPTCP enables seamless handovers between
cellular and Wi-Fi networks while preserving established
connections. This use-case is what pushed Apple to use
MPTCP since 2013 in multiple applications [2]. On dual-stack
hosts, Multipath TCP enables the TCP connection to
automatically use the best performing path, either IPv4
or IPv6. If one path fails, MPTCP automatically uses
the other path.
To benefit from MPTCP, both the client and the server
have to support it. Multipath TCP is a backward-compatible
TCP extension that is enabled by default on recent
Linux distributions (Debian, Ubuntu, Redhat, ...). Multipath
TCP is included in the Linux kernel since version 5.6 [3].
To use it on Linux, an application must explicitly enable
it when creating the socket. No need to change anything
else in the application.
Adding the possibility to create MPTCP sockets would thus
be a really fine addition to httpd, by allowing clients
to make use of their different interfaces.
This patch introduces the possibilty to connect to backend
servers using MPTCP. Note however that these changes are
only available on Linux, as IPPROTO_MPTCP is Linux specific
for the time being.
For proxies, we can connect using MPTCP by passing the
\"multipathtcp\" parameter:
ProxyPass \"/example\" \"http://backend.example.com\" multipathtcp=On
We then store this information in the worker and create sockets
appropriately according to this value.
Link: https://www.rfc-editor.org/rfc/rfc8684.html [1]
Link: https://www.tessares.net/apples-mptcp-story-so-far/ [2]
Link: https://www.mptcp.dev [3]
Add Multipath TCP (MPTCP) support (Core)
Multipath TCP (MPTCP), standardized in RFC8684 [1],
is a TCP extension that enables a TCP connection to
use different paths.
Multipath TCP has been used for several use cases.
On smartphones, MPTCP enables seamless handovers between
cellular and Wi-Fi networks while preserving established
connections. This use-case is what pushed Apple to use
MPTCP since 2013 in multiple applications [2]. On dual-stack
hosts, Multipath TCP enables the TCP connection to
automatically use the best performing path, either IPv4
or IPv6. If one path fails, MPTCP automatically uses
the other path.
To benefit from MPTCP, both the client and the server
have to support it. Multipath TCP is a backward-compatible
TCP extension that is enabled by default on recent
Linux distributions (Debian, Ubuntu, Redhat, ...). Multipath
TCP is included in the Linux kernel since version 5.6 [3].
To use it on Linux, an application must explicitly enable
it when creating the socket. No need to change anything
else in the application.
Adding the possibility to create MPTCP sockets would thus
be a really fine addition to httpd, by allowing clients
to make use of their different interfaces.
This patch introduces the possibility to listen with MPTCP
sockets. Note however that these changes are only available
on Linux, as IPPROTO_MPTCP is Linux specific for the time being.
To do so, we extended the Listen directive to include
a \"multipathtcp\" option, allowing to create MPTCP sockets
instead of regular TCP ones:
Listen 80 options=multipathtcp
We then store this information in flags for the listen directive
and create sockets appropriately according to this value.
Link: https://www.rfc-editor.org/rfc/rfc8684.html [1]
Link: https://www.tessares.net/apples-mptcp-story-so-far/ [2]
Link: https://www.mptcp.dev [3]
Submitted by: Aperence <anthony.doeraene hotmail.com>
Github: closes#476
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1920586 13f79535-47bb-0310-9956-ffa450edef68
With "ProxyPassMatch ^/([^/]+)/(.*)$ https://$1/$2", ap_proxy_get_worker_ex()
should not consider the length of scheme://host part of the given URL because
of the globbing match on the host part.
Fix it by setting worker->s>is_host_matchable when creating a worker with host
substitution and avoiding the min_match check in worker_matches() in this case.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1919617 13f79535-47bb-0310-9956-ffa450edef68
* modules/proxy/proxy_util.c:
Export ap_proxy_fixup_uds_filename() from fix_uds_filename.
Call it from ap_proxy_pre_request() even for rewritten balancer workers.
* modules/proxy/mod_proxy.h:
Declare ap_proxy_fixup_uds_filename()
* modules/proxy/mod_proxy.c:
Fixup UDS filename from r->handler in proxy_handler().
* include/ap_mmn.h:
Bump MMN minor for ap_proxy_fixup_uds_filename()
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1918626 13f79535-47bb-0310-9956-ffa450edef68
Per discussion on PR #449, have a separate state for returning the connection
to the MPM to wait for an IO (namely CONN_STATE_ASYNC_WAITIO), rather than
(ab)using CONN_STATE_PROCESSING.
This removes the need for AGAIN added in r1918257 (for now), and AP_MPMQ_CAN_AGAIN
is renamed to AP_MPMQ_CAN_WAITIO.
This is also the state that mod_status accounts for, so rename ->processing
to ->wait_io in process_score (shows as "wait-io" in mod_status and mod_lua).
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1918482 13f79535-47bb-0310-9956-ffa450edef68
Before r1918022, returning OK with CONN_STATE_PROCESSING to mpm_event was
handled like/by CONN_STATE_LINGER "to not break old or third-party modules
which might return OK w/o touching the state and expect lingering close,
like with worker or prefork MPMs".
So we need a new return code to be allowed to apply the new POLLIN/POLLOUT
behaviour for CONN_STATE_PROCESSING, thus revive AGAIN as introduced by
Graham some times ago for a nonblocking WIP (moved to a branch/PR since then).
MPM event will advertise its ability to handle CONN_STATE_PROCESSING + AGAIN
with AP_MPMQ_CAN_AGAIN, and mod_http2 can use that to know how to return to
the MPM as expected. When !AP_MPMQ_CAN_AGAIN modules/mod_http2 can still use
CONN_STATE_WRITE_COMPLETION + CONN_SENSE_WANT_READ + c->clogging_input_filters
which will work in mpm_even-2.4.x still.
* include/ap_mmn.h:
Bump MMN minor for AP_MPMQ_CAN_AGAIN and AGAIN.
* include/ap_mpm.h:
Define AP_MPMQ_CAN_AGAIN.
* include/httpd.h:
Define AGAIN.
* modules/http2/h2.h:
No need for H2_USE_STATE_PROCESSING anymore with AP_MPMQ_CAN_AGAIN.
* modules/http2/h2_c1.c:
For !keepalive case return to the MPM using CONN_STATE_PROCESSING + AGAIN
or CONN_STATE_WRITE_COMPLETION + c->clogging_input_filters depending on
AP_MPMQ_CAN_AGAIN only.
* modules/http2/h2_session.c:
Can return to the MPM for h2_send_flow_blocked() provided it's async only.
* server/mpm/event/event.c:
Rework process_socket()'s CONN_STATE_PROCESSING to handle AGAIN and preserve
compatibility. Have a lingering_close label to goto there faster when
process_lingering_close() is to be called. Improve relevant comments.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1918257 13f79535-47bb-0310-9956-ffa450edef68
As a follow up to r1918022 which handled the new CONN_STATE_PROCESS(ing) and
existing CONN_STATE_WRITE_COMPLETION in the same async queue, let's now have
two separates ones which allows more relevant async accounting in mod_status.
Rename CONN_STATE_PROCESS to CONN_STATE_PROCESSING as it's how it will be
called in mod_status.
* include/ap_mmn.h:
MMN minor bump for process_score->processing counter.
* include/httpd.h:
Rename CONN_STATE_PROCESS to CONN_STATE_PROCESSING.
* include/scoreboard.h:
Add process_score->processing field.
* include/httpd.h, modules/http/http_core.c, modules/http2/h2_c1.c,
server/mpm/event/event.c, server/mpm/motorz/motorz.c,
server/mpm/simple/simple_io.c:
Rename CONN_STATE_PROCESS to CONN_STATE_PROCESSING.
* server/mpm/event/event.c:
Restore write_completion_q to handle connections in CONN_STATE_WRITE_COMPLETION.
Use processing_q (renamed from process_q) solely for CONN_STATE_PROCESSING.
Update process_score->processing according to the length of processing_q.
* modules/generators/mod_status.c:
Show the value of process_score->processing in the stats.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1918098 13f79535-47bb-0310-9956-ffa450edef68
* include/httpd.h:
Rename CONN_STATE_CHECK_REQUEST_LINE_READABLE to CONN_STATE_KEEPALIVE
and CONN_STATE_READ_REQUEST_LINE to CONN_STATE_PROCESS, keeping the
old enums as aliases. Rework comments about each state.
* server/mpm/event/event.c:
Use the new states names.
Let the process_connection hooks return CONN_STATE_PROCESS for mpm_event
to POLLIN or POLLOUT depending on c->cs->sense being CONN_SENSE_WANT_READ
or CONN_SENSE_WANT_WRITE respectively.
Remove (ab)use of CONN_STATE_WRITE_COMPLETION with CONN_SENSE_WANT_READ to
mean poll() for read (and the need for the obscure c->clogging_input_filters
to make it work as expected). This is what CONN_STATE_PROCESS is for now.
Update the comment about the states that can be returned by process_connection
hooks (and their usage).
Use the same queue (process_q renamed from write_completion_q) for polling
connections in both CONN_STATE_PROCESS and CONN_STATE_WRITE_COMPLETION
states since they both use the same (server_rec's) Timeout. This implies
that both states are accounted as "write-completion" in mod_status for now.
* server/mpm/motorz/motorz.c, server/mpm/simple/simple_io.c, modules/http/http_core.c:
Use the new states names (only).
* include/scoreboard.h:
Change comment about process_score->write_completion to note that the
counter refers to CONN_STATE_PROCESS connections returned to the MPM
too.
* modules/http2/h2_c1.c:
Return the c1 connection with the CONN_STATE_PROCESS state rather than
CONN_STATE_WRITE_COMPLETION when waiting for a window update (i.e. ask
the MPM to poll for read directly). This avoids the transition to
CONN_STATE_KEEPALIVE which could kill the connection under high load.
Github: closes#448
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1918022 13f79535-47bb-0310-9956-ffa450edef68
* include/ap_mmn.h:
Bump MMN minor.
* include/ap_regex.h:
Declare ap_regexec_ex().
* server/util_pcre.c(ap_regexec, ap_regexec_len, ap_regexec_ex):
Reuse existing ap_regexec_len() code to implement ap_regexec_ex() where the
offset is given instead of zero, then implement ap_regexec{,len}() in terms
of ap_regexec_ex().
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1915267 13f79535-47bb-0310-9956-ffa450edef68
The latter requires a pool and returns a non constant string although it may
return worker shared data.
By computing the worker "UDS" name at init time we can return a constant name
in any case with no need for a pool, that's the new ap_proxy_worker_get_name().
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1912461 13f79535-47bb-0310-9956-ffa450edef68
Define a new proxy_address struct holding the current/latest sockaddr in use
by each proxy worker and conn. Since backend addresses can be updated when
their TTL expires and while connections are being processed, each address is
refcounted and freed only when the last worker (or conn) using it grabs the
new one.
The lifetime of the addresses is handled at a single place by the new
ap_proxy_determine_address() function. It guarantees to bind the current/latest
backend address to the passed in conn (or do nothing if it's up to date already).
The function is called indirectly by ap_proxy_determine_connection() for the
proxy modules that use it, or directly by mod_proxy_ftp and mod_proxy_hcheck.
It also is called eventually by ap_proxy_connect_backend() when connect()ing all
the current addresses fails, to check (PROXY_DETERMINE_ADDRESS_CHECK) if some
new addrs are available.
This commit is also a rework of the lifetime of conn->addr, conn->hostname
and conn->forward, using the conn->uds_pool and conn->fwd_pool for the cases
where the backend is connected through a UDS socket and a remote CONNECT proxy
respectively.
* include/ap_mmn.h:
Minor bump for new function/fields.
* modules/proxy/mod_proxy.h (struct proxy_address,
ap_proxy_determine_addresss()):
Declare ap_proxy_determine_addresss() and opaque struct proxy_address,
new fields to structs proxy_conn_rec/proxy_worker_shared/proxy_worker.
* modules/proxy/mod_proxy.c (set_worker_param):
Parse/set the new worker->address_ttl parameter.
* modules/proxy/proxy_util.c (proxy_util_register_hooks(),
ap_proxy_initialize_worker(),
ap_proxy_connection_reusable(),
ap_proxyerror(), proxyerror_core(),
init_conn_pool(), make_conn_subpool(),
connection_make(), connection_cleanup(),
connection_constructor()):
Initialize *proxy_start_time in proxy_util_register_hooks() as the epoch
from which expiration times are relative (i.e. seconds stored in an uint32_t
for atomic changes).
Make sure worker->s->is_address_reusable and worker->s->disablereuse are
consistant in ap_proxy_initialize_worker(), thus no need to check for both
in ap_proxy_connection_reusable().
New proxyerror_core() helper taking an apr_status_t to log, wrap in
ap_proxyerror().
New make_conn_subpool() to create worker->cp->{pool,dns} with their own
allocator.
New connection_make() helper to factorize code in connection_cleanup() and
connection_constructor().
* modules/proxy/proxy_util.c (proxy_address_inc(), proxy_address_dec(),
proxy_address_cleanup(), proxy_address_set_expired(),
worker_address_get(), worker_address_set(),
worker_address_resolve(), proxy_addrs_equal(),
ap_proxy_determine_address(),
ap_proxy_determine_connection(),
ap_proxy_connect_backend()):
Implement ap_proxy_determine_address() using the above helpers for atomic changes,
and call it from ap_proxy_determine_connection() and ap_proxy_connect_backend().
* modules/proxy/mod_proxy_ftp.c (proxy_ftp_handler):
Use ap_proxy_determine_address() and use the returned backend->addr.
* modules/proxy/mod_proxy_hcheck.c (hc_determine_connection, hc_get_backend,
hc_init_worker, hc_watchdog_callback):
Use ap_proxy_determine_address() in hc_determine_connection() and call the
latter from hc_get_backend(), replace hc_init_worker() by hc_init_baton()
which now calls hc_get_hcworker() and hc_get_backend() to resolve the first
address at init time.
* modules/proxy/mod_proxy_http.c (proxy_http_handler):
Use backend->addr and ->hostname instead of worker->cp->addr and
worker->s->hostname_ex respectively.
* modules/proxy/mod_proxy_ajp.c (ap_proxy_ajp_request):
Use backend->addr and ->hostname instead of worker->cp->addr and
worker->s->hostname_ex respectively.
Closes#367
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1912459 13f79535-47bb-0310-9956-ffa450edef68
described in RFC 8441. A new directive 'H2WebSockets on|off' has been
added. The feature is by default not enabled.
As also discussed in the manual, this feature should work for setups
using "ProxyPass backend-url upgrade=websocket" without further changes.
Special server modules for WebSockets will have to be adapted,
most likely, as the handling if IO events is different with HTTP/2.
HTTP/2 WebSockets are supported on platforms with native pipes. This
excludes Windows.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1910507 13f79535-47bb-0310-9956-ffa450edef68
ap_die() to judge if it can send out a response. Bump mmn.
Enable test cases that check errors during response body to
appear as error on client side.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1910161 13f79535-47bb-0310-9956-ffa450edef68
