Define a new proxy_address struct holding the current/latest sockaddr in use
by each proxy worker and conn. Since backend addresses can be updated when
their TTL expires and while connections are being processed, each address is
refcounted and freed only when the last worker (or conn) using it grabs the
new one.
The lifetime of the addresses is handled at a single place by the new
ap_proxy_determine_address() function. It guarantees to bind the current/latest
backend address to the passed in conn (or do nothing if it's up to date already).
The function is called indirectly by ap_proxy_determine_connection() for the
proxy modules that use it, or directly by mod_proxy_ftp and mod_proxy_hcheck.
It also is called eventually by ap_proxy_connect_backend() when connect()ing all
the current addresses fails, to check (PROXY_DETERMINE_ADDRESS_CHECK) if some
new addrs are available.
This commit is also a rework of the lifetime of conn->addr, conn->hostname
and conn->forward, using the conn->uds_pool and conn->fwd_pool for the cases
where the backend is connected through a UDS socket and a remote CONNECT proxy
respectively.
* include/ap_mmn.h:
Minor bump for new function/fields.
* modules/proxy/mod_proxy.h (struct proxy_address,
ap_proxy_determine_addresss()):
Declare ap_proxy_determine_addresss() and opaque struct proxy_address,
new fields to structs proxy_conn_rec/proxy_worker_shared/proxy_worker.
* modules/proxy/mod_proxy.c (set_worker_param):
Parse/set the new worker->address_ttl parameter.
* modules/proxy/proxy_util.c (proxy_util_register_hooks(),
ap_proxy_initialize_worker(),
ap_proxy_connection_reusable(),
ap_proxyerror(), proxyerror_core(),
init_conn_pool(), make_conn_subpool(),
connection_make(), connection_cleanup(),
connection_constructor()):
Initialize *proxy_start_time in proxy_util_register_hooks() as the epoch
from which expiration times are relative (i.e. seconds stored in an uint32_t
for atomic changes).
Make sure worker->s->is_address_reusable and worker->s->disablereuse are
consistant in ap_proxy_initialize_worker(), thus no need to check for both
in ap_proxy_connection_reusable().
New proxyerror_core() helper taking an apr_status_t to log, wrap in
ap_proxyerror().
New make_conn_subpool() to create worker->cp->{pool,dns} with their own
allocator.
New connection_make() helper to factorize code in connection_cleanup() and
connection_constructor().
* modules/proxy/proxy_util.c (proxy_address_inc(), proxy_address_dec(),
proxy_address_cleanup(), proxy_address_set_expired(),
worker_address_get(), worker_address_set(),
worker_address_resolve(), proxy_addrs_equal(),
ap_proxy_determine_address(),
ap_proxy_determine_connection(),
ap_proxy_connect_backend()):
Implement ap_proxy_determine_address() using the above helpers for atomic changes,
and call it from ap_proxy_determine_connection() and ap_proxy_connect_backend().
* modules/proxy/mod_proxy_ftp.c (proxy_ftp_handler):
Use ap_proxy_determine_address() and use the returned backend->addr.
* modules/proxy/mod_proxy_hcheck.c (hc_determine_connection, hc_get_backend,
hc_init_worker, hc_watchdog_callback):
Use ap_proxy_determine_address() in hc_determine_connection() and call the
latter from hc_get_backend(), replace hc_init_worker() by hc_init_baton()
which now calls hc_get_hcworker() and hc_get_backend() to resolve the first
address at init time.
* modules/proxy/mod_proxy_http.c (proxy_http_handler):
Use backend->addr and ->hostname instead of worker->cp->addr and
worker->s->hostname_ex respectively.
* modules/proxy/mod_proxy_ajp.c (ap_proxy_ajp_request):
Use backend->addr and ->hostname instead of worker->cp->addr and
worker->s->hostname_ex respectively.
Closes#367
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1912459 13f79535-47bb-0310-9956-ffa450edef68
In case that AllowEncodedSlashes is set to NoDecode do not double encode
encoded slashes in the URL sent by the reverse proxy to the backend.
* include/ap_mmn.h: Document the addition of ap_proxy_canonenc_ex to the API.
* modules/proxy/mod_proxy.h: Declare ap_proxy_canonenc_ex and define flag
values.
* modules/proxy/proxy_util.c: Implement ap_proxy_canonenc_ex by modifying
ap_proxy_canonenc accordingly and reimplement ap_proxy_canonenc to
use ap_proxy_canonenc_ex with the appropriate flag.
* modules/http2/mod_proxy_http2.c, modules/proxy/mod_proxy_*.c: Set the
correct flag based on the AllowEncodedSlashes configuration and use
ap_proxy_canonenc_ex instead of ap_proxy_canonenc.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1908341 13f79535-47bb-0310-9956-ffa450edef68
Decoding was not done for mapping=servlet only (a subset), but overlooked for
mapping=encoded. To avoid double-encoding in the canon_handler hook, use a new
"proxy-noencode" (similarly to "proxy-nocanon") entry in r->notes.
* proxy/mod_proxy.c(ap_proxy_trans_match):
Set "proxy-noencode" in r->notes for PROXYPASS_MAP_ENCODED, and return DONE
to avoid decoding in ap_process_request_internal().
* proxy/mod_proxy_http.c, proxy/mod_proxy_ajp.c, proxy/mod_proxy_wstunnel.c,
proxy/mod_proxy_fcgi.c, proxy/mod_proxy_ajp.c, http2/mod_proxy_http2.c:
Don't process the url through ap_proxy_canonenc() in canon_handler if
"proxy-noencode" is set.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1907972 13f79535-47bb-0310-9956-ffa450edef68
Let proxy_http_handler() tell ap_proxy_create_hdrbrgd() whether to add or
preserve Expect header or not, through the "proxy-100-continue" note.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1901446 13f79535-47bb-0310-9956-ffa450edef68
Stop returning 417 when mod_proxy has to forward an HTTP/1.1 request with both
"Expect: 100-continue" and "force-proxy-request-1.0" set, mod_proxy can instead
handle the 100-continue by itself before forwarding the request, like in the
"Proxy100Continue Off" case.
Note that this does not change the behaviour of httpd receiving an HTTP/1.0
request with an Expect header, ap_check_request_header() will still correctly
return 417 in this case.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1901420 13f79535-47bb-0310-9956-ffa450edef68
first request on this connection otherwise we assume we have just run
into a keepalive race and the backend is still healthy.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1899584 13f79535-47bb-0310-9956-ffa450edef68
Changing the status code after we sent it to the client causes a status
code being logged that is different from the one sent to the client which
can be confusing.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1895715 13f79535-47bb-0310-9956-ffa450edef68
* modules/proxy/mod_proxy.h,modules/proxy/proxy_util.c:
Add ap_proxy_fill_error_brigade() to factorize proxy error handling
on the client connection side.
* modules/proxy/mod_proxy_{http,ajp,uwsgi}.c:
Use ap_proxy_fill_error_brigade() where needed, including when an
empty brigade is returned on the backend side or when calling
ap_proxy_buckets_lifetime_transform fails.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1893595 13f79535-47bb-0310-9956-ffa450edef68
* modules/proxy/mod_proxy.h:
Restore the interface of ap_proxy_transfer_between_connections() back to
before MMN 20210506.0.
Add ap_proxy_tunnel_conn_bytes_in() and ap_proxy_tunnel_conn_bytes_out().
New proxy_tunnel_conn_t typedef'ed from opaque struct proxy_tunnel_conn.
* modules/proxy/mod_proxy.h(ap_proxy_tunnel_conn_get_read,
ap_proxy_tunnel_conn_get_transferred):
Axed/replaced by ap_proxy_tunnel_conn_bytes_in() and
ap_proxy_tunnel_conn_bytes_out().
* modules/proxy/proxy_util.c(struct proxy_tunnel_conn):
Replace "exchanged" by "bytes_in" and "bytes_out".
* modules/proxy/proxy_util.c(proxy_transfer):
New helper implementing ap_proxy_transfer_between_connections() and
returning both &bytes_in and &bytes_out.
* modules/proxy/proxy_util.c(ap_proxy_transfer_between_connections):
Now calls proxy_transfer().
* modules/proxy/proxy_util.c(ap_proxy_tunnel_conn_bytes_in,
ap_proxy_tunnel_conn_bytes_out):
Return tc->bytes_in and tc->bytes_out respectively.
* modules/proxy/proxy_util.c(proxy_tunnel_forward):
Use proxy_transfer() which updates in->bytes_in, &out->bytes_out.
tunnel->replied will be updated in proxy_tunnel_run().
* modules/proxy/proxy_util.c(proxy_tunnel_forward):
Fall through the "done" label in any case to set tunnel->replied based
on tunnel->client->bytes_out > 0.
* modules/proxy/mod_proxy_http.c(ap_proxy_http_process_response):
Use ap_proxy_tunnel_conn_bytes_{in,out}() for worker->s->{read,transferred}
accounting.
* modules/proxy/mod_proxy_http.c(proxy_http_async_finish):
Update worker->s->{read,transferred} when async too.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1893594 13f79535-47bb-0310-9956-ffa450edef68
* modules/proxy/mod_proxy_http.c (ap_proxy_http_process_response):
Save a call to apr_table_get(r->headers_out, "Transfer-Encoding").
No functional change.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1892563 13f79535-47bb-0310-9956-ffa450edef68
Adding `outgoing` flag to conn_rec, indicating a connection is
initiated by the server to somewhere, in contrast to incoming
connections from clients.
Adding 'ap_ssl_bind_outgoing()` function that marks a connection
as outgoing and is used by mod_proxy instead of the previous
optional function `ssl_engine_set`. This enables other SSL
module to secure proxy connections.
The optional functions `ssl_engine_set`, `ssl_engine_disable` and
`ssl_proxy_enable` are now provided by the core to have backward
compatibility with non-httpd modules that might use them. mod_ssl
itself no longer registers these functions, but keeps them in its
header for backward compatibility.
The core provided optional function wrap any registered function
like it was done for `ssl_is_ssl`.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1890605 13f79535-47bb-0310-9956-ffa450edef68
Since mod_proxy can see CONNECT URIs, "hostname:port" versus "scheme:"
is hardly distinguishable (we don't want to limit the length of hostnames),
and we don't allocate the scheme anymore while parsing, let's simply decline
unrecognized schemes (overlong or not) and be caught by the no-handler case
if there really is no proxy handler interested.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1886151 13f79535-47bb-0310-9956-ffa450edef68
later as (!scheme || u[0] != '/' || u[1] != '/' || u[2] == '\0') is true
for requests with the CONNECT method which we need to decline. But in many
cases requests with the CONNECT method have (u - url) > 14 as in this
case (u - url) is the length of the FQDN the forward proxy should connect
to.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1886141 13f79535-47bb-0310-9956-ffa450edef68
Let mod_proxy_http's canon and scheme handlers accept "ws[s]:" schemes so that
mod_proxy_wstunnel can decline requests when mod_proxy_http is loaded.
* modules/proxy/{mod_proxy.h,proxy_util.c} (ap_proxy_worker_can_upgrade):
Add a "dflt" argument to ap_proxy_worker_can_upgrade() which, if not NULL,
is matched when no worker upgrade= parameter is configured. This allows to
handle the default "Upgrade: websocket" case for "ws[s]:" schemes.
* modules/proxy/mod_proxy_http.c (proxy_http_canon, proxy_http_handler):
Add and use the new get_url_scheme() helper to parse URL schemes handled by
mod_proxy_http and use it in canon and scheme handlers. This helper now
accepts ws[s] schemes.
* modules/proxy/mod_proxy_wstunnel.c (proxy_wstunnel_post_config):
New post_config hook to detect whether mod_proxy_http is loaded and set
global fallback_to_mod_proxy_http flag in this case.
* modules/proxy/mod_proxy_wstunnel.c (proxy_wstunnel_check_trans,
proxy_wstunnel_canon,
proxy_wstunnel_handler):
These hooks now early return DECLINED if fallback_to_mod_proxy_http is set.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1885239 13f79535-47bb-0310-9956-ffa450edef68
Export ap_proxy_prefetch_input(), ap_proxy_spool_input() and
ap_proxy_read_input() from mod_proxy_http to mod_proxy.h/proxy_util.c so
that they are usable by all proxy modules.
mod_proxy_fcgi will use them in a following commit.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1884067 13f79535-47bb-0310-9956-ffa450edef68
When mod_proxy_http prefetches input data it calls the HTTP_IN filter
in nonblocking mode, but since it does not want 100 continue to be sent
for every case (e.g. 100-continue forwarding), it hacks r->expecting_100
(save in req->expecting_100, reset, eventually restore..) all over the
place.
Let's avoid this by making the HTTP_IN filter send 100 continue only
when called in blocking mode (once still), instead of the first time
it's called.
* modules/http/http_filters.c (struct http_filter_ctx): Add the seen_data
bit and rename eos_sent to at_eos (HTTP_IN does not send any EOS).
* modules/http/http_filters.c (ap_http_filter): Move 100 continue
handling outside the initialization/once block, and do it in blocking
mode only. Track in ctx->seen_data whether some data were already
received, and if so don't send 100 continue per RFC 7231 5.1.1.
* modules/proxy/mod_proxy_http.c: Remove req->expecting_100 (and its
danse with r->expecting_100) now that reading from the input filters
does the right thing.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1883639 13f79535-47bb-0310-9956-ffa450edef68
This is cleaner API than using pfds->pool implicitely.
MAJOR bump but reusing the existing hooks (with an API/ABI breakage) because
it's trunk material only.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1879449 13f79535-47bb-0310-9956-ffa450edef68
When supported by the MPM (i.e. "event"), provide async callbacks and let
them be scheduled by ap_mpm_register_poll_callback_timeout(), while the
handler returns SUSPENDED.
The new ProxyAsyncDelay directive (if positive) enables async handling,
while ProxyAsyncIdleTimeout determines the timeout applied on both ends
while tunneling.
Github: closes#126
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1879419 13f79535-47bb-0310-9956-ffa450edef68
modules/proxy/mod_proxy.h:
Rename AP_PROXY_TRANSFER_SHOULD_YIELD to AP_PROXY_TRANSFER_YIELD_PENDING
and add AP_PROXY_TRANSFER_YIELD_MAX_READS.
modules/proxy/mod_proxy_http.c:
modules/proxy/mod_proxy_wstunnel.c:
Removing of reqtimeout filter is now handled by ap_proxy_tunnel_create().
modules/proxy/proxy_util.c:
ap_proxy_transfer_between_connections():
Reorganize loop to break out early.
When AP_PROXY_TRANSFER_YIELD_PENDING, if !ap_filter_should_yield() we
still need to run and check ap_filter_output_pending() since it may
release pending data.
When AP_PROXY_TRANSFER_YIELD_MAX_READS, stop the loop after too much
reads (PROXY_TRANSFER_MAX_READS = 10000) to release the thread and
give the caller a chance to schedule the other direction.
Don't return APR_INCOMPLETE when it comes from an incomplete body
detected by ap_http_filter().
ap_proxy_tunnel_create():
Start with POLLOUT on both directions so that any pending output data
is flushed first.
ap_proxy_tunnel_run():
Remove re-init/clear of the pollset for each call so that the function
is reentrant.
Handle POLLOUT before POLLIN so that we can read in the same pass once
all buffered output data are flushed, using ap_filter_input_pending()
to drain buffered input data.
This is preparatory patch for async websocket tunneling is mod_proxy_http.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1879401 13f79535-47bb-0310-9956-ffa450edef68
Simplify streaming by using Content-Length if the length is available, or
chunked Transfer-Encoding otherwise.
Spooling to memory/file will only be used for HTTP/1.0 requests or if
proxy-sendcl is set.
This removes the handling of proxy-sendchunked and proxy-sendchunks.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1878991 13f79535-47bb-0310-9956-ffa450edef68
If stream_reqbody() detects a Content-Length vs bytes streamed mismatch it
means that a module's filter played bad games, such inconsistency on the client
connection would have been caught by ap_http_filter().
So move AH01087 logic to AH01086, under the same comment and log message, and
always return HTTP_INTERNAL_SERVER because the client is not the culprit here.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1878367 13f79535-47bb-0310-9956-ffa450edef68
To prevent stream_reqbody() from sending the FILE and EOS bucket in separate
brigades, and thus apr_file_setaside() to trigger if network congestion occurs
with the backend, restore the EOS in spool_reqbody_cl() which was stripped
when spooling the request body to a file.
Until APR r1878279 is released (and installed by users), apr_file_setaside()
on a temporary file (mktemp) will simply drop the file cleanup, leaking the
fd and inode..
This fixes BZ 64452.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1878280 13f79535-47bb-0310-9956-ffa450edef68
It helps simplifying a lot of duplicated code based on apr_strtoff(), while
also rejecting leading plus/minus signs which are dissalowed in Content-Length
and (Content-)Range headers.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1877954 13f79535-47bb-0310-9956-ffa450edef68
Also, add a comment about why we forward the Upgrade header in non-101
responses provided the protocol matches the upgrade= configuration.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1877695 13f79535-47bb-0310-9956-ffa450edef68
If the request Upgrade header matches the worker upgrade= parameter and
the backend switches the protocol, do the tunneling in mod_proxy_http.
This allows to keep the protocol to HTTP until the backend really
switches the protocol, and apply usual output filters.
When configured to forward Upgrade mechanism, we want the backend to be
able to announce its Upgrade protocol to the client (e.g. with 426
Upgrade Required response) and thus forward back the Upgrade header that
matches the one(s) configured in the worker upgrade= parameter.
modules/proxy/mod_proxy.h:
modules/proxy/proxy_util.c:
ap_proxy_worker_can_upgrade(): added helper to determine whether a
proxy worker is configured to forward an Upgrade protocol.
include/ap_mmn.h:
Bump MMN minor for ap_proxy_worker_can_upgrade().
modules/proxy/mod_proxy.c:
set_worker_param(): handle worker parameter upgrade=ANY as upgrade=*
(should the "any" protocol scheme be something some day..).
modules/proxy/mod_proxy_wstunnel.c:
proxy_wstunnel_handler(): use ap_proxy_worker_can_upgrade() to match
the Upgrade header. Axe handling of upgrade=NONE, it makes no sense to
Upgrade a connection if the client did not ask for it, nor to configure
mod_proxy_wstunnel to use a worker with upgrade=NONE by the way.
modules/proxy/mod_proxy_http.c:
proxy_http_req_t: add fields force10 (force HTTP/1.0) and upgrade (value
of the Upgrade header sent by the client if it matches the configuration,
NULL otherwise).
proxy_http_handler(): use ap_proxy_worker_can_upgrade() to determine
whether the request is electable for end to end protocol upgrading and set
req->upgrade accordingly.
terminate_headers(): handle Connection and Upgrade headers to send to the
backend, according to req->force10 and req->upgrade set before.
ap_proxy_http_prefetch(): use req->force10 and terminate_headers().
send_continue_body(): added helper to send the body retained for end to
end 100-continue handling.
ap_proxy_http_process_response(): use ap_proxy_worker_can_upgrade() to
match the response Upgrade header and forward it back if it matches the
configured one(s). That is for 101 Switching Protocol obviously but also
any other status code which is not overidden, at the backend wish. If the
protocol is switching, create a proxy tunnel and run it, using the minimal
timeout from the client or backend connection.
Github: closes#125
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1877646 13f79535-47bb-0310-9956-ffa450edef68
Support specifying the http status codes to be considered by ProxyErrorOverride
Submitted By: Martin Drößler <mail martindroessler.de>
Committed By: covener
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1876404 13f79535-47bb-0310-9956-ffa450edef68
possible returned buckets in the brigade in the error case. As we want to
close the connection anyway there is no sensible content we could still
sent and hence doing the needed life time transformation of the buckets
would be a waste.
PR: 64234
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1875353 13f79535-47bb-0310-9956-ffa450edef68
Let's call stream_reqbody() for all rb_methods, no RB_SPOOL_CL special case.
This both simplifies code and allows to keep EOS into the input_brigade until
it's sent, and thus detect whether we already fetched the whole body if/when
proxy_http_handler() re-enters for different balancer members.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1869224 13f79535-47bb-0310-9956-ffa450edef68
