backend socket) instead of the minimum as backend timeouts can be
configured more selectively (per worker if needed) as front end timeouts
and typically the backend timeouts reflect the application requirements
better.
PR: 65886
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1898127 13f79535-47bb-0310-9956-ffa450edef68
It's already NULL per the enclosing "if" test.
Also, while at a non-functional change, break a close long log line to <80 col.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1895986 13f79535-47bb-0310-9956-ffa450edef68
r1895921 changed proxy_detect() to disable forward proxying for URIs with no
hostname which is wrong, there might exist a third-party proxy module handling
the "urn:" scheme for instance (thanks Roy for the catch!).
For this to work, we also need to leave the forward proxied URI alone in
ap_proxy_pre_request() with no UDS special case or alike, a proxy module can
then catch (or not) the original URI as expected.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1895981 13f79535-47bb-0310-9956-ffa450edef68
* modules/proxy/mod_proxy.c(add_pass, add_member, set_proxy_param,
proxysection):
Check return value of ap_proxy_de_socketfy().
* modules/proxy/proxy_util.c(ap_proxy_get_worker_ex):
Check return value of ap_proxy_de_socketfy().
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1895914 13f79535-47bb-0310-9956-ffa450edef68
Some connect/wstunnel protocols might want half-close forwarding while some
might not, let's provide an r->subprocess_env opt-out.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1895304 13f79535-47bb-0310-9956-ffa450edef68
It seems that mod_proxy_connect has never applied any timeout in its tunneling
loop. Address this by setting a default timeout in ap_proxy_tunnel_create()
since mod_proxy_connect does not overwrite tunnel->timeout (while proxy_http
and proxy_wstunnel do).
This default timeout is set to the smallest of the backend side or the client
side timeout.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1894290 13f79535-47bb-0310-9956-ffa450edef68
* modules/proxy/mod_proxy.h, modules/proxy/mod_proxy.c:
Declare/implement the hook.
* modules/proxy/proxy_util.c(proxy_transfer):
Run tunnel_forward hooks when called by the tunneling loop.
Simpler input/output brigade cleanup on exit.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1893603 13f79535-47bb-0310-9956-ffa450edef68
* modules/proxy/mod_proxy.h,modules/proxy/proxy_util.c:
Add ap_proxy_fill_error_brigade() to factorize proxy error handling
on the client connection side.
* modules/proxy/mod_proxy_{http,ajp,uwsgi}.c:
Use ap_proxy_fill_error_brigade() where needed, including when an
empty brigade is returned on the backend side or when calling
ap_proxy_buckets_lifetime_transform fails.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1893595 13f79535-47bb-0310-9956-ffa450edef68
* modules/proxy/mod_proxy.h:
Restore the interface of ap_proxy_transfer_between_connections() back to
before MMN 20210506.0.
Add ap_proxy_tunnel_conn_bytes_in() and ap_proxy_tunnel_conn_bytes_out().
New proxy_tunnel_conn_t typedef'ed from opaque struct proxy_tunnel_conn.
* modules/proxy/mod_proxy.h(ap_proxy_tunnel_conn_get_read,
ap_proxy_tunnel_conn_get_transferred):
Axed/replaced by ap_proxy_tunnel_conn_bytes_in() and
ap_proxy_tunnel_conn_bytes_out().
* modules/proxy/proxy_util.c(struct proxy_tunnel_conn):
Replace "exchanged" by "bytes_in" and "bytes_out".
* modules/proxy/proxy_util.c(proxy_transfer):
New helper implementing ap_proxy_transfer_between_connections() and
returning both &bytes_in and &bytes_out.
* modules/proxy/proxy_util.c(ap_proxy_transfer_between_connections):
Now calls proxy_transfer().
* modules/proxy/proxy_util.c(ap_proxy_tunnel_conn_bytes_in,
ap_proxy_tunnel_conn_bytes_out):
Return tc->bytes_in and tc->bytes_out respectively.
* modules/proxy/proxy_util.c(proxy_tunnel_forward):
Use proxy_transfer() which updates in->bytes_in, &out->bytes_out.
tunnel->replied will be updated in proxy_tunnel_run().
* modules/proxy/proxy_util.c(proxy_tunnel_forward):
Fall through the "done" label in any case to set tunnel->replied based
on tunnel->client->bytes_out > 0.
* modules/proxy/mod_proxy_http.c(ap_proxy_http_process_response):
Use ap_proxy_tunnel_conn_bytes_{in,out}() for worker->s->{read,transferred}
accounting.
* modules/proxy/mod_proxy_http.c(proxy_http_async_finish):
Update worker->s->{read,transferred} when async too.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1893594 13f79535-47bb-0310-9956-ffa450edef68
It was reported to me (privately) that r1893101 broke existing settings like:
SetHandler "proxy:unix:///path/to/uds.sock|fcgi://localhost/"
RFC 3986 (section 3.2.2) says that:
If the URI scheme defines a default for host, then that default
applies when the host subcomponent is undefined or when the
registered name is empty (zero length). For example, the "file" URI
scheme is defined so that no authority, an empty host, and
"localhost" all mean the end-user's machine, whereas the "http"
scheme considers a missing authority or empty host invalid.
Let's consider that the "unix" scheme is closer to the "file" scheme than
the "http" one, and accept "unix:///path/to/uds.sock" as a valid URI.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1893519 13f79535-47bb-0310-9956-ffa450edef68
c->outgoing shouldn't be set by mod_ssl, ap_create_connection() allows that
and this commit also replaces all the calls to ap_run_create_connection() in
mod_proxy modules (not in the MPMs which create incoming connections only).
* include/http_connection.h, server/connection.c:
Declare and implement ap_create_connection().
* modules/proxy/proxy_util.c, modules/proxy/mod_proxy_connect.c,
modules/proxy/mod_proxy_ftp.c:
Use ap_create_connection() instead of ap_run_create_connection(), and don't
provide a connection_id a scoreboard handle for outgoing connection.
* server/log.c(do_errorlog_default):
Use c->outgoing instead of c->sbh to determine if it's a "client" or "remote"
connection.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1893184 13f79535-47bb-0310-9956-ffa450edef68
* modules/proxy/proxy_util.c(fix_uds_filename):
Sanity checks on the configured UDS path, fail with 500 if invalid since
continuing through proxy processing wouldn't work as expected.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1892986 13f79535-47bb-0310-9956-ffa450edef68
Unless I missed something, it is broken since r1516930 (Aug 23 2013).
So no-one has compiled this for more than 8 years.
Sounds like dead code.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1892921 13f79535-47bb-0310-9956-ffa450edef68
The actual r->filename format is "[proxy:]unix:path|url" for UDS, no need to
strstr(,"unix:") since it's at the start of the string.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1892814 13f79535-47bb-0310-9956-ffa450edef68
PRs 65521 and 65519.
* modules/proxy/proxy_util.c(ap_proxy_tunnel_run):
Avoid an infinite loop by shutting down the connection for write when poll()
returns POLLHUP and read is already down. PR 65521.
* modules/proxy/proxy_util.c(ap_proxy_tunnel_run):
When write completion is finished don't check for ap_filter_input_pending()
before proxy_tunnel_forward() to flush input data, this is a nonblocking read
already which will do the same thing implicitely. ap_filter_input_pending()
is broken in 2.4.x without the whole pending data mechanism (not backported
yet), so let's align here. PR 65519.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1892740 13f79535-47bb-0310-9956-ffa450edef68
Define WC bucket semantics as:
/**
* @brief Write Completion (WC) bucket
*
* A WC bucket is a FLUSH bucket with special ->data == &ap_bucket_wc_data,
* still both AP_BUCKET_IS_WC() and APR_BUCKET_IS_FLUSH() hold for them so
* they have the same semantics for most filters, namely:
* Everything produced before shall be passed to the next filter, including
* the WC/FLUSH bucket itself.
* The distinction between WC and FLUSH buckets is only for filters that care
* about write completion (calling ap_filter_reinstate_brigade() with non-NULL
* flush_upto), those can setaside WC buckets and the preceding data provided
* they have first determined that the next filter(s) have pending data
* already, usually by calling ap_filter_should_yield(f->next).
*/
The only filters that care about write completion for now are
ap_core_output_filter() and ssl_io_filter_output(), which try to fill
in the pipe as much as possible, using ap_filter_reinstate_brigade(&flush_upto)
to determine whether they should flush (blocking) or setaside their remaining
data.
So ap_filter_reinstate_brigade() is made to not treat WC as FLUSH buckets and
keep the above filters working as before (and correctly w.r.t. above WC bucket
semantics).
* include/ap_mmn.h, include/util_filter.h:
Axe specific ap_bucket_type_wc and define global &ap_bucket_wc_data address to
mark WC buckets checked by AP_BUCKET_IS_WC().
* server/util_filter.c (ap_filter_reinstate_brigade):
Don't treat WC buckets as FLUSH buckets.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1892468 13f79535-47bb-0310-9956-ffa450edef68
Clear the workers created in ap_proxy_sync_balancer(), notably ->local_status
for below ap_proxy_initialize_worker() to initialize all the child structures
like ->cp and ->cp->reslist, avoiding a possible crash when the workers are
used at runtime.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1891477 13f79535-47bb-0310-9956-ffa450edef68
ap_proxy_get_worker() needs to know whether it should lookup for prefix or
match or both matching workers, depending on the context.
For instance <Proxy[Match]> or ProxyPass[Match] directives need to lookup for
an existing worker with the same type as the directive (*Match or not), because
they will define one with that matching type if none exists.
On the contrary, "ProxySet <url>" at load time or ap_proxy_pre_request() at run
time need to find a worker matching an url whether it's by prefix or by regex.
So this commit adds ap_proxy_get_worker_ex() which takes a bitmask for the
matching type and calls it appropriately where needed.
For consistency, ap_proxy_define_worker_ex() is also added, using the same
bitmask flags, deprecating ap_proxy_define_match_worker().
Follow up to r1891206.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1891284 13f79535-47bb-0310-9956-ffa450edef68
We can't truncate ProxyMatch's worker name/url to the first '$' substitution
without possibly colliding with other workers. This also makes the matching
done at runtime by ap_proxy_strcmp_ematch() completely pointless.
To fix this and still address r1878467 (i.e. make http://host:port$1 a "valid"
URL), we need to remove '$' substitutions from the :port part of the URL only
since it's allowed anywhere else by apr_uri_parse().
So let's strip them before apr_uri_parse() and prepend them back in the path
before apr_uri_unparse() to restore the original URL. Non-matchable workers are
not concerned so ap_proxy_define_worker() is made a local helper (w/o the ap_
prefix) which takes "matchable" as argument and can then be called by both
ap_proxy_define_[match_]worker() functions.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1891206 13f79535-47bb-0310-9956-ffa450edef68
A WC bucket is meant to prevent buffering/coalescing filters from retaining
data, but unlike a FLUSH bucket it won't cause the core output filter to
block trying to flush anything before.
It can be passed by async handlers which want to never block, followed by
ap_filter_should_yield() to check for pending data and eventually suspend
processing until MPM/asynchronous write completion finishes.
In this commit it's used that way by the tunneling loop of mod_proxy to
prevent SSL coaslescing.
gh: closes#200
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1891148 13f79535-47bb-0310-9956-ffa450edef68
Let mod_proxy_http's canon and scheme handlers accept "ws[s]:" schemes so that
mod_proxy_wstunnel can decline requests when mod_proxy_http is loaded.
* modules/proxy/{mod_proxy.h,proxy_util.c} (ap_proxy_worker_can_upgrade):
Add a "dflt" argument to ap_proxy_worker_can_upgrade() which, if not NULL,
is matched when no worker upgrade= parameter is configured. This allows to
handle the default "Upgrade: websocket" case for "ws[s]:" schemes.
* modules/proxy/mod_proxy_http.c (proxy_http_canon, proxy_http_handler):
Add and use the new get_url_scheme() helper to parse URL schemes handled by
mod_proxy_http and use it in canon and scheme handlers. This helper now
accepts ws[s] schemes.
* modules/proxy/mod_proxy_wstunnel.c (proxy_wstunnel_post_config):
New post_config hook to detect whether mod_proxy_http is loaded and set
global fallback_to_mod_proxy_http flag in this case.
* modules/proxy/mod_proxy_wstunnel.c (proxy_wstunnel_check_trans,
proxy_wstunnel_canon,
proxy_wstunnel_handler):
These hooks now early return DECLINED if fallback_to_mod_proxy_http is set.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1885239 13f79535-47bb-0310-9956-ffa450edef68
Export ap_proxy_prefetch_input(), ap_proxy_spool_input() and
ap_proxy_read_input() from mod_proxy_http to mod_proxy.h/proxy_util.c so
that they are usable by all proxy modules.
mod_proxy_fcgi will use them in a following commit.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1884067 13f79535-47bb-0310-9956-ffa450edef68
Consistently use pconf for ap_proxy_define_{worker,balancer}() and pchild for
ap_proxu_initialize_{worker,balancer}() in mod_proxy [child_]init code.
pchild is needed in _initialize() for mutexes/shms' child_init and cleanup,
and to avoid a crash on shutdown (i.e. ap_terminate) in ONE_PROCESS mode,
where worker->cp->pool is destroyed twice, let's register conn_pool_cleanup()
as a pre_cleanup of pchild.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1883744 13f79535-47bb-0310-9956-ffa450edef68
to configure the Basic auth credentials to send to the remote proxy.
(Note that credentials are always sent w/o waiting for a challenge as
with proxy-chain-auth, and only Basic is supported - both of which are
not exactly ideal - but better than nothing.)
* modules/proxy/mod_proxy.h (struct proxy_remote): Add creds field.
* modules/proxy/mod_proxy.c (proxy_handler): Pass forward proxy
credentials via r->notes.
(add_proxy): Take credentials and base64-encode into ->creds field if
passed.
(add_proxy_noregex, add_proxy_regex): Take optional creds argument.
* modules/proxy/proxy_util.c (ap_proxy_determine_connection):
Use proxy credentials from r->notes if available.
(ap_proxy_create_hdrbrgd): Set Proxy-Authorization header from
credentials in r->notes if present.
PR: 37355
Github: closes#135
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1881790 13f79535-47bb-0310-9956-ffa450edef68
proxy_util.c:
Set POLLERR in reqevents for pollset providers that require it to detect
socket errors (like select() based one).
Call filters to read/write on POLLERR socket event, so that they know about
the error by experiencing the failure. If no POLLIN|POLLOUT is returned
with POLLERR (depending on the system or pollset provider), go with the
requested read or write event handling.
Restore ap_proxy_transfer_between_connections() so that it always tries to
read first (i.e. move yielding conditions afterward).
Add proxy_tunnel_forward() helper that calls transfer_between_connections()
and handles errors pollset updates.
Call proxy_tunnel_forward() when write completion finishes and there are
pending input data.
mod_proxy.h:
Add read_buf_size to proxy_tunnel_rec (trunk only, no MMN minor bump).
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1880200 13f79535-47bb-0310-9956-ffa450edef68
The purpose of ap_filter_output_pending() is to flush pending data, so
ssl_io_filter_coalesce() should honor that.
This allows mod_proxy to not care about mod_ssl coalescing filters when
tunneling between connections.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1879416 13f79535-47bb-0310-9956-ffa450edef68
modules/proxy/mod_proxy.h:
Rename AP_PROXY_TRANSFER_SHOULD_YIELD to AP_PROXY_TRANSFER_YIELD_PENDING
and add AP_PROXY_TRANSFER_YIELD_MAX_READS.
modules/proxy/mod_proxy_http.c:
modules/proxy/mod_proxy_wstunnel.c:
Removing of reqtimeout filter is now handled by ap_proxy_tunnel_create().
modules/proxy/proxy_util.c:
ap_proxy_transfer_between_connections():
Reorganize loop to break out early.
When AP_PROXY_TRANSFER_YIELD_PENDING, if !ap_filter_should_yield() we
still need to run and check ap_filter_output_pending() since it may
release pending data.
When AP_PROXY_TRANSFER_YIELD_MAX_READS, stop the loop after too much
reads (PROXY_TRANSFER_MAX_READS = 10000) to release the thread and
give the caller a chance to schedule the other direction.
Don't return APR_INCOMPLETE when it comes from an incomplete body
detected by ap_http_filter().
ap_proxy_tunnel_create():
Start with POLLOUT on both directions so that any pending output data
is flushed first.
ap_proxy_tunnel_run():
Remove re-init/clear of the pollset for each call so that the function
is reentrant.
Handle POLLOUT before POLLIN so that we can read in the same pass once
all buffered output data are flushed, using ap_filter_input_pending()
to drain buffered input data.
This is preparatory patch for async websocket tunneling is mod_proxy_http.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1879401 13f79535-47bb-0310-9956-ffa450edef68
The list can be rather long, speed up runtime by sorting the status codes in
error_override_codes and using binary search from ap_proxy_should_override().
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1877728 13f79535-47bb-0310-9956-ffa450edef68
If the request Upgrade header matches the worker upgrade= parameter and
the backend switches the protocol, do the tunneling in mod_proxy_http.
This allows to keep the protocol to HTTP until the backend really
switches the protocol, and apply usual output filters.
When configured to forward Upgrade mechanism, we want the backend to be
able to announce its Upgrade protocol to the client (e.g. with 426
Upgrade Required response) and thus forward back the Upgrade header that
matches the one(s) configured in the worker upgrade= parameter.
modules/proxy/mod_proxy.h:
modules/proxy/proxy_util.c:
ap_proxy_worker_can_upgrade(): added helper to determine whether a
proxy worker is configured to forward an Upgrade protocol.
include/ap_mmn.h:
Bump MMN minor for ap_proxy_worker_can_upgrade().
modules/proxy/mod_proxy.c:
set_worker_param(): handle worker parameter upgrade=ANY as upgrade=*
(should the "any" protocol scheme be something some day..).
modules/proxy/mod_proxy_wstunnel.c:
proxy_wstunnel_handler(): use ap_proxy_worker_can_upgrade() to match
the Upgrade header. Axe handling of upgrade=NONE, it makes no sense to
Upgrade a connection if the client did not ask for it, nor to configure
mod_proxy_wstunnel to use a worker with upgrade=NONE by the way.
modules/proxy/mod_proxy_http.c:
proxy_http_req_t: add fields force10 (force HTTP/1.0) and upgrade (value
of the Upgrade header sent by the client if it matches the configuration,
NULL otherwise).
proxy_http_handler(): use ap_proxy_worker_can_upgrade() to determine
whether the request is electable for end to end protocol upgrading and set
req->upgrade accordingly.
terminate_headers(): handle Connection and Upgrade headers to send to the
backend, according to req->force10 and req->upgrade set before.
ap_proxy_http_prefetch(): use req->force10 and terminate_headers().
send_continue_body(): added helper to send the body retained for end to
end 100-continue handling.
ap_proxy_http_process_response(): use ap_proxy_worker_can_upgrade() to
match the response Upgrade header and forward it back if it matches the
configured one(s). That is for 101 Switching Protocol obviously but also
any other status code which is not overidden, at the backend wish. If the
protocol is switching, create a proxy tunnel and run it, using the minimal
timeout from the client or backend connection.
Github: closes#125
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1877646 13f79535-47bb-0310-9956-ffa450edef68
Support specifying the http status codes to be considered by ProxyErrorOverride
Submitted By: Martin Drößler <mail martindroessler.de>
Committed By: covener
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1876404 13f79535-47bb-0310-9956-ffa450edef68
Support half closed connections and pending data draining (for protocols like
rsync). PR 61616.
When reading on one side goes faster than writing on the other side, the output
filters chain may start buffering data and finally block, which will break
bidirectional tunneling for some protocols.
To avoid this, proxy_tunnel_run() now stops polling/reading until pending data
are drained, and recovers appropriately.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1869420 13f79535-47bb-0310-9956-ffa450edef68
This commit adds struct proxy_tunnel_rec that contains the fields needed for a
poll() loop through the filters chains, plus functions ap_proxy_tunnel_create()
and ap_proxy_tunnel_run() to respectively initialize a tunnel and (re)start it.
Proxy connect and wstunnel modules now make use of this new API to avoid
duplicating logic and code.
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1869338 13f79535-47bb-0310-9956-ffa450edef68
Create a subpool of the connection pool for worker scoped DNS resolutions.
This is needed to avoid race conditions in using the connection pool by multiple
threads during ramp up.
Recheck after obtaining the lock if we still need to do things or if they
were already done by another thread while we were waiting on the lock.
* modules/proxy/proxy_util.c: Create a subpool of the connection pool for worker
scoped DNS resolutions and use it.
* modules/proxy/mod_proxy.h: Define AP_VOLATILIZE_T and add dns_pool to
struct proxy_conn_pool.
* modules/proxy/mod_proxy_ftp.c: Use dns_pool and consider that
worker->cp->addr is volatile in this location of the code.
PR: 63503
git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1868296 13f79535-47bb-0310-9956-ffa450edef68
