Summary:
Finishes the part of D120517427 that was dropped. Picks up the `setParserPaused` half of D122722345 by willwang1; the `getTransportDirection` half already landed in D120517427 through the cached codec traits.
`HTTPSession::processReadData` calls `codec_.setParserPaused(false)` before every `onIngress`, and the call used to walk every filter in the chain, each one just forwarding `call_->setParserPaused(paused)`. Now `HTTPCodecFilterChain::setParserPaused` calls the codec at the end of the chain directly.
To keep that correct, filters no longer take part in `setParserPaused`:
- `HTTPCodecFilter` is now a class on top of `GenericFilter` with an empty `final` `setParserPaused`. No filter can override it, whatever it derives from, and a filter called directly ignores it.
- `HTTPCodecFilterChain::add` and `addFilters` require `std::derived_from<Filter, HTTPCodecFilter>`, so a raw `GenericFilter` cannot get into the chain. `foreach` hands out `HTTPCodecFilter*`, so existing `FilterIteratorFn` callers are unchanged.
- `chainEnd_` changes only in `setDestination`, so the chain always reaches the current codec.
No filter in fbsource overrides `setParserPaused`, and nothing outside the chain calls it on a filter.
Perf (from D122722345): Strobelight puts stacks through `PassThroughHTTPCodecFilter::setParserPaused` at about $148K a year fleet-wide, about 96% of it the forwarding itself. On proxygen/olb_internet, `HTTPCodecFilterChain::setParserPaused` is about 0.165% of CPU.
Reviewed By: hanidamlaj
Differential Revision: D123003021
fbshipit-source-id: ed2aeafd7c22d4ad4ba35cd469d5f7ede217dc8b
Summary:
Updates the generated common-header files.
Diff resulted from pipeline (D115638777) and is first in plan to generate a monthly bot to update the common headers lists. Diff consists of updates to the headers list files, and resulting build errors from callers of it.
In HTTPCommonHeaders.txt: Added 4 headers, removed 16
In FacebookHTTPHeaders.txt: Added 63 Headers, removed 51
The net total between the two lists stayed as 254 headers.
For details on the decision of how headers were chosen to be removed this diff explains (D115508426), but tldr is:
- Headers counts were aggregated (1/10^6 sampling) to a scuba table
- Through the actual count of each header itself, the tiers each header appearance occurred on (slightly more value for edge and fna tiers), and the amount a header enum appears in code (for listed headers), a score was assigned to each header (higher score -> more valuable)
- The scores were sorted such that if [highest scored non-list header / lowest scored listed header] was greater than a certain ratio (chose 2.0 for first run), the header would replace it on the list
- This kept going until all of the non-listed headers were checked with the lowest scored non-swapped listed headers
Additionally: Theres a DENY_LIST for headers that likely shouldn't be removed (some psuedoheaders, connection headers, etc...) and for headers used more than 50 times in code as I deemed that traffic over a few weeks period of time likely isn't enough to justify removing them.
If you believe that a header was removed that you don't think should be removed from the list or if you believe anything gets broken from this change please lmk.
But advantages of this change can be seen below in the test plan when testing on different canary teirs.
Reviewed By: hanidamlaj
Differential Revision: D116961031
fbshipit-source-id: 9d6c281ba22ed3732fddb3f860527deb20f756fa
Summary:
Advertise the current H3 WebTransport enable setting from the sample HQ server so coroutine H3 clients can negotiate WebTransport and receive the associated stream limits.
___
landed-with-radar-review
Differential Revision: D122455664
fbshipit-source-id: 55977796bd6f4d07cf15324b62e7b1afcf1ce441
Summary:
Avoid rebuilding a `BufQueue` and walks the whole chain to count its bytes again. Calling the `.move()`
Old code:
1. `bodyBuf.move()` hands back a bare unique_ptr<IOBuf> and the calculated `chainLength_` is recalculated walking the chain again (this is unnecessary)
New code:
1. std::move copies nothing on its own. It only marks bodyBuf as an rvalue BufQueue which then use the move constructor
Cowboy edge task T289680342: this walk is ≈ 96 % of the measured cost (≈ $60 K a year on edge hosts).
Reviewed By: hanidamlaj
Differential Revision: D122754751
fbshipit-source-id: fe5ee78f318332de6f1b842c786e8c30e4b80c3e
Summary: adds a TLSParams::ClientCertKey struct to allow clients to (soon, see next diff) supply certificates that have already been loaded/buffered into memory
Differential Revision: D114226409
fbshipit-source-id: bd0205ef908dff10e08fd2da9ca8cbcc9c01b660
Summary: Reject malformed Content-Length values in the HTTP/1 codec before they can be framed leniently and forwarded verbatim.
Reviewed By: hanidamlaj, samdlinn
Differential Revision: D122834844
fbshipit-source-id: 73e1922a36bfd33b844f5aa44d09e36e40b57e37
Summary:
`AsyncServerSocket::bind()` has an overload taking a list of addresses and a single port, but the coro `HTTPServer` only ever calls the single-address one, so a server cannot listen on more than one address.
Adds `Config::bindAddresses`, forwarded to that overload. Empty keeps today's behaviour.
___
Differential Revision: D122935114
fbshipit-source-id: c2ae81065e554bfa682b2d8c0f68ba3ce887f61e
Summary: GitHub-hosted Ubuntu runners expose kernel-specific bpftool packaging that can lag the active Azure kernel. Follow the portable Katran approach by adding bpftool as a pinned getdeps source dependency, installing it into the getdeps bin directory, and removing distro-specific bpftool packages. Regenerate the socket-lb workflow so GitHub Actions explicitly fetches the new dependency.
Reviewed By: kainanpeace666
Differential Revision: D122838798
fbshipit-source-id: 5778f0621ce75c8ca237dfe306888b0f78950de3
Summary:
Memory-capped builds stopped sccache and let the first parallel compiler clients restart it lazily.
Under real `-j36` compiler fanout this exhausted the container PID cgroup, causing intermittent startup timeouts and `unable to fork (EAGAIN)`.
Restart sccache serially under the same `preexec_fn` memory cap before launching CMake. Preserve the uncapped, and missing-sccache behavior, and fail early if the replacement server cannot start.
Reviewed By: srikrishnagopu
Differential Revision: D122615787
fbshipit-source-id: 26120f150e12c8611a4213dad3df585249d5cf10
Summary: Make `supportsH3Wt()` check only local endpoint H3 WebTransport settings. This lets clients send CONNECT before peer SETTINGS arrive and leaves peer capability enforcement to the CONNECT response.
Reviewed By: hanidamlaj
Differential Revision: D121641086
fbshipit-source-id: 694a0c49cea46fa85fed896cf6e21bc3631b3b31
Summary: When logging things such as the URL on errors, they may contain sensitive query parameters that should not actually be logged. This change strips them from related logging.
Reviewed By: hanidamlaj
Differential Revision: D121881132
fbshipit-source-id: 4bf59eb49625fd88685f44901caa21d46d9bde25
Summary: Teach the HTTP/3 codec to serialize and parse the existing WebTransport initial uni- and bidirectional-stream limits. This lets an H3 WebTransport client open streams after negotiating the peer limits.
Reviewed By: hanidamlaj
Differential Revision: D121628277
fbshipit-source-id: 2611c1c9aea5beb91fee64e00a55c923c650ab10
Summary: Add HTTP/3 WebTransport upstream negotiation by reusing the HTTP/2 CONNECT validation and response flow, then constructing an `H3CoroWtSession` after a successful upgrade. Associate the CONNECT stream with that session so peer QUIC streams and RFC datagrams are routed to the WebTransport handler while unknown streams are rejected.
Reviewed By: hanidamlaj
Differential Revision: D120738506
fbshipit-source-id: 090d299925f560f53bfc833e9ae5dfef1a0151de
Summary:
proxygen's open-source macOS build has been red since 2026-09-03: 27 of 27 `main` runs failed this week, each filing ~4 oncall tasks. Internal and Linux OSS builds are unaffected.
Homebrew's libc++ 23 stopped providing `<vector>` transitively, and `HTTPWebTransport.h` and `SafePathUtils.h` had silently relied on that while declaring `std::vector` parameters. Add the includes they already use.
Identical to external PR facebook/proxygen#648 (T290230920), filed by an outside contributor. Landing here because OSS trunk is red now; please close#648 with credit to its author once this exports. Filed over the diff cap for the same reason.
Sentinel-Council-Run: council_1790568195_a4e76414
Sentinel-Council-Run: council_1790596993_36dca3c6
Sentinel-Harness: claude
*Modify your team agent prompt, check stats, and leave feedback: https://www.internalfb.com/sentinel_agent/rotations/traffic_protocols*
Model used: Claude Opus 5 (claude-opus-5[1m])
Reviewed By: hanidamlaj
Differential Revision: D122097017
fbshipit-source-id: 9bc7745ca8d004f3a1e9fa925071dcb0fc94c35b
Summary:
Skeleton for H3CoroWtSession Class to provide H3 WebTransport support of upstream.
___
Differential Revision: D100044283
fbshipit-source-id: 26570c02c190462f42c945041fca0d4c320f43b7
Summary:
X-link: https://github.com/facebookincubator/axel/pull/363
as of the previous diff, `HttpTxnHandler::onPushedTransaction` is no longer being invoked anywhere -- so we're removing all overrides across the codebase in preparation of removing the virtual method entirely
Differential Revision: D121237794
fbshipit-source-id: f76245d67ca109ca590aeff42e726483e363b514
Summary:
`ParseURL::hostAndPort()` now appends the host, `':'` and the port to a `std::string`, formatting the port with `std::to_chars`, instead of building a `std::stringstream`.
Why???
`ParseURL::parse()` calls it for every absolute URL that carries a port, and on olb_internet that stream is the largest single cost of URL parsing.
**Context:**
- D84177632 replaced `fmt::format` with `std::stringstream` here to keep `fmt` out of the Android (Oxygen) preloads. This keeps that constraint: `std::to_chars` and `std::array` are standard library only, and `ParseURL.cpp` already uses `std::from_chars` from `<charconv>`.
- `std::to_string` would be shorter, but clang-tidy (`facebook-hte-StdToStringUse`) flags it as unavailable on some Android toolchains.
**Motivation:**
- On olb_internet (revproxy), the stream inside `hostAndPort()` costs 0.52 % of the tier's CPU (Strobelight, production builds, 24 h to 2026-09-27 13:24 PDT): 38 % of all URL parsing.
- Expected saving: 0.4–0.5 percentage points of olb_internet CPU
**This diff:**
- `ParseURL::hostAndPort()`: the same string, with the port written by `std::to_chars` into a 5-byte buffer (a `uint16_t` port has at most five digits).
- New test `ParseURL.HostAndPort`: no host, no port, one- and five-digit ports, and IPv4 and bracketed IPv6 hosts in authority form. The existing `ParseURL.FullyFormedURL` cases cover it through `authority()`.
- The same change in the byte-identical Liger copies under `xplat/proxygen/lib/utils/`, as earlier `ParseURL` diffs did.
Reviewed By: hanidamlaj
Differential Revision: D122382543
fbshipit-source-id: d0998a44f576a1a68ace81bf23dc4383af21a95a
Summary: We need to uninstall the StopSendingCallback from a stream when fin is written, QuicSocket::resetStream is invoked, or when ::closeSession is invoked.
Reviewed By: afrind
Differential Revision: D121256188
fbshipit-source-id: a9826e8bb0ec0e23b7ad67deba8608b3f307d033
Summary:
Bump `fbcode/opensource/fbcode_builder/manifests/c-ares` to the 1.34.7
release archive so ShipIt-exported Proxygen builds fetch the patched
version. URL, SHA-256 and `subdir` change; `patchfile =
c-ares-mintimeout.patch` is deliberately left as-is.
Split out of D121051347 because getdeps is a different build system from
buck -- nothing under `third-party-buck` or `tools/build_defs` references
this manifest, so it neither needs nor benefits from landing atomically
with the buck dependency flip, and its verification is entirely separate
(archive digest plus patch reapplication, not a buck build).
The retained `patchfile` line is the part worth a reviewer's attention:
the Meta `ARES_OPT_MINTIMEOUTMS` patch is applied by getdeps at build time
against the downloaded archive, so the upgrade silently loses the
configured DNS timeout floor if that patch stops applying. Verified below
that it still does.
Reviewed By: oargon
Differential Revision: D121207734
fbshipit-source-id: 885c4b15f9f75cfe703c7eb072e6c5ae0a67f818
Summary:
WARNING: Generated by Autopilot (alpha) — review carefully, verify the underlying claim before accepting.
Agent: Sentinel Agent | Trajectory: https://www.internalfb.com/intern/devai/devmate/inspector/231f6c76-da01-4ccd-bd60-e5a86e79a9f2/ | SC job: https://www.internalfb.com/intern/sandcastle/instance/58546795851523006/
---
## The issue
- D122196773 changed the default `HTTPCoroConnector::ConnectionParams::identityValidation` from `Logging` to `Enforcing`. The revproxy `Pool::getConnectionParams` (`proxygen/facebook/revproxy/pools/Pool.cpp`) keeps that default, and BigCache fetches from upstream through it. BigCache dials origin VIPs by IP with no SNI, so the connector now requires the VIP IP in the peer cert's SANs. Those certs carry only the host name and host IPs, so every TLS connect to origin fails.
- BigCache release R13708 (traffic_static/bigcache), node `canary_binary_efficiency`, failed the `read_through_5xx_increase` healthcheck on `man2c01/ti/edge.bigcache.man2c01`. On the canary tasks, 82% of ReadThrough misses returned 5xx (control: 0.6%). The task logs show `certificate identity verification failed: expected=2401:db00:...:face:b00c:0:55a, cert_sans=[twsharedNNN...facebook.com, <host IPs>]` for the oil-proxy and anystore-everstore-proxy pools.
## The fix
- Back out D122196773 to restore the `Logging` default in the fbcode and xplat copies of `HTTPCoroConnector.h`. Callers that set `identityValidation` explicitly are unchanged. That includes D122198548, which sets `Enforcing` for storage clients.
- Before enforcement is turned back on, IP-dialed revproxy pools such as the BigCache upstream pools need either an expected identity that matches their upstream certs or an explicit opt-out.
Reviewed By: hanidamlaj, sandarsh
Differential Revision: D122333076
fbshipit-source-id: 98352fa9eabc930fdb3902b3c6dcbcf2e33ada38
Summary:
The main goal of this diff is to have WtStreamManager solely control the insertion, updating, and removing of streams from the PriorityQueue. When bypassing the `WtStreamManager` and directly manipulating streams via the `PriorityQueue`, there's high potential of corrupting the internal state of WtStreamManager.
To achieve this we utilize the `WtStreamManager::Config::peerMaxStreamData.*` to control how much data we can pull before WtStreamManager removes the stream from the PriorityQueue. This effectively delegates all PriorityQueue operations to WtStreamManager. We know use a default of `kDefaultWtEgress` instead of `kMaxVarint` for peer stream flow control (uni, bidi local & bidi remote). When the limit is saturated, we install the `onStreamWriteReady` callback and invoke `WtStreamManager::onMaxData` to reinsert the stream.
Reviewed By: afrind
Differential Revision: D119222318
fbshipit-source-id: ee07c5ccab07be3204aced3ca7e653de2b196c70
Summary:
c-ares 1.34.7 marks callback parameters as const (upstream PR #1060):
`ares_callback` buffers become `const unsigned char*` and the
`ares_host_callback` hostent becomes `const struct hostent*`. Adapt every
in-repo c-ares callback to a version-gated typedef so the same source
compiles against both 1.34.6 and 1.34.7.
Every change here sits inside `#if ARES_VERSION >= 0x012207`, so against
the 1.34.6 that is still selected at this point in the stack it is a
no-op. Nothing switches version in this diff -- that is D121051347.
Split out of D121051347 so each owner reviews only their own callback
rather than a 25-file version bump, and so the actual switch is reduced to
one-line dependency edits that are trivial to revert.
Consumers adapted:
- proxygen `CAresResolver` (fbcode and xplat cells): `AresCallbackBuffer`
typedef, `Query::queryCallback`, and the `detail::parse{Txt,Srv,Mx}Records`
helpers
- `c2p/secure_thrift` `AresDnsResolver`: `Query::onDone` and its static
trampoline
- `rl_pe/rlti/lab-peer` and `hedwig-peer-oss` `ServiceDiscovery` -- these
two files are byte-identical copies of each other, so they take
byte-identical hunks
- three vendored gRPC trees (`grpc`, `grpc/v1.83.0`, `grpc-v1.8`):
`on_hostbyname_done_locked`, `on_srv_query_done_locked`,
`on_txt_done_locked`. These additionally need an `AresHostent` typedef
for the hostent callback.
Reviewed By: oargon
Differential Revision: D121207733
fbshipit-source-id: 718f288ec8dff6b823aaec984cad6248897dfa8b
Summary: The comment near the top of the packsim getdeps manifest explains why it has no keys for building the project out of a monorepo checkout. It now describes those keys (the export project name, its path map and its strip list) instead of quoting them. The manifest's settings are unchanged, so no build changes.
Differential Revision: D122093195
fbshipit-source-id: f338cf7d508db35ccf25eecc3c1c2ce333c1ac73
Summary:
getdeps caps each project's `cmake --build` with an `RLIMIT_AS` of `job_weight_mib * 10` MiB. With `sccache` as the launcher, compiles run inside the long-lived sccache server rather than under ninja, so a project inherits the cap of whichever earlier build started the server (5, 10 or 15 GiB for dependencies) instead of its own.
FBOSS's largest FSDB instantiation TUs now peak at up to 17.2 GiB, so local and GitHub Actions OSS builds fail with `LLVM ERROR: out of memory` depending on which dependency last started the server, even with plenty of host memory free. This is the FSDB OOM targeted by D120549290's stack, whose ninja pool bounds concurrency rather than the per-process cap.
Stopping the server before each capped build makes the next compile respawn it under the current project's cap. Sandcastle builds skip sccache and are unchanged.
Reviewed By: AarjunC
Differential Revision: D121882709
fbshipit-source-id: 0a6d7a1cf309c8bf4bbf23d6f5fb3b5ed6dae2fe
Summary:
- getdeps.py vendor wrote fetcher.hash() next to each vendored project,
which for a git project without a pinned rev is the branch name, so a
project that ships no build/deps/github_hashes (mcrouter) got 'folly
main' recorded, identifying nothing. Ask the checkout for HEAD instead.
- Also record a version as a third field where one can be determined:
for a git checkout the nearest tag (git describe, deepening the
shallow clone as needed), as the tag itself when HEAD is tagged or
<tag>^<distance>.<commit> otherwise; for a downloaded archive the
version in its file name. Distributions bundling the vendored tree
need 'Provides: bundled(<name>) = <version>' (Fedora guidelines).
- Internal adjustments on import: BLACK/import sorting via arc lint -a,
E731 lambda fix in the new test.
Reviewed By: bigfootjon
Differential Revision: D121677377
fbshipit-source-id: f4a8a01925b8c8f9f431fd44ba348fb7bdfb4b7a
Summary:
closeSession() queues a CLOSE_WEBTRANSPORT_SESSION capsule and an EOM, then aborts the transaction. On HQ that abort reset both directions, so the capsule and the FIN were discarded and the peer saw HTTP_REQUEST_CANCELLED, making a normal client exit look like an internal error. A NO_ERROR abort now stops sending on ingress only when an EOM is queued or already written, leaving egress to flush and close the stream. HTTPTransaction also allowed NO_ERROR aborts on downstream transactions only, so WebTransport CONNECT streams are now included; classic CONNECT tunnels keep the old abort code. Transactions taking this path detach after the EOM's delivery ack rather than immediately.
Pull Request resolved: https://github.com/facebook/proxygen/pull/647
Reviewed By: hanidamlaj
Differential Revision: D121216613
Pulled By: afrind
fbshipit-source-id: 7f49fc6b486a76c2ac180acf71384509a7a03d5c
Summary:
The internal `oss-proxygen-linux-getdeps` job fails one test,
`HQUpstreamSessionTestWebTransport.BidirectionalStream` (T289289236).
`MockQuicSocketDriver` delivers ACK byte events 50ms after a write. The test relies on a
fixed number of `loopOnce()` calls to cover that delay, but `EventBase::loopOnce()` skips
waiting whenever a loop callback is queued. On the OSS runner it never waits, so
`closeWTSession()` resets the stream and cancels both callbacks (`onByteEventCanceled`)
instead of delivering them.
Deliver the already-written ACKs before closing the session.
Scope correction: the GitHub Actions `build` failures (T289334546, T289408775) are a
separate cross-repo hash-bump issue, not this test.
Sentinel-Council-Run: council_1789891406_717f889e, council_1790042605_522bfedb
Sentinel-Harness: claude
*Modify your team agent prompt, check stats, and leave feedback: https://www.internalfb.com/sentinel_agent/rotations/traffic_protocols*
Model used: Claude Opus 5 (claude-opus-5[1m])
Reviewed By: jbeshay
Differential Revision: D120892081
fbshipit-source-id: b68f50e7512b3a9c4a8b0fbae22e80a582174dd0
Summary:
When `--allow-system-packages` is enabled on Ubuntu, Katran’s `[debs]` entry makes getdeps treat the project itself as satisfied by `libssl-dev`. As a result, dependent projects do not fetch or build Katran and cannot find `katranConfig.cmake`.
Remove the misleading package replacement. Folly’s OpenSSL dependency continues to pull in `libssl-dev` transitively.
Reviewed By: p-shah256
Differential Revision: D121691384
fbshipit-source-id: f68680312e5c6f2211c1e28699d9aae2dafeb12d
Summary:
`HQSession::HQStreamTransportBase::initIngress` set its ingress callback with
`codecFilterChain.call()->setCallback(this)`. `call()` is the head of the *call*
chain, and `FilterChain` documents that method explicitly:
Returns the head of the call chain. Do *not* call T1::set_callback() on
this member. To change the callback of this entire filter chain, use the
separate setCallback() method.
Going through `call()` writes the callback straight onto whatever sits at the
head, bypassing the chain's own callback wiring and leaving `FilterChain`'s
`callback_` stale. It happens to work today only because `HQSession` never adds
filters to `codec_`, so the head of the call chain *is* the concrete codec --
unlike `HTTPSession` and `HTTPCoroSession`, which stack `HTTPChecks`,
`FlowControlFilter` and `RateLimitFilter`. The moment HQ grows a filter, the
callback would be installed in the wrong place.
Use the chain-level `setCallback()`, which is what `HQStreamBase::setActiveCodec`
two frames up already does for the same object.
No behaviour change today: with no filters in the chain, `callbackSource_` is the
concrete codec, so both forms end at `codec->setCallback()`. The chain-level call
additionally keeps `callback_` consistent, which is what makes the filter case
work.
Reviewed By: hanidamlaj
Differential Revision: D120697370
fbshipit-source-id: 12129f9c225094fefc73c5f48072162fed0376a4
