Summary:
Pull Request resolved: https://github.com/facebook/proxygen/pull/644
Instead of calling `codec_->` and all the chain methods, we want to encapsulate them together.
That way we can optimize them in the diffs to follow.
This diff just does the encapsulation.
This is mechanical and behaviour-preserving. The bulk of the diff is
`codec_->foo()` becoming `codec_.foo()` across `HTTPSession`, `HQSession`,
`HTTPCoroSession` and their tests.
The only problem with exposing both Chain and Codec methods was with `setCallback`
that is available on both today.
The only call site `HQSession::initIngress` where `->setCallback` was used is behaviourally identical in this diff.
(it will be fixed on top)
Reviewed By: hanidamlaj
Differential Revision: D120697369
fbshipit-source-id: 3a84d9c42b2358eb9aaa1e56b72e3052468e9717
Summary: now that PrestoDb's usage of `h2cEnabled` has been removed, we now fully the `h2cEnabled` member of HTTPServerOptions
Reviewed By: zhangyuannie
Differential Revision: D121691328
fbshipit-source-id: d6890ec10236e515dc68059d9ed5cd57e3075388
Summary: uses the recently added enqueuedFin_ flag for the XCHECK condition
Reviewed By: afrind
Differential Revision: D119378123
fbshipit-source-id: 73a72a97b6ef78430973bca857e4603429ebc306
Summary: We're currently deprecating PUSH_PROMISE support in http/2 and http/3 for proxygen/lib – this diff is focused on the removal of `HttpTxn::onPushedTransaction`
Reviewed By: kwaugh
Differential Revision: D121228773
fbshipit-source-id: 27bda0777353587964a369333d54bc57fa93b82b
Summary:
The QPACK decoder keeps a raw pointer to a stream's codec while a header block
waits for encoder data. Only the abort path cleared that pointer, so a peer
GOAWAY destroys the codec with the pointer still live and the block later
decodes through freed memory. Clear the queue where the codec is actually
destroyed instead, in `eraseStream` and `removeCodec`. The purge touches no
wire state, so Stream Cancellation still goes out only where a stream is
abandoned. It does not adjust the queued byte counter, which is a pre-existing
gap.
Reviewed By: kvtsoy
Differential Revision: D120193223
fbshipit-source-id: 7cf995b526fc24b4770331c5f65126ad9b4748ff
Summary:
Remove the unused `HTTPServerOptions::enableExHeaders` field and its remaining assignment in `ProxyBaseHandler`.
landed-with-radar-review
Reviewed By: jbeshay
Differential Revision: D121224863
fbshipit-source-id: cf3cb9e94e3cfdb43995011ad1b1a9cc8c326607
Summary:
This diff hardens the expectations around WtStreamManager and the state of the PriorityQueue – it fixes two small issues:
* A fin-only stream should not be inserted into ConnFcBlockedStreams. This isn't a big issue since it would have still been sitting in WritableStreams and that is what drives the write loop (either directly by reading the top of PriorityQueue or by accessing WtStreamManager::nextWritable)
* If the backing transport invokes WtStreamManager::dequeue with atMost=0 (e.g. http/2 has a fixed size write buffer of <= 64KiB for egress backpressure, and only re-enters the write loop once that write is successful), we should not remove that stream from the PriorityQueue
Reviewed By: SUJP123
Differential Revision: D120632884
fbshipit-source-id: e4940b47305fe1110c9ebc99d725ed8b26a5c4d5
Summary: Cleanup diff -- Http2Codec::parseExHeaders was removed a while back but I forgot to remove this method declaration in Http2Codec
Reviewed By: SUJP123
Differential Revision: D121072294
fbshipit-source-id: 78669b3663602985423edd92d819027637aec196
Summary:
Why?
Four statements the export makes to an external reader are wrong or
incomplete, and nothing internal checks any of them — no test reads the
READMEs, no job runs the GitHub workflow, and `oss_check`'s content lint
only looks for internal tokens.
1. `.github/ISSUE_TEMPLATE/bug_report.md` points at `examples/incast-2host.json`.
The exported path is `packsim/examples/incast-2host.json`; a bare
`examples/` resolves to a root directory that does not exist. The same
file gets the sibling reference right two comments later, so this is an
inconsistency inside one file rather than a convention.
2. `packsim/README.md` tells a maintainer to regenerate the GitHub workflow
with a command that does not reproduce it. `--free-up-disk` and
`--free-up-disk-before-build` both default to `False`
(`getdeps/workflow_generator.py`), and the three steps they gate —
deleting the Android SDK, dotnet, powershell and ghcup — are load-bearing
on a job that builds boost, openssl, folly and fbthrift from source and
then compiles every unit-test executable. Following the documented command
silently drops them and yields a disk-exhaustion failure with nothing in
the diff to explain it.
3. `packsim/README.md` names `libdwarf`, `libaio` and `libiberty` as
"copyleft" without saying what that asks of the reader, or that all three
arrive through folly rather than through PackSim. It also does not
mention that the CMake path against pre-installed dependencies avoids
them entirely. Note `libunwind` is in the same folly Linux dependency
block but is MIT, not copyleft, so it is deliberately NOT added here.
4. `CONTRIBUTING.md` sends security reports to `facebook.com/whitehat`
while `bug_report.md` and the shared `SECURITY.md` both use
`bugbounty.meta.com`.
How?
Fixes all four, and adds two things the export was missing:
- `.github/ISSUE_TEMPLATE/config.yml`, which turns off the blank-issue link
so the templates are the only route in and the security notice cannot be
bypassed, and adds a contact link for security reports. No Configerator
change was needed — `oss_check classify` confirms the existing
`ISSUE_TEMPLATE/*.(md|ya?ml)` exception already admits it.
- A Contributing section on the root `README.md`. Neither `CONTRIBUTING.md`
nor `CODE_OF_CONDUCT.md` was linked from the front page.
Two internal-only corrections ride along because they are the same class of
staleness and are cheaper to fix than to explain:
- `CMakeLists.txt` said its test sources are globbed out of 23 `oss_tests/`
directories. There are 24. The count is dropped rather than incremented,
since the sentence reads identically without it and cannot then drift.
- The getdeps manifest hard-coded "187 test executables" in a comment that
ships publicly. Replaced with a pointer at the figure the configure step
prints live.
- `.llms/rules/packsim-oss-build.md` claimed five files ship from this
directory and that nothing under `.github/` but a workflow does. The
contributor templates have shipped since D120640286. Its `apply_to_path`
is widened to match, so the rule now actually fires on the files it
describes.
Differential Revision: D121131700
fbshipit-source-id: 75439e492a46616443c4b1ab56285158f5e4b955
Summary: Ubuntu exposes `bpftool` as a virtual APT package, which makes getdeps fail because APT cannot choose a provider. Select `linux-tools-common` for Ubuntu while retaining `bpftool` for non-Ubuntu environments.
Reviewed By: tagrawal03
Differential Revision: D121095850
fbshipit-source-id: 170d670d725da798fcc1feed925408d54a1115d9
Summary:
Same attribution for HTTP/3, and the largest gap of the three.
`HQFramedCodec::checkConnectionError` built every frame-layer `HTTPException`
from the hardcoded literal `Connection error`, so `HQControlCodec`'s five
indistinguishable `HTTP_FRAME_UNEXPECTED` returns and `HQFramer`'s message-less
returns all collapsed into one string. Only the numeric HTTP/3 code
differentiated them.
Connection error [File]=HQControlCodec.cpp
[Context]=settings-not-first-frame
The frame loop lives in the base class while the checks live in the derived
codecs, so `setParseErrorContext` takes the caller's file and the origin points
at the real rejecting file rather than the base. The base clears the context
before `checkFrameAllowed` and falls back to a generic reason only when the
derived codec named none, so a specific attribution is never overwritten.
Instrumented `HQFramedCodec`, `HQControlCodec` and `HQStreamCodec`.
Reviewed By: afrind
Differential Revision: D114943895
fbshipit-source-id: 47123a6e08d1ecca411f505f500f3bee3f285680
Summary:
`mcrouter`'s buck targets, sources, and CMake `find_package` list never reference `mvfst`. The single hit is a test string literal (`set_umember3() = "mvfst"`) exercising serialization round-trips — pure test data, no headers used, no libraries linked, and it would pass identically with any other string.
So the manifest's direct `mvfst` entry is dropped. Note: `mvfst` stays in the resolved closure transitively via `fbthrift`; this changes declared direct deps only, not what gets built.
Reviewed By: bigfootjon
Differential Revision: D121020246
fbshipit-source-id: d9fe8f7934af7cdcc2d8be271c4151f246ba75c0
The internal and external repositories are out of sync. This Pull Request attempts to brings them back in sync by patching the GitHub repository. Please carefully review this patch. You must disable ShipIt for your project in order to merge this pull request. DO NOT IMPORT this pull request. Instead, merge it directly on GitHub using the MERGE BUTTON. Re-enable ShipIt after merging.
fbshipit-source-id: e24e32453662b79b80308343ceefcb628225c94c
Co-authored-by: Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com>
Summary:
Bump the OSS/getdeps default SAI version to 1.19.0 (libsai manifest, CMakeLists, run-getdeps.py, build-helper.py, fboss_oss_build.bzl, container docs).
Sites updated:
- opensource/fbcode_builder/manifests/libsai: url, sha256, subdir
- fboss/github/CMakeLists.txt: default SAI_VERSION + status message
- fboss/oss/scripts/run-getdeps.py: new SAI_VERSION_SHAS entry
- fboss/oss/scripts/build-helper.py: url + sha tables, choices, const/default, help
- fboss/facebook/oss_ci/fboss_oss_build.bzl: DEFAULT_SAI_VERSION + its docstring
- fboss/github/docs/static/code_snips/build_helper.sh: default text + supported list
The v1.19.0 tarball sha256 (199d2cd32408b6470ce0c43beee660cc468e29e72921f1e93e64d4a8eb80246a)
was obtained by downloading the release tarball and hashing it; it is used
identically in all three places that record it.
Building_FBOSS_on_containers.md carries no SAI version reference, so it needed
no change despite being listed in the integration runbook.
Reviewed By: ashutoshgrewal, shri-khare
Differential Revision: D120953509
fbshipit-source-id: 01072090c5844062d31f4ec2ab4aff6ed421d9b9
Summary:
Same attribution for HTTP/2. `checkConnectionError` reported every framer
failure as the single literal `Connection error`, so roughly thirty distinct
rejection paths in `HTTP2Framer` were indistinguishable in `debug_data`.
Frame parsing now records a reason via `setParseErrorContext`:
GOAWAY error: excessively large frame len=32768
[File]=HTTP2Codec.cpp [Context]=frame-exceeds-max-size
Reviewed By: afrind
Differential Revision: D114943825
fbshipit-source-id: e2e0ae29fed562b96e8fabaeb38641eef558cc62
Summary: Change `HTTPCoroSession` so it only enables push if that was specified in the construction settings.
Reviewed By: lnicco, hanidamlaj
Differential Revision: D120591826
fbshipit-source-id: 911e12cce94640e14059f64d62599f4d3b4bd5da
Summary:
## What
Adds a `[github.actions]` section to `opensource/fbcode_builder/manifests/packsim`
setting `timeout_minutes = 180`, and regenerates
`ai_simulation/.github/workflows/getdeps_linux.yml` from it.
## Why
The job had 60 minutes, which is the workflow generator's fallback
(`workflow_generator.py`: `timeout_minutes = gh("timeout_minutes") or "60"`) and
not a figure anyone picked for this project. One run has to build boost,
openssl, folly and fbthrift from source, then compile 187 test executables, then
run them -- and each test process defaults to 4 PDES worker threads in the
open-source build while ctest is already running jobs in parallel, so the test
phase oversubscribes a standard two-core runner several times over.
180 is a ceiling, not an estimate, and the manifest comment says so. Nothing is
known about the real wall clock because the workflow has never executed: nothing
has been pushed to the repository yet, and it only triggers on push and pull
request against `main`. Its first run will be its first measurement. A job that
is killed at 60 minutes reports as a red badge on the repository's front page
and tells you nothing about why; one that finishes in 70 tells you the number to
set. Lower it once a green run has reported one.
Deliberately not changed: the runner size. `--cpu-cores` would also help the
oversubscription, but that is a standing cost decision rather than a one-off
ceiling, and it belongs with whoever owns the project's CI budget.
## Why the manifest and not the workflow
The workflow file carries an `generated by getdeps.py` header and is rebuilt by
`opensource/fbcode_builder/getdeps/facebook/update-all-github-actions.sh`, which
already has an entry for packsim. Editing the YAML directly would be reverted by
the next sweep; `[github.actions]` keys in the manifest are the supported way to
change it, and they survive.
Differential Revision: D120642627
fbshipit-source-id: 4ea8d4a5bcc8533e6eec0cfb41dde63871c72f9b
Summary:
a test was recently added, this simply refactors it to use the test fixture to simplify code
___
Differential Revision: D120731929
fbshipit-source-id: 5f0910f7ab6ee9d32168a3b3b9c371c7754f3caf
Summary:
Revproxy codec errors reach the `proxygen_codec_errors` Scuba dataset via
`CodecErrorFilter`, whose `debug_data` column is `HTTPException::what()`.
For HTTP/1, the generic `http_parser` callback failure does not identify which
branch rejected the message.
Record a static reason for ingress parser callback failures and append it to
the parser error. Header-value failures also use
`CodecUtil::validateHeaderValueDetail` from the parent diff to include the
header name and rejected validation rule.
Reviewed By: hanidamlaj
Differential Revision: D120020224
fbshipit-source-id: 0664905ae48263552cde1ed33ddadc29463287d2
Summary:
Enable getdeps-based builds on every Enterprise Linux distro (RHEL, AlmaLinux, Rocky Linux, CentOS Stream) by selecting EL-specific manifest sections with distro_family instead of individual distro names.
The manifests that mean "any Enterprise Linux 9" (or 8) spell it as distro=centos_stream, because CentOS Stream is what CI runs. That never matched AlmaLinux or Rocky, and after the previous change gave RHEL its own distro name it would not match RHEL either. The per-distro enumeration is the bug class itself: each file was incomplete in a different way (libaio hand-listed alma+rocky but missed RHEL; everything else listed only centos_stream), so the fix has to apply everywhere the pattern occurs -- splitting it per project would just re-create the drift. Every project touched here builds against the same Folly-stack toolchain (gcc toolsets via the shared gcc12/gcc14 manifests, clang19, python3.12 for sapling), so distro coverage has to move together. The most consequential case is cachelib's gcc-toolset-14: without it an EL 9 build uses the default GCC 11 and fails to compile C++20 code although the toolset is right there in the repositories.
Rewrite those sections with the distro_family variable introduced in the previous change, preserving each selector's meaning while extending it to the whole family:
```
- all(distro=centos_stream,distro_vers=9) -> all(distro_family=rhel,distro_vers=9)
cachelib, gcc12, gcc14, openr, python, python-setuptools,
rebalancer, rebalancer-explorer
- all(distro=centos_stream,distro_vers=8) -> all(distro_family=rhel,distro_vers=8)
boost, boost-python
- not(all(any(distro=centos_stream,distro=centos),distro_vers=8))
-> not(all(distro_family=rhel,distro_vers=8))
libmnl
- not(any(all(distro=centos,distro_vers=8),all(distro=centos_stream,distro_vers=9)))
-> not(all(distro_family=rhel,any(distro_vers=8,distro_vers=9)))
lz4, lz4-python
- rpms.any(distro=centos_stream,...,distro=fedora) -> rpms
libaio, numa (every rpm distro ships libaio-devel and numactl-devel)
- rpms.distro=centos_stream -> rpms.distro_family=rhel
libaio-python
```
"centos" was the old CentOS Linux 8 name and folds into the family. The wider exclusions this produces (e.g. lz4-static now also excluded on centos_stream 8 and centos 9) only touch EOL or practically unused combos. No manifest still selects on an individual EL distro name. libaio-devel and numactl-devel are confirmed present on RHEL, CentOS and Fedora, the full rpm universe getdeps targets.
This must land after the distro_family change: the expression parser rejects unknown selector variables, so on a getdeps without it these manifests fail to load.
Tested with both changes applied: every manifest passes validate-manifest, and `install-system-deps --recursive --dry-run --os-type linux --distro rhel --distro-version 9 cachelib` now yields the same package set as --distro centos_stream, including gcc-toolset-14, libaio-devel and numactl-devel, where before it lacked all three. The openr, rebalancer and rebalancer-explorer hunks are the same mechanical rewrite but those projects were not built here.
**Depends on https://github.com/facebook/CacheLib/issues/495** (`distro_family`): must land after it, since the expression parser rejects unknown selector variables. Submitted separately so each can be reviewed on its own; the two are otherwise independent of https://github.com/facebook/CacheLib/issues/494.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
X-link: https://github.com/facebook/CacheLib/pull/496
Reviewed By: likewhatevs
Differential Revision: D120501944
Pulled By: michel-slm
fbshipit-source-id: fdfc8b65770736ed190223df5e4dc2d0b2b417ba
Summary:
# Context
`CoroSSLTransport::writeImpl` assumes a successful `SSL_write` consumes the entire input buffer. `SSL_MODE_ENABLE_PARTIAL_WRITE` permits positive partial returns, so writes larger than one TLS record can hit a fatal `XCHECK`.
# Mitigation
Advance the input range after each successful write and continue until all bytes are consumed. Preserve the original size for `WriteInfo` and add a regression covering a 64 KiB write with partial-write mode enabled.
Reviewed By: hanidamlaj, sandarsh
Differential Revision: D120634805
fbshipit-source-id: 3d0daf65f924c26d3c388d68503471c5105c67fe
Summary:
The goal here is just safety, when the WriteHandle is done we should unconditionally erase the handle from WritableStreams in ::finish
This is similar to what we currently do with connFcBlockedStreams & finOnlyStreams
Reviewed By: SUJP123
Differential Revision: D120439884
fbshipit-source-id: b29723ff99446789152037b93ec1cb3cbea3fcc5
Summary: Export public build scaffolding at the repository root and place the implementation under `socket-lb/`, matching Katran and other Meta OSS projects. Keep the top-level CMake and build scripts usable from both the internal and exported layouts, and update public build instructions accordingly.
Reviewed By: p-shah256
Differential Revision: D118875318
fbshipit-source-id: 996c0735d98fa093aa01bf08fce590e9e072d8f5
Summary:
Fedora retired liboqs in Fedora 45 ("Obsoleted by openssl 3.5": OpenSSL 3.5 ships ML-KEM natively), so the `[rpms.distro=fedora]` mapping added in facebook/CacheLib#488 only resolves on Fedora 44 and older. On Rawhide a build with --allow-system-packages that generates its BuildRequires from the manifests fails at dependency resolution with "No match for argument: liboqs-devel". fizz still hard-depends on liboqs, so let getdeps build it from source (or the vendor tree) on Fedora as it does everywhere else.
Found while building the Fedora cachelib package on Rawhide with BuildRequires generated from the manifests; the mapping was added in https://github.com/facebook/CacheLib/issues/488 and only checked against Fedora 44 at the time. CentOS Stream 10 does ship liboqs-devel, so a `distro_family=rhel` mapping for EL 10 could be considered separately once https://github.com/facebook/CacheLib/issues/495 lands.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
X-link: https://github.com/facebook/CacheLib/pull/497
Reviewed By: likewhatevs
Differential Revision: D120517415
Pulled By: michel-slm
fbshipit-source-id: 1386a4418536d59a36feffb24f85f48cf19f11f3
Summary:
get_linux_type() derives the distro name from NAME in /etc/os-release, which on Red Hat Enterprise Linux yields "red_hat_enterprise". Nothing knows that name: HostType.get_package_manager() returns None for it, so --allow-system-packages silently satisfies nothing and every dependency is built from source, and the manifests with distro=centos_stream selectors do not apply either. RHEL's VERSION_ID also carries a minor ("9.8"), as do AlmaLinux's and Rocky's, so even a matching distro would miss the distro_vers=9 selectors written for CentOS Stream.
This matters for EPEL: Fedora's Koji builds EPEL 9 packages in a buildroot with redhat-release (real RHEL), while EPEL 10 buildroots and the local mock configs use centos-stream-release. A getdeps-based EPEL 9 build therefore behaves differently in Koji than in local mock. Nobody noticed because getdeps has not been used for distro packaging before and RHEL is rare among people building from source.
Rather than pretend RHEL is CentOS Stream, model what the manifests actually mean. Following Chef's platform_family, introduce a distro_family alongside distro: "rhel" for rhel, centos, centos_stream, alma and rocky; "fedora"; "debian" for debian, ubuntu, pop!_os and mint; "arch". Fedora is deliberately its own family, unlike Ansible's os_family which folds it into RedHat: its base repositories carry many dependencies that the EL family only has in EPEL or not at all (glog, gflags, fmt, fast_float, googletest, benchmark, liboqs, jq), zlib is zlib-ng-compat there, and CentOS Stream 9 needs gcc-toolset; nineteen manifest sections already select on exactly that difference.
Concretely: - RHEL reports distro "rhel", the same short name as its os-release ID; - within the rhel family distro_vers is the major version only, so RHEL 9.8, Alma 9.6 and Rocky 9.6 all match distro_vers=9 as CentOS Stream 9 does; Fedora and Ubuntu keep their full versions; - distro_family is available to manifest selectors and drives get_package_manager(), which now returns rpm for the whole rhel family; - `install-system-deps --distro` accepts rhel; - os-release parsing moves into parse_os_release() so it can be unit tested.
Existing selectors keep their meaning: distro=centos_stream still matches only CentOS Stream. A follow-up will switch the sections that mean "any EL 9" to distro_family=rhel so that RHEL, Alma and Rocky pick up gcc-toolset, libaio and numactl too. Until then a RHEL host already gains every plain [rpms] mapping and the not(distro=fedora) ones.
Tested: `install-system-deps --recursive --dry-run --os-type linux --distro rhel --distro-version 9 cachelib` now produces a dnf command (previously "I don't know how to install any packages on this system"), and unit tests cover the os-release parsing for Fedora, CentOS Stream, RHEL, Alma, Rocky and Ubuntu plus the family and package-manager mapping. Verified against synthetic os-release contents, not a live RHEL host.
A follow-up PR will switch the manifest selectors that mean "any EL 9" (gcc-toolset, libaio, numactl and friends) to `distro_family=rhel`; it must land after this one because the expression parser rejects unknown selector variables.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
X-link: https://github.com/facebook/CacheLib/pull/495
Reviewed By: likewhatevs
Differential Revision: D120501703
Pulled By: michel-slm
fbshipit-source-id: f24b01e46207a08ca09b9f09500ba0fb78a502fa
Summary:
Fedora and EPEL package sparse-map as tsl-sparse-map-devel: 0.7.0 on F45 and Rawhide (stable since 2026-09-17), 0.6.2 on F44 and EPEL 9, and an EPEL 10 build is under way. With --allow-system-packages this takes sparsemap out of the set of projects built from source, leaving only the Meta stack (folly, fizz, wangle, mvfst, fbthrift) for a vendored distro build. A plain `[rpms]` section is used, as for libsodium, xxhash and snappy: where the package is not installed (CentOS Stream without EPEL, say) getdeps simply falls back to building sparsemap itself.
The F44 version is older than the 0.7.0 the manifest pins, so the difference was checked rather than assumed: the public API of sparse_map and sparse_set is identical between 0.6.2 and 0.7.0 down to template defaults, the tsl::detail_sparse_hash::sparse_array template cachelib reaches into has the same parameters, and the serialization format is unchanged. 0.7.0's only compatibility change is requiring C++17, irrelevant to cachelib. cachelib locates the headers by path (cmake/FindSparsemap.cmake: find_path tsl/sparse_map.h), which the package satisfies at /usr/include/tsl.
Tested on Fedora 44 with tsl-sparse-map-devel 0.6.2 installed: `getdeps.py --allow-system-packages build --no-tests --src-dir=. cachelib` no longer fetches or builds sparsemap (no Assessing/Building step for it in the log), cachelib configures with SPARSEMAP_INCLUDE_DIRS=/usr/include, SparseMapIndex.cpp compiles against the system header with -Werror on, cachebench links, and the build exits 0 after 58 minutes at two jobs.
Manifest-only follow-up to https://github.com/facebook/CacheLib/issues/488 and https://github.com/facebook/CacheLib/issues/493. RHEL hosts only benefit once https://github.com/facebook/CacheLib/issues/495 (distro_family) lands, since getdeps does not currently recognise RHEL as an rpm distro; the two are independent otherwise.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
X-link: https://github.com/facebook/CacheLib/pull/494
Reviewed By: likewhatevs
Differential Revision: D120501594
Pulled By: michel-slm
fbshipit-source-id: db5b7ad6514b56984aa3d92a24dbde1614ff1ed4
Summary: Convert os.path joins, basenames, dirnames, and exists/isfile/isdir/islink checks in the getdeps test suite to pathlib with os.fspath at the str boundaries, matching slices 1-3b. Deliberately kept: os.path.abspath (no exact Path equivalent; still used inside Path() for the fixtures/manifests dirs), os.path.realpath replaced only where resolve() is exactly equivalent, os.unlink/symlink/chmod/listdir (not os.path), and vendor_test.py (owned by unlanded D120204081). Verified: arc lint clean, pyre clean, buck test 113/113 pass.
Reviewed By: vitaut
Differential Revision: D120383078
fbshipit-source-id: 94b41927b849bd1667793648f5351476fa49fa98
Summary: Convert remaining os.path joins, basenames, dirnames, and exists/isfile checks to pathlib with os.fspath at the str-contract boundaries, matching slices 1-3a. Deliberately kept: os.path.normcase/relpath (no Path equivalent), os.fsdecode, os.path.sep, listdir/lstat traversal, os.walk in include_rewriter (rglob would change prune semantics), the abspath site in workflow_generator, and all test files (no mock seams affected; vendor_test stays owned by D120204081). Verified: arc lint clean, pyre clean, buck test 113/113 pass.
Reviewed By: bigfootjon
Differential Revision: D120383077
fbshipit-source-id: 37741cd8a8a042e40589b2925b08885f5bca8e7c
Summary: fetcher.py path manipulation now uses pathlib (joins, exists, makedirs, archive validation via resolve()+relative_to instead of a raw startswith prefix check). get_src_dir() stays -> str and constructor path fields stay str (builders/loaders consume them as str; retyped in the core slice). ShipitPathMap is deliberately untouched: its posix-matching semantics are load-bearing per the comment on add_mapping. retry_test now neutralizes Path.mkdir instead of os.makedirs (the seam _download_dir uses). New LocalDirFetcher/copy_if_different Path-input tests.
Reviewed By: likewhatevs
Differential Revision: D120383075
fbshipit-source-id: 8908611c0f79d853b09dcc86f9da09e4782e389b
Summary: Adds lint_cmake_floors.py, an arc lint wrapper holding the single cmake_minimum_required floor (3.14, from folly) for the folly-family OSS CMake projects, registered in fbsource-lint-engine.toml as OSSCMAKEFLOOR. It checks 13 effective roots (both sides of fbcode<->xplat mirrors, so skew is caught too) with at-or-above semantics so fbthrift 3.24 and mcrouter 3.16 stay legal, and offers the bump as an arc lint -a autofix. Any future dip below the floor fails lint on the offending diff instead of shipping a deprecation warning (or worse) to OSS consumers. Related: facebook/folly#2703.
Reviewed By: likewhatevs
Differential Revision: D120432856
fbshipit-source-id: 74895b760e4e73238ae0321f6e7eb5f966e39215
Summary: CMake 4.x removed compatibility with minimums below 3.5 and deprecates minimums below 3.10 ahead of removing that too. folly already requires 3.14, which is the effective floor for every folly-dependent OSS project: declaring lower no longer excludes any supported consumer (lowest CMake in any supported config is 3.22) and only buys deprecation warnings today plus hard configure failures once the next removal lands. Aligned here: fizz 3.5, wangle 3.5, mvfst 3.10, proxygen 3.10, cachelib 3.12, fb303 3.8. Left alone above the floor: fbthrift 3.24, mcrouter 3.16; folly itself already 3.14. Related: facebook/folly#2703.
Reviewed By: likewhatevs
Differential Revision: D120432855
fbshipit-source-id: 0734e2b4e4e0c8866de52bc8657d4bf0d81e7c57
Summary:
job_weight_mib is the per-compiler memory estimate getdeps divides available RAM by to pick the build concurrency, and the base of the RLIMIT_AS safety net it puts on each build process. cachelib and fbthrift declared 2048 MiB, which two measured builds show is low.
Peak resident memory of every cc1plus process was sampled every two seconds during full builds on Fedora 44 (aarch64, gcc 15), once at four jobs and once at two jobs with tests disabled; 1182 compiler processes were observed across the fbthrift and cachelib phases of the second run:
| percentile | peak RSS |
|------------|----------|
| p50 | 489 MiB |
| p90 | 1321 MiB |
| p95 | 1626 MiB |
| p99 | 2580 MiB |
| max | 4027 MiB |
24 translation units exceeded 2048 MiB and 4 exceeded 3072 MiB. The heaviest were cachebench's Stressor.cpp (4.0 GiB) and fbthrift's ThriftServer.cpp (3.8 GiB), ThriftQuicServer.cpp (3.1 GiB), ThriftRocketServerHandler.cpp and Cpp2Connection.cpp (2.8 GiB each).
3072 MiB matches what proxygen, fboss, openr, bgp and moxygen already declare and keeps a builder sized for N jobs from over-committing when several of the heavy units land together, which on a machine whose scratch dir sits on a tmpfs was enough to push it into swap. The weight is a typical-case budget rather than a ceiling, so the remaining 4 GiB outliers are left to the 10x RLIMIT_AS margin. fbthrift-python builds the same sources and moves with fbthrift.
Follow-up to https://github.com/facebook/CacheLib/issues/488/#490/https://github.com/facebook/CacheLib/issues/491, measured on the same host; manifest-only change.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
X-link: https://github.com/facebook/CacheLib/pull/492
Reviewed By: likewhatevs
Differential Revision: D120414253
Pulled By: michel-slm
fbshipit-source-id: f93eab02ecb3d3aa6a5b4e1a45a31cd7b376c366
Summary:
sparse-map 0.6.2 declares cmake_minimum_required(VERSION 3.1). CMake 4 removed compatibility with anything below 3.5, so on hosts whose system CMake is 4.x (Fedora 44, Homebrew, the GitHub macOS runners, and soon Ubuntu 26.04 and Debian 14) the sparsemap step fails to configure:
```
CMake Error at CMakeLists.txt:1 (cmake_minimum_required):
Compatibility with CMake < 3.5 has been removed from CMake.
```
Upstream fixed this in 0.7.0 (released 2025-11-02), which requires CMake 3.10, so take the release rather than carry a patch. 0.7.0 also moves to C++17, fixes iterator invalidation on a non-inserting insert and the serialization of small hashes, and adds a no-exceptions mode; the headers install to the same include/tsl layout. cachelib is the only manifest that depends on sparsemap, and the one internal type it reaches into, tsl::detail_sparse_hash::sparse_array, keeps the same template parameters.
Tested on Fedora 44 with system CMake 4.3.0 and gcc 15, no CMAKE_POLICY_VERSION_MINIMUM anywhere on the command line: `getdeps.py --allow-system-packages build --no-tests --src-dir=. cachelib` configured and installed sparsemap 0.7.0 with no CMake error (the only CMake output of note being the < 3.10 deprecation warnings from fizz and wangle, tracked in facebook/folly#2703), compiled SparseMapIndex.cpp against the new headers with -Werror on, and linked cachebench; 51 minutes at two jobs.
Manifest-only change; follow-up to https://github.com/facebook/CacheLib/issues/488/#490/https://github.com/facebook/CacheLib/issues/491. The CMake version matrix behind the platform list is in facebook/folly#2703.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
X-link: https://github.com/facebook/CacheLib/pull/493
Reviewed By: bigfootjon
Differential Revision: D120425085
Pulled By: michel-slm
fbshipit-source-id: 9802415458f4d6ba20fab21630ec9047d7ba45d5
Summary:
https://fb.workplace.com/groups/proxygendevelopers/posts/3993434290963932
A handler that sets its own Content-Length and then sends a complete response silently loses compression due to duplicate Content-Length.
Full chain of events:
(1) The handler sets Content-Length on the response.
(2) On the non-chunked path ResponseBuilder::send() appends a second one instead of replacing it, so the message now carries two values.
https://www.internalfb.com/code/fbsource/fbcode/proxygen/httpserver/ResponseBuilder.h?lines=161
(3) HTTPHeaders::getSingleOrEmpty returns an empty string for a multi-valued header, so isMinimumCompressibleSize reads the length as 0.
https://www.internalfb.com/code/fbsource/fbcode/proxygen/lib/utils/CompressionFilterUtils.h?lines=96-106
(4) The size test therefore fails and shouldCompress returns false, whatever the real body size.
https://www.internalfb.com/code/fbsource/fbcode/proxygen/lib/utils/CompressionFilterUtils.h?lines=91-92
(5) The response goes out uncompressed. The codec collapses the duplicate to a single value on the wire, so the response looks correct to the client and nothing is logged.
The chunked path never hits this, because ResponseBuilder sets no Content-Length there and the getIsChunked() arm short-circuits the size test. That is why the parent diff is still required: disabling it makes the chunked case regress immediately.
Fix is to treat the field as a singleton.
Reviewed By: hanidamlaj
Differential Revision: D120387888
fbshipit-source-id: 0c7e89df5deb02f9a4346c7bf1e01a8d45f478b4
Summary: katran_tester is moving to a CLI11 command interface, so the getdeps build needs CLI11 installed and KatranTesterCli.cpp compiled. Nothing calls it yet; the next diff adds the dispatch. Split out because edits under fbcode/opensource trigger every open source build.
Reviewed By: tagrawal03
Differential Revision: D119598499
fbshipit-source-id: 9db46020fc36e3e00f828a4b35ba725fd1a149eb
