aff37b9380
Summary: # Context Fife authorizes connections for vDPUs, and callers punt the opening TCP SYN with `authorize` so Fife can forward it (D125621587). We need a simple client to inject that traffic by hand, and the vDPU-side client has to be open source, so this tool is too. # Product decisions 1. `askBarney --fife_ip X --fife_port Y --cert_path C [--key_path K] --ca_path A <host> <port>` works like netcat over TCP, except it first punts the connection's SYN to Fife. 2. The punt is `authorize(local_ip, dst_ip, syn)` over mTLS. askBarney prints Fife's `decision` and `forward` to stderr. Anything other than `ALLOWED` exits 1. If allowed, it relays stdin and stdout over the connection until the server closes it. # Code decisions - **The kernel owns the connection.** If askBarney made up its own SYN and only Fife sent it, the server's SYN-ACK would reach a kernel with no matching socket and be reset. So askBarney starts a normal non-blocking `connect()`, captures the SYN the kernel sends with an `AF_PACKET` socket, and punts those exact bytes. Because the punted SYN has the kernel's ports and sequence number, the server's SYN-ACK completes the kernel's connection whichever copy of the SYN reached the server. The kernel's own SYN also goes out directly. - Capturing needs `CAP_NET_RAW`, so run it as root. - **Open source only.** It uses folly, fbthrift, fmt, gflags and `folly::Init`, has Apache-2.0 headers, and doesn't depend on Fife's internal server code. That's why the SYN matcher (`TcpSyn`) is its own small parser rather than reusing `fife/server/IpPacket`. There are no Meta-specific default paths; the certificate, key and CA are required flags. Its only internal dependency is the generated client for `fife/if/fife.thrift`, which has to be published alongside it before it can ship. # Test listener `fife/askbarney/listen.py` (also `buck2 run fbcode//fife/askbarney:listen`) is a standard-library-only destination for askBarney tests. It listens on `--port` (default 17172, IPv6 and IPv4), replies to each connection with `--reply`, prints everything the peer sends, and keeps serving until Ctrl-C, or one connection with `--once`. It ships with askBarney in the open-source repo. # Open-source build - BUCK stays for the fbcode build. The open-source build is CMake plus getdeps, the standard pattern for Meta's open-source C++. All the dependencies (folly, fbthrift, fizz, wangle, fb303) are built and consumed that way, and a Makefile would mean hand-maintaining their include and link flags plus a Thrift code-generation rule. - `fife/askbarney/public_tld/CMakeLists.txt` becomes the open-source repo's top level. It finds the dependencies and generates the `Fife` client from `fife/if/fife.thrift` with `add_fbthrift_cpp_library`, depending on `fb303::fb303_thrift_cpp` for the `fb303_core.thrift` include. `fife/askbarney/CMakeLists.txt` builds `askBarney`, `askbarney_tcp_syn` and the test. - `opensource/fbcode_builder/manifests/askbarney` is the getdeps manifest. It maps `public_tld` to the repo root, ships `fife/askbarney/` and only `fife.thrift` from `fife/if`, and strips BUCK files. The open-source repo mirrors the fbcode layout so generated `#include` paths (`fife/if/gen-cpp2/...`) are the same in both builds. - Before publishing: there's no GitHub repo yet, so the manifest has no `[git]` section and only works inside fbsource. `fife.thrift` still has the internal "Confidential and proprietary" header and needs an open-source license header through the open-source review. Reviewed By: IvanTopolcic Differential Revision: D125707320 fbshipit-source-id: 74c83cf24f46c5b9028d662847f473afa2b3b053
Easy builds for Facebook projects
This directory contains tools designed to simplify continuous-integration (and other builds) of Facebook open source projects. In particular, this helps manage builds for cross-project dependencies.
The main entry point is the getdeps.py script. This script has several
subcommands, but the most notable is the build command. This will download
and build all dependencies for a project, and then build the project itself.
getdeps.py requires Python 3.6 or newer and enforces this at startup.
Deployment
This directory is copied literally into a number of different Facebook open
source repositories. Any change made to code in this directory will be
automatically be replicated by our open source tooling into all GitHub hosted
repositories that use fbcode_builder. Typically this directory is copied
into the open source repositories as build/fbcode_builder/.
Vendoring dependencies for offline builds
Distributions typically require builds to run without network access and to
ship third-party sources alongside the project. getdeps.py vendor --output-dir DIR project copies the source tree of every dependency that is
not satisfied by system packages to DIR/<project> and records what it
vendored in DIR/getdeps-vendor.txt. A later getdeps.py --vendor-dir DIR build project then takes those trees instead of fetching, and fails if a
dependency is missing from DIR rather than downloading it. Pass the same
--allow-system-packages and --no-tests options to both commands so they
agree on the dependency set.
Project Configuration Files
The manifests subdirectory contains configuration files for many different
projects, describing how to build each project. These files also list
dependencies between projects, enabling getdeps.py to build all dependencies
for a project before building the project itself.
Manifest features
A manifest may declare named features that toggle parts of its build. Each
feature appears as a feature_<name> variable that conditional sections such
as [dependencies.feature_<name>=on] or [cmake.defines.feature_<name>=off]
can gate on. Consumers of a project pick which features they want by attaching
a feature spec to the dep entry.
A project that supports features looks like:
[features]
default = rpc
rpc =
[cmake.defines.feature_rpc=off]
THRIFT_RPC=OFF
[dependencies.feature_rpc=on]
fizz
wangle
default lists features enabled when no consumer overrides; the other entries
declare valid feature names. There is no CLI flag for selecting features —
they are configured per dependency edge inside manifests.
A consumer's dep entry uses comma-separated tokens after =:
| Spec | Effect |
|---|---|
fbthrift |
request the dep's default features |
fbthrift = extra |
defaults plus extra |
fbthrift = !default |
this edge contributes no features |
fbthrift = !rpc |
prohibit rpc (this edge and globally for the build) |
fbthrift = !default, extra |
only extra, none of the dep's defaults |
Resolution composes feature requests by union across all edges in the dep
graph, then checks prohibitions against the union; a conflict (one consumer
requests a feature another prohibits) is a hard error. Each project produces a
single build per invocation, so its resolved feature set is global to that
invocation. The resolved set is folded into the project's cache hash, so a
serialization-only build of fbthrift caches separately from a full build.
Since this directory is copied into many Facebook open source repositories,
it is also used to help share some CMake utility files across projects. The
CMake/ subdirectory contains a number of .cmake files that are shared by
the CMake-based build systems across several different projects.
Older Build Scripts
This directory also still contains a handful of older build scripts that
pre-date the current getdeps.py build system. Most of the other .py files
in this top directory, apart from getdeps.py itself, are from this older
build system. This older system is only used by a few remaining projects, and
new projects should generally use the newer getdeps.py script, by adding a
new configuration file in the manifests/ subdirectory.