1
0
mirror of https://github.com/facebookincubator/mvfst.git synced 2026-10-23 17:08:55 +03:00

48 Commits

Author SHA1 Message Date
Sandarsh Srivastava
801c459cdf Use default Xcode for macOS CI
Summary: Update the generated mvfst macOS workflows to point `DEVELOPER_DIR` at `/Applications/Xcode.app/Contents/Developer` instead of the retired Xcode 16.2 path. This matches the current workflow generator and prevents `xcrun` from failing when GitHub rotates runner images.

Reviewed By: jbeshay

Differential Revision: D115296030

fbshipit-source-id: 261ab7581d893747e5da44795407c89de97ece50
2026-08-10 16:06:31 -07:00
Rob Lyerly
67330b4b76 Upgrade getdeps template's mozilla-actions/sccache-action from v0.0.9 -> v0.0.10
Summary: Someone flagged this on the Cachelib repo in [this PR](https://github.com/facebook/CacheLib/pull/461).  This apparently fixes a deprecation warning for node20.js by bumping to node24 ([relevant PR](https://github.com/Mozilla-Actions/sccache-action/pull/245) for v0.0.10).

Reviewed By: xavierd

Differential Revision: D108750058

fbshipit-source-id: b17971b6bd5c17d56eddfa56a9d03d295aa2e882
2026-06-16 09:49:09 -07:00
Richard Barnes
44df22e346 getdeps generate-github-actions: remove --recursive from build cmd
Summary:
The `build` subcommand already builds all transitive deps unconditionally (`manifests_in_dependency_order()` always returns the full dep graph). The `--recursive` flag added to `final_build_cmd` in D106028158 is not a recognized argument for `build`, so argparse exits non-zero and every GHA job fails before building anything.

Fix: remove `--recursive` from the generated build step in `workflow_generator.py`. Regenerated all affected workflow files via `update-all-github-actions.sh` and updated test fixtures with `UPDATE_FIXTURES=1 buck run`.

The `--recursive` flag remains correct on `install-system-deps` and `query-paths` steps, where it is a defined and meaningful argument.

Supersedes D106119149.

Reviewed By: bigfootjon

Differential Revision: D106122539

fbshipit-source-id: f10dc25bee75b64171844d94cd35eec0f37e8445
2026-05-22 13:08:44 -07:00
Richard Barnes
9d036914bc getdeps generate-github-actions: collapse per-dep build pyramid into single recursive step
Summary:
Replace the per-dep `Build <dep>` / `Cache <dep>` step pyramid in generated GitHub Actions workflows with a single `Build <project> --recursive` step. With sccache enabled by default (since 6c880b17) we do not need per-dep builds.

Changes:
- `workflow_generator.py`: remove the per-dep deps loop, `has_same_repo_dep`, `no_deps_arg`, `free_up_disk_arg`, and `--use-build-cache` CLI flag; `final_build_cmd` now always passes `--recursive --src-dir=.`
- `templates/workflow.yml.j2`: remove `cache_restore`/`cache_save` macros and the `for dep in deps` loop
- `test/workflow_generator_test.py`: drop `use_build_cache` from base args
- Golden fixtures and all OSS project workflows regenerated via `update-all-github-actions.sh`

Reviewed By: bigfootjon

Differential Revision: D106028158

fbshipit-source-id: e1d69616fe85553b0d00b6822988dac58bec9c3c
2026-05-22 10:41:41 -07:00
Richard Barnes
f3a955d9e5 getdeps: eliminate per-dep cache pyramid; enable sccache by default
Summary:
Our GHA workflows did not previously use sccache, so they had a complex per-dep cache pyramid. Now they do use sccache.

sccache works on a per-file level so it can accelerate builds even when a dependency has changed.

Now that we have this new caching mechanism we can retire the cache pyramid for a significant reduction in workflow complexity. Trying that here.

Flip `--use-build-cache` to off by default. The pyramid is now opt-in via `--use-build-cache`; sccache remains on by default for all non-Windows platforms (it was already the default — this change makes the pyramid the non-default).

Golden test fixtures are regenerated to match the new output shape.

Reviewed By: bigfootjon

Differential Revision: D105903053

fbshipit-source-id: 8eb0fa3afa0de8975752710bb695619d2453d2c4
2026-05-21 16:19:42 -07:00
Richard Barnes
169bf7d602 fbcode_builder: regenerate all OSS CI workflows; fix test step cmake defines; auto-inject sccache
Summary:
Regenerate all OSS GitHub Actions workflow files via `update-all-github-actions.sh`. Three fixes bundled:

1. **Test step cmake defines** (`D104914460` follow-up): the generator fix was not applied to the already-committed workflow files. The test step was missing `--extra-cmake-defines`, so it computed different dep-install-dir hashes than the build steps — `CMAKE_CXX_COMPILER_LAUNCHER=sccache` was folded into every dep's hash via `GETDEPS_CMAKE_DEFINES`, so the test step looked in the wrong directories and LD_LIBRARY_PATH was broken at test time. CacheLib was the known-failing case; openr, rebalancer, folly-shared-lib, and others with test steps had the same latent bug.

2. **sccache cmake define removed from generator**: `workflow_generator.py` was manually injecting `CMAKE_CXX_COMPILER_LAUNCHER=sccache` into `--extra-cmake-defines`, which put it into `GETDEPS_CMAKE_DEFINES` and thus into every dep's install-dir hash. This is the root cause of the hash-mismatch bug above. `builder.py`'s `_compute_cmake_define_args` already calls `path_search(env, "sccache")` and injects `CMAKE_CXX_COMPILER_LAUNCHER` automatically when sccache is in PATH — outside the hash — so the explicit injection is redundant and harmful. Remove it.

3. **Regen script cmake side-channel removed**: `update-all-github-actions.sh` had a `--package-extra-cmake-defines "fbthrift=..."` cmake side-channel for the rebalancer. The rebalancer manifest already declares `fbthrift = !rpc, !benchmark`, and the feature spec drives the cmake defines at runtime — no side-channel needed.

Reviewed By: AlnisM

Differential Revision: D105100509

fbshipit-source-id: 6be93076a3034d4919dc32d99987e1bfac582d68
2026-05-13 22:56:16 -07:00
Richard Barnes
3330606e2c fbthrift CMake: split RPC and benchmarks into their own options
Summary:
The CMake var was named `THRIFT_SERIALIZATION_ONLY`. That name made it sound like a hard mode switch, but really it just turned RPC off. Drop it and replace with two positive options:

- `THRIFT_RPC` (default ON) — when OFF, skip the RPC link graph (transport, async, thriftcpp2 RPC pieces) and the fizz/wangle/mvfst `find_package` calls.
- `THRIFT_BENCHMARKS` (default ON) — when OFF, skip the benchmark executables (`ProtocolBench`, `VarintUtilsBench`).

Example, before:

```
cmake -DTHRIFT_SERIALIZATION_ONLY=ON ...
```

After:

```
cmake -DTHRIFT_RPC=OFF ...           # RPC off; benchmarks auto-off
cmake -DTHRIFT_BENCHMARKS=OFF ...    # just skip benchmarks
```

`THRIFT_RPC=OFF` force-clears `THRIFT_BENCHMARKS=OFF`, since the benchmark targets link `thriftcpp2`. Asking for benchmarks without RPC would otherwise produce a confusing link error. The clamp prints a status message so the user sees what happened. The gate on each benchmark target is then a single check on `THRIFT_BENCHMARKS`.

This diff also opts the fbthrift manifest in to fbcode_builder's per-dependency feature spec (parent diff), declaring `rpc` and `benchmark` features that map to the new CMake options:

```ini
# manifests/fbthrift
[features]
default = rpc, benchmark
rpc =
benchmark =

[cmake.defines.feature_rpc=off]
THRIFT_RPC=OFF

[cmake.defines.feature_benchmark=off]
THRIFT_BENCHMARKS=OFF

[dependencies.feature_rpc=on]
fizz
libsodium
wangle
mvfst

[dependencies.feature_benchmark=on]
benchmark
```

Default features stay ON, so consumers see no change. A consumer that says `fbthrift = !rpc` skips both the CMake RPC stack and the fetch of fizz/wangle/mvfst. A consumer that says `fbthrift = !benchmark` skips the benchmark fetch and the benchmark targets.

Reviewed By: bigfootjon

Differential Revision: D104487188

fbshipit-source-id: 318eb99602747244d6c1993c67a357e6a7d85c2d
2026-05-11 11:38:19 -07:00
Richard Barnes
b26a89aba3 Fix getdeps introducting a space after a colon which confuses the parser
Summary:
Fix YAML syntax error in the generated GitHub Actions workflow cachelib/public_tld/.github/workflows/getdeps_linux.yml that prevented the linux CI job from loading (Invalid workflow file ... line 161).

The breakage was on every run: step containing --extra-cmake-defines '{"CMAKE_CXX_COMPILER_LAUNCHER": "sccache"}'. The run: value is an unquoted YAML plain scalar, and the ": " (colon + space) inside the JSON was parsed as a mapping key/value separator, producing mapping values are not allowed here. GitHub reported the failing step's first line (161); the actual offending byte was on line 168 col 196, with the same pattern repeated on all 36 build steps.

Two changes:

  1. Quick fix to the generated workflow (cachelib/public_tld/.github/workflows/getdeps_linux.yml) — rewrite the inline JSON in compact form ({"CMAKE_CXX_COMPILER_LAUNCHER":"sccache"}, no space after :) across all 36 occurrences. This unblocks CI immediately.
  2. Durable fix in the generator (opensource/fbcode_builder/getdeps.py, cmake_arg_for) — pass separators=(",", ":") to json.dumps, so future regenerations of any project's workflow (cachelib, folly, fbthrift, watchman, etc.) emit YAML-safe compact JSON.

Both layers are needed: without (1) CI fails today; without (2) the next time anyone runs update-all-github-actions.sh the bug returns.

Reviewed By: vj7-byte, alikhtarov

Differential Revision: D104263446

fbshipit-source-id: 9ff90b341ba5d1aaf64d10ab77b3f6e3552c2398
2026-05-08 09:56:48 -07:00
Richard Barnes
44b624c363 fbcode_builder: regenerate OSS CI workflows after D96800077 backout
Summary:
Make security happy.

Removes the workflow_dispatch tmate_enabled input and the trailing
"Setup tmate session" step from every generated GitHub Actions
workflow, matching what the reverted generator now emits.

Reviewed By: bigfootjon

Differential Revision: D104103189

fbshipit-source-id: 2bb2c36b9cd58e80d08618e569c574772283b481
2026-05-06 16:33:12 -07:00
Richard Barnes
7aece54adb fbcode_builder: regenerate non-rebalancer OSS CI workflows
Summary:
Refresh the regenerated GitHub Actions yml files for cachelib, eden(fs)/mononoke/sapling, edencommon, fb303, fizz, folly (incl. shared-libs), katran, openr, proxygen, quic (mvfst), rust-shed, wangle, watchman, and xplat fbthrift to match what `fbcode/opensource/fbcode_builder/getdeps/facebook/update-all-github-actions.sh` currently emits. No source manifest or generator changes — just `update-all-github-actions.sh` output that had drifted out of sync with the committed yml.

Net effect across these workflows:
- Enables sccache (`mozilla-actions/sccache-action@v0.0.9`) and threads `--extra-cmake-defines '{"CMAKE_CXX_COMPILER_LAUNCHER":"sccache"}'` through every dep build step. None of these projects had sccache previously.
- Adds `Show runner info` step (CPU model, core count, free memory) at the top of each build job to help diagnose runner heterogeneity.
- Adds `Show sccache stats` and `Show GitHub Actions cache usage` post-build diagnostic steps.
- Adds the `actions: read` permission, `SCCACHE_GHA_ENABLED: on` env, and `timeout-minutes: 60` where the generator now emits them.

Cachelib's `actions/upload-artifact@v6` was preserved at `v7` — the generator default is `v6` and everywhere else already matches, but cachelib was hand-bumped to `v7` and we're not rolling that back as part of this regen pass.

Differential Revision: D103882416

fbshipit-source-id: 431dff1c6c429c73e9bda4bf5ef0d348d5b5c312
2026-05-05 14:51:42 -07:00
Fei Chen
b87d7676cc Regen github actions
Summary:
as per title
ran opensource/fbcode_builder/getdeps/facebook/update-all-github-actions.sh  without any change

Reviewed By: bigfootjon

Differential Revision: D90409978

fbshipit-source-id: 866cac33b978ab82847a3a36bfa74afabea28881
2026-01-12 10:26:09 -08:00
Harsh Chokshi
ca88367846 Update Github actions checkout and upload-artifact to v6 for Node 24 compatibility
Summary:
Upgrade GitHub Actions to their latest versions to ensure compatibility with Node 24, as Node 20 EOL is April 2026.

| Action | Old Version(s) | New Version |
|--------|---------------|-------------|
| `actions/checkout` | [`v4`](https://github.com/actions/checkout/releases/tag/v4) | [`v6`](https://github.com/actions/checkout/releases/tag/v6) |
| `actions/upload-artifact` | [`v4`](https://github.com/actions/upload-artifact/releases/tag/v4) | [`v6`](https://github.com/actions/upload-artifact/releases/tag/v6) |

[GitHub announced](https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/) that Node 20 is being deprecated and runners will begin using Node 24 by default starting 2026-03-04.

**Node 20 EOL**: April 2026
**Node 24 becomes default on Github Actions**: 2026-03-04

Reviewed By: bigfootjon

Differential Revision: D89332222

fbshipit-source-id: b8793f6a7a2aaac6f93cd8419e9afa49c5489bee
2025-12-17 13:42:14 -08:00
Alex Hornby
b24bf1ca2a getdeps: update github windows runner and internal CI to VS 2022
Summary:
X-link: https://github.com/facebookincubator/fizz/pull/163

The github [windows-2019 actions image was retired by github](https://github.com/actions/runner-images/issues/12045),  so all jobs on it fail

Update to windows-2022 to get them running again

windows-2022 has [different tools and versions than windows-2019](https://github.com/actions/runner-images/issues/3949).  Notably it moves from Visual Studio 2019 (aka msvc 16.x) to Visual Studio 2022 (aka msvc 17.x), hence the update to buildopts.py discovery

In the course of regenerating the github actions I also fixed a couple of issues that stopped regeneration matching repo contents
  * a few workflows were using workflow_dispatch, added support
  * there were a trailing and double spaces for project_prefix, fixed (use ignore whitespace to remove this from review!)

Reviewed By: bigfootjon, yfeldblum

Differential Revision: D78019509

fbshipit-source-id: f8b0e9438bfc6b481b4207ad82bc1002e496a2d9
2025-07-09 22:23:08 -07:00
Alex Hornby
e4cfb0631a preserve http_proxy in generated github actions sudo invocatons
Summary:
X-link: https://github.com/facebookincubator/zstrong/pull/1102

make it possible to run the generated github actions locally with [act](https://github.com/nektos/act) when behind a proxy by preserving http_proxy in generated github actions sudo invocatons

also updated to consistently write `apt-get` instead of the interactive `apt` (we had a mix)

example usage (substitute myproxy for your actual proxy):
```
mkdir $HOME/act-artifacts
cd ~/fizz
time act -r -j build -W .github/workflows/getdeps_linux.yml --artifact-server-path=$HOME/act-artifacts --artifact-server-addr=127.0.0.1 --cache-server-addr=127.0.0.1 --env http_proxy=http://myproxy:8080 --env https_proxy=myproxy:8080 --action-offline-mode
```

Reviewed By: bigfootjon

Differential Revision: D67864689

fbshipit-source-id: 45695675ca5672e56104c8c33803afe004ff98da
2025-01-07 02:43:19 -08:00
Alex Hornby
10343e2952 add github actions caching
Summary:
X-link: https://github.com/facebook/sapling/pull/1005

Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps.

Cached fizz windows build completed in 22% of original time,  linux build in 54% of the original time,  and mac in 72% of original time.  This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one.  Speed ups will vary across the different Meta OSS projects.

Windows benefits most from caching dependencies as we don't use system packages there at all,  linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners).

Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity.  You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc),  looks like our usage is pretty small so far.

More background:
Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys.

Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build.  This allows us to benefit from caching on main and PRs where dependency versions match.

The final item being built, and dependencies inside the same repo as the final are not cached.  A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output).  This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file.

When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708

Reviewed By: bigfootjon

Differential Revision: D67839730

fbshipit-source-id: c384a216eb27ccd3f816e3c31b167232bda571a6
2025-01-06 15:15:49 -08:00
Alex Hornby
8b7a794f1d skip unnecessary github actions steps
Summary:
X-link: https://github.com/facebookincubator/zstrong/pull/1100

Update generated github actions to only run the fetch and and build steps when there are sources expected for a manifest

For local github actions testing using `act` this speeds up the test runs, and in real github CI it makes it clearer which steps are actually doing something on the given runner (we don't know exactly what it has installed beforehand)

Also set the windows git config the same as on internal CI

Reviewed By: bigfootjon

Differential Revision: D67839708

fbshipit-source-id: 0a60c6fc89e8c6abb2464f879459aa23d5aec969
2025-01-06 11:16:30 -08:00
Alex Hornby
6bd25ff775 regenerate github actions
Summary: regenerate the github actions before making any real changes

Reviewed By: bigfootjon

Differential Revision: D67839617

fbshipit-source-id: 87dd1bf4e5923a85b9e2078f8c9144b1da0643dd
2025-01-06 11:16:30 -08:00
Alex Hornby
427d8ec34a regenerate github actions to update actions/upload-artifact version
Summary:
X-link: https://github.com/facebook/sapling/pull/946

Fix github jobs [failing with errors](https://github.com/facebook/mvfst/actions/runs/10855890595/job/30129430199)  like `Error: This request has been automatically failed because it uses a deprecated version of `actions/upload-artifact: v2`. Learn more: https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/`

Regenerated actions with: `./opensource/fbcode_builder/getdeps/facebook/update-all-github-actions.sh`

This also shows some changes from D62399390 as actions weren't fully regenerated on that diff.

Reviewed By: JakobDegen

Differential Revision: D62685074

fbshipit-source-id: b6735022e7f58274b3f78f289b72ae7b2c8d9d7b
2024-09-14 15:42:11 -07:00
Yedidya Feldblum
3a05dda2fb bump repo ci to ubuntu-22.04
Summary:
X-link: https://github.com/facebook/folly/pull/2189

X-link: https://github.com/facebookincubator/zstrong/pull/782

Now that Ubuntu 24.04 LTS [has been released](https://ubuntu.com/blog/canonical-releases-ubuntu-24-04-noble-numbat), it is a suitable time to bump the version of Ubuntu used in CI to Ubuntu 22.04 LTS, which is the prior LTS.

Ubuntu 22.04 LTS ships with GCC 11.2, and is the first Ubuntu release to ship with a version of GCC that implements C++20 coroutines.

Reviewed By: chadaustin

Differential Revision: D57017204

fbshipit-source-id: ce5754e7dfc6cb066739bf164e725de8e21f8d24
2024-05-07 09:51:01 -07:00
Alex Hornby
71dd653581 regenerate workflows
Summary: Regenerate github workflows to pick up new folly deps

Reviewed By: zpao

Differential Revision: D56698744

fbshipit-source-id: 7b0b72d47351bd9fd9afc9f97d2369f79934d584
2024-04-30 10:26:32 -07:00
Joseph Beshay
a55625f508 update github workflow for windows getdeps build
Summary: As title. This syncs the windows github actions with the latest getdeps changes.

Reviewed By: hanidamlaj

Differential Revision: D54863503

fbshipit-source-id: 673e45e3788f7848f3926d77a2801bd6d2a53537
2024-03-13 16:16:14 -07:00
Alex Hornby
cb42b717d0 use system deps in github CI for mac and linux (#330)
Summary:
use system deps in github CI for mac and linux

speed up the github CI by not rebuilding boost every time

Regenerated tha actions with
```
./build/fbcode_builder/getdeps.py generate-github-actions --allow-system-packages --src-dir=. --os-type=darwin --output-dir=.github/workflows mvfst
./build/fbcode_builder/getdeps.py generate-github-actions --allow-system-packages --src-dir=. --os-type=linux --output-dir=.github/workflows mvfst
```

Pull Request resolved: https://github.com/facebook/mvfst/pull/330

Test Plan:
CI

 ---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/mvfst/pull/330).
* __->__ https://github.com/facebook/mvfst/issues/330
* https://github.com/facebook/mvfst/issues/315

Reviewed By: hanidamlaj

Differential Revision: D54860939

Pulled By: jbeshay

fbshipit-source-id: 8d30fcb83e31c583e75a10152293d5b83352a9a5
2024-03-13 14:40:50 -07:00
Chad Austin
f580811442 upgrade getdeps GitHub actions to Ubuntu 20 (#516)
Summary:
X-link: https://github.com/facebook/fbthrift/pull/516

X-link: https://github.com/facebook/watchman/pull/1050

X-link: https://github.com/facebook/proxygen/pull/426

X-link: https://github.com/facebook/folly/pull/1842

X-link: https://github.com/facebook/fboss/pull/117

Sadly, even though Ubuntu 18.04 is still in LTS, GitHub is deprecating
its runner image.

Migrate the generated GitHub Actions to 20.04.

https://github.com/actions/runner-images/issues/6002

https://github.blog/changelog/2022-08-09-github-actions-the-ubuntu-18-04-actions-runner-image-is-being-deprecated-and-will-be-removed-by-12-1-22/

Reviewed By: fanzeyi

Differential Revision: D38877286

fbshipit-source-id: 85f3324d6666eacb190a43985585b438de69d545
2022-08-22 10:05:10 -07:00
Zeyi (Rice) Fan
ceb2c01788 disable autocrlf on Windows jobs
Summary:
In D36905191 (c7613aa43c), getdeps gained the ability to apply patches on Windows.

However, on GitHub Action workers, `core.autocrlf` is turned on by default. This means the patch files we checked in will end up having CRLF endings while the source code we downloaded are still in LF endings, and `git apply` doesn't like that.

This diff teaches getdeps to disable `core.autocrlf` in GitHub Action Windows workers.

Reviewed By: vitaut

Differential Revision: D37008387

fbshipit-source-id: 34c9f89e8783d0613040d190c4ad3477bd7bfd53
2022-06-08 13:18:38 -07:00
Chad Austin
235780b3c9 update all Windows jobs to windows-2019 (#1714)
Summary:
Pull Request resolved: https://github.com/facebook/folly/pull/1714

windows-2016 will be removed in March, and now that we've updated
Boost, we can upgrade to windows-2019.

Reviewed By: genevievehelsel

Differential Revision: D34119171

fbshipit-source-id: caad432bebf6b8282844bf79f82b1fbc34f2df9e
2022-02-09 20:02:13 -08:00
Alex Hornby
857d039445 regenerate github actions
Summary: Fix the getdeps.py  generation sub command and regenerate github actions before making changes to support actions generation for eden_scm

Reviewed By: fanzeyi

Differential Revision: D34044243

fbshipit-source-id: 0039d04f25af4c842145dc142dae6b9996fc8046
2022-02-08 03:01:35 -08:00
Chad Austin
dbed033064 update to checkout@v2
Summary:
checkout@v2 is the current action version, and the old one may have
security issues. Let's just run with the latest version.

Reviewed By: ahornby

Differential Revision: D33834847

fbshipit-source-id: fba0733577bf23c208c9824aa3815bd01a813d9a
2022-02-01 09:51:05 -08:00
Chad Austin
2d9d06f988 update all github action workflows
Summary:
I'm about to upgrade from checkout v1 to checkout v2. First, update
the GitHub action workflows for all projects.

Reviewed By: ahornby

Differential Revision: D33834784

fbshipit-source-id: f4cba24f1a589588dbfaf7ad7ada7122dd8284cb
2022-02-01 09:51:05 -08:00
Alex Hornby
8170b09078 change to upload-artifact@v2
Summary: Update to new version of upload-artifact without branch name as that's a separate repo that our dynamic branch settings should not apply to.

Reviewed By: zpao

Differential Revision: D32717494

fbshipit-source-id: 968908d1072d6d4550705e7df0b3828528e057dc
2021-12-01 01:18:02 -08:00
Konstantin Tsoy
77b5497a1a Proper branch name for mvfst
Summary:
Need to fix branch name for OSS mvfst so CI continues on.

Ran opensource/fbcode_builder/getdeps/facebook/update-all-github-actions.sh

Reviewed By: lnicco

Differential Revision: D32041656

fbshipit-source-id: f72aab91a3084814a5d3fa8643ae5f405908c313
2021-11-02 15:37:58 -07:00
Shai Szulanski
aa619fc122 Regen github actions (#1614)
Summary: Pull Request resolved: https://github.com/facebook/folly/pull/1614

Reviewed By: yfeldblum

Differential Revision: D29496725

fbshipit-source-id: 7c726e3e310eb28cf33603ebd83df6d832488369
2021-06-30 13:58:26 -07:00
Luca Niccolini
7b59115b1a github actions: build tests but don't run them
Reviewed By: mjoras

Differential Revision: D27305676

fbshipit-source-id: f84e6b30010d8589d7101c73e23aac323fc58764
2021-03-24 14:08:32 -07:00
Luca Niccolini
6f63fa59ca disable --allow-system-packages for generate-github-actions
Summary: Fix github actions builds for projects depending on zstd

Reviewed By: wez

Differential Revision: D26743251

fbshipit-source-id: a3fd8a14750227d025bff763cca8aa683b26a422
2021-03-03 07:26:28 -08:00
Zeyi (Rice) Fan
1a69536923 GitHub Action: fix Windows build
Summary:
Watchman's GitHub Action has been failing because of the use of `set-env` is deprecated due to security issues. See https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/ for details.

See https://github.com/facebook/watchman/runs/1520165260?check_suite_focus=true for example of failures.

To fix it we just need to write to the special environment file.

Reviewed By: chadaustin

Differential Revision: D25412442

fbshipit-source-id: d31ef025189c8aebfa08d434d9373f5da7d4690e
2020-12-09 17:48:09 -08:00
Zeyi (Rice) Fan
d71d6fa1d9 regenerate all GitHub Actions
Reviewed By: chadaustin

Differential Revision: D25430913

fbshipit-source-id: 3bdef6346788deb3977167b89affb5a388a7d739
2020-12-09 17:48:09 -08:00
Luca Niccolini
80c6af1155 update github action workflows
Summary:
- googletest version
- disable running tests for linux and mac
- disable copy artifact for mac

generated with
```
opensource/fbcode_builder/getdeps.py generate-github-actions --output-dir proxygen/public_tld/.github/workflows/ proxygen
opensource/fbcode_builder/getdeps.py generate-github-actions --output-dir fizz/public_tld/.github/workflows/ fizz
opensource/fbcode_builder/getdeps.py generate-github-actions --output-dir quic/public_root/.github/workflows/ mvfst
```

plus some manual edits

Reviewed By: afrind

Differential Revision: D23707251

fbshipit-source-id: 7e189bf482229c1ce1c95a33583dc3153eb00ff5
2020-09-15 09:24:22 -07:00
Zeyi (Rice) Fan
eb9a7ee955 update GitHub actions
Summary:
The GitHub actions have been failing because `rsocket-cpp` has been deleted.

https://github.com/facebook/watchman/runs/843777632

This commit is generated with `./opensource/fbcode_builder/getdeps/facebook/update-all-github-actions.sh`

Reviewed By: wez

Differential Revision: D22417304

fbshipit-source-id: e64a8d011c753bf8ac90ed2c9c90036baa8cf950
2020-07-07 19:36:30 -07:00
Wez Furlong
bbd7e74dd9 getdeps: GH actions: strip artifacts before capturing them (#809)
Summary:
On Linux the debug info sections in projects downstream from folly and
thrift are huge due to a lot of C++ template usage.

We build in RelWithDebInfo mode as that is most useful when debugging
locally and because the build times are long, but most casual consumers
of the binaries via GH actions really don't care about debug info,
and this should save ~180MB of size in (for example) the watchman
executables.

Pull Request resolved: https://github.com/facebook/watchman/pull/809
Refs: https://github.com/facebook/watchman/issues/804

Test Plan:
Review the artifact size on the linux build in this PR (see
https://github.com/facebook/watchman/actions/runs/91688952) and confirm that it
is a bit smaller than 180MB; now under 3 MB.

Reviewed By: simpkins

Differential Revision: D21318302

Pulled By: wez

fbshipit-source-id: f78bc5e735dd78771e0604abae64c0a23cf9158d
2020-04-29 19:09:23 -07:00
Wez Furlong
127ff7495f getdeps: fix artifact generation (#808)
Summary:
The mismatched project output settings were causing
the dyndeps fixup to fail to find any objects and thus none of
the artifacts are populated for posix platforms.

Pull Request resolved: https://github.com/facebook/watchman/pull/808

Test Plan:
The PR published artifacts to:

* https://github.com/facebook/watchman/actions/runs/91617994
* https://github.com/facebook/watchman/actions/runs/91617996
* https://github.com/facebook/watchman/actions/runs/91617997

Reviewed By: simpkins

Differential Revision: D21315090

Pulled By: wez

fbshipit-source-id: 60461809f55e73119f7206e3f392d5b237722f85
2020-04-29 18:45:27 -07:00
Wez Furlong
465ca12e30 getdeps: split monolithic GH actions CI spec up (#1360)
Summary:
Rather than have a single main.yml file containing all off the different
builds, split that up so that we have one file per build environment
(linux, mac, windows).

This has a couple of advantages:

* It is quicker to see the status of just one of the platforms
* Artifact collection for one platform is not blocked pending completion
  of the builds for all systems
* It's a little easier to understand what is happening for a single platform

To support having multiple files I've changed the output-file option to
be an output-dir.

I've included the rm of main.yml in this commit.  Once this gets
imported back to the FB internal system I'll amend in an update to
the helper script that updates all of our opensource builds and run
and amend that.
Pull Request resolved: https://github.com/facebook/folly/pull/1360

Test Plan:
the GH action status on this PR should show three different
actions running, one for each platform.

I updated and ran
`fbcode/opensource/fbcode_builder/getdeps/facebook/update-all-github-actions.sh`
to regenerate all the actions files for FB.

Reviewed By: yfeldblum

Differential Revision: D21310991

Pulled By: wez

fbshipit-source-id: 604ef652c8f746781a4b410c6b996cdee4524e0d
2020-04-29 15:40:46 -07:00
Wez Furlong
a82367fe2c GH actions: fixup boost windows environment (#1359)
Summary:
The environment changed since I tested  D20740410 and now it appears
that we'll need to re-export a versioned variable in order for cmake
to detect boost on the GH actions hosts.
Pull Request resolved: https://github.com/facebook/folly/pull/1359

Test Plan:
the GH actions status of this diff.

The workflow was updated via:

```
python3 build/fbcode_builder/getdeps.py generate-github-actions folly --output-file .github/workflows/main.yml
```

Reviewed By: yfeldblum

Differential Revision: D21307640

Pulled By: yfeldblum

fbshipit-source-id: 1555cbcade822775379cd9054be37fdbc17b4d93
2020-04-29 13:11:48 -07:00
Wez Furlong
05d15ad981 getdeps: allow satisfying deps from system packages
Summary:
From the outset, we wanted to be sure that getdeps was able
to source and build the dependencies so that we knew that we'd have
a repeatable build.  This came at the cost of build times: having
to build boost on each CI run is a bit of a chore.

This commit adds three new elements to the manifest files:

* `rpms` - a list of RPM names that are all required to be present
  in order to consider the dependency satisfied
* `debs` - like `rpms` above, but scoped to debian package names
* `preinstalled.env` - a list of environment variables that if they
  are all set and non-empty will satisfy the dependency.

A new `--allow-system-packages` option to getdeps enables the new
logic that looks for system packages; it is off by default, but
enabled in the generated GitHub Actions workflows.

A new `install-system-deps` subcommand is provided that will attempt
to install the system packages needed to satisfy the build.  This
typically needs to be run via sudo and is thus broken out separately
from the main getdeps build flow.

I made a pass over the manifest files and added package names that
satisfy the build on ubuntu-18 and fedora-31.

shri-khare: I renamed the `Python3.7.6` manifest to just `python` as
part of this change; the version of python that it pulls in through
the normal build is the same and I believe that an equal or newer
version of python3 is available in the GH actions builder.

The `preinstalled.env` is used only by the boost manifest: it references
the name of an environment variable that is set by the github
windows hosts and that points to a pre-built and pre-installed
copy of boost.  Since there is no package manager that we can
easily query for this sort of thing, probing from the environment
seems like a reasonable and fast way to check for this.  We
may need to evolve this over time to become more feature rich,
but this seems like a good starting point.

This commit has the potential to save 20 minutes of build time
from each public CI build just due to the boost dependency alone!

Refs: https://github.com/facebook/watchman/pull/797

Reviewed By: yfeldblum

Differential Revision: D20740410

fbshipit-source-id: 6c38019449c54465127656c3d18a6ff1f30adaea
2020-04-29 11:18:20 -07:00
Wez Furlong
70d8d13d0f getdeps: allow setting per-project install prefix for DESTDIR installs
Summary:
We have a global `--install-prefix` argument that can be used to set
the prefix for all projects, but that is only suitable if you are running with
sufficient privileges to install each of the deps to that location during the
build.  Cmake dependency resolution won't work from the build directory in that
situation; it can only see the final installed location and it will error out
if those files are not present, or link against the currently installed version
instead of the version we just built; not great!

This commit adds a project specific `--project-install-prefix` that can be used
on just the leaf project in a set of deps.  That sidesteps the dependency
concern because only the last stage is built in that mode.  This option
can technically be applied to an arbitrary set of projects, but in light
of the above, in practice it only makes sense to use it for the final
cmake project.  Only the CMakeBuilder respects this option.

In the watchman repo, this commit adjusts the autogen.sh script to allow
specifying the installation prefix; it defaults to `/usr/local` as you
might expect.

refs: https://github.com/facebook/watchman/issues/760

Reviewed By: yfeldblum

Differential Revision: D20674439

fbshipit-source-id: 52799dbd47f3c295e2d6469ee2b74cedeaa20138
2020-03-31 12:11:25 -07:00
Wez Furlong
3d01962493 getdeps: fixup python3 usage on GitHub Actions CI
Summary:
My recent change to ensure that we were using python3 to launch everything failed on windows: the GH actions environment has `python.exe` in the path and it is python version 3.  There is no such thing as `python3` in that environment, although there is a `python2`.

Refs: https://github.com/facebook/watchman/pull/797

Reviewed By: chadaustin

Differential Revision: D20740411

fbshipit-source-id: 0e40590ccedc18e327ebb84901e2509588fdd0ff
2020-03-30 12:53:07 -07:00
Wez Furlong
4b6bb62b15 getdeps: fix git configuration on windows
Summary:
The GH actions defaults for git prevent it from being able
to checkout the fbthrift repo due the length of the java related
files in the fbthrift repo.

This commit tells git to use long filenames and allows the checkout
to succeed.

Reviewed By: fanzeyi

Differential Revision: D20659750

fbshipit-source-id: 060b36d312d52a0769cf2f2dd9af60f7446f94a8
2020-03-26 05:51:03 -07:00
Wez Furlong
4f230ad30d getdeps: embrace python3 a bit more
Summary:
Ensure that we are referencing python3 in the paths
that we generate for the github actions workflows, and remove
some shebangs that influence how our internal linters process
the python code.

Reviewed By: fanzeyi

Differential Revision: D20659747

fbshipit-source-id: 6f300f8e91edf7701bb27babc7b1418958cf0a10
2020-03-26 05:46:21 -07:00
Luca Niccolini
f4b4ec6d7e windows build
Summary: ^

Reviewed By: mjoras

Differential Revision: D18256970

fbshipit-source-id: 4d07b961ab5ab145b620d9e10fa67116236bbf80
2019-10-31 19:03:58 -07:00
Luca Niccolini
40b808d09a Github workflows (#50)
Summary:
workflows-based build
Pull Request resolved: https://github.com/facebookincubator/mvfst/pull/50

Reviewed By: mjoras

Differential Revision: D17733986

Pulled By: lnicco

fbshipit-source-id: c0f5639c7e7de961d11760598ead752f74018532
2019-10-03 09:18:46 -07:00