1
0
mirror of https://github.com/facebook/proxygen.git synced 2025-08-10 05:22:59 +03:00

Enable ccache to speed up Travis Docker-based builds 3-4x

Summary:
Unfortunately, Travis is constantly in a capacity crunch. Plus, building `folly` + `wangle` + `proxygen` + `thrift` takes upwards of 40 minutes on Travis hosts, and the timeout for builds is 50 minutes. That creates several problems:
 - Builds tend to get queued up, and sometimes take a long time to produce signal.
 - Adding more platform, or project, or dependency coverage to our Travis builds would just make the problem worse (so I have not pushed much on the adoption of `fbcode_builder`).
 - Building a service of any complexity will likely pull in all of the above dependencies, and then it gets 10 minutes for build & tests. So higher-up-the-foodchain projects like `bistro` just time out.

To help accelerate builds, Travis supports caching build artifacts between runs: https://docs.travis-ci.com/user/caching/#Arbitrary-directories
Each build variant gets its own cache, as one would want: https://docs.travis-ci.com/user/caching/#Caches-and-build-matrices

This is great for us because -- especially with all the `Updating submodules` commits -- we actually keep rebuilding nearly the same tree, most of the time.

Better yet, https://ccache.samba.org provides transparent compile caching -- just set your cache directory (and a few options), and use `ccache {gcc,g++}` instead of the vanilla compiler. It does the rest, and is quite robust -- the only issues I had were:
 - different platforms have different versions, so one has to target ccache 3.1-ish,
 - the max cache size started out too small, which tended to eliminate all speedup.

In this diff, I enable `ccache` for all projects using the default `.travis.yml` (jstrizich, you'll need to patch the custom `.travis.yml` files of `openr` and `fbzmq` if you want to try caching).

Notably, even if a project would time out on Travis without `ccache`, the caching accumulates across minor revisions, and should keep re-build times quite short for most changes.

This also significantly improves the experience of iterating on a Docker-based build on your own machine -- thoughtful use of Docker layer caching & ccache dramatically speeds up rebuilds after certain changes.

It works like this:
 - Start with an empty `ccache.tgz`, or a pre-existing one provided by Travis from its cache.
 - The `Dockerfile` extracts it into the Docker image under `/ccache`, and sets some environment variables to enable ccache for C & C++ builds.
 - The build runs as usual.
 - The build either finishes successfully, or it fails, or the new option of `docker_build_timeout` kills the container about 10 minutes before Travis would time out the build.
 - A small script runs inside the container to prepare the updated `ccache.tgz`. This includes automatic cache sizing based on the working set size of a full successful build.
 - Travis uploads `ccache.tgz` to S3 for the next build.

The rest is plumbing, CLI tweaks, and documentation to make manual iteration using `ccache` fairly pleasant.

The overhead of my `ccache` implementation is hard to measure exactly, but based on my indirect measurements, it's on the order of 2 minutes for `bistro`, which would otherwise take 52+ minutes to build on Travis.

Reviewed By: zpao

Differential Revision: D5648461

fbshipit-source-id: 207b621ba6bc6f827b20a5746b4dc4f28c1c96e2
This commit is contained in:
Alexey Spiridonov
2017-08-21 16:43:01 -07:00
committed by Facebook Github Bot
parent 999b9eb4a7
commit 4d5d194635
7 changed files with 357 additions and 8 deletions

View File

@@ -241,8 +241,13 @@ class FBCodeBuilder(object):
'apt-get upgrade -yq cmake'
)))
actions.extend(self.debian_ccache_setup_steps())
return self.step('Install packages for Debian-based OS', actions)
def debian_ccache_setup_steps(self):
raise [] # It's ok to ship a renderer without ccache support.
def github_project_workdir(self, project, path):
# Only check out a non-default branch if requested. This especially
# makes sense when building from a local repo.