Summary:
In 0ae204a978c11ddefafd81bd319a078239a44c1c the 'projects_dir' option
became a required constructor argument since it is called within the
constructor. However, it has not been adjusted in the subclasses that
used to set the option after instantiation. This commit fixes the
'shell_builder' and the 'debian_system_builder'.
Pull Request resolved: https://github.com/facebook/openr/pull/50
Test Plan:
1. Go to build directory: `cd build`
2. Run the `shell_builder` & `debian_system_builder`:
- `python fbcode_builder/shell_builder.py`
- `python debian_system_builder/debian_system_builder.py`
`shell_builder` output before:
```
Traceback (most recent call last):
File "fbcode_builder/shell_builder.py", line 102, in <module>
builder = ShellFBCodeBuilder()
File "/home/butjar/tu/ma/openr/build/fbcode_builder/fbcode_builder.py", line 93, in __init__
self._github_dir = self.option('projects_dir')
File "/home/butjar/tu/ma/openr/build/fbcode_builder/fbcode_builder.py", line 108, in option
raise RuntimeError('Option {0} is required'.format(name))
RuntimeError: Option projects_dir is required
```
`shell_builder` output after:
```
set -exo pipefail
export CCACHE_DIR='/home/butjar/.fbcode_builder-sZshomesZsbutjarsZstusZsmasZsopenr/.ccache' CC="ccache ${CC:-gcc}" CXX="ccache ${CXX:-g++}"
### Diagnostics ###
# Builder ShellFBCodeBuilder(google/googletest:cmake_defines={u'BUILD_GTEST': u'ON', u'BUILD_SHARED_LIBS': u'OFF'}, google/googletest:git_hash=u'release-1.8.1', facebook/openr:local_repo_dir='/home/butjar/tu/ma/openr', facebook/zstd:git_hash=ShellQuoted(u'$(git describe --abbrev=0 --tags origin/master)'), openr/build:cmake_defines={u'ADD_ROOT_TESTS': u'OFF'}, thom311/libnl:git_hash=u'libnl3_2_25', projects_dir=u'/home/butjar/.fbcode_builder-sZshomesZsbutjarsZstusZsmasZsopenr', fmtlib/fmt:git_hash=u'5.3.0', wangle/wangle/build:cmake_defines={u'BUILD_TESTS': u'OFF'}, prefix=u'/home/butjar/.fbcode_builder-sZshomesZsbutjarsZstusZsmasZsopenr/installed', fizz/fizz/build:cmake_defines={u'BUILD_TESTS': u'ON'}, ccache_dir=u'/home/butjar/.fbcode_builder-sZshomesZsbutjarsZstusZsmasZsopenr/.ccache', zeromq/libzmq:git_hash=u'v4.2.2', make_parallelism=4, jedisct1/libsodium:git_hash=u'stable')
hostname
cat /etc/issue || echo no /etc/issue
g++ --version || echo g++ not installed
cmake --version || echo cmake not installed
### Check out fmtlib/fmt, workdir build ###
mkdir -p '/home/butjar/.fbcode_builder-sZshomesZsbutjarsZstusZsmasZsopenr' && cd '/home/butjar/.fbcode_builder-sZshomesZsbutjarsZstusZsmasZsopenr'
git clone https://github.com/'fmtlib/fmt'
mkdir -p '/home/butjar/.fbcode_builder-sZshomesZsbutjarsZstusZsmasZsopenr'/'fmt'/'build' && cd '/home/butjar/.fbcode_builder-sZshomesZsbutjarsZstusZsmasZsopenr'/'fmt'/'build'
git checkout '5.3.0'
### Build and install fmtlib/fmt ###
...
```
Reviewed By: steven1327
Differential Revision: D21865881
Pulled By: saifhhasan
fbshipit-source-id: dfd78127d3b2c78721f84a3ecafe0b7198c38f06
Summary:
This should fix the Travis CI builds. It adds rust toolchain support inside docker and sets the required THRIFT env variable.
Pull Request resolved: https://github.com/facebookexperimental/rust-shed/pull/3
Reviewed By: krallin
Differential Revision: D18905608
Pulled By: lukaspiatkowski
fbshipit-source-id: 5db1eff6f215a6617d8acaa0c99a62d45225956b
Summary:
This is an update to the fbcode_builder codebase to allow setting up the python virtualenv with python dependencies installed. I've included wheel and cython (with a pinned version to 0.28.6 which is the only version that works with thriftpy3 at the moment, due to https://github.com/cython/cython/issues/2985) as standard packages since these are required by some of our top-level dependencies (folly and thrift)
As far as I know, there are no other projects that use PYTHON_VENV at the moment except LogDevice so the impact should be minimal.
Reviewed By: lucaspmelo
Differential Revision: D18758383
fbshipit-source-id: 264941311c5e3a19dc4ef2bb78c9a1baa34dfd8c
Summary:
Needs to be enabled by option PYTHON_VENV in the config.
shell_builder.py sets up the venv and uses it once; calling activate
For docker we set ENV; resulting in the virtual environment being present
when the resulting container is run as well as at build time. This is also cleaner
and easier to follow than re-asserting on each RUN step.
For Lego builder we need to source activate on each command as environment
will not persist between commands.
While man on the posts say it makes no sense to use virtualenv within docker
container, this method simplifies the process considerably as we can rely on the
name pip being valid and we don't need to either ensure we are root or pass the
--user flag to pip and setuptools.
Pull Request resolved: https://github.com/facebookincubator/LogDevice/pull/76
Reviewed By: wez
Differential Revision: D14875633
Pulled By: calebmarchent
fbshipit-source-id: aabbcdd509d2a59fa36f8004032a052f014ce1ba
Summary:
These were failing in the watchman build
```
$ cd build && python fbcode_builder/shell_builder.py > ~/run.sh && bash ~/run.sh
Traceback (most recent call last):
File "fbcode_builder/shell_builder.py", line 110, in <module>
steps = make_steps(builder)
File "/home/travis/build/facebook/watchman/build/fbcode_builder/utils.py", line 94, in <lambda>
steps_for_spec(builder, config['fbcode_builder_spec'](builder))
File "/home/travis/build/facebook/watchman/build/fbcode_builder/fbcode_builder.py", line 144, in build
return [self.setup(), self.diagnostics()] + steps
File "fbcode_builder/shell_builder.py", line 61, in setup
).format(ccache_dir=ccache_dir)
File "/home/travis/build/facebook/watchman/build/fbcode_builder/shell_quoting.py", line 64, in format
(k, shell_quote(v).do_not_use_raw_str) for k, v in kwargs.items()
KeyError: u'CC'
```
Reviewed By: snarkmaster
fbshipit-source-id: 614723e631a82f277739765a920731c872700c45
Summary:
`shell_builder.py` allows running the fbcode_builder logic
on the host rather than in a container.
It emits a bash script with `set -exo pipefail` configured such that
any failing step will cause the script to exit with failure.
Refs: https://github.com/facebook/watchman/pull/639
Reviewed By: simpkins
Differential Revision: D9552411
fbshipit-source-id: c7835deedf07ea342dcb3de61d576a4fb5439985
