mirror of
https://github.com/facebook/zstd.git
synced 2025-08-01 09:47:01 +03:00
Publish artifacts with CircleCI
* Updates CircleCI to use workflows. We can now specify any number of test jobs to run in parallel. * Switch the image to `buildpack-deps:trusty` which is only 500 MB instead of 7 GB, so that saves 7 minutes to download it if it isn't already cached on the host. * Publish the source tarball and sha256sum as artifacts. * If the `GITHUB_TOKEN` environment variable is set, we will also add the tarball + sha256sum to the tagged release, after manual approval.
This commit is contained in:
Nick Terrell
@ -1,108 +1,111 @@
|
|||||||
# This configuration was automatically generated from a CircleCI 1.0 config.
|
|
||||||
# It should include any build commands you had along with commands that CircleCI
|
|
||||||
# inferred from your project structure. We strongly recommend you read all the
|
|
||||||
# comments in this file to understand the structure of CircleCI 2.0, as the idiom
|
|
||||||
# for configuration has changed substantially in 2.0 to allow arbitrary jobs rather
|
|
||||||
# than the prescribed lifecycle of 1.0. In general, we recommend using this generated
|
|
||||||
# configuration as a reference rather than using it in production, though in most
|
|
||||||
# cases it should duplicate the execution of your original 1.0 config.
|
|
||||||
version: 2
|
version: 2
|
||||||
|
|
||||||
|
references:
|
||||||
|
# Install the dependencies required for tests.
|
||||||
|
# Add the step "- *install-dependencies" to the beginning of your job to run
|
||||||
|
# this command.
|
||||||
|
install-dependencies: &install-dependencies
|
||||||
|
run:
|
||||||
|
name: Install dependencies
|
||||||
|
# TODO: We can split these dependencies up by job to reduce installation
|
||||||
|
# time.
|
||||||
|
command: |
|
||||||
|
sudo dpkg --add-architecture i386
|
||||||
|
sudo apt-get -y -qq update
|
||||||
|
sudo apt-get -y install \
|
||||||
|
gcc-multilib-powerpc-linux-gnu gcc-arm-linux-gnueabi \
|
||||||
|
libc6-dev-armel-cross gcc-aarch64-linux-gnu libc6-dev-arm64-cross \
|
||||||
|
libc6-dev-ppc64-powerpc-cross
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
build:
|
# the first half of the jobs are in this test
|
||||||
working_directory: ~/facebook/zstd
|
short-tests-0:
|
||||||
parallelism: 1
|
# TODO: Create a small custom docker image with all the dependencies we need
|
||||||
shell: /bin/bash --login
|
# preinstalled to reduce installation time.
|
||||||
# CircleCI 2.0 does not support environment variables that refer to each other the same way as 1.0 did.
|
docker:
|
||||||
# If any of these refer to each other, rewrite them so that they don't or see https://circleci.com/docs/2.0/env-vars/#interpolating-environment-variables-to-set-other-environment-variables .
|
- image: circleci/buildpack-deps:bionic
|
||||||
|
steps:
|
||||||
|
- checkout
|
||||||
|
- *install-dependencies
|
||||||
|
- run:
|
||||||
|
name: Test
|
||||||
|
command: |
|
||||||
|
cc -v; CFLAGS="-O0 -Werror" make all && make clean
|
||||||
|
make c99build ; make clean
|
||||||
|
make c11build ; make clean
|
||||||
|
make aarch64build ; make clean
|
||||||
|
make -j regressiontest; make clean
|
||||||
|
make shortest ; make clean
|
||||||
|
make cxxtest ; make clean
|
||||||
|
# the second half of the jobs are in this test
|
||||||
|
short-tests-1:
|
||||||
|
docker:
|
||||||
|
- image: circleci/buildpack-deps:bionic
|
||||||
|
steps:
|
||||||
|
- checkout
|
||||||
|
- *install-dependencies
|
||||||
|
- run:
|
||||||
|
name: Test
|
||||||
|
command: |
|
||||||
|
make gnu90build; make clean
|
||||||
|
make gnu99build; make clean
|
||||||
|
make ppc64build; make clean
|
||||||
|
make ppcbuild ; make clean
|
||||||
|
make armbuild ; make clean
|
||||||
|
make -C tests test-legacy test-longmatch test-symbols; make clean
|
||||||
|
make -C lib libzstd-nomt; make clean
|
||||||
|
# This step is only run on release tags.
|
||||||
|
# It publishes the source tarball as artifacts and if the GITHUB_TOKEN
|
||||||
|
# environment variable is set it will publish the source tarball to the
|
||||||
|
# tagged release.
|
||||||
|
publish-github-release:
|
||||||
|
docker:
|
||||||
|
- image: cibuilds/github:0.12.0
|
||||||
environment:
|
environment:
|
||||||
CIRCLE_ARTIFACTS: /tmp/circleci-artifacts
|
CIRCLE_ARTIFACTS: /tmp/circleci-artifacts
|
||||||
CIRCLE_TEST_REPORTS: /tmp/circleci-test-results
|
|
||||||
# In CircleCI 1.0 we used a pre-configured image with a large number of languages and other packages.
|
|
||||||
# In CircleCI 2.0 you can now specify your own image, or use one of our pre-configured images.
|
|
||||||
# The following configuration line tells CircleCI to use the specified docker image as the runtime environment for you job.
|
|
||||||
# We have selected a pre-built image that mirrors the build environment we use on
|
|
||||||
# the 1.0 platform, but we recommend you choose an image more tailored to the needs
|
|
||||||
# of each job. For more information on choosing an image (or alternatively using a
|
|
||||||
# VM instead of a container) see https://circleci.com/docs/2.0/executor-types/
|
|
||||||
# To see the list of pre-built images that CircleCI provides for most common languages see
|
|
||||||
# https://circleci.com/docs/2.0/circleci-images/
|
|
||||||
docker:
|
|
||||||
- image: circleci/build-image:ubuntu-14.04-XXL-upstart-1189-5614f37
|
|
||||||
command: /sbin/init
|
|
||||||
steps:
|
steps:
|
||||||
# Machine Setup
|
- checkout
|
||||||
# If you break your build into multiple jobs with workflows, you will probably want to do the parts of this that are relevant in each
|
- run:
|
||||||
# The following `checkout` command checks out your code to your working directory. In 1.0 we did this implicitly. In 2.0 you can choose where in the course of a job your code should be checked out.
|
name: Install dependencies
|
||||||
- checkout
|
command: |
|
||||||
# Prepare for artifact and test results collection equivalent to how it was done on 1.0.
|
apk add -q gzip coreutils
|
||||||
# In many cases you can simplify this from what is generated here.
|
- run:
|
||||||
# 'See docs on artifact collection here https://circleci.com/docs/2.0/artifacts/'
|
name: Publish
|
||||||
- run: mkdir -p $CIRCLE_ARTIFACTS $CIRCLE_TEST_REPORTS
|
command: |
|
||||||
# Dependencies
|
export VERSION=$(echo $CIRCLE_TAG | tail -c +2)
|
||||||
# This would typically go in either a build or a build-and-test job when using workflows
|
export ZSTD_VERSION=zstd-$VERSION
|
||||||
# Restore the dependency cache
|
git archive $CIRCLE_TAG --prefix $ZSTD_VERSION/ --format tar \
|
||||||
- restore_cache:
|
-o $ZSTD_VERSION.tar
|
||||||
keys:
|
gzip -9 $ZSTD_VERSION.tar
|
||||||
# This branch if available
|
sha256sum $ZSTD_VERSION.tar.gz > $ZSTD_VERSION.tar.gz.sha256sum
|
||||||
- v1-dep-{{ .Branch }}-
|
mkdir -p $CIRCLE_ARTIFACTS
|
||||||
# Default branch if not
|
cp $ZSTD_VERSION.tar.gz{,.sha256sum} $CIRCLE_ARTIFACTS
|
||||||
- v1-dep-dev-
|
- store_artifacts:
|
||||||
# Any branch if there are none on the default branch - this should be unnecessary if you have your default branch configured correctly
|
path: /tmp/circleci-artifacts
|
||||||
- v1-dep-
|
|
||||||
# This is based on your 1.0 configuration file or project settings
|
workflows:
|
||||||
- run: sudo dpkg --add-architecture i386
|
version: 2
|
||||||
- run: sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test; sudo apt-get -y -qq update
|
commit:
|
||||||
- run: sudo apt-get -y install gcc-powerpc-linux-gnu gcc-arm-linux-gnueabi libc6-dev-armel-cross gcc-aarch64-linux-gnu libc6-dev-arm64-cross
|
jobs:
|
||||||
# Save dependency cache
|
# Run the tests in parallel
|
||||||
- save_cache:
|
- short-tests-0:
|
||||||
key: v1-dep-{{ .Branch }}-{{ epoch }}
|
filters:
|
||||||
paths:
|
tags:
|
||||||
# This is a broad list of cache paths to include many possible development environments
|
only: /.*/
|
||||||
# You can probably delete some of these entries
|
- short-tests-1:
|
||||||
- vendor/bundle
|
filters:
|
||||||
- ~/virtualenvs
|
tags:
|
||||||
- ~/.m2
|
only: /.*/
|
||||||
- ~/.ivy2
|
# Only run on release tags.
|
||||||
- ~/.bundle
|
- publish-github-release:
|
||||||
- ~/.go_workspace
|
requires:
|
||||||
- ~/.gradle
|
- short-tests-0
|
||||||
- ~/.cache/bower
|
- short-tests-1
|
||||||
# Test
|
filters:
|
||||||
# This would typically be a build job when using workflows, possibly combined with build
|
branches:
|
||||||
# This is based on your 1.0 configuration file or project settings
|
ignore: /.*/
|
||||||
- run: |
|
tags:
|
||||||
if [[ "$CIRCLE_NODE_INDEX" == "0" ]] ; then cc -v; CFLAGS="-O0 -Werror" make all && make clean; fi &&
|
only: /^v\d+\.\d+\.\d+$/
|
||||||
if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make gnu90build && make clean; fi
|
|
||||||
- run: |
|
|
||||||
if [[ "$CIRCLE_NODE_INDEX" == "0" ]] ; then make c99build && make clean; fi &&
|
|
||||||
if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make gnu99build && make clean; fi
|
|
||||||
- run: |
|
|
||||||
if [[ "$CIRCLE_NODE_INDEX" == "0" ]] ; then make c11build && make clean; fi &&
|
|
||||||
if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make ppc64build && make clean; fi
|
|
||||||
- run: |
|
|
||||||
if [[ "$CIRCLE_NODE_INDEX" == "0" ]] ; then make aarch64build && make clean; fi &&
|
|
||||||
if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make ppcbuild && make clean; fi
|
|
||||||
- run: |
|
|
||||||
if [[ "$CIRCLE_NODE_INDEX" == "0" ]] ; then make -j regressiontest && make clean; fi &&
|
|
||||||
if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make armbuild && make clean; fi
|
|
||||||
- run: |
|
|
||||||
if [[ "$CIRCLE_NODE_INDEX" == "0" ]] ; then make shortest && make clean; fi &&
|
|
||||||
if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make -C tests test-legacy test-longmatch test-symbols && make clean; fi
|
|
||||||
- run: |
|
|
||||||
if [[ "$CIRCLE_NODE_INDEX" == "0" ]] ; then make cxxtest && make clean; fi &&
|
|
||||||
if [[ "$CIRCLE_NODE_TOTAL" < "2" ]] || [[ "$CIRCLE_NODE_INDEX" == "1" ]]; then make -C lib libzstd-nomt && make clean; fi
|
|
||||||
# This is based on your 1.0 configuration file or project settings
|
|
||||||
- run: echo Circle CI tests finished
|
|
||||||
# Teardown
|
|
||||||
# If you break your build into multiple jobs with workflows, you will probably want to do the parts of this that are relevant in each
|
|
||||||
# Save test results
|
|
||||||
- store_test_results:
|
|
||||||
path: /tmp/circleci-test-results
|
|
||||||
# Save artifacts
|
|
||||||
- store_artifacts:
|
|
||||||
path: /tmp/circleci-artifacts
|
|
||||||
- store_artifacts:
|
|
||||||
path: /tmp/circleci-test-results
|
|
||||||
|
|
||||||
# Longer tests
|
# Longer tests
|
||||||
#- make -C tests test-zstd-nolegacy && make clean
|
#- make -C tests test-zstd-nolegacy && make clean
|
||||||
|
|||||||
@ -13,6 +13,7 @@ import argparse
|
|||||||
import contextlib
|
import contextlib
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
|
import shlex
|
||||||
import shutil
|
import shutil
|
||||||
import subprocess
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
@ -349,11 +350,11 @@ def build(args):
|
|||||||
targets = args.TARGET
|
targets = args.TARGET
|
||||||
cc = args.cc
|
cc = args.cc
|
||||||
cxx = args.cxx
|
cxx = args.cxx
|
||||||
cppflags = [args.cppflags]
|
cppflags = shlex.split(args.cppflags)
|
||||||
cflags = [args.cflags]
|
cflags = shlex.split(args.cflags)
|
||||||
ldflags = [args.ldflags]
|
ldflags = shlex.split(args.ldflags)
|
||||||
cxxflags = [args.cxxflags]
|
cxxflags = shlex.split(args.cxxflags)
|
||||||
mflags = [args.mflags] if args.mflags else []
|
mflags = shlex.split(args.mflags)
|
||||||
# Flags to be added to both cflags and cxxflags
|
# Flags to be added to both cflags and cxxflags
|
||||||
common_flags = []
|
common_flags = []
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user