OpenE2K/rust
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
0a3eb5c508
rust/.travis.yml
Alex Crichton a09e9e9a2a ci: Don't use Travis caches for docker images 
This commit moves away from caching on Travis to our own caching on S3 for
caching docker layers between builds. Unfortunately the Travis caches have over
time had a few critical pain points:

* Caches are only updated for successful builds, meaning that if a build times
  out or fails in a different location the sucessfully-created docker images
  isn't always cached. While this makes sense as a general rule of caches it
  hurts our use cases.

* Caches are per-branch and builder which means that we don't have a separate
  cache on each release channel. All our merges go through the `auto` branch
  which means that they're all sharing the same cache, even those for merging to
  master/beta. This means that PRs which switch between master/beta will keep
  rebuilting and having cache misses.

* Caches have historically been invaliated somewhat regularly a little more
  aggressively than we'd want (I think).

* We don't always need to update the contents of the cache if the Docker image
  didn't change at all, and saving off the docker layers can sometimes be quite
  expensive.

For all these reasons this commit drops the usage of Travis's built-in caching
support. Instead our own caching is used by storing blobs to S3. Normally this
would be a very risky endeavour but we're basically priming a cache for a cache
(docker) so if we get this wrong the failure mode is longer builds, not stale
caches. We'll notice that pretty quickly and hopefully fix it!

The logic here is inserted directly into the `src/ci/docker/run.sh` script to
download an image based on a shasum of the `Dockerfile` and other assorted files.
This blob, if found, is loaded into docker and we record what layers were
inserted. After docker finishes the build (hopefully quickly with lots of cache
hits) we then see the sha of the final image. If it's one of the layers we
loaded then there's no need to update the cache. Otherwise we upload our layers
to the global cache, possibly overwriting what we previously just downloaded.

This is hopefully a step towards mitigating #49278 although it doesn't
completely fix it as it means we'll still probably have to retry builds that
bust the cache.
2018-03-22 18:31:45 -07:00

370 lines
13 KiB
YAML
Raw Blame History

language: shell
sudo: required
dist: trusty
services:
- docker
git:
depth: 2
submodules: false
matrix:
fast_finish: true
include:
# Images used in testing PR and try-build should be run first.
- env: IMAGE=x86_64-gnu-llvm-3.9 RUST_BACKTRACE=1
if: type = pull_request OR branch = auto
- env: IMAGE=dist-x86_64-linux DEPLOY=1
if: branch = try OR branch = auto
# "alternate" deployments, these are "nightlies" but have LLVM assertions
# turned on, they're deployed to a different location primarily for
# additional testing.
- env: IMAGE=dist-x86_64-linux DEPLOY_ALT=1
if: branch = try OR branch = auto
- env: >
RUST_CHECK_TARGET=dist
RUST_CONFIGURE_ARGS="--enable-extended --enable-profiler"
SRC=.
DEPLOY_ALT=1
RUSTC_RETRY_LINKER_ON_SEGFAULT=1
SCCACHE_ERROR_LOG=/tmp/sccache.log
MACOSX_DEPLOYMENT_TARGET=10.7
NO_LLVM_ASSERTIONS=1
NO_DEBUG_ASSERTIONS=1
os: osx
osx_image: xcode9.3-moar
if: branch = auto
# macOS builders. These are placed near the beginning because they are very
# slow to run.
# OSX builders running tests, these run the full test suite.
#
# Note that the compiler is compiled to target 10.8 here because the Xcode
# version that we're using, 8.2, cannot compile LLVM for OSX 10.7.
- env: >
RUST_CHECK_TARGET=check
RUST_CONFIGURE_ARGS="--build=x86_64-apple-darwin --enable-sanitizers --enable-profiler"
SRC=.
RUSTC_RETRY_LINKER_ON_SEGFAULT=1
SCCACHE_ERROR_LOG=/tmp/sccache.log
MACOSX_DEPLOYMENT_TARGET=10.8
MACOSX_STD_DEPLOYMENT_TARGET=10.7
NO_LLVM_ASSERTIONS=1
NO_DEBUG_ASSERTIONS=1
os: osx
osx_image: xcode9.3-moar
if: branch = auto
- env: >
RUST_CHECK_TARGET=check
RUST_CONFIGURE_ARGS=--build=i686-apple-darwin
SRC=.
RUSTC_RETRY_LINKER_ON_SEGFAULT=1
SCCACHE_ERROR_LOG=/tmp/sccache.log
MACOSX_DEPLOYMENT_TARGET=10.8
MACOSX_STD_DEPLOYMENT_TARGET=10.7
NO_LLVM_ASSERTIONS=1
NO_DEBUG_ASSERTIONS=1
os: osx
osx_image: xcode9.3-moar
if: branch = auto
# OSX builders producing releases. These do not run the full test suite and
# just produce a bunch of artifacts.
#
# Note that these are running in the `xcode7` image instead of the
# `xcode8.2` image as above. That's because we want to build releases for
# OSX 10.7 and `xcode7` is the latest Xcode able to compile LLVM for 10.7.
- env: >
RUST_CHECK_TARGET=dist
RUST_CONFIGURE_ARGS="--build=i686-apple-darwin --enable-full-tools --enable-profiler"
SRC=.
DEPLOY=1
RUSTC_RETRY_LINKER_ON_SEGFAULT=1
SCCACHE_ERROR_LOG=/tmp/sccache.log
MACOSX_DEPLOYMENT_TARGET=10.7
NO_LLVM_ASSERTIONS=1
NO_DEBUG_ASSERTIONS=1
os: osx
osx_image: xcode9.3-moar
if: branch = auto
- env: >
RUST_CHECK_TARGET=dist
RUST_CONFIGURE_ARGS="--target=aarch64-apple-ios,armv7-apple-ios,armv7s-apple-ios,i386-apple-ios,x86_64-apple-ios --enable-full-tools --enable-sanitizers --enable-profiler"
SRC=.
DEPLOY=1
RUSTC_RETRY_LINKER_ON_SEGFAULT=1
SCCACHE_ERROR_LOG=/tmp/sccache.log
MACOSX_DEPLOYMENT_TARGET=10.7
NO_LLVM_ASSERTIONS=1
NO_DEBUG_ASSERTIONS=1
os: osx
osx_image: xcode9.3-moar
if: branch = auto
# Linux builders, remaining docker images
- env: IMAGE=arm-android
if: branch = auto
- env: IMAGE=armhf-gnu
if: branch = auto
- env: IMAGE=dist-various-1 DEPLOY=1
if: branch = auto
- env: IMAGE=dist-various-2 DEPLOY=1
if: branch = auto
- env: IMAGE=dist-aarch64-linux DEPLOY=1
if: branch = auto
- env: IMAGE=dist-android DEPLOY=1
if: branch = auto
- env: IMAGE=dist-arm-linux DEPLOY=1
if: branch = auto
- env: IMAGE=dist-armhf-linux DEPLOY=1
if: branch = auto
- env: IMAGE=dist-armv7-linux DEPLOY=1
if: branch = auto
- env: IMAGE=dist-i586-gnu-i586-i686-musl DEPLOY=1
if: branch = auto
- env: IMAGE=dist-i686-freebsd DEPLOY=1
if: branch = auto
- env: IMAGE=dist-i686-linux DEPLOY=1
if: branch = auto
- env: IMAGE=dist-mips-linux DEPLOY=1
if: branch = auto
- env: IMAGE=dist-mips64-linux DEPLOY=1
if: branch = auto
- env: IMAGE=dist-mips64el-linux DEPLOY=1
if: branch = auto
- env: IMAGE=dist-mipsel-linux DEPLOY=1
if: branch = auto
- env: IMAGE=dist-powerpc-linux DEPLOY=1
if: branch = auto
- env: IMAGE=dist-powerpc64-linux DEPLOY=1
if: branch = auto
- env: IMAGE=dist-powerpc64le-linux DEPLOY=1
if: branch = auto
- env: IMAGE=dist-s390x-linux DEPLOY=1
if: branch = auto
- env: IMAGE=dist-x86_64-freebsd DEPLOY=1
if: branch = auto
- env: IMAGE=dist-x86_64-musl DEPLOY=1
if: branch = auto
- env: IMAGE=dist-x86_64-netbsd DEPLOY=1
if: branch = auto
- env: IMAGE=asmjs
if: branch = auto
- env: IMAGE=i686-gnu
if: branch = auto
- env: IMAGE=i686-gnu-nopt
if: branch = auto
- env: IMAGE=wasm32-unknown
if: branch = auto
- env: IMAGE=x86_64-gnu
if: branch = auto
- env: IMAGE=x86_64-gnu-full-bootstrap
if: branch = auto
- env: IMAGE=x86_64-gnu-aux
if: branch = auto
- env: IMAGE=x86_64-gnu-tools
if: branch = auto
- env: IMAGE=x86_64-gnu-debug
if: branch = auto
- env: IMAGE=x86_64-gnu-nopt
if: branch = auto
- env: IMAGE=x86_64-gnu-distcheck
if: branch = auto
- env: IMAGE=x86_64-gnu-incremental
if: branch = auto
- stage: publish toolstate
if: branch = master AND type = push
before_install: []
install: []
sudo: false
script:
MESSAGE_FILE=$(mktemp -t msg.XXXXXX);
. src/ci/docker/x86_64-gnu-tools/repo.sh;
commit_toolstate_change "$MESSAGE_FILE" "$TRAVIS_BUILD_DIR/src/tools/publish_toolstate.py" "$(git rev-parse HEAD)" "$(git log --format=%s -n1 HEAD)" "$MESSAGE_FILE" "$TOOLSTATE_REPO_ACCESS_TOKEN";
env:
global:
- SCCACHE_BUCKET=rust-lang-ci-sccache2
- SCCACHE_REGION=us-west-1
- AWS_ACCESS_KEY_ID=AKIAJAMV3QAMMA6AXHFQ
# AWS_SECRET_ACCESS_KEY=...
- secure: "j96XxTVOSUf4s4r4htIxn/fvIa5DWbMgLqWl7r8z2QfgUwscmkMXAwXuFNc7s7bGTpV/+CgDiMFFM6BAFLGKutytIF6oA02s9b+usQYnM0th7YQ2AIgm9GtMTJCJp4AoyfFmh8F2faUICBZlfVLUJ34udHEe35vOklix+0k4WDo="
# TOOLSTATE_REPO_ACCESS_TOKEN=...
- secure: "cFh8thThqEJLC98XKI5pfqflUzOlxsYPRW20AWRaYOOgYHPTiGWypTXiPbGSKaeAXTZoOA+DpQtEmefc0U6lt9dHc7a/MIaK6isFurjlnKYiLOeTruzyu1z7PWCeZ/jKXsU2RK/88DBtlNwfMdaMIeuKj14IVfpepPPL71ETbuk="
before_install:
# We'll use the AWS cli to download/upload cached docker layers, so install
# that here.
- if [ "$TRAVIS_OS_NAME" = linux ]; then
pip install --user awscli;
export PATH=$PATH:$HOME/.local/bin;
fi
- mkdir -p $HOME/rustsrc
# FIXME(#46924): these two commands are required to enable IPv6,
# they shouldn't exist, please revert once more official solutions appeared.
# see https://github.com/travis-ci/travis-ci/issues/8891#issuecomment-353403729
- if [ "$TRAVIS_OS_NAME" = linux ]; then
echo '{"ipv6":true,"fixed-cidr-v6":"fd9a:8454:6789:13f7::/64"}' | sudo tee /etc/docker/daemon.json;
sudo service docker restart;
fi
install:
- case "$TRAVIS_OS_NAME" in
linux)
travis_retry curl -fo $HOME/stamp https://s3-us-west-1.amazonaws.com/rust-lang-ci2/rust-ci-mirror/2017-03-17-stamp-x86_64-unknown-linux-musl &&
chmod +x $HOME/stamp &&
export PATH=$PATH:$HOME
;;
osx)
if [[ "$RUST_CHECK_TARGET" == dist ]]; then
travis_retry brew update &&
travis_retry brew install xz;
fi &&
travis_retry curl -fo /usr/local/bin/sccache https://s3-us-west-1.amazonaws.com/rust-lang-ci2/rust-ci-mirror/2017-05-12-sccache-x86_64-apple-darwin &&
chmod +x /usr/local/bin/sccache &&
travis_retry curl -fo /usr/local/bin/stamp https://s3-us-west-1.amazonaws.com/rust-lang-ci2/rust-ci-mirror/2017-03-17-stamp-x86_64-apple-darwin &&
chmod +x /usr/local/bin/stamp
;;
esac
before_script:
- >
echo "#### Disk usage before running script:";
df -h;
du . | sort -nr | head -n100
- >
RUN_SCRIPT="src/ci/init_repo.sh . $HOME/rustsrc";
if [ "$TRAVIS_OS_NAME" = "osx" ]; then
export RUN_SCRIPT="$RUN_SCRIPT && src/ci/run.sh";
else
export RUN_SCRIPT="$RUN_SCRIPT && src/ci/docker/run.sh $IMAGE";
fi
# Log time information from this machine and an external machine for insight into possible
# clock drift. Timezones don't matter since relative deltas give all the necessary info.
script:
- >
date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
- stamp sh -x -c "$RUN_SCRIPT"
- >
date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
after_success:
- >
echo "#### Build successful; Disk usage after running script:";
df -h;
du . | sort -nr | head -n100
after_failure:
- >
echo "#### Build failed; Disk usage after running script:";
df -h;
du . | sort -nr | head -n100
# One of these is the linux sccache log, one is the OSX sccache log. Instead
# of worrying about what system we are just cat both. One of these commands
# will fail but that's ok, they'll both get executed.
- cat obj/tmp/sccache.log
- cat /tmp/sccache.log
# Random attempt at debugging currently. Just poking around in here to see if
# anything shows up.
- ls -lat $HOME/Library/Logs/DiagnosticReports/
- find $HOME/Library/Logs/DiagnosticReports
-type f
-name '*.crash'
-not -name '*.stage2-*.crash'
-not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash'
-exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \;
-exec head -750 {} \;
-exec echo travis_fold":"end:crashlog \; || true
# attempt to debug anything killed by the oom killer on linux, just to see if
# it happened
- dmesg | grep -i kill
notifications:
email: false
before_deploy:
- mkdir -p deploy/$TRAVIS_COMMIT
- >
if [ "$TRAVIS_OS_NAME" == "osx" ]; then
rm -rf build/dist/doc &&
cp -r build/dist/* deploy/$TRAVIS_COMMIT;
else
rm -rf obj/build/dist/doc &&
cp -r obj/build/dist/* deploy/$TRAVIS_COMMIT;
fi
- travis_retry gem update --system
- ls -la deploy/$TRAVIS_COMMIT
deploy:
- provider: s3
bucket: rust-lang-ci2
skip_cleanup: true
local_dir: deploy
upload_dir: rustc-builds
acl: public_read
region: us-west-1
access_key_id: AKIAJVBODR3IA4O72THQ
secret_access_key:
secure: "kUGd3t7JcVWFESgIlzvsM8viZgCA9Encs3creW0xLJaLSeI1iVjlJK4h/2/nO6y224AFrh/GUfsNr4/4AlxPuYb8OU5oC5Lv+Ff2JiRDYtuNpyQSKAQp+bRYytWMtrmhja91h118Mbm90cUfcLPwkdiINgJNTXhPKg5Cqu3VYn0="
on:
branch: auto
condition: $DEPLOY = 1
# this is the same as the above deployment provider except that it uploads to
# a slightly different directory and has a different trigger
- provider: s3
bucket: rust-lang-ci2
skip_cleanup: true
local_dir: deploy
upload_dir: rustc-builds-alt
acl: public_read
region: us-west-1
access_key_id: AKIAJVBODR3IA4O72THQ
secret_access_key:
secure: "kUGd3t7JcVWFESgIlzvsM8viZgCA9Encs3creW0xLJaLSeI1iVjlJK4h/2/nO6y224AFrh/GUfsNr4/4AlxPuYb8OU5oC5Lv+Ff2JiRDYtuNpyQSKAQp+bRYytWMtrmhja91h118Mbm90cUfcLPwkdiINgJNTXhPKg5Cqu3VYn0="
on:
branch: auto
condition: $DEPLOY_ALT = 1
# These two providers are the same as the two above, except deploy on the
# try branch. Travis does not appear to provide a way to use "or" in these
# conditions.
- provider: s3
bucket: rust-lang-ci2
skip_cleanup: true
local_dir: deploy
upload_dir: rustc-builds
acl: public_read
region: us-west-1
access_key_id: AKIAJVBODR3IA4O72THQ
secret_access_key:
secure: "kUGd3t7JcVWFESgIlzvsM8viZgCA9Encs3creW0xLJaLSeI1iVjlJK4h/2/nO6y224AFrh/GUfsNr4/4AlxPuYb8OU5oC5Lv+Ff2JiRDYtuNpyQSKAQp+bRYytWMtrmhja91h118Mbm90cUfcLPwkdiINgJNTXhPKg5Cqu3VYn0="
on:
branch: try
condition: $DEPLOY = 1
- provider: s3
bucket: rust-lang-ci2
skip_cleanup: true
local_dir: deploy
upload_dir: rustc-builds-alt
acl: public_read
region: us-west-1
access_key_id: AKIAJVBODR3IA4O72THQ
secret_access_key:
secure: "kUGd3t7JcVWFESgIlzvsM8viZgCA9Encs3creW0xLJaLSeI1iVjlJK4h/2/nO6y224AFrh/GUfsNr4/4AlxPuYb8OU5oC5Lv+Ff2JiRDYtuNpyQSKAQp+bRYytWMtrmhja91h118Mbm90cUfcLPwkdiINgJNTXhPKg5Cqu3VYn0="
on:
branch: try
condition: $DEPLOY_ALT = 1
Reference in New Issue View Git Blame Copy Permalink