OpenE2K
/
rust
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
rust/src/ci/docker/android/Dockerfile

59 lines
1.6 KiB
Docker
Raw Normal View History

Move all Linux/OSX CI infastructure to Travis This commit configures our `.travis.yml` to test the full suite of tests we have on Buildbot right now. A whole mess of docker images are added to the `src/ci` directory which represent all the build environments for each configuration. Each of these environments is then configured in `.travis.yml` to run on the auto branch. Note that the full matrix of tests aren't intended to be run on all PRs. Instead, we continue to run only one entry in the matrix, forcing all others to finish quickly. Only the `auto` branch should run the full matrix of builds. Also note that the infrastructure hasn't quite been allocated yet to the rust-lang/rust repository, so everything is disabled for now except for the one build that happens on PRs. Once that infrastructure is allocated though we can enable this and let it fly! Notable modifications from the current test suite today: * Android tests are run in rustbuild instead of the makefiles, for whatever reason I couldn't get the makefiles to work on Travis. * A debuginfo test was updated to work with the current version of the Android NDK. * Some dependencies in `mk/tests.mk` were fixed to allow running tests in parallel.
2016-10-11 21:21:40 +02:00
FROM ubuntu:16.04
RUN dpkg --add-architecture i386 && \
apt-get update && \
apt-get install -y --no-install-recommends \
g++ \
make \
file \
curl \
ca-certificates \
python2.7 \
git \
cmake \
unzip \
expect \
openjdk-9-jre \
sudo \
rustbuild: Add sccache support This commit adds support for sccache, a ccache-like compiler which works on MSVC and stores results into an S3 bucket. This also switches over all Travis and AppVeyor automation to using sccache to ensure a shared and unified cache over time which can be shared across builders. The support for sccache manifests as a new `--enable-sccache` option which instructs us to configure LLVM differently to use a 'sccache' binary instead of a 'ccache' binary. All docker images for Travis builds are updated to download Mozilla's tooltool builds of sccache onto various containers and systems. Additionally a new `rust-lang-ci-sccache` bucket is configured to hold all of our ccache goodies.
2016-12-12 20:36:52 +01:00
libstdc++6:i386 \
Automatically vendor Cargo deps when building the source tarballs.
2017-02-10 21:59:40 +01:00
xz-utils \
libssl-dev \
pkg-config
Move all Linux/OSX CI infastructure to Travis This commit configures our `.travis.yml` to test the full suite of tests we have on Buildbot right now. A whole mess of docker images are added to the `src/ci` directory which represent all the build environments for each configuration. Each of these environments is then configured in `.travis.yml` to run on the auto branch. Note that the full matrix of tests aren't intended to be run on all PRs. Instead, we continue to run only one entry in the matrix, forcing all others to finish quickly. Only the `auto` branch should run the full matrix of builds. Also note that the infrastructure hasn't quite been allocated yet to the rust-lang/rust repository, so everything is disabled for now except for the one build that happens on PRs. Once that infrastructure is allocated though we can enable this and let it fly! Notable modifications from the current test suite today: * Android tests are run in rustbuild instead of the makefiles, for whatever reason I couldn't get the makefiles to work on Travis. * A debuginfo test was updated to work with the current version of the Android NDK. * Some dependencies in `mk/tests.mk` were fixed to allow running tests in parallel.
2016-10-11 21:21:40 +02:00
WORKDIR /android/
ENV PATH=$PATH:/android/ndk-arm-9/bin:/android/sdk/tools:/android/sdk/platform-tools
COPY install-ndk.sh install-sdk.sh accept-licenses.sh /android/
RUN sh /android/install-ndk.sh
RUN sh /android/install-sdk.sh
Fix travis builds After reading some articles [1] [2] yesterday about Docker and the "init" process I got to thinking about the problems that we've been seeing on Travis. The basic problem is that a Linux system may need an "init" process to work properly when processes become zombies. Docker by default doesn't handle this and the root process typically isn't an init process, so this can occasionally cause quite a few problems. We've been seeing spurious errors on Travis inside containers which look like OOM and such, but my guess is that zombie processes were being reparented to the top-level shell. The shell didn't expect the zombies and then behaved very strangely. This commit fixes these problems by using Yelp's "dumb-init" program [2] as the init process in all of our containers. This ensures that there's a valid init ready to reap children when they're reparented, which our test suite apparently generates a bunch of throughout the tests and such. [1]: https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/ [2]: https://engineeringblog.yelp.com/2016/01/dumb-init-an-init-for-docker.html
2016-12-13 05:01:13 +01:00
RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
dpkg -i dumb-init_*.deb && \
rm dumb-init_*.deb
Move all Linux/OSX CI infastructure to Travis This commit configures our `.travis.yml` to test the full suite of tests we have on Buildbot right now. A whole mess of docker images are added to the `src/ci` directory which represent all the build environments for each configuration. Each of these environments is then configured in `.travis.yml` to run on the auto branch. Note that the full matrix of tests aren't intended to be run on all PRs. Instead, we continue to run only one entry in the matrix, forcing all others to finish quickly. Only the `auto` branch should run the full matrix of builds. Also note that the infrastructure hasn't quite been allocated yet to the rust-lang/rust repository, so everything is disabled for now except for the one build that happens on PRs. Once that infrastructure is allocated though we can enable this and let it fly! Notable modifications from the current test suite today: * Android tests are run in rustbuild instead of the makefiles, for whatever reason I couldn't get the makefiles to work on Travis. * A debuginfo test was updated to work with the current version of the Android NDK. * Some dependencies in `mk/tests.mk` were fixed to allow running tests in parallel.
2016-10-11 21:21:40 +02:00
COPY start-emulator.sh /android/
Fix travis builds After reading some articles [1] [2] yesterday about Docker and the "init" process I got to thinking about the problems that we've been seeing on Travis. The basic problem is that a Linux system may need an "init" process to work properly when processes become zombies. Docker by default doesn't handle this and the root process typically isn't an init process, so this can occasionally cause quite a few problems. We've been seeing spurious errors on Travis inside containers which look like OOM and such, but my guess is that zombie processes were being reparented to the top-level shell. The shell didn't expect the zombies and then behaved very strangely. This commit fixes these problems by using Yelp's "dumb-init" program [2] as the init process in all of our containers. This ensures that there's a valid init ready to reap children when they're reparented, which our test suite apparently generates a bunch of throughout the tests and such. [1]: https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/ [2]: https://engineeringblog.yelp.com/2016/01/dumb-init-an-init-for-docker.html
2016-12-13 05:01:13 +01:00
ENTRYPOINT ["/usr/bin/dumb-init", "--", "/android/start-emulator.sh"]
Move all Linux/OSX CI infastructure to Travis This commit configures our `.travis.yml` to test the full suite of tests we have on Buildbot right now. A whole mess of docker images are added to the `src/ci` directory which represent all the build environments for each configuration. Each of these environments is then configured in `.travis.yml` to run on the auto branch. Note that the full matrix of tests aren't intended to be run on all PRs. Instead, we continue to run only one entry in the matrix, forcing all others to finish quickly. Only the `auto` branch should run the full matrix of builds. Also note that the infrastructure hasn't quite been allocated yet to the rust-lang/rust repository, so everything is disabled for now except for the one build that happens on PRs. Once that infrastructure is allocated though we can enable this and let it fly! Notable modifications from the current test suite today: * Android tests are run in rustbuild instead of the makefiles, for whatever reason I couldn't get the makefiles to work on Travis. * A debuginfo test was updated to work with the current version of the Android NDK. * Some dependencies in `mk/tests.mk` were fixed to allow running tests in parallel.
2016-10-11 21:21:40 +02:00
Update sccache binaries on CI Currently CI builds can fail spuriously during the LLVM build (#39003). I believe this is due to sccache, and I believe that in turn was due to the fact that the sccache server used to just be a raw mio server. Historically raw mio servers are quite complicated to get right, but this is why we built Tokio! The sccache server has been migrated to Tokio which I suspect would fix any latent issues. I have no confirmation of this (never been able to reproduce the deadlock locally), but my hunch is that updating sccache to the master branch will fix the timeouts during the LLVM build. The binaries previously came from Gecko's infrastructure, but I've built new ones by hand for Win/Mac/Linux and uploaded them to our CI bucket.
2017-02-24 22:16:54 +01:00
RUN curl -o /usr/local/bin/sccache \
appveyor: Use sccache on pc-windows-gnu for caching Now that mozilla/sccache#43 is fixed the caching works for MinGW on Windows. We still can't use it for MSVC just yet, but I'll try to revive that branch at some point.
2017-02-25 09:53:12 +01:00
https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-02-25-sccache-x86_64-unknown-linux-musl && \
Update sccache binaries on CI Currently CI builds can fail spuriously during the LLVM build (#39003). I believe this is due to sccache, and I believe that in turn was due to the fact that the sccache server used to just be a raw mio server. Historically raw mio servers are quite complicated to get right, but this is why we built Tokio! The sccache server has been migrated to Tokio which I suspect would fix any latent issues. I have no confirmation of this (never been able to reproduce the deadlock locally), but my hunch is that updating sccache to the master branch will fix the timeouts during the LLVM build. The binaries previously came from Gecko's infrastructure, but I've built new ones by hand for Win/Mac/Linux and uploaded them to our CI bucket.
2017-02-24 22:16:54 +01:00
chmod +x /usr/local/bin/sccache
rustbuild: Add sccache support This commit adds support for sccache, a ccache-like compiler which works on MSVC and stores results into an S3 bucket. This also switches over all Travis and AppVeyor automation to using sccache to ensure a shared and unified cache over time which can be shared across builders. The support for sccache manifests as a new `--enable-sccache` option which instructs us to configure LLVM differently to use a 'sccache' binary instead of a 'ccache' binary. All docker images for Travis builds are updated to download Mozilla's tooltool builds of sccache onto various containers and systems. Additionally a new `rust-lang-ci-sccache` bucket is configured to hold all of our ccache goodies.
2016-12-12 20:36:52 +01:00
Move all Linux/OSX CI infastructure to Travis This commit configures our `.travis.yml` to test the full suite of tests we have on Buildbot right now. A whole mess of docker images are added to the `src/ci` directory which represent all the build environments for each configuration. Each of these environments is then configured in `.travis.yml` to run on the auto branch. Note that the full matrix of tests aren't intended to be run on all PRs. Instead, we continue to run only one entry in the matrix, forcing all others to finish quickly. Only the `auto` branch should run the full matrix of builds. Also note that the infrastructure hasn't quite been allocated yet to the rust-lang/rust repository, so everything is disabled for now except for the one build that happens on PRs. Once that infrastructure is allocated though we can enable this and let it fly! Notable modifications from the current test suite today: * Android tests are run in rustbuild instead of the makefiles, for whatever reason I couldn't get the makefiles to work on Travis. * A debuginfo test was updated to work with the current version of the Android NDK. * Some dependencies in `mk/tests.mk` were fixed to allow running tests in parallel.
2016-10-11 21:21:40 +02:00
ENV TARGETS=arm-linux-androideabi
ENV TARGETS=$TARGETS,i686-linux-android
ENV TARGETS=$TARGETS,aarch64-linux-android
ENV TARGETS=$TARGETS,armv7-linux-androideabi
ENV RUST_CONFIGURE_ARGS \
--target=$TARGETS \
--arm-linux-androideabi-ndk=/android/ndk-arm-9 \
--armv7-linux-androideabi-ndk=/android/ndk-arm-9 \
--i686-linux-android-ndk=/android/ndk-x86-9 \
mk: Switch rustbuild to the default build system This commit switches the default build system for Rust from the makefiles to rustbuild. The rustbuild build system has been in development for almost a year now and has become quite mature over time. This commit is an implementation of the proposal on [internals] which slates deletion of the makefiles on 2016-01-02. [internals]: https://internals.rust-lang.org/t/proposal-for-promoting-rustbuild-to-official-status/4368 This commit also updates various documentation in `README.md`, `CONTRIBUTING.md`, `src/bootstrap/README.md`, and throughout the source code of rustbuild itself. Closes #37858
2016-11-16 21:31:19 +01:00
--aarch64-linux-android-ndk=/android/ndk-aarch64
travis: Start uploading artifacts on commits This commit starts adding the infrastructure for uploading release artifacts from AppVeyor/Travis on each commit. The idea is that eventually we'll upload a full release to AppVeyor/Travis in accordance with plans [outlined earlier]. Right now this configures Travis/Appveyor to upload all tarballs in the `dist` directory, and various images are updated to actually produce tarballs in these directories. These are nowhere near ready to be actual release artifacts, but this should allow us to play around with it and test it out. Once this commit lands we should start seeing artifacts uploaded on each commit. [outlined earlier]: https://internals.rust-lang.org/t/rust-ci-release-infrastructure-changes/4489
2017-01-01 02:42:40 +01:00
# Just a smoke test in dist to see if this works for now, we should expand this
# to all the targets above eventually.
ENV SCRIPT \
python2.7 ../x.py test --target arm-linux-androideabi && \
travis: Expand the `cross` linux image This expands the `cross` travis matrix entry with a few more targets that our nightlies are building: * x86_64-rumprun-netbsd * arm-unknown-linux-musleabi * arm-unknown-linux-musleabihf * armv7-unknown-linux-musleabihf * mips-unknown-linux-musl * mipsel-unknown-linux-musl This commit doesn't compile custom toolchains like our current cross-image does, but instead compiles musl manually and then compiles libunwind manually (like x86_64) for use for the ARM targets and just uses openwrt toolchains for the mips targets.
2017-01-16 23:15:12 +01:00
python2.7 ../x.py dist --target $TARGETS