OpenE2K
/
rust
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
rust/src/ci/docker/dist-s390x-linux-netbsd/Dockerfile

87 lines
2.3 KiB
Docker
Raw Normal View History

Move all Linux/OSX CI infastructure to Travis This commit configures our `.travis.yml` to test the full suite of tests we have on Buildbot right now. A whole mess of docker images are added to the `src/ci` directory which represent all the build environments for each configuration. Each of these environments is then configured in `.travis.yml` to run on the auto branch. Note that the full matrix of tests aren't intended to be run on all PRs. Instead, we continue to run only one entry in the matrix, forcing all others to finish quickly. Only the `auto` branch should run the full matrix of builds. Also note that the infrastructure hasn't quite been allocated yet to the rust-lang/rust repository, so everything is disabled for now except for the one build that happens on PRs. Once that infrastructure is allocated though we can enable this and let it fly! Notable modifications from the current test suite today: * Android tests are run in rustbuild instead of the makefiles, for whatever reason I couldn't get the makefiles to work on Travis. * A debuginfo test was updated to work with the current version of the Android NDK. * Some dependencies in `mk/tests.mk` were fixed to allow running tests in parallel.
2016-10-11 21:21:40 +02:00
FROM ubuntu:16.04
RUN apt-get update && apt-get install -y --no-install-recommends \
travis: move IBM backwards in time Using Ubuntu's cross-toolchains for powerpc* and s390x meant they were depending on glibc symbols from Ubuntu 16.04. And if that host is ever updated to a new release, the toolchains would raise the bar too. This switches powerpc, powerpc64, and s390x to use crosstool-ng toolchains, configured approximately like RHEL6 with kernel 2.6.32 and glibc 2.12. This ABI level should also be compatible with Debian 7 (wheezy) and Ubuntu 12.04 (precise). For powerpc64le, the challenge was that only glibc-2.19 officially added support, but RHEL7 backported those changes to glibc-2.17. The backport patches are complex and numerous, so instead of trying to push those into crosstool-ng, this just uses glibc binaries directly from CentOS 7 and builds the toolchain manually. This is ported from rust-lang/rust-buildbot#149. r? @alexcrichton
2017-01-29 06:25:11 +01:00
automake \
bison \
bzip2 \
ca-certificates \
cmake \
curl \
file \
flex \
Move all Linux/OSX CI infastructure to Travis This commit configures our `.travis.yml` to test the full suite of tests we have on Buildbot right now. A whole mess of docker images are added to the `src/ci` directory which represent all the build environments for each configuration. Each of these environments is then configured in `.travis.yml` to run on the auto branch. Note that the full matrix of tests aren't intended to be run on all PRs. Instead, we continue to run only one entry in the matrix, forcing all others to finish quickly. Only the `auto` branch should run the full matrix of builds. Also note that the infrastructure hasn't quite been allocated yet to the rust-lang/rust repository, so everything is disabled for now except for the one build that happens on PRs. Once that infrastructure is allocated though we can enable this and let it fly! Notable modifications from the current test suite today: * Android tests are run in rustbuild instead of the makefiles, for whatever reason I couldn't get the makefiles to work on Travis. * A debuginfo test was updated to work with the current version of the Android NDK. * Some dependencies in `mk/tests.mk` were fixed to allow running tests in parallel.
2016-10-11 21:21:40 +02:00
g++ \
travis: move IBM backwards in time Using Ubuntu's cross-toolchains for powerpc* and s390x meant they were depending on glibc symbols from Ubuntu 16.04. And if that host is ever updated to a new release, the toolchains would raise the bar too. This switches powerpc, powerpc64, and s390x to use crosstool-ng toolchains, configured approximately like RHEL6 with kernel 2.6.32 and glibc 2.12. This ABI level should also be compatible with Debian 7 (wheezy) and Ubuntu 12.04 (precise). For powerpc64le, the challenge was that only glibc-2.19 officially added support, but RHEL7 backported those changes to glibc-2.17. The backport patches are complex and numerous, so instead of trying to push those into crosstool-ng, this just uses glibc binaries directly from CentOS 7 and builds the toolchain manually. This is ported from rust-lang/rust-buildbot#149. r? @alexcrichton
2017-01-29 06:25:11 +01:00
gawk \
gdb \
git \
gperf \
help2man \
libncurses-dev \
libtool-bin \
Move all Linux/OSX CI infastructure to Travis This commit configures our `.travis.yml` to test the full suite of tests we have on Buildbot right now. A whole mess of docker images are added to the `src/ci` directory which represent all the build environments for each configuration. Each of these environments is then configured in `.travis.yml` to run on the auto branch. Note that the full matrix of tests aren't intended to be run on all PRs. Instead, we continue to run only one entry in the matrix, forcing all others to finish quickly. Only the `auto` branch should run the full matrix of builds. Also note that the infrastructure hasn't quite been allocated yet to the rust-lang/rust repository, so everything is disabled for now except for the one build that happens on PRs. Once that infrastructure is allocated though we can enable this and let it fly! Notable modifications from the current test suite today: * Android tests are run in rustbuild instead of the makefiles, for whatever reason I couldn't get the makefiles to work on Travis. * A debuginfo test was updated to work with the current version of the Android NDK. * Some dependencies in `mk/tests.mk` were fixed to allow running tests in parallel.
2016-10-11 21:21:40 +02:00
make \
travis: move IBM backwards in time Using Ubuntu's cross-toolchains for powerpc* and s390x meant they were depending on glibc symbols from Ubuntu 16.04. And if that host is ever updated to a new release, the toolchains would raise the bar too. This switches powerpc, powerpc64, and s390x to use crosstool-ng toolchains, configured approximately like RHEL6 with kernel 2.6.32 and glibc 2.12. This ABI level should also be compatible with Debian 7 (wheezy) and Ubuntu 12.04 (precise). For powerpc64le, the challenge was that only glibc-2.19 officially added support, but RHEL7 backported those changes to glibc-2.17. The backport patches are complex and numerous, so instead of trying to push those into crosstool-ng, this just uses glibc binaries directly from CentOS 7 and builds the toolchain manually. This is ported from rust-lang/rust-buildbot#149. r? @alexcrichton
2017-01-29 06:25:11 +01:00
patch \
Move all Linux/OSX CI infastructure to Travis This commit configures our `.travis.yml` to test the full suite of tests we have on Buildbot right now. A whole mess of docker images are added to the `src/ci` directory which represent all the build environments for each configuration. Each of these environments is then configured in `.travis.yml` to run on the auto branch. Note that the full matrix of tests aren't intended to be run on all PRs. Instead, we continue to run only one entry in the matrix, forcing all others to finish quickly. Only the `auto` branch should run the full matrix of builds. Also note that the infrastructure hasn't quite been allocated yet to the rust-lang/rust repository, so everything is disabled for now except for the one build that happens on PRs. Once that infrastructure is allocated though we can enable this and let it fly! Notable modifications from the current test suite today: * Android tests are run in rustbuild instead of the makefiles, for whatever reason I couldn't get the makefiles to work on Travis. * A debuginfo test was updated to work with the current version of the Android NDK. * Some dependencies in `mk/tests.mk` were fixed to allow running tests in parallel.
2016-10-11 21:21:40 +02:00
python2.7 \
sudo \
travis: move IBM backwards in time Using Ubuntu's cross-toolchains for powerpc* and s390x meant they were depending on glibc symbols from Ubuntu 16.04. And if that host is ever updated to a new release, the toolchains would raise the bar too. This switches powerpc, powerpc64, and s390x to use crosstool-ng toolchains, configured approximately like RHEL6 with kernel 2.6.32 and glibc 2.12. This ABI level should also be compatible with Debian 7 (wheezy) and Ubuntu 12.04 (precise). For powerpc64le, the challenge was that only glibc-2.19 officially added support, but RHEL7 backported those changes to glibc-2.17. The backport patches are complex and numerous, so instead of trying to push those into crosstool-ng, this just uses glibc binaries directly from CentOS 7 and builds the toolchain manually. This is ported from rust-lang/rust-buildbot#149. r? @alexcrichton
2017-01-29 06:25:11 +01:00
texinfo \
travis: Expand dist builder coverage This commit adds six new travis matrix entires for doing cross-compiled distribution builds of the compiler. The support added in #38731 allows us to quickly compile a complete suite of distribution artifacts for cross-compiled platforms, and currently each matrix entry (when fully cached) clocks in around an hour to finish. Note that a full test run typically takes about two hours right now. With further optimizations coming down the pike in #39026 this commit also starts doubling up cross-compiled distribution builders on each matrix entry. We initially planned to do one build per entry, but it's looking like we may be able to get by with more than one in each entry. Depending on how long these builds take we may even be able to up it to three, but we'll start with two first. This commit then completes the suite of cross-compiled compilers that we're going to compile, adding it for a whole litany of platforms detailed in the changes to the docker files here. The existing `cross` image is also trimmed down quite a bit to avoid duplicate work, and we'll eventually provision it for far more cross compilation as well. Note that the gcc toolchains installed to compile most of these compilers are inappropriate for actualy distribution. The glibc they pull in is much newer than we'd like, so before we turn nightlies off we'll need to tweak these docker files to custom build toolchains like the current `linux-cross` docker image does.
2017-01-13 18:18:09 +01:00
wget \
Automatically vendor Cargo deps when building the source tarballs.
2017-02-10 21:59:40 +01:00
xz-utils \
libssl-dev \
pkg-config
Move all Linux/OSX CI infastructure to Travis This commit configures our `.travis.yml` to test the full suite of tests we have on Buildbot right now. A whole mess of docker images are added to the `src/ci` directory which represent all the build environments for each configuration. Each of these environments is then configured in `.travis.yml` to run on the auto branch. Note that the full matrix of tests aren't intended to be run on all PRs. Instead, we continue to run only one entry in the matrix, forcing all others to finish quickly. Only the `auto` branch should run the full matrix of builds. Also note that the infrastructure hasn't quite been allocated yet to the rust-lang/rust repository, so everything is disabled for now except for the one build that happens on PRs. Once that infrastructure is allocated though we can enable this and let it fly! Notable modifications from the current test suite today: * Android tests are run in rustbuild instead of the makefiles, for whatever reason I couldn't get the makefiles to work on Travis. * A debuginfo test was updated to work with the current version of the Android NDK. * Some dependencies in `mk/tests.mk` were fixed to allow running tests in parallel.
2016-10-11 21:21:40 +02:00
Update sccache binaries on CI Currently CI builds can fail spuriously during the LLVM build (#39003). I believe this is due to sccache, and I believe that in turn was due to the fact that the sccache server used to just be a raw mio server. Historically raw mio servers are quite complicated to get right, but this is why we built Tokio! The sccache server has been migrated to Tokio which I suspect would fix any latent issues. I have no confirmation of this (never been able to reproduce the deadlock locally), but my hunch is that updating sccache to the master branch will fix the timeouts during the LLVM build. The binaries previously came from Gecko's infrastructure, but I've built new ones by hand for Win/Mac/Linux and uploaded them to our CI bucket.
2017-02-24 22:16:54 +01:00
RUN curl -o /usr/local/bin/sccache \
travis: Update sccache binary I've built a local copy with mozilla/sccache#79 and mozilla/sccache#78. Let's see if that helps #40240!
2017-03-16 15:55:15 +01:00
https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-03-16-sccache-x86_64-unknown-linux-musl && \
Update sccache binaries on CI Currently CI builds can fail spuriously during the LLVM build (#39003). I believe this is due to sccache, and I believe that in turn was due to the fact that the sccache server used to just be a raw mio server. Historically raw mio servers are quite complicated to get right, but this is why we built Tokio! The sccache server has been migrated to Tokio which I suspect would fix any latent issues. I have no confirmation of this (never been able to reproduce the deadlock locally), but my hunch is that updating sccache to the master branch will fix the timeouts during the LLVM build. The binaries previously came from Gecko's infrastructure, but I've built new ones by hand for Win/Mac/Linux and uploaded them to our CI bucket.
2017-02-24 22:16:54 +01:00
chmod +x /usr/local/bin/sccache
Move all Linux/OSX CI infastructure to Travis This commit configures our `.travis.yml` to test the full suite of tests we have on Buildbot right now. A whole mess of docker images are added to the `src/ci` directory which represent all the build environments for each configuration. Each of these environments is then configured in `.travis.yml` to run on the auto branch. Note that the full matrix of tests aren't intended to be run on all PRs. Instead, we continue to run only one entry in the matrix, forcing all others to finish quickly. Only the `auto` branch should run the full matrix of builds. Also note that the infrastructure hasn't quite been allocated yet to the rust-lang/rust repository, so everything is disabled for now except for the one build that happens on PRs. Once that infrastructure is allocated though we can enable this and let it fly! Notable modifications from the current test suite today: * Android tests are run in rustbuild instead of the makefiles, for whatever reason I couldn't get the makefiles to work on Travis. * A debuginfo test was updated to work with the current version of the Android NDK. * Some dependencies in `mk/tests.mk` were fixed to allow running tests in parallel.
2016-10-11 21:21:40 +02:00
Fix travis builds After reading some articles [1] [2] yesterday about Docker and the "init" process I got to thinking about the problems that we've been seeing on Travis. The basic problem is that a Linux system may need an "init" process to work properly when processes become zombies. Docker by default doesn't handle this and the root process typically isn't an init process, so this can occasionally cause quite a few problems. We've been seeing spurious errors on Travis inside containers which look like OOM and such, but my guess is that zombie processes were being reparented to the top-level shell. The shell didn't expect the zombies and then behaved very strangely. This commit fixes these problems by using Yelp's "dumb-init" program [2] as the init process in all of our containers. This ensures that there's a valid init ready to reap children when they're reparented, which our test suite apparently generates a bunch of throughout the tests and such. [1]: https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/ [2]: https://engineeringblog.yelp.com/2016/01/dumb-init-an-init-for-docker.html
2016-12-13 05:01:13 +01:00
RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
dpkg -i dumb-init_*.deb && \
rm dumb-init_*.deb
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
travis: move IBM backwards in time Using Ubuntu's cross-toolchains for powerpc* and s390x meant they were depending on glibc symbols from Ubuntu 16.04. And if that host is ever updated to a new release, the toolchains would raise the bar too. This switches powerpc, powerpc64, and s390x to use crosstool-ng toolchains, configured approximately like RHEL6 with kernel 2.6.32 and glibc 2.12. This ABI level should also be compatible with Debian 7 (wheezy) and Ubuntu 12.04 (precise). For powerpc64le, the challenge was that only glibc-2.19 officially added support, but RHEL7 backported those changes to glibc-2.17. The backport patches are complex and numerous, so instead of trying to push those into crosstool-ng, this just uses glibc binaries directly from CentOS 7 and builds the toolchain manually. This is ported from rust-lang/rust-buildbot#149. r? @alexcrichton
2017-01-29 06:25:11 +01:00
# Ubuntu 16.04 (this contianer) ships with make 4, but something in the
# toolchains we build below chokes on that, so go back to make 3
RUN curl https://ftp.gnu.org/gnu/make/make-3.81.tar.gz | tar xzf - && \
cd make-3.81 && \
./configure --prefix=/usr && \
make && \
make install && \
cd .. && \
rm -rf make-3.81
RUN curl http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.22.0.tar.bz2 | \
tar xjf - && \
cd crosstool-ng && \
./configure --prefix=/usr/local && \
make -j$(nproc) && \
make install && \
cd .. && \
rm -rf crosstool-ng
RUN groupadd -r rustbuild && useradd -m -r -g rustbuild rustbuild
RUN mkdir /x-tools && chown rustbuild:rustbuild /x-tools
USER rustbuild
WORKDIR /tmp
COPY patches/ /tmp/patches/
COPY s390x-linux-gnu.config build-s390x-toolchain.sh /tmp/
RUN ./build-s390x-toolchain.sh
COPY build-netbsd-toolchain.sh /tmp/
RUN ./build-netbsd-toolchain.sh
ci/netbsd: use the "official" cross compiler
2017-03-17 20:52:44 +01:00
USER root
ENV PATH=$PATH:/x-tools/s390x-ibm-linux-gnu/bin:/x-tools/x86_64-unknown-netbsd/bin
rustbuild: Add sccache support This commit adds support for sccache, a ccache-like compiler which works on MSVC and stores results into an S3 bucket. This also switches over all Travis and AppVeyor automation to using sccache to ensure a shared and unified cache over time which can be shared across builders. The support for sccache manifests as a new `--enable-sccache` option which instructs us to configure LLVM differently to use a 'sccache' binary instead of a 'ccache' binary. All docker images for Travis builds are updated to download Mozilla's tooltool builds of sccache onto various containers and systems. Additionally a new `rust-lang-ci-sccache` bucket is configured to hold all of our ccache goodies.
2016-12-12 20:36:52 +01:00
Move all Linux/OSX CI infastructure to Travis This commit configures our `.travis.yml` to test the full suite of tests we have on Buildbot right now. A whole mess of docker images are added to the `src/ci` directory which represent all the build environments for each configuration. Each of these environments is then configured in `.travis.yml` to run on the auto branch. Note that the full matrix of tests aren't intended to be run on all PRs. Instead, we continue to run only one entry in the matrix, forcing all others to finish quickly. Only the `auto` branch should run the full matrix of builds. Also note that the infrastructure hasn't quite been allocated yet to the rust-lang/rust repository, so everything is disabled for now except for the one build that happens on PRs. Once that infrastructure is allocated though we can enable this and let it fly! Notable modifications from the current test suite today: * Android tests are run in rustbuild instead of the makefiles, for whatever reason I couldn't get the makefiles to work on Travis. * A debuginfo test was updated to work with the current version of the Android NDK. * Some dependencies in `mk/tests.mk` were fixed to allow running tests in parallel.
2016-10-11 21:21:40 +02:00
ENV \
ci/netbsd: use the "official" cross compiler
2017-03-17 20:52:44 +01:00
AR_x86_64_unknown_netbsd=x86_64--netbsd-ar \
CC_x86_64_unknown_netbsd=x86_64--netbsd-gcc-sysroot \
CXX_x86_64_unknown_netbsd=x86_64--netbsd-g++-sysroot \
travis: move IBM backwards in time Using Ubuntu's cross-toolchains for powerpc* and s390x meant they were depending on glibc symbols from Ubuntu 16.04. And if that host is ever updated to a new release, the toolchains would raise the bar too. This switches powerpc, powerpc64, and s390x to use crosstool-ng toolchains, configured approximately like RHEL6 with kernel 2.6.32 and glibc 2.12. This ABI level should also be compatible with Debian 7 (wheezy) and Ubuntu 12.04 (precise). For powerpc64le, the challenge was that only glibc-2.19 officially added support, but RHEL7 backported those changes to glibc-2.17. The backport patches are complex and numerous, so instead of trying to push those into crosstool-ng, this just uses glibc binaries directly from CentOS 7 and builds the toolchain manually. This is ported from rust-lang/rust-buildbot#149. r? @alexcrichton
2017-01-29 06:25:11 +01:00
CC_s390x_unknown_linux_gnu=s390x-ibm-linux-gnu-gcc \
AR_s390x_unknown_linux_gnu=s390x-ibm-linux-gnu-ar \
CXX_s390x_unknown_linux_gnu=s390x-ibm-linux-gnu-g++
Move all Linux/OSX CI infastructure to Travis This commit configures our `.travis.yml` to test the full suite of tests we have on Buildbot right now. A whole mess of docker images are added to the `src/ci` directory which represent all the build environments for each configuration. Each of these environments is then configured in `.travis.yml` to run on the auto branch. Note that the full matrix of tests aren't intended to be run on all PRs. Instead, we continue to run only one entry in the matrix, forcing all others to finish quickly. Only the `auto` branch should run the full matrix of builds. Also note that the infrastructure hasn't quite been allocated yet to the rust-lang/rust repository, so everything is disabled for now except for the one build that happens on PRs. Once that infrastructure is allocated though we can enable this and let it fly! Notable modifications from the current test suite today: * Android tests are run in rustbuild instead of the makefiles, for whatever reason I couldn't get the makefiles to work on Travis. * A debuginfo test was updated to work with the current version of the Android NDK. * Some dependencies in `mk/tests.mk` were fixed to allow running tests in parallel.
2016-10-11 21:21:40 +02:00
travis: Expand the `cross` linux image This expands the `cross` travis matrix entry with a few more targets that our nightlies are building: * x86_64-rumprun-netbsd * arm-unknown-linux-musleabi * arm-unknown-linux-musleabihf * armv7-unknown-linux-musleabihf * mips-unknown-linux-musl * mipsel-unknown-linux-musl This commit doesn't compile custom toolchains like our current cross-image does, but instead compiles musl manually and then compiles libunwind manually (like x86_64) for use for the ARM targets and just uses openwrt toolchains for the mips targets.
2017-01-16 23:15:12 +01:00
ENV HOSTS=x86_64-unknown-netbsd
ENV HOSTS=$HOSTS,s390x-unknown-linux-gnu
rustbuild: Start building --enable-extended This commit adds a new flag to the configure script, `--enable-extended`, which is intended for specifying a desire to compile the full suite of Rust tools such as Cargo, the RLS, etc. This is also an indication that the build system should create combined installers such as the pkg/exe/msi artifacts. Currently the `--enable-extended` flag just indicates that combined installers should be built, and Cargo is itself not compiled just yet but rather only downloaded from its location. The intention here is to quickly get to feature parity with the current release process and then we can start improving it afterwards. All new files in this PR inside `src/etc/installer` are copied from the rust-packaging repository.
2017-01-21 02:03:06 +01:00
ENV RUST_CONFIGURE_ARGS --host=$HOSTS --enable-extended
travis: Expand the `cross` linux image This expands the `cross` travis matrix entry with a few more targets that our nightlies are building: * x86_64-rumprun-netbsd * arm-unknown-linux-musleabi * arm-unknown-linux-musleabihf * armv7-unknown-linux-musleabihf * mips-unknown-linux-musl * mipsel-unknown-linux-musl This commit doesn't compile custom toolchains like our current cross-image does, but instead compiles musl manually and then compiles libunwind manually (like x86_64) for use for the ARM targets and just uses openwrt toolchains for the mips targets.
2017-01-16 23:15:12 +01:00
ENV SCRIPT python2.7 ../x.py dist --host $HOSTS --target $HOSTS