Commit Graph

55 Commits

Author SHA1 Message Date
Marco Castelluccio ce56daf37b Don't enable profiler on MinGW targets, as compiler-rt can't currently be compiled with MinGW 2017-06-07 12:50:50 +01:00
Marco Castelluccio 4c908a9d37 Enable profiler selectively on some builders 2017-06-06 18:09:33 +01:00
Alex Crichton 7644508b4e appveyor: Wrap handle.exe download in a retry
Should help deal with spurious download failures.
2017-05-15 12:01:09 -07:00
Alex Crichton c30e007b86 Update sccache binaries to mozilla/sccache@d3627d766
This commit updates the sccache binaries to fix a cache load failure seen
on #41926, fixed by mozilla/sccache#119
2017-05-12 09:07:10 -07:00
Alex Crichton aa891a57a1 ci: Update sccache build
Pulls in mozilla/sccache@ef0d77543 to fix #40240 again after the builds included
in #41447 forgot to include the mio fixed included in #41076.

Closes #40240
2017-04-29 00:29:54 -07:00
Alex Crichton 2e72bcb934 appveyor: Use Ninja/sccache on MSVC
Now that the final bug fixes have been merged into sccache we can start
leveraging sccache on the MSVC builders on AppVeyor instead of relying on the
ad-hoc caching strategy of trigger files and whatnot.
2017-04-27 07:19:34 -07:00
Alex Crichton 499b84aca8 travis: Update sccache build used
This build is no longer a forked version with temporary bugfixes, everything
should be upstreamed!
2017-04-27 07:18:01 -07:00
Alex Crichton d1db74b63e appveyor: Upgrade to gcc for mingw 6.3.0
This commit sort of brings back #40777 by upgrading back to 6.3.0. While
investigating #40546 it was discovered that 6.3.0 appears to not spurious
fail in the same way that 6.2.0 does (which we're currently using). The
workaround for #40184 contained in #40777 did not work so this commit also
contains a different workaround for the gdb issue. We will not download the
6.2.0 version of gdb and use that instead of the default version that comes with
6.3.0.

I'm going to optimistically say...

Closes #40546
2017-04-26 06:58:50 -07:00
bors 0777c757a6 Auto merge of #40123 - TimNN:llvm40, r=alexcrichton
LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [x] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [x] cleanup `.gitmodules`
* [x] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

Everything is green 🎉
2017-04-24 22:18:16 +00:00
Tim Neumann 67560f6fae FIN: switch appveryor to mingw with posix threads 2017-04-23 22:00:03 +02:00
Tim Neumann 5bd4e27db2 FIN: update appveyor mingw description 2017-04-23 22:00:02 +02:00
Aidan Hobson Sayers 24640b306e Disable git caches again 2017-04-19 16:29:47 +01:00
Aidan Hobson Sayers bb843582ca Add a comment for disabling errexit 2017-04-14 15:30:01 +01:00
Corey Farwell a2c032631b Rollup merge of #41075 - aidanhs:aphs-enable-appveyor-cache, r=alexcrichton
Re-enable appveyor cache

After breaking the queue last time, I'm cautiously back with a PR to re-enable caching on appveyor.

If you look at https://ci.appveyor.com/project/rust-lang/rust/build/1.0.2623/job/46o90by4ari6gege (one of the multiple runs that started failed, there are actually two errors - one for restoring the cache, one right at the bottom for creating a directory. I only noticed the restore error at the time as I was a bit rushed to revert and didn't stop to wonder why it continued - turns out appveyor [does not abort on cache restore failure](https://github.com/appveyor/ci/issues/723).

Turns out the cause of the build failures was the cache directory existing and me being thinking that because mkdir on windows is [recursive by default](http://stackoverflow.com/a/905239/2352259), it ignores the error if the directory already exists. Apparently this is not true, so now it checks if the directory exists before attempting to create.

In addition, I've added some more paranoia to double check everything is sane.
2017-04-07 09:20:07 -04:00
Alex Crichton e60ea55f66 travis: Update sccache binaries
I've tracked down what I believe is the last spurious sccache failure on #40240
to behavior in mio (carllerche/mio#583), and this commit updates the binaries to
a version which has that fix incorporated.
2017-04-04 15:55:23 -07:00
Aidan Hobson Sayers b970bc2a79 Enable appveyor cache, add more paranoia 2017-04-04 23:41:38 +01:00
Aidan Hobson Sayers fe0fbbff5c Disable appveyor cache 2017-03-30 01:33:45 +01:00
Corey Farwell a9dc8ac7ac Rollup merge of #40780 - aidanhs:aphs-cache-git-modules, r=alexcrichton
Attempt to cache git modules

Partial resolution of #40772, appveyor remains to be done once travis looks like it's working ok.

The approach in this PR is based on the `--reference` flag to `git-clone`/`git-submodule --update` and is a compromise based on the current limitations of the tools we're using.

The ideal would be:
1. have a cached pristine copy of rust-lang/rust master in `$HOME/rustsrc` with all submodules initialised
2. clone the PR branch with `git clone --recurse-submodules --reference $HOME/rustsrc git@github.com:rust-lang/rust.git`

This would (in the nonexistent ideal world) use the pristine copy as an object cache for the top level repo and all submodules, transferring over the network only the changes on the branch. Unfortunately, a) there is no way to manually control the initial clone with travis and b) even if there was, cloned submodules don't use the submodules of the reference as an object cache. So the steps we end up with are:

1. have a cached pristine copy of rust-lang/rust master in `$HOME/rustsrc` with all submodules initialised
2. have a cloned PR branch
3. extract the path of each submodule, and explicitly `git submodule update --init --reference $HOME/rustsrc/$module $module` (i.e. point directly to the location of the pristine submodule repo) for each one

I've also taken some care to make this forward compatible, both for adding and removing submodules.

r? @alexcrichton
2017-03-29 16:53:29 -04:00
Aidan Hobson Sayers 0347ff5823 Attempt to cache git modules 2017-03-29 02:51:56 +01:00
Alex Crichton e87dd42cfe appveyor: Downgrade MinGW to 6.2.0
It looks like the 6.3.0 MinGW comes with a gdb which has issues (#40184) that an
attempted workaround (#40777) does not actually fix (#40835). The original
motivation for upgradin MinGW was to fix build flakiness (#40546) due to newer
builds not exhibiting the same bug, so let's hope that 6.2.0 isn't too far back
in time and still contains the fix we need.

Closes #40835
2017-03-27 13:07:33 -07:00
Alex Crichton bff332e0a2 travis: Update sccache again
Looks like the last version was built with mio 0.6.5 which now has known bugs
against it. This build includes mio 0.6.6
2017-03-24 19:00:35 -07:00
bors 8cf8fc9d89 Auto merge of #40777 - alexcrichton:update-mingw-compilers, r=aturon
appveyor: Upgrade MinGW toolchains we use

In debugging #40546 I was able to reproduce locally finally using
the literal toolchain that the bots were using. I reproduced the error maybe 4
in 10 builds. I also have the 6.3.0 toolchain installed through `pacman` which
has yet to have a failed build.

When attempting to reproduce the bug with the toolchain that this commit
switches to I was unable to reproduce anything after a few builds. I have no
idea what the original problem was, but I'm hoping that it was just some random
bug fixed somewhere along the way.

I don't currently know of a technical reason to stick to the 4.9.2 toolchains we
were previously using. Historcal 5.3.* toolchains would cause llvm to segfault
(maybe a miscompile?) but this seems to have been fixed recently. To me if it
passes CI then I think we're good.

Closes #40546
2017-03-24 18:40:57 +00:00
Alex Crichton e6e8c91961 appveyor: Upgrade MinGW toolchains we use
In debugging #40546 I was able to reproduce locally finally using
the literal toolchain that the bots were using. I reproduced the error maybe 4
in 10 builds. I also have the 6.3.0 toolchain installed through `pacman` which
has yet to have a failed build.

When attempting to reproduce the bug with the toolchain that this commit
switches to I was unable to reproduce anything after a few builds. I have no
idea what the original problem was, but I'm hoping that it was just some random
bug fixed somewhere along the way.

I don't currently know of a technical reason to stick to the 4.9.2 toolchains we
were previously using. Historcal 5.3.* toolchains would cause llvm to segfault
(maybe a miscompile?) but this seems to have been fixed recently. To me if it
passes CI then I think we're good.

Closes #40546
2017-03-24 09:59:18 -07:00
bors 366386c46d Auto merge of #40676 - alexcrichton:update-sccache, r=brson
travis: Update sccache binary

Pick up mozilla/sccache#82
2017-03-24 10:11:05 +00:00
Alex Crichton 05c4051f64 travis: Update sccache binary 2017-03-23 10:16:57 -07:00
Alex Crichton 823715c18e appveyor: Leverage auto-retry to upload to S3
This was recently implemented (appveyor/ci#1387) in response to one of our
feature requests, so let's take advantage of it! I'm going to optimistically
say...

Closes #39074
2017-03-23 07:07:02 -07:00
Alex Crichton 6ae22346b8 appveyor: Use Ninja to build LLVM on MinGW
I have a suspicion that MinGW's make is the cause of #40546 rather than anything
else, but that's purely a suspicion without any facts to back it up. In any case
we'll eventually be moving the MSVC build over to Ninja in order to leverage
sccache regardless, so this commit simply jumpstarts that process by downloading
Ninja for use by MinGW anyway.

I'm not sure if this closes #40546 for real, but this is my current best shot at
closing it out, so...

Closes #40546
2017-03-21 19:18:56 -07:00
Alex Crichton 7305ca3852 travis: Don't set `RUST_LOG` globally
I have a suspicion that this caused a large regression in cycle times by forcing
the compiler to perform more checks on every `debug!` statement, so let's test
this out by removing the `RUST_LOG` env var globally.

This regression in cycle time was witnessed between [two] [builds] where the
[PR] in question didn't do much suspicious. Judging by how the stage0 times
*also* regressed though then this is my best guess.

[two]: https://travis-ci.org/rust-lang/rust/builds/210149932
[builds]: https://travis-ci.org/rust-lang/rust/builds/210179995
[PR]: https://github.com/rust-lang/rust/pull/40446
2017-03-20 14:32:36 -07:00
Alex Crichton ec829fe00e travis: Update sccache binary
I've built a local copy with mozilla/sccache#79 and mozilla/sccache#78. Let's
see if that helps #40240!
2017-03-16 07:55:15 -07:00
Alex Crichton f27bd73fd2 appveyor: Turn down sccache logging
Let's not have it be quite so verbose
2017-03-13 12:47:37 -07:00
Vadim Petrochenkov 9b8b3b2b03 rustbuild: Add option for enabling partial LLVM rebuilds 2017-03-12 14:02:49 +03:00
Alex Crichton c60a58b6d1 Attempt to debug sccache in more locations
This should hopefully add support for debugging OSX and Windows presumed sccache
failures instead of just the Linux ones.
2017-03-11 09:07:50 -08:00
Alex Crichton 65b46098b7 appveyor: Use sccache on pc-windows-gnu for caching
Now that mozilla/sccache#43 is fixed the caching works for MinGW on Windows. We
still can't use it for MSVC just yet, but I'll try to revive that branch at some
point.
2017-02-27 11:51:44 -08:00
Alex Crichton c08f3824cd travis: Make more network requests retryable
This commit attempts to move more network operations to being retryable through
various operations. For example git submodule updates, downloading snapshots,
etc, are now all in retryable steps.

Hopefully this commit can cut down on the number of network failures we've been
seeing!
2017-02-25 21:28:54 -08:00
Alex Crichton 255a87499d Update sccache binaries on CI
Currently CI builds can fail spuriously during the LLVM build (#39003). I
believe this is due to sccache, and I believe that in turn was due to the fact
that the sccache server used to just be a raw mio server. Historically raw mio
servers are quite complicated to get right, but this is why we built Tokio! The
sccache server has been migrated to Tokio which I suspect would fix any latent
issues.

I have no confirmation of this (never been able to reproduce the deadlock
locally), but my hunch is that updating sccache to the master branch will fix
the timeouts during the LLVM build.

The binaries previously came from Gecko's infrastructure, but I've built new
ones by hand for Win/Mac/Linux and uploaded them to our CI bucket.
2017-02-24 13:16:54 -08:00
Alex Crichton 5286c681a9 appveyor: Move MSVC dist builds to their own builder
In the long run we want to separate out the dist builders from the test
builders. This provides us leeway to expand the dist builders with more tools
(e.g. Cargo and the RLS) without impacting cycle times.

Currently the Travis dist builders double-up the platforms they provide builds
for, so I figured we could try that out for MSVC as well. This commit adds a new
AppVeyor builder which runs a dist for all the MSVC targets:

* x86_64-pc-windows-msvc
* i686-pc-windows-msvc
* i586-pc-windows-msvc

If this takes too long and/or times out we'll need to split this up. In any case
we're going to need more capacity from AppVeyor no matter what becaue the two
pc-windows-gnu targets can't cross compile so we need at least 2 more builders
no matter what.
2017-02-16 12:18:36 -08:00
Corey Farwell c9737af4ed Rollup merge of #39754 - alexcrichton:less-assertions, r=brson
travis: Add builders without assertions

This commit adds three new builders, one OSX, one Linux, and one MSVC, which
will produce "nightlies" with LLVM assertions disabled. Currently all nightly
releases have LLVM assertions enabled to catch bugs before they reach the
beta/stable channels. The beta/stable channels, however, do not have LLVM
assertions enabled.

Unfortunately though projects like Servo are stuck on nightlies for the near
future at least and are also suffering very long compile times. The purpose of
this commit is to provide artifacts to these projects which are not distributed
through normal channels (e.g. rustup) but are provided for developers to use
locally if need be.

Logistically these builds will all be uploaded to `rustc-builds-alt` instead of
the `rustc-builds` folder of the `rust-lang-ci` bucket. These builds will stay
there forever (until cleaned out if necessary) and there are no plans to
integrate this with rustup and/or the official release process.
2017-02-14 10:07:31 -05:00
Eduard-Mihai Burtescu d29f0bc8fa Automatically vendor Cargo deps when building the source tarballs. 2017-02-14 01:52:03 +02:00
Alex Crichton 0340ddeb3b travis: Add builders without assertions
This commit adds three new builders, one OSX, one Linux, and one MSVC, which
will produce "nightlies" with LLVM assertions disabled. Currently all nightly
releases have LLVM assertions enabled to catch bugs before they reach the
beta/stable channels. The beta/stable channels, however, do not have LLVM
assertions enabled.

Unfortunately though projects like Servo are stuck on nightlies for the near
future at least and are also suffering very long compile times. The purpose of
this commit is to provide artifacts to these projects which are not distributed
through normal channels (e.g. rustup) but are provided for developers to use
locally if need be.

Logistically these builds will all be uploaded to `rustc-builds-alt` instead of
the `rustc-builds` folder of the `rust-lang-ci` bucket. These builds will stay
there forever (until cleaned out if necessary) and there are no plans to
integrate this with rustup and/or the official release process.
2017-02-11 17:38:09 -08:00
Alex Crichton ce4abc3515 Delete Travis/AppVeyor makefile builders
We no longer need these builders as we're no longer testing the old build
system.
2017-02-06 08:42:54 -08:00
Alex Crichton 45d203df30 travis: Tweak artifact uploads
* Don't upload `*.wixpdb` files by accident
* Don't upload `doc` dir by accident
* Fix level of indirection on Travis
2017-01-30 08:56:30 -08:00
Alex Crichton 8944582d1d travis: Upload all artifacts in build/dist
Previously we only uploaded tarballs, but this modifies Travis/AppVeyor to
upload everything. We shouldn't have anything else in there to worry about and
otherwise we need to be sure to pick up pkg/msi/exe installers.
2017-01-25 11:55:40 -08:00
Alex Crichton f3dfcae202 rustbuild: Start building --enable-extended
This commit adds a new flag to the configure script,
`--enable-extended`, which is intended for specifying a desire to
compile the full suite of Rust tools such as Cargo, the RLS, etc. This
is also an indication that the build system should create combined
installers such as the pkg/exe/msi artifacts.

Currently the `--enable-extended` flag just indicates that combined
installers should be built, and Cargo is itself not compiled just yet
but rather only downloaded from its location. The intention here is to
quickly get to feature parity with the current release process and then
we can start improving it afterwards.

All new files in this PR inside `src/etc/installer` are copied from the
rust-packaging repository.
2017-01-24 14:48:03 -08:00
Alex Crichton 8bee3983f8 appveyor: Don't test i586 MSVC binaries yet
I was hoping
2017-01-20 16:35:58 -08:00
Alex Crichton 72c3148bb3 More test fixes from rollup 2017-01-20 13:49:16 -08:00
Alex Crichton a759406dce Rollup merge of #39167 - alexcrichton:no-more-sha, r=brson
travis: Stop uploading sha256 files

We'll generate these later in the build process and otherwise they could just
cause spurious failures with files overwriting one another.

cc #38531
2017-01-20 08:35:49 -08:00
Alex Crichton c457b819d7 travis: Stop uploading sha256 files
We'll generate these later in the build process and otherwise they could just
cause spurious failures with files overwriting one another.
2017-01-18 16:10:05 -08:00
Alex Crichton 2d52cb9349 appveyor: Test/Dist i586 MSVC
This is a target that we're shipping today, so this commit adds this matrix
entry to AppVeyor. This reuses the existing i686 MSVC matrix entry as it's
currently finishing about a half hour under two hours, which should hopefully
give it enough extra time to run this test suite.
2017-01-16 16:51:13 -08:00
Alex Crichton 318767266f travis: Start uploading artifacts on commits
This commit starts adding the infrastructure for uploading release artifacts
from AppVeyor/Travis on each commit. The idea is that eventually we'll upload a
full release to AppVeyor/Travis in accordance with plans [outlined earlier].

Right now this configures Travis/Appveyor to upload all tarballs in the `dist`
directory, and various images are updated to actually produce tarballs in these
directories. These are nowhere near ready to be actual release artifacts, but
this should allow us to play around with it and test it out. Once this commit
lands we should start seeing artifacts uploaded on each commit.

[outlined earlier]: https://internals.rust-lang.org/t/rust-ci-release-infrastructure-changes/4489
2017-01-12 15:29:04 -08:00
Alex Crichton 3eb459ff5f Merge branch 'aux-tests' of https://github.com/alexcrichton/rust into rollup 2016-12-29 17:29:32 -08:00