Add an optional --logfile argument to std::test::test_main and to
compiletest.
Use this features and the new 'check-summary.py' script to
summarise all the tests performed by the 'check' target. This is
a short term fix for #2075.
`make check` executes `tidy` after compile. It reminds me that I've left
long lines or trailing whitespaces only after compilation finshed. That
is too late since I have to recompile only because fixing the trivial
formatting issues.
Run tidy first to avoid potentially unnecessary re-compilation.
Each stage is organized more according to Unix standards and to
accommodate multiple target architectures.
stageN/
bin - rustc lives here
lib - libraries that rustc needs
lib/rustc/$(target_triple/ - target libraries
This was having the effect of scrubbing failure error codes. The only affect
of removing this should be that the .out file isn't generated, so subsequent
make invocations will re-run the tests (which is how our other tests work
anyway).
Add a new src/test/pretty directory to hold just source files for testing the
pretty-printer.
Add a new pp-exact directive. When this directive is followed by a file name
it specifies a file containing the output that the pretty-printer should
generate. When pp-exact is not followed by a filename it says that the file
should pretty-print as written.
This will reduce the valgrind deluge when a test fails. The tests themselves
are still run under valgrind. Leave a CTEST_VALGRIND environment variable for
running with the old behavior.
This replaces the make-based test runner with a set of Rust-based test
runners. I believe that all existing functionality has been
preserved. The primary objective is to dogfood the Rust test
framework.
A few main things happen here:
1) The run-pass/lib-* tests are all moved into src/test/stdtest. This
is a standalone test crate intended for all standard library tests. It
compiles to build/test/stdtest.stageN.
2) rustc now compiles into yet another build artifact, this one a test
runner that runs any tests contained directly in the rustc crate. This
allows much more fine-grained unit testing of the compiler. It
compiles to build/test/rustctest.stageN.
3) There is a new custom test runner crate at src/test/compiletest
that reproduces all the functionality for running the compile-fail,
run-fail, run-pass and bench tests while integrating with Rust's test
framework. It compiles to build/test/compiletest.stageN.
4) The build rules have been completely changed to use the new test
runners, while also being less redundant, following the example of the
recent stageN.mk rewrite.
It adds two new features to the cfail/rfail/rpass/bench tests:
1) Tests can specify multiple 'error-pattern' directives which must be
satisfied in order.
2) Tests can specify a 'compile-flags' directive which will make the
test runner provide additional command line arguments to rustc.
There are some downsides, the primary being that Rust has to be
functioning pretty well just to run _any_ tests, which I imagine will
be the source of some frustration when the entire test suite
breaks. Will also cause some headaches during porting.
Not having individual make rules, each rpass, etc test no longer
remembers between runs whether it completed successfully. As a result,
it's not possible to incrementally fix multiple tests by just running
'make check', fixing a test, and repeating without re-running all the
tests contained in the test runner. Instead you can filter just the
tests you want to run by using the TESTNAME environment variable.
This also dispenses with the ability to run stage0 tests, but they
tended to be broken more often than not anyway.
This will link to std and compile with the --test flag. Eventually the
run-pass/lib* tests will move here.
We could also put the std tests directly into the library and compile both a
library version and a test version, but I think this way will make for faster
builds.
Issue #428
