mirror of
https://github.com/rust-lang/rust.git
synced 2024-10-31 22:41:50 +00:00
mk: Don't run benchmarks with make check
The current suite of benchmarks for the standard distribution take a significant amount of time to run, but it's unclear whether we're gaining any benefit from running them. Some specific pain points: * No one is looking at the data generated by the benchmarks. We have no graphs or analysis of what's happening, so all the data is largely being cast into the void. * No benchmark has ever uncovered a bug, they have always run successfully. * Benchmarks not only take a significant amount of time to run, but also take a significant amount of time to compile. I don't think we should mitigate this for now because it's useful to ensure that they do indeed still compile. This commit disables --bench test runs by default as part of `make check`, flipping the NO_BENCH environment variable to a PLEASE_BENCH variable which will manually enable benchmarking. If and when a dedicated bot is set up for benchmarking, this flag can be enabled on that bot.
This commit is contained in:
parent
ba5f53009a
commit
bfbd732dae
@ -68,7 +68,7 @@
|
||||
#
|
||||
# * `TESTNAME=...` - Specify the name of tests to run
|
||||
# * `CHECK_IGNORED=1` - Run normally-ignored tests
|
||||
# * `NO_BENCH=1` - Don't run crate benchmarks (disable `--bench` flag)
|
||||
# * `PLEASE_BENCH=1` - Run crate benchmarks (enable `--bench` flag)
|
||||
#
|
||||
# * `CFG_ENABLE_VALGRIND=1` - Run tests under valgrind
|
||||
# * `VALGRIND_COMPILE=1` - Run the compiler itself under valgrind
|
||||
|
@ -32,7 +32,7 @@ ifdef CHECK_IGNORED
|
||||
TESTARGS += --ignored
|
||||
endif
|
||||
|
||||
TEST_BENCH = --bench
|
||||
TEST_BENCH =
|
||||
|
||||
# Arguments to the cfail/rfail/rpass/bench tests
|
||||
ifdef CFG_VALGRIND
|
||||
@ -40,8 +40,8 @@ ifdef CFG_VALGRIND
|
||||
TEST_BENCH =
|
||||
endif
|
||||
|
||||
ifdef NO_BENCH
|
||||
TEST_BENCH =
|
||||
ifdef PLEASE_BENCH
|
||||
TEST_BENCH = --bench
|
||||
endif
|
||||
|
||||
# Arguments to the perf tests
|
||||
|
Loading…
Reference in New Issue
Block a user