nix/mk/tests.mk

# Run program $1 as part of ‘make installcheck’.

test-deps =

define run-bash

  .PHONY: $1
  $1: $2
	@env BASH=$(bash) $(bash) $3 < /dev/null

endef

define run-test

  $(eval $(call run-bash,$1.test,$1 $(test-deps),mk/run-test.sh $1))
  $(eval $(call run-bash,$1.test-debug,$1 $(test-deps),mk/debug-test.sh $1))

endef

define run-test-group

  .PHONY: $1.test-group

endef

.PHONY: check installcheck

print-top-help += \
  echo "  check: Run unit tests"; \
  echo "  installcheck: Run functional tests";
-												Revert "Get rid of unicode quotes (#1140)"

This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There
really is no need for such a massive change...

											
										
										
											2016-11-25 23:37:43 +00:00
+								# Run program $1 as part of ‘make installcheck’.
-												Fix the test dependencies

Reuse the pre-existing list rather than the one written as part of #3777

											
										
										
											2020-07-02 15:15:02 +00:00
 								test-deps =
-												Refactor bash test build system a bit

The basic idea here is to separate a few intertwined notions:

1. Not all "run bash tests" are "install tests"

2. Not all "run bash tests" use `tests/functional/init.sh`, or any
   pre-test initialization at all.

This will used in the next commit when we have a test that check unit
test golden master data.

Also, move our custom `PS4` from the test to the test runner, as it is
part of how we want to display the tests, not the test themselves.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>

											
										
										
											2023-11-08 05:30:55 +00:00
+								define run-bash
-												Initial commit (imported from the Nix repo)

											
										
										
											2013-12-10 14:54:34 +00:00
-												Refactor bash test build system a bit

The basic idea here is to separate a few intertwined notions:

1. Not all "run bash tests" are "install tests"

2. Not all "run bash tests" use `tests/functional/init.sh`, or any
   pre-test initialization at all.

This will used in the next commit when we have a test that check unit
test golden master data.

Also, move our custom `PS4` from the test to the test runner, as it is
part of how we want to display the tests, not the test themselves.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>

											
										
										
											2023-11-08 05:30:55 +00:00
+								  .PHONY: $1
 								  $1: $2
 									@env BASH=$(bash) $(bash) $3 < /dev/null
-												Make `./mk/run-test.sh` work by itself; add `mk/debug-test.sh`

First, logic is consolidated in the shell script instead of being spread
between them and makefiles. That makes understanding what is going on a
little easier.

This would not be super interesting by itself, but it gives us a way to
debug tests more easily. *That* in turn I hope is much more compelling.
See the updated manual for details.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>

											
										
										
											2022-12-16 01:17:08 +00:00
-												Refactor bash test build system a bit

The basic idea here is to separate a few intertwined notions:

1. Not all "run bash tests" are "install tests"

2. Not all "run bash tests" use `tests/functional/init.sh`, or any
   pre-test initialization at all.

This will used in the next commit when we have a test that check unit
test golden master data.

Also, move our custom `PS4` from the test to the test runner, as it is
part of how we want to display the tests, not the test themselves.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>

											
										
										
											2023-11-08 05:30:55 +00:00
+								endef
 								define run-test
-												tests: simplify initialisation and wiring

pararameterisation is not actually needed the way things are currently
set up, and it confused me when trying to understand what the code does.

all but one test sources vars-and-functions.sh, which nominally only
defines variables, but in practice is always coupled with the actual
initialisation. while the cleaner way of making this more legible would
be to source variables and initialisation separately, this would produce
a huge diff.

the change requires a few small fixes to keep the tests working:

- only create test home directory during initialisation

  that vars-and-functions.sh wrote to the file system seems not write

- fix creation of the test directory

  due to statefulness, the test home directory was implicitly creating
  the test root, too. decoupling that made it apparent that this was
  probably not intentional, and certainly confusing.

- only source vars-and-functions.sh if init.sh is not needed

  there is one test case that only needs a helper function but no
  initialisation side effects

- remove some unnecessary cleanups and split parts of re-used test code

  there were confusing bits in how initialisation code was repurposed,
  which break if trying to refactor the outer layers naively...

											
										
										
											2024-05-06 13:42:49 +00:00
+								  $(eval $(call run-bash,$1.test,$1 $(test-deps),mk/run-test.sh $1))
 								  $(eval $(call run-bash,$1.test-debug,$1 $(test-deps),mk/debug-test.sh $1))
-												Initial commit (imported from the Nix repo)

											
										
										
											2013-12-10 14:54:34 +00:00
 								endef
-												Refactor bash test build system a bit

The basic idea here is to separate a few intertwined notions:

1. Not all "run bash tests" are "install tests"

2. Not all "run bash tests" use `tests/functional/init.sh`, or any
   pre-test initialization at all.

This will used in the next commit when we have a test that check unit
test golden master data.

Also, move our custom `PS4` from the test to the test runner, as it is
part of how we want to display the tests, not the test themselves.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>

											
										
										
											2023-11-08 05:30:55 +00:00
+								define run-test-group
-												Introduce notion of a test group, use for CA tests

Grouping our tests should make it easier to understand the intent than
one long poorly-arranged list. It also is convenient for running just
the tests for a specific component when working on that component.

We need at least one test group so this isn't dead code; I decided to
collect the tests for the `ca-derivations` and `dynamic-derivations`
experimental features in groups. Do
```bash
make ca.test-group -jN
```
and
```bash
make dyn-drv.test-group -jN
```
to try running just them.

I originally did this as part of #8397 for being able to just the local
overlay store alone. I am PRing it separately now so we can separate
general infra from new features.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>

											
										
										
											2023-07-10 02:24:51 +00:00
 								  .PHONY: $1.test-group
 								endef
-												Initial commit (imported from the Nix repo)

											
										
										
											2013-12-10 14:54:34 +00:00
+								.PHONY: check installcheck
-												mk/tests.mk: document 'check' and 'installcheck' in 'make help'

											
										
										
											2021-10-02 10:09:30 +00:00
 								print-top-help += \
 								  echo "  check: Run unit tests"; \
 								  echo "  installcheck: Run functional tests";