The tests started to fail after the repo-wide python 3.10 -> 3.11 update.
This is caused by Bazel's py_binary rule setting the [`PYTHONSAFEPATH`][1]
environment variable, which only has an effect for Python >= 3.11.
Setting this variable avoids prepending the current working directory and the
script's directory. The current test code relied on this behavior and thus
failed with:
```
Traceback (most recent call last):
File "/build/.cache/bazel/_bazel_build/8bcfff1c77854f2a2b07d1413b0fc106/execroot/our_workspace/bazel-out/k8-fastbuild/bin/python/bin.runfiles/our_workspace/python/bin.py", line 6, in <module>
from lib import foo
ModuleNotFoundError: No module named 'lib'
```
See also [bazelbuild/bazel#7091][2]
[1]: https://docs.python.org/3.11/using/cmdline.html#envvar-PYTHONSAFEPATH
[2]: https://github.com/bazelbuild/bazel/issues/7091
Nix's version of bazel has custom patches to make bazel more hermetic when used with nix. One of those is limiting the default
environment of Bash. However, this default environment is insufficient because lots of programs rely on diff being in the path like buildifier, golangci-lint, and other things since golang diff implementations aren't very good.
Ideally we'd include "diffutils" as part of this list for rules that assume the presence of "diff" and don't allow nice knobs to configure otherwise.
Furthermore, bazel has a lot of internal facilities that utilize patch (like when downloading workspaces, we have the option of patching them), so include gnupatch as well.
Fixes#249846
In case bazel dist archive includes .bazelversion (like in 5.4.1 https://github.com/bazelbuild/bazel/issues/18189 or 6.3.1rc1) running bazel commands in dist worktree will require using version specified in .bazelversion. If .bazelversion is absent there any available version will be used.
Let's make sure we don't fail installCheckPhase for dist archives that do include .bazelversion (note: bazel bootstrapping from sources not via bazel works regardless of that file, so that's only about test/check phases)
Since bazel 6.1.0 there was a change in `install_name_tool` invocation
on darwin: `/usr/bin/install_name_tool` -> `/usr/bin/xcrun install_name_tool`
https://github.com/bazelbuild/bazel/pull/17253
So interception of `install_name_tool` broke and build in nixpkgs became
less hermetic. It was either picking up host `xcode` or failing if
xcode isn't installed.
Let's update patching script in nixpkgs.
For errors like https://hydra.nixos.org/build/219158800/nixlog/3
```
ERROR: /private/tmp/nix-build-bazel-6.1.2.drv-0/bazel_src/examples/cpp/BUILD:13:8: Linking examples/cpp/hello-success_test failed: (Exit 1): cc_wrapper.sh failed: error executing command (from target //examples/cpp:hello-success_test)
(cd /private/tmp/nix-build-bazel-6.1.2.drv-0/_bazel__nixbld12/bee562cde99c3bcadaa4e82f18c698fe/sandbox/processwrapper-sandbox/44/execroot/io_bazel && \
exec env - \
PATH=/nix/store/v6f6y9mwmqj9v1jffiy4g96ili4pj5la-python3-3.10.11/bin:/nix/store/mda7jac839k0z33n80pglkhfk3s8panz-unzip-6.0/bin:/nix/store/r9ynkgvbn9lp0x3dvrhr0dskss8wwhc2-which-2.21/bin:/nix/store/1jfsv6gcw2lzzavr3jhydzlmpy3iw9b7-zip-3.0/bin:/nix/store/w66qbvdwgpgbn3ikwyxkhpx1zwaw5rln-cctools-port-973.0.1/bin:/nix/store/99xl1hg0pvscfn7aszp0df034jnprn0j-clang-wrapper-11.1.0/bin:/nix/store/ngbj0z5mr5z7dhi0q11csq1k7rxvfcf7-clang-11.1.0/bin:/nix/store/lhbkcjmrv547plyiybfxqddlpn2l13yl-coreutils-9.1/bin:/nix/store/72fxghdpx8732zizx6jg5d74kblpz7ic-cctools-binutils-darwin-wrapper-973.0.1/bin:/nix/store/qm48dbbhmqk70hm35s9hzbmgvwd6slvs-cctools-binutils-darwin-973.0.1/bin:/nix/store/dv87q4384pyhg8f0hkw3l4lgkiqxwk94-zulu11.48.21-ca-jdk-11.0.11/bin:/nix/store/1alnbw2p2s4wfk3j52mxrbwvcrwh8c1b-bash-5.2-p15/bin:/nix/store/shzh26kvfirgl441lkv7vcw9lgylrgy7-file-5.44/bin:/nix/store/4dzdpald6db495rflbwfldvzn0vaqp7k-findutils-4.9.0/bin:/nix/store/2k3k7wg1azk8mr3393aj8xxj5pv3098r-gawk-5.2.1/bin:/nix/store/3whcyzf1r7cvjx30mfdy6h3krnvfd176-gnugrep-3.7/bin:/nix/store/8j0jryir74m945154zxw3zs17ig51v89-gnused-4.9/bin:/nix/store/0zy67qd1vd6643p0p6hw93kikz11lbml-gnutar-1.34/bin:/nix/store/79617rkc430q3rm6v4c5fs2raixq43qi-gzip-1.12/bin:/nix/store/lhbkcjmrv547plyiybfxqddlpn2l13yl-coreutils-9.1/bin:/nix/store/4dzdpald6db495rflbwfldvzn0vaqp7k-findutils-4.9.0/bin:/nix/store/6fx4a1125c3xfs7alrr0jlyh5cwkshz8-diffutils-3.9/bin:/nix/store/8j0jryir74m945154zxw3zs17ig51v89-gnused-4.9/bin:/nix/store/3whcyzf1r7cvjx30mfdy6h3krnvfd176-gnugrep-3.7/bin:/nix/store/2k3k7wg1azk8mr3393aj8xxj5pv3098r-gawk-5.2.1/bin:/nix/store/ym7jyaml2zj2x6y6mriyh6ch8z5abqr1-gnutar-1.34/bin:/nix/store/79617rkc430q3rm6v4c5fs2raixq43qi-gzip-1.12/bin:/nix/store/m8yqab399pakajchm99nq42qdcxiw1n0-bzip2-1.0.8-bin/bin:/nix/store/iiasa4fvs0d5ijj0qbj9nl7p31bznxah-gnumake-4.4.1/bin:/nix/store/1alnbw2p2s4wfk3j52mxrbwvcrwh8c1b-bash-5.2-p15/bin:/nix/store/w9x4wz6i8y7cy8hfvp5is86fyh2bh6cp-patch-2.7.6/bin:/nix/store/kx2n8m3j2pfq0v9jydav1qpdy1fd22im-xz-5.4.2-bin/bin:/nix/store/41qmp6qwwz3vvf94pccqnvcx1zdp5z9i-file-5.44/bin \
PWD=/proc/self/cwd \
external/local_config_cc/cc_wrapper.sh @bazel-out/darwin-fastbuild/bin/examples/cpp/hello-success_test-2.params)
Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
xcode-select: error: No developer tools were found and no install could be requested (perhaps no UI is present).
If developer tools are located at a non-default location on disk, use `sudo xcode-select --switch path/to/Xcode.app` to specify the Xcode that you wish to use for command line developer tools.
Use `xcode-select --install` to install the standalone command line developer tools, or visit http://adc.apple.com to download Xcode or the standalone command line tools installation package.
See `man xcode-select` for more details.
```
Updating the updater script:
- `//external:all` -> `//external:*` to make it work with recent bazel versions
Avoiding chicken-egg problem by explicitly asking to first build previous version,
then update sources version, then use updater and previous version to update src-deps.
`bazel_self` may not necessarily build out of the box after just the sources version
bump and may depend on updater being run first, which can't run without `bazel_self`.
So listing steps to use old version to help updating to new version
Done with the help of https://github.com/Mindavi/nixpkgs-mark-broken
Tool is still WIP but this is one of the first results.
I manually audited the results and removed some results that were not valid.
Note that some of these packages maybe should have more constrained platforms set
instead of broken set, but I think not being perfectly correct is better than
just keep trying to build all these things and never succeeding.
Some observations:
- Some darwin builds require XCode tools
- aarch64-linux builds sometimes suffer from using gcc9
- gcc9 is getting older and misses some new libraries/features
- Sometimes tools try to do system detection or expect some explicit settings for
platforms that are not x86_64-linux
