Bazel 5 is going to be a second long term support release.
Changes from bazel_4:
- compile java_tools (such as ijar) using nix-provided jdk
- local jvm and jdk are now passed using --tool_java_runtime_version, --tool_java_runtime_version and --extra_toolchains flags
see https://docs.bazel.build/versions/5.0.0/bazel-and-java.html for details
- update-srcDeps.py: distdir_tar function renamed to _distdir_tar to properly parse bazel query output executed against bazel 5 sources
- bazel_5.updater: uses bazel_4 to generate src-deps.json. This change was required to handle json.decode calls in bazel 5 dependencies
- protobuf-test and java-test adjusted to take into consideration both jdk 8 (for Bazel < 5.0.0) and jdk 11 (for Bazel >= 5.0.0)
https://blog.bazel.build/2022/01/19/bazel-5.0.htmlhttps://github.com/bazelbuild/bazel/releases/tag/5.0.0
This resolves an issue where the path to the bazel-rc file is sometimes
optimised so the full hash is not present, and hence not picked up by
the scan as a runtime dependency.
Previously the customBash wrapper added the default tools to the PATH of commands from repository rules (which are run in the same environment as Bazel).
In order to remove duplicates in PATH when run_shell is called, the customBash script is removed.
This is consistent with how other platform behave: only look in the local environment if PATH is not set, but it may break builds that explicitly expects a different local environment.
Also begin to start work on cross compilation, though that will have to
be finished later.
The patches are based on the first version of
https://reviews.llvm.org/D99484. It's very annoying to do the
back-porting but the review has uncovered nothing super major so I'm
fine sticking with what I've got.
Beyond making the outputs work, I also strove to re-sync the packages,
as they have been drifting pointlessly apart for some time.
----
Other misc notes, highly incomplete
- lvm-config-native and llvm-config are put in `dev` because they are
tools just for build time.
- Clang no longer has an lld dep. That was introduced in
db29857eb3, but if clang needs help
finding lld when it is used we should just pass it flags / put in the
resource dir. Providing it at build time increases critical path
length for no good reason.
----
A note on `nativeCC`:
`stdenv` takes tools from the previous stage, so:
1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.stdenv.cc`: `(?0, ?1, x)`
while:
1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.targetPackages`: `(x, x, ?2)`
3. `pkgsBuildBuild.targetPackages.stdenv.cc`: `(?1, x, x)`
Use the same JDK for building bazel and for its runtime.
Effectively, the former `toolchain_hostjdk8` java toolchain has been deprecated
and should no longer be used (in newer bazel)[1]:
```
default_java_toolchain(
name = "toolchain_hostjdk8",
...
)
```
[1]: 4fc4868065/tools/jdk/BUILD.tools (L384-L387)
* use default stdenv (clang 7)
* add no-arc.patch to make the xcode_locate tool compile without libarc-lite
* remove the `-mmacosx-version-min=10.9` flag from the bootstrap compile script
- bump protobuf to 3.13.0
- use recent rules_proto
- use recent bazel-skylib
- remove use of native.bind
Hopefully this is backward-compatible with bazel_3 and below
Bazel 4 is going to be a long term support release.
Latest version in NixPkgs so far was 3.3.1
There's a need for more recent version
https://github.com/NixOS/nixpkgs/issues/97497
All versions from 3.5.0 to 3.7.1 had some reproducibility issues
as noted in issue above, but there also seems to be
a working PR for 3.7.1 now at
https://github.com/NixOS/nixpkgs/pull/105439
Notable changes from bazel_3 setup:
- put python to default bash path
For autodetecting python toolchain
with strict action_env on and without this change
bazel would fail to autodetect host python.
There are some repos that define hermetic python
toolchains, but they aren't easy to use yet. Also
telling python paths to bazel isn't a 1-liner it
seems:
- action_env=PATH would affect cache
- declaring toolchain via BUILD&WORKSPACE files
is not per-user but more like per-repo and
affects cache too
Using python from nixpkgs shouldn't be too bad
in the lack of simpler hermetic python toolchain
options
- bazel_4.updater is bazel on `bazel query` to support
new constructs in WORKSPACE (load of vars, transitive
load etc). This is more robust but requires bazel
to run the updater, using bazel_3 for now. This is
only needed to bump package version, doesn't introduce
bazel_4 build dependency on bazel_3
https://blog.bazel.build/2020/11/10/bazel-4.0-announce.htmlhttps://blog.bazel.build/2020/11/10/long-term-support-release.htmlhttps://github.com/bazelbuild/bazel/issues/12455https://github.com/bazelbuild/bazel/releases/tag/4.0.0https://blog.bazel.build/2021/01/19/bazel-4-0.html
* use default stdenv (clang 7)
* add no-arc.patch to make the xcode_locate tool compile without libarc-lite
* remove the `-mmacosx-version-min=10.9` flag from the bootstrap compile script
Use the same JDK for building bazel and for its runtime.
Effectively, the former `toolchain_hostjdk8` java toolchain has been deprecated
and should no longer be used (in newer bazel)[1]:
```
# Deprecated, do not use.
# It will be removed after migration to Java toolchain resolution.
default_java_toolchain(
name = "toolchain_hostjdk8",
...
)
```
[1]: 4fc4868065/tools/jdk/BUILD.tools (L384-L387)
These are fixes for problems I ran into with:
- `bazel test //example:cpp-test`
This needed `build --host_javabase='@local_jdk//:jdk'`
- `bazel query 'deps(//example:cpp-test)'`
This needed the same flags as `build`.
Is it contentious to (partially?) configure the default java toolchain? I don't see it as much different than providing the bazel server's java.
It would continue to be configurable/overridable by overriding the flags.
---
And a random notes from this escapade, but https://github.com/bazelbuild/bazel/blob/master/WORKSPACE#L144-L308 looks a little different from https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/tools/build-managers/bazel/bazel_3/src-deps.json so one of them is probably wrong :)
Limit the resources Bazel is allowed to use during the build to 1/2 the
available RAM and 3/4 the available CPU cores. This should help avoid
overwhelming the build machine.
