Document josh subtrees and update subtree repository list (#2182)

This commit is contained in:
Jakub Beránek 2025-01-02 08:18:32 +01:00
parent 1c4ad64a44
commit 047e26bcad

View File

@ -3,24 +3,22 @@
The `rust-lang/rust` git repository depends on several other repos in the `rust-lang` organization. The `rust-lang/rust` git repository depends on several other repos in the `rust-lang` organization.
There are three main ways we use dependencies: There are three main ways we use dependencies:
1. As a Cargo dependency through crates.io (e.g. `rustc-rayon`) 1. As a Cargo dependency through crates.io (e.g. `rustc-rayon`)
2. As a git subtree (e.g. `clippy`) 2. As a git (e.g. `clippy`) or a [josh][josh] (e.g. `miri`) subtree
3. As a git submodule (e.g. `cargo`) 3. As a git submodule (e.g. `cargo`)
As a general rule, use crates.io for libraries that could be useful for others in the ecosystem; use As a general rule, use crates.io for libraries that could be useful for others in the ecosystem; use
subtrees for tools that depend on compiler internals and need to be updated if there are breaking subtrees for tools that depend on compiler internals and need to be updated if there are breaking
changes; and use submodules for tools that are independent of the compiler. changes; and use submodules for tools that are independent of the compiler.
## External Dependencies (subtree) ## External Dependencies (subtrees)
As a developer to this repository, you don't have to treat the following external projects The following external projects are managed using some form of a `subtree`:
differently from other crates that are directly in this repo:
* [Clippy](https://github.com/rust-lang/rust-clippy) * [clippy](https://github.com/rust-lang/rust-clippy)
* [Miri] * [miri](https://github.com/rust-lang/miri)
* [rustfmt](https://github.com/rust-lang/rustfmt) * [rustfmt](https://github.com/rust-lang/rustfmt)
* [rust-analyzer](https://github.com/rust-lang/rust-analyzer) * [rust-analyzer](https://github.com/rust-lang/rust-analyzer)
* [rustc_codegen_cranelift](https://github.com/rust-lang/rustc_codegen_cranelift)
[Miri]: https://github.com/rust-lang/miri
In contrast to `submodule` dependencies In contrast to `submodule` dependencies
(see below for those), the `subtree` dependencies are just regular files and directories which can (see below for those), the `subtree` dependencies are just regular files and directories which can
@ -29,6 +27,20 @@ to these tools should be filed against the tools directly in their respective
upstream repositories. The exception is that when rustc changes are required to upstream repositories. The exception is that when rustc changes are required to
implement a new tool feature or test, that should happen in one collective rustc PR. implement a new tool feature or test, that should happen in one collective rustc PR.
`subtree` dependencies are currently managed by two distinct approaches:
* Using `git subtree`
* `clippy` ([sync guide](https://doc.rust-lang.org/nightly/clippy/development/infrastructure/sync.html#performing-the-sync-from-rust-langrust-to-clippy))
* `rustfmt`
* `rustc_codegen_cranelift` ([sync script](https://github.com/rust-lang/rustc_codegen_cranelift/blob/113af154d459e41b3dc2c5d7d878e3d3a8f33c69/scripts/rustup.sh#L7))
* Using the [josh][josh] tool
* `miri` ([sync guide](https://github.com/rust-lang/miri/blob/master/CONTRIBUTING.md#advanced-topic-syncing-with-the-rustc-repo))
* `rust-analyzer` ([sync script](https://github.com/rust-lang/rust-analyzer/blob/2e13684be123eca7181aa48e043e185d8044a84a/xtask/src/release.rs#L147))
The [josh][josh] tool is an alternative to git subtrees, which manages git history in a different way and scales better to larger repositories. Specific tooling is required to work with josh, you can check out the `miri` or `rust-analyzer` scripts linked above for inspiration. If you want to migrate a subtree from `git subtree` to josh, you can check out [this guide](https://hackmd.io/7pOuxnkdQDaL1Y1FQr65xg).
Below you can find a guide on how to perform push and pull synchronization with the main rustc repo using `git subtree`, although these instructions might differ repo from repo.
### Synchronizing a subtree ### Synchronizing a subtree
Periodically the changes made to subtree based dependencies need to be synchronized between this Periodically the changes made to subtree based dependencies need to be synchronized between this
@ -84,7 +96,6 @@ Now you're done, the `src/tools/clippy` directory behaves as if Clippy were
part of the rustc monorepo, so no one but you (or others that synchronize part of the rustc monorepo, so no one but you (or others that synchronize
subtrees) actually needs to use `git subtree`. subtrees) actually needs to use `git subtree`.
## External Dependencies (submodules) ## External Dependencies (submodules)
Building Rust will also use external git repositories tracked using [git Building Rust will also use external git repositories tracked using [git
@ -111,3 +122,4 @@ the week leading up to the beta cut.
[The Rust Reference]: https://github.com/rust-lang/reference/ [The Rust Reference]: https://github.com/rust-lang/reference/
[toolstate website]: https://rust-lang-nursery.github.io/rust-toolstate/ [toolstate website]: https://rust-lang-nursery.github.io/rust-toolstate/
[Toolstate chapter]: https://forge.rust-lang.org/infra/toolstate.html [Toolstate chapter]: https://forge.rust-lang.org/infra/toolstate.html
[josh]: https://josh-project.github.io/josh/intro.html