nordic-dev.net/rust
nordic-dev.net/rust
2
0
Fork 0
mirror of https://github.com/rust-lang/rust.git synced 2024-11-25 08:13:41 +00:00
Code Issues Packages Projects Releases Wiki Activity

Replace many of the last references to readmes

Browse Source
This commit is contained in:
Mark Mansi 2018-03-16 12:43:22 -05:00
parent 3b6412b943
commit 50f2884dcd
14 changed files with 56 additions and 327 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

297
src/librustc/dep_graph/README.md
View File

@ -1,295 +1,4 @@
# Dependency graph for incremental compilation
To learn more about how dependency tracking works in rustc, see the [rustc
guide].
This module contains the infrastructure for managing the incremental
compilation dependency graph. This README aims to explain how it ought
to be used. In this document, we'll first explain the overall
strategy, and then share some tips for handling specific scenarios.
The high-level idea is that we want to instrument the compiler to
track which parts of the AST and other IR are read/written by what.
This way, when we come back later, we can look at this graph and
determine what work needs to be redone.
### The dependency graph
The nodes of the graph are defined by the enum `DepNode`. They represent
one of three things:
1. HIR nodes (like `Hir(DefId)`) represent the HIR input itself.
2. Data nodes (like `TypeOfItem(DefId)`) represent some computed
information about a particular item.
3. Procedure nodes (like `CoherenceCheckTrait(DefId)`) represent some
procedure that is executing. Usually this procedure is
performing some kind of check for errors. You can think of them as
computed values where the value being computed is `()` (and the
value may fail to be computed, if an error results).
An edge `N1 -> N2` is added between two nodes if either:
- the value of `N1` is used to compute `N2`;
- `N1` is read by the procedure `N2`;
- the procedure `N1` writes the value `N2`.
The latter two conditions are equivalent to the first one if you think
of procedures as values.
### Basic tracking
There is a very general strategy to ensure that you have a correct, if
sometimes overconservative, dependency graph. The two main things you have
to do are (a) identify shared state and (b) identify the current tasks.
### Identifying shared state
Identify "shared state" that will be written by one pass and read by
another. In particular, we need to identify shared state that will be
read "across items" -- that is, anything where changes in one item
could invalidate work done for other items. So, for example:
1. The signature for a function is "shared state".
2. The computed type of some expression in the body of a function is
not shared state, because if it changes it does not itself
invalidate other functions (though it may be that it causes new
monomorphizations to occur, but that's handled independently).
Put another way: if the HIR for an item changes, we are going to
recompile that item for sure. But we need the dep tracking map to tell
us what *else* we have to recompile. Shared state is anything that is
used to communicate results from one item to another.
### Identifying the current task, tracking reads/writes, etc
FIXME(#42293). This text needs to be rewritten for the new red-green
system, which doesn't fully exist yet.
#### Dependency tracking map
`DepTrackingMap` is a particularly convenient way to correctly store
shared state. A `DepTrackingMap` is a special hashmap that will add
edges automatically when `get` and `insert` are called. The idea is
that, when you get/insert a value for the key `K`, we will add an edge
from/to the node `DepNode::Variant(K)` (for some variant specific to
the map).
Each `DepTrackingMap` is parameterized by a special type `M` that
implements `DepTrackingMapConfig`; this trait defines the key and value
types of the map, and also defines a fn for converting from the key to
a `DepNode` label. You don't usually have to muck about with this by
hand, there is a macro for creating it. You can see the complete set
of `DepTrackingMap` definitions in `librustc/middle/ty/maps.rs`.
As an example, let's look at the `adt_defs` map. The `adt_defs` map
maps from the def-id of a struct/enum to its `AdtDef`. It is defined
using this macro:
```rust
dep_map_ty! { AdtDefs: ItemSignature(DefId) -> ty::AdtDefMaster<'tcx> }
// ~~~~~~~ ~~~~~~~~~~~~~ ~~~~~ ~~~~~~~~~~~~~~~~~~~~~~
// | | Key type Value type
// | DepNode variant
// Name of map id type
```
this indicates that a map id type `AdtDefs` will be created. The key
of the map will be a `DefId` and value will be
`ty::AdtDefMaster<'tcx>`. The `DepNode` will be created by
`DepNode::ItemSignature(K)` for a given key.
Once that is done, you can just use the `DepTrackingMap` like any
other map:
```rust
let mut map: DepTrackingMap<M> = DepTrackingMap::new(dep_graph);
map.insert(key, value); // registers dep_graph.write
map.get(key; // registers dep_graph.read
```
#### Memoization
One particularly interesting case is memoization. If you have some
shared state that you compute in a memoized fashion, the correct thing
to do is to define a `RefCell<DepTrackingMap>` for it and use the
`memoize` helper:
```rust
map.memoize(key, || /* compute value */)
```
This will create a graph that looks like
... -> MapVariant(key) -> CurrentTask
where `MapVariant` is the `DepNode` variant that the map is associated with,
and `...` are whatever edges the `/* compute value */` closure creates.
In particular, using the memoize helper is much better than writing
the obvious code yourself:
```rust
if let Some(result) = map.get(key) {
return result;
}
let value = /* compute value */;
map.insert(key, value);
```
If you write that code manually, the dependency graph you get will
include artificial edges that are not necessary. For example, imagine that
two tasks, A and B, both invoke the manual memoization code, but A happens
to go first. The resulting graph will be:
... -> A -> MapVariant(key) -> B
~~~~~~~~~~~~~~~~~~~~~~~~~~~ // caused by A writing to MapVariant(key)
~~~~~~~~~~~~~~~~~~~~ // caused by B reading from MapVariant(key)
This graph is not *wrong*, but it encodes a path from A to B that
should not exist. In contrast, using the memoized helper, you get:
... -> MapVariant(key) -> A
|
+----------> B
which is much cleaner.
**Be aware though that the closure is executed with `MapVariant(key)`
pushed onto the stack as the current task!** That means that you must
add explicit `read` calls for any shared state that it accesses
implicitly from its environment. See the section on "explicit calls to
read and write when starting a new subtask" above for more details.
### How to decide where to introduce a new task
Certainly, you need at least one task on the stack: any attempt to
`read` or `write` shared state will panic if there is no current
task. But where does it make sense to introduce subtasks? The basic
rule is that a subtask makes sense for any discrete unit of work you
may want to skip in the future. Adding a subtask separates out the
reads/writes from *that particular subtask* versus the larger
context. An example: you might have a 'meta' task for all of borrow
checking, and then subtasks for borrow checking individual fns. (Seen
in this light, memoized computations are just a special case where we
may want to avoid redoing the work even within the context of one
compilation.)
The other case where you might want a subtask is to help with refining
the reads/writes for some later bit of work that needs to be memoized.
For example, we create a subtask for type-checking the body of each
fn. However, in the initial version of incr. comp. at least, we do
not expect to actually *SKIP* type-checking -- we only expect to skip
trans. However, it's still useful to create subtasks for type-checking
individual items, because, otherwise, if a fn sig changes, we won't
know which callers are affected -- in fact, because the graph would be
so coarse, we'd just have to retrans everything, since we can't
distinguish which fns used which fn sigs.
### Testing the dependency graph
There are various ways to write tests against the dependency graph.
The simplest mechanism are the
`#[rustc_if_this_changed]` and `#[rustc_then_this_would_need]`
annotations. These are used in compile-fail tests to test whether the
expected set of paths exist in the dependency graph. As an example,
see `src/test/compile-fail/dep-graph-caller-callee.rs`.
The idea is that you can annotate a test like:
```rust
#[rustc_if_this_changed]
fn foo() { }
#[rustc_then_this_would_need(TypeckTables)] //~ ERROR OK
fn bar() { foo(); }
#[rustc_then_this_would_need(TypeckTables)] //~ ERROR no path
fn baz() { }
```
This will check whether there is a path in the dependency graph from
`Hir(foo)` to `TypeckTables(bar)`. An error is reported for each
`#[rustc_then_this_would_need]` annotation that indicates whether a
path exists. `//~ ERROR` annotations can then be used to test if a
path is found (as demonstrated above).
### Debugging the dependency graph
#### Dumping the graph
The compiler is also capable of dumping the dependency graph for your
debugging pleasure. To do so, pass the `-Z dump-dep-graph` flag. The
graph will be dumped to `dep_graph.{txt,dot}` in the current
directory. You can override the filename with the `RUST_DEP_GRAPH`
environment variable.
Frequently, though, the full dep graph is quite overwhelming and not
particularly helpful. Therefore, the compiler also allows you to filter
the graph. You can filter in three ways:
1. All edges originating in a particular set of nodes (usually a single node).
2. All edges reaching a particular set of nodes.
3. All edges that lie between given start and end nodes.
To filter, use the `RUST_DEP_GRAPH_FILTER` environment variable, which should
look like one of the following:
```
source_filter // nodes originating from source_filter
-> target_filter // nodes that can reach target_filter
source_filter -> target_filter // nodes in between source_filter and target_filter
```
`source_filter` and `target_filter` are a `&`-separated list of strings.
A node is considered to match a filter if all of those strings appear in its
label. So, for example:
```
RUST_DEP_GRAPH_FILTER='-> TypeckTables'
```
would select the predecessors of all `TypeckTables` nodes. Usually though you
want the `TypeckTables` node for some particular fn, so you might write:
```
RUST_DEP_GRAPH_FILTER='-> TypeckTables & bar'
```
This will select only the `TypeckTables` nodes for fns with `bar` in their name.
Perhaps you are finding that when you change `foo` you need to re-type-check `bar`,
but you don't think you should have to. In that case, you might do:
```
RUST_DEP_GRAPH_FILTER='Hir&foo -> TypeckTables & bar'
```
This will dump out all the nodes that lead from `Hir(foo)` to
`TypeckTables(bar)`, from which you can (hopefully) see the source
of the erroneous edge.
#### Tracking down incorrect edges
Sometimes, after you dump the dependency graph, you will find some
path that should not exist, but you will not be quite sure how it came
to be. **When the compiler is built with debug assertions,** it can
help you track that down. Simply set the `RUST_FORBID_DEP_GRAPH_EDGE`
environment variable to a filter. Every edge created in the dep-graph
will be tested against that filter -- if it matches, a `bug!` is
reported, so you can easily see the backtrace (`RUST_BACKTRACE=1`).
The syntax for these filters is the same as described in the previous
section. However, note that this filter is applied to every **edge**
and doesn't handle longer paths in the graph, unlike the previous
section.
Example:
You find that there is a path from the `Hir` of `foo` to the type
check of `bar` and you don't think there should be. You dump the
dep-graph as described in the previous section and open `dep-graph.txt`
to see something like:
Hir(foo) -> Collect(bar)
Collect(bar) -> TypeckTables(bar)
That first edge looks suspicious to you. So you set
`RUST_FORBID_DEP_GRAPH_EDGE` to `Hir&foo -> Collect&bar`, re-run, and
then observe the backtrace. Voila, bug fixed!
[rustc guide]: https://rust-lang-nursery.github.io/rustc-guide/query.html

5
src/librustc/infer/higher_ranked/mod.rs
View File

@ -580,7 +580,10 @@ impl<'a, 'gcx, 'tcx> InferCtxt<'a, 'gcx, 'tcx> {
/// the pop occurs as part of the rollback, so an explicit call is not
/// needed (but is also permitted).
///
/// See `README.md` for more details.
/// For more information about how skolemization for HRTBs works, see
/// the [rustc guide].
///
/// [rustc guide]: https://rust-lang-nursery.github.io/rustc-guide/trait-hrtb.html
pub fn skolemize_late_bound_regions<T>(&self,
binder: &ty::Binder<T>,
snapshot: &CombinedSnapshot<'a, 'tcx>)

8
src/librustc/infer/lexical_region_resolve/README.md
View File

@ -1,14 +1,16 @@
# Region inference
> WARNING: This README is obsolete and will be removed soon! For
> more info on how the current borrowck works, see the [rustc guide].
[rustc guide]: https://rust-lang-nursery.github.io/rustc-guide/mir-borrowck.html
## Terminology
Note that we use the terms region and lifetime interchangeably.
## Introduction
See the [general inference README](../README.md) for an overview of
how lexical-region-solving fits into the bigger picture.
Region inference uses a somewhat more involved algorithm than type
inference. It is not the most efficient thing ever written though it
seems to work well enough in practice (famous last words). The reason

9
src/librustc/infer/region_constraints/README.md
View File

@ -1,18 +1,25 @@
# Region constraint collection
> WARNING: This README is obsolete and will be removed soon! For
> more info on how the current borrowck works, see the [rustc guide].
[rustc guide]: https://rust-lang-nursery.github.io/rustc-guide/mir-borrowck.html
## Terminology
Note that we use the terms region and lifetime interchangeably.
## Introduction
As described in the [inference README](../README.md), and unlike
As described in the rustc guide [chapter on type inference][ti], and unlike
normal type inference, which is similar in spirit to H-M and thus
works progressively, the region type inference works by accumulating
constraints over the course of a function. Finally, at the end of
processing a function, we process and solve the constraints all at
once.
[ti]: https://rust-lang-nursery.github.io/rustc-guide/type-inference.html
The constraints are always of one of three possible forms:
- `ConstrainVarSubVar(Ri, Rj)` states that region variable Ri must be

6
src/librustc/infer/region_constraints/mod.rs
View File

@ -468,8 +468,10 @@ impl<'tcx> RegionConstraintCollector<'tcx> {
/// the APIs in `higher_ranked/mod.rs`, such as
/// `skolemize_late_bound_regions` and `plug_leaks`, which will
/// guide you on this path (ensure that the `SkolemizationMap` is
/// consumed and you are good). There are also somewhat extensive
/// comments in `higher_ranked/README.md`.
/// consumed and you are good). For more info on how skolemization
/// for HRTBs works, see the [rustc guide].
///
/// [rustc guide]: https://rust-lang-nursery.github.io/rustc-guide/trait-hrtb.html
///
/// The `snapshot` argument to this function is not really used;
/// it's just there to make it explicit which snapshot bounds the

8
src/librustc/middle/liveness.rs
View File

@ -402,12 +402,10 @@ fn visit_local<'a, 'tcx>(ir: &mut IrMaps<'a, 'tcx>, local: &'tcx hir::Local) {
fn visit_arm<'a, 'tcx>(ir: &mut IrMaps<'a, 'tcx>, arm: &'tcx hir::Arm) {
for pat in &arm.pats {
// for struct patterns, take note of which fields used shorthand (`x`
// rather than `x: x`)
// for struct patterns, take note of which fields used shorthand (`x` rather than `x: x`)
//
// FIXME: according to the rust-lang-nursery/rustc-guide book and
// librustc/README.md, `NodeId`s are to be phased out in favor of
// `HirId`s; however, we need to match the signature of `each_binding`,
// FIXME: according to the rust-lang-nursery/rustc-guide book, `NodeId`s are to be phased
// out in favor of `HirId`s; however, we need to match the signature of `each_binding`,
// which uses `NodeIds`.
let mut shorthand_field_ids = NodeSet();
if let hir::PatKind::Struct(_, ref fields, _) = pat.node {

6
src/librustc/middle/region.rs
View File

@ -11,8 +11,10 @@
//! This file builds up the `ScopeTree`, which describes
//! the parent links in the region hierarchy.
//!
//! Most of the documentation on regions can be found in
//! `middle/infer/region_constraints/README.md`
//! For more information about how MIR-based region-checking works,
//! see the [rustc guide].
//!
//! [rustc guide]: https://rust-lang-nursery.github.io/rustc-guide/mir-borrowck.html
use ich::{StableHashingContext, NodeIdHashingMode};
use util::nodemap::{FxHashMap, FxHashSet};

4
src/librustc/mir/mod.rs
View File

@ -8,9 +8,9 @@
// option. This file may not be copied, modified, or distributed
// except according to those terms.
//! MIR datatypes and passes. See the module-level [README] for details.
//! MIR datatypes and passes. See the [rustc guide] for more info.
//!
//! [README]: https://github.com/rust-lang/rust/blob/master/src/librustc/mir/README.md
//! [rustc guide]: https://rust-lang-nursery.github.io/rustc-guide/mir.html
use graphviz::IntoCow;
use middle::const_val::ConstVal;

3
src/librustc/ty/sty.rs
View File

@ -991,10 +991,11 @@ pub type Region<'tcx> = &'tcx RegionKind;
/// the inference variable is supposed to satisfy the relation
/// *for every value of the skolemized region*. To ensure that doesn't
/// happen, you can use `leak_check`. This is more clearly explained
/// by infer/higher_ranked/README.md.
/// by the [rustc guide].
///
/// [1]: http://smallcultfollowing.com/babysteps/blog/2013/10/29/intermingled-parameter-lists/
/// [2]: http://smallcultfollowing.com/babysteps/blog/2013/11/04/intermingled-parameter-lists/
/// [rustc guide]: https://rust-lang-nursery.github.io/rustc-guide/trait-hrtb.html
#[derive(Clone, PartialEq, Eq, Hash, Copy, RustcEncodable, RustcDecodable, PartialOrd, Ord)]
pub enum RegionKind {
// Region bound in a type or fn declaration which will be

8
src/librustc_back/README.md
View File

@ -1,6 +1,6 @@
NB: This crate is part of the Rust compiler. For an overview of the
compiler as a whole, see
[the README.md file found in `librustc`](../librustc/README.md).
`librustc_back` contains some very low-level details that are
specific to different LLVM targets and so forth.
For more information about how trans works, see the [rustc guide].
[rustc guide]: https://rust-lang-nursery.github.io/rustc-guide/trans.html

5
src/librustc_borrowck/borrowck/README.md
View File

@ -1,5 +1,10 @@
% The Borrow Checker
> WARNING: This README is more or less obsolete, and will be removed
> soon! The new system is described in the [rustc guide].
[rustc guide]: https://rust-lang-nursery.github.io/rustc-guide/mir-borrowck.html
This pass has the job of enforcing memory safety. This is a subtle
topic. This docs aim to explain both the practice and the theory
behind the borrow checker. They start with a high-level overview of

6
src/librustc_driver/README.md
View File

@ -1,7 +1,3 @@
NB: This crate is part of the Rust compiler. For an overview of the
compiler as a whole, see
[the README.md file found in `librustc`](../librustc/README.md).
The `driver` crate is effectively the "main" function for the rust
compiler. It orchestrates the compilation process and "knits together"
the code from the other crates within rustc. This crate itself does
@ -9,4 +5,6 @@ not contain any of the "main logic" of the compiler (though it does
have some code related to pretty printing or other minor compiler
options).
For more information about how the driver works, see the [rustc guide].
[rustc guide]: https://rust-lang-nursery.github.io/rustc-guide/rustc-driver.html

8
src/librustc_trans/README.md
View File

@ -1,7 +1,7 @@
NB: This crate is part of the Rust compiler. For an overview of the
compiler as a whole, see
[the README.md file found in `librustc`](../librustc/README.md).
The `trans` crate contains the code to convert from MIR into LLVM IR,
and then from LLVM IR into machine code. In general it contains code
that runs towards the end of the compilation process.
For more information about how trans works, see the [rustc guide].
[rustc guide]: https://rust-lang-nursery.github.io/rustc-guide/trans.html

10
src/libsyntax/README.md
View File

@ -1,7 +1,9 @@
NB: This crate is part of the Rust compiler. For an overview of the
compiler as a whole, see
[the README.md file found in `librustc`](../librustc/README.md).
The `syntax` crate contains those things concerned purely with syntax
– that is, the AST ("abstract syntax tree"), parser, pretty-printer,
lexer, macro expander, and utilities for traversing ASTs.
For more information about how these things work in rustc, see the
rustc guide:
- [Parsing](https://rust-lang-nursery.github.io/rustc-guide/the-parser.html)
- [Macro Expansion](https://rust-lang-nursery.github.io/rustc-guide/macro-expansion.html)