Commit Graph

75 Commits

Author SHA1 Message Date
bors
a7e2e33960 Auto merge of #91919 - Aaron1011:query-recursive-read, r=michaelwoerister
Don't perform any new queries while reading a query result on disk

In addition to being very confusing, this can cause us to add dep node edges between two queries that would not otherwise have an edge.

We now panic if any new dep node edges are created during the deserialization of a query result. This requires serializing the full `AdtDef` to disk, instead of just serializing the `DefId` and invoking the `adt_def` query during deserialization.

I'll probably split this up into several smaller PRs for perf runs.
2022-01-08 18:32:31 +00:00
Aaron Hill
27ed52c0a2
Adjust wording of comment 2021-12-23 13:44:04 -05:00
Aaron Hill
28f19f62c7
Address review comments 2021-12-23 13:38:54 -05:00
Aaron Hill
75181dc22f
Error if we try to read dep during deserialization 2021-12-23 13:38:53 -05:00
Aaron Hill
f1d682334d
Add #[rustc_clean(loaded_from_disk)] to assert loading of query result
Currently, you can use `#[rustc_clean]` to assert to that a particular
query (technically, a `DepNode`) is green or red. However, a green
`DepNode` does not mean that the query result was actually deserialized
from disk - we might have never re-run a query that needed the result.

Some incremental tests are written as regression tests for ICEs that
occured during query result decoding. Using
`#[rustc_clean(loaded_from_disk="typeck")]`, you can now assert
that the result of a particular query (e.g. `typeck`) was actually
loaded from disk, in addition to being green.
2021-12-21 16:34:12 -05:00
Camille GILLOT
8785b70774 Inline DepNodeParams methods. 2021-10-20 18:46:25 +02:00
Camille GILLOT
69a3594635 Store node_intern_event_id in CurrentDepGraph. 2021-10-20 18:37:11 +02:00
Camille GILLOT
bd5c107672 Build jump table at runtime. 2021-10-20 18:32:29 +02:00
Camille GILLOT
602d3cbce3 Invoke callbacks from rustc_middle. 2021-10-20 18:29:33 +02:00
Camille GILLOT
b09de95fab Merge two query callbacks arrays. 2021-10-20 18:29:27 +02:00
Camille GILLOT
dc7143367c Drop has_params. 2021-10-20 18:29:22 +02:00
Camille GILLOT
aa404c24dd Make hash_result an Option. 2021-10-20 18:29:18 +02:00
Yuki Okushi
3d95330230
Rollup merge of #87404 - rylev:artifact-size-profiling, r=wesleywiser
Add support for artifact size profiling

This adds support for profiling artifact file sizes (incremental compilation artifacts and query cache to begin with).

Eventually we want to track this in perf.rlo so we can ensure that file sizes do not change dramatically on each pull request.

This relies on support in measureme: https://github.com/rust-lang/measureme/pull/169. Once that lands we can update this PR to not point to a git dependency.

This was worked on together with `@michaelwoerister.`

r? `@wesleywiser`
2021-10-20 04:35:11 +09:00
Ryan Levick
947a33bf20 Add support for artifact size profiling 2021-10-07 14:22:29 +02:00
Mark Rousskov
6f78eed1c7 Query the fingerprint style during key reconstruction
Keys can be reconstructed from fingerprints that are not DefPathHash, but then
we cannot extract a DefId from them.
2021-10-06 22:19:48 -04:00
Camille GILLOT
fedd7785fe Access StableHashingContext in rustc_query_system. 2021-10-03 16:08:55 +02:00
bors
d6cd2c6c87 Auto merge of #82183 - michaelwoerister:lazier-defpathhash-loading2, r=wesleywiser
Simplify lazy DefPathHash decoding by using an on-disk hash table.

This PR simplifies the logic around mapping `DefPathHash` values encountered during incremental compilation to valid `DefId`s in the current session. It is able to do so by using an on-disk hash table encoding that allows for looking up values directly, i.e. without deserializing the entire table.

The main simplification comes from not having to keep track of `DefPathHashes` being used during the compilation session.
2021-09-18 14:37:39 +00:00
Tomasz Miąsko
b6b19f3b6c Use explicit log level in tracing instrument macro
Specify a log level in tracing instrument macro explicitly.

Additionally reduce the used log level from a default info level to a
debug level (all of those appear to be developer oriented logs, so there
should be no need to include them in release builds).
2021-09-15 19:02:10 +02:00
Michael Woerister
5445715c20 Remove RawDefId tracking infrastructure from incr. comp. framework.
This infrastructure is obsolete now with the new encoding scheme for
the DefPathHash->DefIndex maps in crate metadata.
2021-09-14 13:56:33 +02:00
Camille GILLOT
eeb3c8f4b7 Unify with_task functions.
Remove with_eval_always_task.
2021-08-22 20:23:32 +02:00
Camille GILLOT
45d6decc19 Remove try_mark_green_and_read. 2021-08-22 20:23:29 +02:00
Camille GILLOT
c3bf3969d4 Move assertion inwards.
`with_taks_impl` is only called from `with_eval_always_task` and
`with_task` . The former is only used in query invocation, while the
latter is also used to start the `tcx` and to trigger codegen.

This move should not change significantly the number of calls to this
assertion.
2021-08-22 20:23:29 +02:00
Aaron Hill
e6a5231238
Create QuerySideEffects and use it for diagnostics 2021-07-25 20:27:58 -05:00
Ryan Levick
b5bec17184 Add docs to new methods 2021-07-07 11:14:14 +02:00
Ryan Levick
6e33dce9c2 Profile incremental hashing 2021-07-07 10:43:30 +02:00
bors
e4a6032706 Auto merge of #85903 - bjorn3:rustc_serialize_cleanup, r=varkor
Remove unused functions and arguments from rustc_serialize
2021-06-07 14:40:26 +00:00
Camille GILLOT
b51f24f021 Make the reasoning more explicit. 2021-06-01 21:46:30 +02:00
Camille GILLOT
3a6d5c2beb Avoid creating anonymous nodes with zero or one dependency. 2021-06-01 21:43:30 +02:00
bjorn3
a2c4affe86 Remove unused functions and arguments from rustc_serialize 2021-06-01 19:29:11 +02:00
bors
f60a670256 Auto merge of #85319 - cjgillot:query-simp, r=Mark-Simulacrum
Simplification of query forcing

Extracted from #78780
2021-05-30 10:11:23 +00:00
Camille GILLOT
a50f1e949b Get rid of PreviousDepGraph. 2021-05-22 14:14:23 +02:00
Camille GILLOT
eb82187b13 Make the fast path faster. 2021-05-15 10:36:37 +02:00
Camille GILLOT
91444af87a Refactor try_mark_previous_green. 2021-05-15 10:27:27 +02:00
pierwill
0019ca9141 Fix outdated crate names in compiler docs
Changes `librustc_X` to `rustc_X`, only in documentation comments.
Plain code comments are left unchanged.

Also fix incorrect file paths.
2021-04-08 11:12:14 -05:00
Camille GILLOT
f3dde45d2a Enable debugging the dep-graph without debug-assertions.
It may also be useful in these cases,
and some CI configurations test without debug assertions.
2021-03-31 17:12:06 +02:00
Camille GILLOT
8ee9322c10 Also profile finishing the encoding. 2021-03-30 18:10:08 +02:00
Camille GILLOT
df24315ddf Adjust profiling. 2021-03-30 18:10:08 +02:00
Camille GILLOT
fe89f3236c Address review. 2021-03-30 18:10:08 +02:00
Camille GILLOT
65a8681a17 Add documentation. 2021-03-30 18:10:07 +02:00
Camille GILLOT
c5c935af92 Simplify tracking the encoder state. 2021-03-30 18:10:07 +02:00
Camille GILLOT
e1c99e5fcc Remove the parallel version. 2021-03-30 18:10:06 +02:00
Camille GILLOT
8208872fa2 Fix parallel compiler. 2021-03-30 18:10:06 +02:00
Camille GILLOT
cfe786e5e0 Fix tests.
Avoid invoking queries inside `check_paths`, since we are holding a lock
to the reconstructed graph.
2021-03-30 18:10:06 +02:00
Camille GILLOT
39b306a53d Do not allocate in decoder. 2021-03-30 18:10:05 +02:00
Camille GILLOT
6bfaf3a9cb Stream the dep-graph to a file. 2021-03-30 18:09:59 +02:00
Joshua Nelson
441dc3640a Remove (lots of) dead code
Found with https://github.com/est31/warnalyzer.

Dubious changes:
- Is anyone else using rustc_apfloat? I feel weird completely deleting
  x87 support.
- Maybe some of the dead code in rustc_data_structures, in case someone
  wants to use it in the future?
- Don't change rustc_serialize

  I plan to scrap most of the json module in the near future (see
  https://github.com/rust-lang/compiler-team/issues/418) and fixing the
  tests needed more work than I expected.

TODO: check if any of the comments on the deleted code should be kept.
2021-03-27 22:16:33 -04:00
Camille GILLOT
f96e960ccf Access the session directly from DepContext. 2021-02-20 22:53:46 +01:00
Camille GILLOT
b27266fdb2 Use a QueryContext for try_mark_green. 2021-02-19 17:51:56 +01:00
Camille GILLOT
ea3d465c95 Move try_load_from_on_disk_cache to the QueryContext. 2021-02-19 17:51:55 +01:00
Camille GILLOT
49c1b07a9e Decouple QueryContext from DepContext. 2021-02-19 17:51:49 +01:00