Commit Graph

108 Commits

Author SHA1 Message Date
Nicholas Nethercote
416399dc10 Make Decodable and Decoder infallible.
`Decoder` has two impls:
- opaque: this impl is already partly infallible, i.e. in some places it
  currently panics on failure (e.g. if the input is too short, or on a
  bad `Result` discriminant), and in some places it returns an error
  (e.g. on a bad `Option` discriminant). The number of places where
  either happens is surprisingly small, just because the binary
  representation has very little redundancy and a lot of input reading
  can occur even on malformed data.
- json: this impl is fully fallible, but it's only used (a) for the
  `.rlink` file production, and there's a `FIXME` comment suggesting it
  should change to a binary format, and (b) in a few tests in
  non-fundamental ways. Indeed #85993 is open to remove it entirely.

And the top-level places in the compiler that call into decoding just
abort on error anyway. So the fallibility is providing little value, and
getting rid of it leads to some non-trivial performance improvements.

Much of this commit is pretty boring and mechanical. Some notes about
a few interesting parts:
- The commit removes `Decoder::{Error,error}`.
- `InternIteratorElement::intern_with`: the impl for `T` now has the same
  optimization for small counts that the impl for `Result<T, E>` has,
  because it's now much hotter.
- Decodable impls for SmallVec, LinkedList, VecDeque now all use
  `collect`, which is nice; the one for `Vec` uses unsafe code, because
  that gave better perf on some benchmarks.
2022-01-22 10:38:31 +11:00
Aaron Hill
70d36a05bc
Show a more informative panic message when DefPathHash does not exist
This should hopefully make it easier to debug incremental compilation
bugs like #93096 without affecting performance.
2022-01-19 17:36:44 -05:00
Josh Stone
f3b8812f24 Update rayon and rustc-rayon 2022-01-10 11:34:07 -08:00
Aaron Hill
d9220924dc
Import SourceFiles from crate before decoding foreign Span
Fixes #92163
Fixes #92014

When writing to the incremental cache, we encode all `Span`s
we encounter, regardless of whether or not their `SourceFile`
comes from the local crate, or from a foreign crate.

When we decode a `Span`, we use the `StableSourceFileId` we encoded
to locate the matching `SourceFile` in the current session. If this
id corresponds to a `SourceFile` from another crate, then we need to
have already imported that `SourceFile` into our current session.

This usually happens automatically during resolution / macro expansion,
when we try to resolve definitions from other crates. In certain cases,
however, we may try to load a `Span` from a transitive dependency
without having ever imported the `SourceFile`s from that crate, leading
to an ICE.

This PR fixes the issue by calling `imported_source_files()`
when we encounter a `SourceFile` with a foreign `CrateNum`.
This ensures that all `SourceFile`s from that crate are imported
into the current session.
2021-12-23 12:56:12 -05:00
bors
a41a6925ba Auto merge of #91957 - nnethercote:rm-SymbolStr, r=oli-obk
Remove `SymbolStr`

This was originally proposed in https://github.com/rust-lang/rust/pull/74554#discussion_r466203544. As well as removing the icky `SymbolStr` type, it allows the removal of a lot of `&` and `*` occurrences.

Best reviewed one commit at a time.

r? `@oli-obk`
2021-12-19 09:31:37 +00:00
Nicholas Nethercote
8cddcd39ba Remove SymbolStr.
By changing `as_str()` to take `&self` instead of `self`, we can just
return `&str`. We're still lying about lifetimes, but it's a smaller lie
than before, where `SymbolStr` contained a (fake) `&'static str`!
2021-12-15 13:30:26 +11:00
LegionMammal978
77a0c65264 Remove in_band_lifetimes from rustc_query_impl
See #91867 for more information.
2021-12-14 12:13:07 -05:00
Deadbeef
42963f4d50
Query modifier 2021-12-12 12:35:00 +08:00
est31
15de4cbc4b Remove redundant [..]s 2021-12-09 00:01:29 +01:00
Mark Rousskov
3215eeb99f
Revert "Add rustc lint, warning when iterating over hashmaps" 2021-10-28 11:01:42 -04:00
bjorn3
f5c3e83013 Avoid a branch on key being local for queries that use the same local and extern providers 2021-10-25 13:36:23 +02:00
bors
28d0e75269 Auto merge of #90210 - cjgillot:qarray2, r=Mark-Simulacrum
Build the query vtable directly.

Continuation of https://github.com/rust-lang/rust/pull/89978.

This shrinks the query interface and attempts to reduce the amount of function pointer calls.
2021-10-25 01:10:50 +00:00
Matthias Krüger
87822b27ee
Rollup merge of #89558 - lcnr:query-stable-lint, r=estebank
Add rustc lint, warning when iterating over hashmaps

r? rust-lang/wg-incr-comp
2021-10-24 15:48:42 +02:00
Camille GILLOT
138e96b719 Do not require QueryCtxt for cache_on_disk. 2021-10-23 18:12:43 +02:00
Camille GILLOT
7c0920f5fb Build the query vtable directly. 2021-10-23 16:59:19 +02:00
Camille GILLOT
0a5666b838 Do not depend on the stored value when trying to cache on disk. 2021-10-21 20:00:45 +02:00
Camille GILLOT
bd5c107672 Build jump table at runtime. 2021-10-20 18:32:29 +02:00
Camille GILLOT
602d3cbce3 Invoke callbacks from rustc_middle. 2021-10-20 18:29:33 +02:00
Camille GILLOT
b09de95fab Merge two query callbacks arrays. 2021-10-20 18:29:27 +02:00
Camille GILLOT
aa404c24dd Make hash_result an Option. 2021-10-20 18:29:18 +02:00
Camille GILLOT
e53404cca6 Move def_path_hash_to_def_id to rustc_middle. 2021-10-20 18:28:54 +02:00
Yuki Okushi
3d95330230
Rollup merge of #87404 - rylev:artifact-size-profiling, r=wesleywiser
Add support for artifact size profiling

This adds support for profiling artifact file sizes (incremental compilation artifacts and query cache to begin with).

Eventually we want to track this in perf.rlo so we can ensure that file sizes do not change dramatically on each pull request.

This relies on support in measureme: https://github.com/rust-lang/measureme/pull/169. Once that lands we can update this PR to not point to a git dependency.

This was worked on together with `@michaelwoerister.`

r? `@wesleywiser`
2021-10-20 04:35:11 +09:00
lcnr
00e5abe9b6 allow potential_query_instability everywhere 2021-10-15 10:58:18 +02:00
Mark Rousskov
127373822e Remove built-in cache_hit tracking
This was already only enabled in debug_assertions builds. Generally, it seems
like most use cases that would use this could also use the -Zself-profile flag
which also tracks cache hits (in all builds), and so the extra cfg's and such
are not really necessary.

This is largely just a small cleanup though, which primarily is intended to make
other changes easier by avoiding the need to deal with this field.
2021-10-11 16:33:49 -04:00
bors
15491d7b6b Auto merge of #89343 - Mark-Simulacrum:no-args-queries, r=cjgillot
Refactor fingerprint reconstruction

This PR replaces can_reconstruct_query_key with fingerprint_style, which returns the style of the fingerprint for that query. This allows us to avoid trying to extract a DefId (or equivalent) from keys which *are* reconstructible because they're () but not as DefIds.

This is done with the goal of fixing -Zdump-dep-graph, which seems to have broken a while ago (I didn't try to bisect). Currently even on a `fn main() {}` file it'll ICE (you need to also pass -Zquery-dep-graph for it to work at all), and this patch indirectly fixes the cause of that ICE. This also adds a test for it continuing to work.
2021-10-09 13:13:07 +00:00
bors
44995f7afb Auto merge of #89619 - michaelwoerister:incr-vtables, r=nagisa
Turn vtable_allocation() into a query

This PR removes the untracked vtable-const-allocation cache from the `tcx` and turns the `vtable_allocation()` method into a query.

The change is pretty straightforward and should be backportable without too much effort.

Fixes https://github.com/rust-lang/rust/issues/89598.
2021-10-08 09:04:06 +00:00
Michael Woerister
b7cc99142a Turn tcx.vtable_allocation() into a query. 2021-10-07 20:03:00 +02:00
Ryan Levick
757f76ef73 Update to measureme v10 2021-10-07 15:08:44 +02:00
Ryan Levick
947a33bf20 Add support for artifact size profiling 2021-10-07 14:22:29 +02:00
Mark Rousskov
6f78eed1c7 Query the fingerprint style during key reconstruction
Keys can be reconstructed from fingerprints that are not DefPathHash, but then
we cannot extract a DefId from them.
2021-10-06 22:19:48 -04:00
Camille GILLOT
daf8903e8e Do not re-hash foreign spans. 2021-10-06 19:10:07 +02:00
Camille GILLOT
ce21756ed3 Access Session while decoding expn_id. 2021-10-06 19:06:20 +02:00
bors
55111d656f Auto merge of #89266 - cjgillot:session-ich, r=michaelwoerister
Move ICH to rustc_query_system

Based on https://github.com/rust-lang/rust/pull/89183

The StableHashingContext does not need to be in rustc_middle.

This PR moves it to rustc_query_system. This will avoid a dependency between rustc_ast_lowering and rustc_middle in https://github.com/rust-lang/rust/pull/89124.
2021-10-05 09:45:11 +00:00
Camille GILLOT
fedd7785fe Access StableHashingContext in rustc_query_system. 2021-10-03 16:08:55 +02:00
Camille GILLOT
02025d86ac Remove re-export. 2021-10-03 16:08:54 +02:00
bjorn3
e2d3e09b9c Prevent macro ambiguity errors
The previous macro_rules! parsers failed when an additional modifier was added
with ambiguity errors. The error is pretty unclear as to what exactly the cause
here is, but this change simplifies the argument parsing code such that the
error is avoided.
2021-10-02 13:00:19 -04:00
bors
197fc8591e Auto merge of #89120 - In-line:remove_unneded_visible_parents_map, r=estebank
Disable visible path calculation for PrettyPrinter in Ok path of compiler
2021-09-24 05:29:49 +00:00
Mark Rousskov
c746be2219 Migrate to 2021 2021-09-20 22:21:42 -04:00
Alik Aslanyan
9da27f0429
Disable visible path calculation for PrettyPrinter in Ok path of compiler 2021-09-21 00:41:44 +04:00
bors
91198820d7 Auto merge of #88575 - eddyb:fn-abi-queries, r=nagisa
Querify `FnAbi::of_{fn_ptr,instance}` as `fn_abi_of_{fn_ptr,instance}`.

*Note: opening this PR as draft because it's based on #88499*

This more or less replicates the `LayoutOf::layout_of` setup from #88499, to replace `FnAbi::of_{fn_ptr,instance}` with `FnAbiOf::fn_abi_of_{fn_ptr,instance}`, and also route them through queries (which `layout_of` has used for a while).

The two changes at the use sites (other than the names) are:
* return type is now wrapped in `&'tcx`
  * the value *is* interned, which may affect performance
* the `extra_args` list is now an interned `&'tcx ty::List<Ty<'tcx>>`
  * should be cheap (it's empty for anything other than C variadics)

Theoretically, a `FnAbiOfHelpers` implementer could choose to keep the `Result<...>` instead of eagerly erroring, but the only existing users of these APIs are codegen backends, so they don't (want to) take advantage of this.
At least miri could make use of this, since it prefers propagating errors (it "just" doesn't use `FnAbi` yet - cc `@RalfJung).`

The way this is done is probably less efficient than what is possible, because the queries handle the correctness-oriented API (i.e. the split into `fn` pointers vs instances), whereas a lower-level query could end up with more reuse between different instances with identical signatures.

r? `@nagisa` cc `@oli-obk` `@bjorn3`
2021-09-19 21:39:47 +00:00
Eduard-Mihai Burtescu
c1837ef1c5 Querify fn_abi_of_{fn_ptr,instance}. 2021-09-18 04:41:33 +03:00
Michael Woerister
66cf8ea1af Replace cnum_map with tcx.stable_crate_id_to_crate_num() in OnDiskCache. 2021-09-14 13:56:33 +02:00
Michael Woerister
021c0520e3 Fix up comment about OnDiskCache::foreign_expn_data. 2021-09-14 13:56:33 +02:00
Michael Woerister
2b60338ee9 Make DefPathHash->DefId panic for if the mapping fails.
We only use this mapping for cases where we know that it must succeed.
Letting it panic otherwise makes it harder to use the API in unsupported
ways.
2021-09-14 13:56:33 +02:00
Michael Woerister
5445715c20 Remove RawDefId tracking infrastructure from incr. comp. framework.
This infrastructure is obsolete now with the new encoding scheme for
the DefPathHash->DefIndex maps in crate metadata.
2021-09-14 13:56:33 +02:00
Michael Woerister
960893c50a Store DefPathHash->DefIndex map in on-disk-hash-table format in crate metadata.
This encoding allows for random access without an expensive upfront decoding
state which in turn allows simplifying the DefPathIndex lookup logic without
regressing performance.
2021-09-14 13:56:33 +02:00
Manish Goregaokar
f5ac5cadd3
Rollup merge of #88709 - BoxyUwU:thir-abstract-const, r=lcnr
generic_const_exprs: use thir for abstract consts instead of mir

Changes `AbstractConst` building to use `thir` instead of `mir` so that there's less chance of consts unifying when they shouldn't because lowering to mir dropped information (see `abstract-consts-as-cast-5.rs` test)

r? `@lcnr`
2021-09-12 03:44:56 -07:00
Camille GILLOT
940fa9251e Rename decode to data_untracked. 2021-09-10 20:18:22 +02:00
Camille GILLOT
b19ae20aad Track span dependency using a callback. 2021-09-10 20:18:18 +02:00
Camille GILLOT
e85ddeb474 Encode spans relative to their parent. 2021-09-10 20:18:11 +02:00