Commit Graph

336 Commits

Author SHA1 Message Date
Matthias Krüger
42010a23f5
Rollup merge of #104675 - SarthakSingh31:issue-101666, r=jyn514
Unsupported query error now specifies if its unsupported for local or external crate

Fixes #101666.
I had to move `keys.rs` from `rustc_query_impl` to `rustc_middle`. I don't know if that is problematic. I couldn't think of any other way to get the needed information inside `rustc_middle`.

r? ```@jyn514```
2022-11-26 10:39:10 +01:00
kadmin
f9750c1554 Add empty ConstKind::Abstract
Initial pass at expr/abstract const/s

Address comments

Switch to using a list instead of &[ty::Const], rm `AbstractConst`

Remove try_unify_abstract_consts

Update comments

Add edits

Recurse more

More edits

Prevent equating associated consts

Move failing test to ui

Changes this test from incremental to ui, and mark it as failing and a known bug.
Does not cause the compiler to ICE, so should be ok.
2022-11-25 09:28:43 +00:00
Sarthak Singh
1767f9f9bf Unsupported query error now specifies if its unsupported for local or external crate 2022-11-24 19:05:17 +05:30
bors
341d8b8a2c Auto merge of #103808 - cjgillot:vec-cache, r=TaKO8Ki
Use an IndexVec to cache queries with index-like key

Revival of an old idea. Let's see if it has more effect.

r? `@ghost`
2022-11-24 06:32:23 +00:00
Camille GILLOT
c49e2501bf Make AbsoluteBytePos a u64. 2022-11-08 18:47:26 +00:00
Nilstrieb
df3187260f
Remove one lifetime from QueryKeyStringBuilder 2022-11-06 13:10:35 +01:00
Nilstrieb
36be251a35
Merge QueryDescription into QueryConfig
`QueryDescription` has gone through a lot of refactoring and doesn't
make sense anymore.
2022-11-05 16:24:13 +01:00
Michael Howell
03968a802c rustdoc: use ThinVec for cleaned generics 2022-11-02 16:17:22 -07:00
Camille GILLOT
ade5cffc2b Move keys module. 2022-11-01 17:02:51 +00:00
Michael Woerister
9117ea9758 Introduce UnordMap, UnordSet, and UnordBag (see MCP 533)
MCP 533: https://github.com/rust-lang/compiler-team/issues/533

Also, as an example, substitute UnordMap for FxHashMap in
used_trait_imports query result.
2022-10-27 13:23:26 +00:00
Patrick Walton
da630ac79d Introduce deduced parameter attributes, and use them for deducing readonly on
indirect immutable freeze by-value function parameters.

Right now, `rustc` only examines function signatures and the platform ABI when
determining the LLVM attributes to apply to parameters. This results in missed
optimizations, because there are some attributes that can be determined via
analysis of the MIR making up the function body. In particular, `readonly`
could be applied to most indirectly-passed by-value function arguments
(specifically, those that are freeze and are observed not to be mutated), but
it currently is not.

This patch introduces the machinery that allows `rustc` to determine those
attributes. It consists of a query, `deduced_param_attrs`, that, when
evaluated, analyzes the MIR of the function to determine supplementary
attributes. The results of this query for each function are written into the
crate metadata so that the deduced parameter attributes can be applied to
cross-crate functions. In this patch, we simply check the parameter for
mutations to determine whether the `readonly` attribute should be applied to
parameters that are indirect immutable freeze by-value.  More attributes could
conceivably be deduced in the future: `nocapture` and `noalias` come to mind.

Adding `readonly` to indirect function parameters where applicable enables some
potential optimizations in LLVM that are discussed in [issue 103103] and [PR
103070] around avoiding stack-to-stack memory copies that appear in functions
like `core::fmt::Write::write_fmt` and `core::panicking::assert_failed`. These
functions pass a large structure unchanged by value to a subfunction that also
doesn't mutate it. Since the structure in this case is passed as an indirect
parameter, it's a pointer from LLVM's perspective. As a result, the
intermediate copy of the structure that our codegen emits could be optimized
away by LLVM's MemCpyOptimizer if it knew that the pointer is `readonly
nocapture noalias` in both the caller and callee. We already pass `nocapture
noalias`, but we're missing `readonly`, as we can't determine whether a
by-value parameter is mutated by examining the signature in Rust. I didn't have
much success with having LLVM infer the `readonly` attribute, even with fat
LTO; it seems that deducing it at the MIR level is necessary.

No large benefits should be expected from this optimization *now*; LLVM needs
some changes (discussed in [PR 103070]) to more aggressively use the `noalias
nocapture readonly` combination in its alias analysis. I have some LLVM patches
for these optimizations and have had them looked over. With all the patches
applied locally, I enabled LLVM to remove all the `memcpy`s from the following
code:

```rust
fn main() {
    println!("Hello {}", 3);
}
```

which is a significant codegen improvement over the status quo. I expect that
if this optimization kicks in in multiple places even for such a simple
program, then it will apply to Rust code all over the place.

[issue 103103]: https://github.com/rust-lang/rust/issues/103103

[PR 103070]: https://github.com/rust-lang/rust/pull/103070
2022-10-21 02:33:15 -07:00
nils
24ce4cfa20
Remove the describe method from the QueryDescription trait
It was called directly already, but now it's even more useless since it
just forwards to the free function. Call it directly.
2022-10-14 22:35:56 +02:00
Nilstrieb
167b3bd3b2
Get rid of rustc_query_description!
Queries can provide an arbitrary expression for their description and
their caching behavior. Before, these expressions where stored in a
`rustc_query_description` macro emitted by the `rustc_queries` macro,
and then used in `rustc_query_impl` to fill out the methods for the
`QueryDescription` trait.

Instead, we now emit two new modules from `rustc_queries` containing the
functions with the expressions. `rustc_query_impl` calls these functions
now instead of invoking the macro.

Since we are now defining some of the functions in
`rustc_middle::query`, we now need all the imports for the key types
there as well.
2022-10-14 22:35:56 +02:00
Cameron Steffen
ff940db666 Rewrite representability 2022-10-07 09:33:46 -05:00
Nicholas Nethercote
9110d925d0 Remove -Ztime option.
The compiler currently has `-Ztime` and `-Ztime-passes`. I've used
`-Ztime-passes` for years but only recently learned about `-Ztime`.

What's the difference? Let's look at the `-Zhelp` output:
```
  -Z        time=val -- measure time of rustc processes (default: no)
  -Z time-passes=val -- measure time of each rustc pass (default: no)
```
The `-Ztime-passes` description is clear, but the `-Ztime` one is less so.
Sounds like it measures the time for the entire process?

No. The real difference is that `-Ztime-passes` prints out info about passes,
and `-Ztime` does the same, but only for a subset of those passes. More
specifically, there is a distinction in the profiling code between a "verbose
generic activity" and an "extra verbose generic activity". `-Ztime-passes`
prints both kinds, while `-Ztime` only prints the first one. (It took me
a close reading of the source code to determine this difference.)

In practice this distinction has low value. Perhaps in the past the "extra
verbose" output was more voluminous, but now that we only print stats for a
pass if it exceeds 5ms or alters the RSS, `-Ztime-passes` is less spammy. Also,
a lot of the "extra verbose" cases are for individual lint passes, and you need
to also use `-Zno-interleave-lints` to see those anyway.

Therefore, this commit removes `-Ztime` and the associated machinery. One thing
to note is that the existing "extra verbose" activities all have an extra
string argument, so the commit adds the ability to accept an extra argument to
the "verbose" activities.
2022-10-06 15:49:44 +11:00
Camille GILLOT
6e76599f5f Correct Key impl for HirId. 2022-10-01 16:20:08 +02:00
Deadbeef
3cb1811e45 Compute lint_levels by definition 2022-10-01 16:12:50 +02:00
bors
3288d3a305 Auto merge of #101785 - jyn514:query-struct-fn-ptrs, r=cjgillot
Use function pointers instead of macro-unrolled loops in rustc_query_impl

By making these standalone functions, we
a) allow making them extensible in the future with a new `QueryStruct`
b) greatly decrease the amount of code in each individual function, avoiding exponential blowup in llvm

Helps with https://github.com/rust-lang/rust/issues/96524. Based on https://github.com/rust-lang/rust/pull/101173; only the last commit is relevant.

r? `@cjgillot`
2022-09-26 00:17:59 +00:00
Joshua Nelson
00cde6d4b9 Move the codegen_unit debug assert from rustc_query_system to query_impl
This allows removing a function from the `DepKind` trait.
2022-09-25 12:08:36 -05:00
Joshua Nelson
7a8369096c Use function pointers instead of macro-unrolled loops in rustc_query_impl
By making these standalone functions, we
a) allow making them extensible in the future with a new `QueryStruct`
b) greatly decrease the amount of code in each individual function, avoiding exponential blowup in llvm
2022-09-25 11:27:12 -05:00
Takayuki Maeda
8fe936099a separate definitions and HIR owners
fix a ui test

use `into`

fix clippy ui test

fix a run-make-fulldeps test

implement `IntoQueryParam<DefId>` for `OwnerId`

use `OwnerId` for more queries

change the type of `ParentOwnerIterator::Item` to `(OwnerId, OwnerNode)`
2022-09-24 23:21:19 +09:00
bors
bb5a016175 Auto merge of #102064 - cjgillot:revert, r=Mark-Simulacrum
Revert perf-regression 101620

Reverts #101862 #101620

r? `@Mark-Simulacrum`
2022-09-24 09:36:29 +00:00
b-naber
a705e65605 rename Unevaluated to UnevaluatedConst 2022-09-23 14:27:34 +02:00
Camille GILLOT
fc43df0333 Revert "Auto merge of #101620 - cjgillot:compute_lint_levels_by_def, r=oli-obk"
This reverts commit 2cb9a65684, reversing
changes made to 750bd1a7ff.
2022-09-22 19:36:11 +02:00
b-naber
9f3784df89 introduce mir::Unevaluated 2022-09-22 12:35:28 +02:00
Matthias Krüger
4757d2d57e
Rollup merge of #101801 - SparrowLii:query_depth_note, r=estebank
add note for `layout_of` when query depth overflows

Fixes #101747
Added `try_find_layout_root` function to add a note for `layout_of` when query depth overflows. This would make the error in #101747 look like this:
```
error: queries overflow the depth limit!
   |
note: Query depth increased by 66 when computing layout of `core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<core::option::Option<alloc::boxed::Box<alloc::string::String>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>`!
  --> D:\rust-backup\parallel_rust\query_depth.rs:40:1
   |
40 | fn main() {
   | ^^^^^^^^^

error: aborting due to previous error
```

cc ``@semicoleon``
2022-09-17 19:27:06 +02:00
Dylan DPC
28b4c62382
Rollup merge of #101787 - compiler-errors:cache-rpitit, r=petrochenkov
cache `collect_trait_impl_trait_tys`

Micro-optimization for RPITITs
2022-09-16 11:17:01 +05:30
bors
294f0eef73 Auto merge of #101173 - jyn514:simplify-macro-arguments, r=cjgillot
Further simplify the macros generated by `rustc_queries`

This doesn't actually move anything outside the macros, but it makes them simpler to read.

- Add a new `rustc_query_names` macro. This allows a much simpler syntax for the matchers in the macros passed to it as a callback.
- Convert `define_dep_nodes` and `alloc_once` to use `rustc_query_names`. This is possible because they only use the names
  (despite the quite complicated matchers in `define_dep_nodes`, none of the other arguments are used).
- Get rid of `rustc_dep_node_append`.

r? `@cjgillot`
2022-09-15 11:54:03 +00:00
SparrowLii
89fd6ae458 correct span, add help message and add UI test when query depth overflows 2022-09-15 16:05:44 +08:00
SparrowLii
44506f38e0 add note for layout_of when query depth overflows 2022-09-15 16:05:00 +08:00
Michael Goulet
4cdf264e6f cache collect_trait_impl_trait_tys 2022-09-14 20:50:52 +00:00
Camille GILLOT
42a92eb54b Correct Key impl for HirId. 2022-09-14 19:06:48 +02:00
Deadbeef
eb19a8a620 Compute lint_levels by definition 2022-09-14 19:02:44 +02:00
bors
a5b58addae Auto merge of #101307 - jyn514:simplify-storage, r=cjgillot
Simplify caching and storage for queries

I highly recommend reviewing commit-by-commit; each individual commit is quite small but it can be hard to see looking at the overall diff that the behavior is the same. Each commit depends on the previous.

r? `@cjgillot`
2022-09-14 02:39:51 +00:00
Dylan DPC
b0455e9ab4
Rollup merge of #101635 - jyn514:queries-new-derived, r=cjgillot
Move `Queries::new` out of the macro

Split out from https://github.com/rust-lang/rust/pull/101178 to make sure it's not contributing to the perf impact.

r? `@cjgillot`
2022-09-10 18:56:10 +05:30
Joshua Nelson
0a9d7dbca2 Remove unnecessary TRY_LOAD_FROM_DISK constant 2022-09-09 20:24:02 -05:00
Joshua Nelson
9273782d55 Move TRY_LOAD_FROM_DISK out of rustc_queries to rustc_query_impl
We want to refer to `crate::plumbing::try_load_from_disk` in the const, but hard-coding it in
rustc_queries, where we don't yet know the crate this macro will be called in, seems kind of hacky.
Do it in query_impl instead.
2022-09-09 20:21:59 -05:00
Joshua Nelson
7208bdee33 Remove cache_on_disk from QueryVTable
This is not only simpler, but removes a generic function and unwrap.
I have hope it will see compile time and bootstrap time improvements.
2022-09-09 20:21:58 -05:00
Joshua Nelson
b164dbc271 Don't create a new try_load_from_disk closure for each query
Instead, define a single function, parameterized only by the return type.
2022-09-09 20:20:12 -05:00
Joshua Nelson
3a4e3c7788 Get rid of the emitted rustc_query_names and rustc_cached_queries macro
We can avoid these by adding slightly more information to `rustc_query_append` instead.
2022-09-06 21:46:31 -05:00
Joshua Nelson
c630c87ceb Support doc-comments in define_dep_nodes 2022-09-06 21:43:15 -05:00
Joshua Nelson
05886e28a4 Further simplify the macros generated by rustc_queries
- Add a new `rustc_query_names` macro. This allows a much simpler syntax for the matchers in the macros passed to it as a callback.
- Convert `define_dep_nodes` and `alloc_once` to use `rustc_query_names`. This is possible because they only use the names
  (despite the quite complicated matchers in `define_dep_nodes`, none of the other arguments are used).
- Get rid of `rustc_dep_node_append`.
2022-09-06 21:43:15 -05:00
Joshua Nelson
d2c53caee2 Move Queries::new out of the macro 2022-09-06 21:41:01 -05:00
Joshua Nelson
4856affd90 Make HandleCycleError an enum instead of a macro-generated closure
- Add a `HandleCycleError` enum to rustc_query_system, along with a `handle_cycle_error` function
- Move `Value` to rustc_query_system, so `handle_cycle_error` can use it
- Move the `Value` impls from rustc_query_impl to rustc_middle. This is necessary due to orphan rules.
2022-09-06 19:26:08 -05:00
Joshua Nelson
4e09a13bb8 Don't create two new closures for each query
- Parameterize DepKindStruct over `'tcx`

    This allows passing in an invariant function pointer in `query_callback`,
    rather than having to try and make it work for any lifetime.

- Add a new `execute_query` function to `QueryDescription` so we can call `tcx.$name` without needing to be in a macro context
2022-09-01 18:47:54 -05:00
Joshua Nelson
4fcc745266 Simplify try_load_from_on_disk_cache 2022-09-01 18:47:54 -05:00
Joshua Nelson
83b6dc9588 Move almost all of the function in query_callbacks to a generic function 2022-09-01 18:47:54 -05:00
Joshua Nelson
8f442e8ded Get rid of fn recover 2022-09-01 18:42:31 -05:00
Joshua Nelson
70f20ac40e Move force_with_dep_node outside the giant macro 2022-09-01 18:42:30 -05:00
Joshua Nelson
375d78012f Move try_on_disk_cache out of the giant macro 2022-09-01 18:42:26 -05:00
Joshua Nelson
1cf7bcc388 Get rid of make_query module 2022-09-01 18:34:37 -05:00
Oli Scherer
1fc9ef1edd tracing::instrument cleanup 2022-09-01 14:54:27 +00:00
Nicholas Nethercote
b38106b6d8 Replace rustc_data_structures::thin_vec::ThinVec with thin_vec::ThinVec.
`rustc_data_structures::thin_vec::ThinVec` looks like this:
```
pub struct ThinVec<T>(Option<Box<Vec<T>>>);
```
It's just a zero word if the vector is empty, but requires two
allocations if it is non-empty. So it's only usable in cases where the
vector is empty most of the time.

This commit removes it in favour of `thin_vec::ThinVec`, which is also
word-sized, but stores the length and capacity in the same allocation as
the elements. It's good in a wider variety of situation, e.g. in enum
variants where the vector is usually/always non-empty.

The commit also:
- Sorts some `Cargo.toml` dependency lists, to make additions easier.
- Sorts some `use` item lists, to make additions easier.
- Changes `clean_trait_ref_with_bindings` to take a
  `ThinVec<TypeBinding>` rather than a `&[TypeBinding]`, because this
  avoid some unnecessary allocations.
2022-08-29 15:42:13 +10:00
bors
4065b89b1e Auto merge of #100946 - jyn514:query-system-3, r=cjgillot
Simplify the arguments to macros generated by the `rustc_queries` proc macro

Very small cleanup. Based on https://github.com/rust-lang/rust/pull/100436 which modifies some of the same code.

r? `@cjgillot`
2022-08-27 08:53:24 +00:00
bors
cfb5ae26a4 Auto merge of #100748 - SparrowLii:query_depth, r=cjgillot
add `depth_limit` in `QueryVTable` to avoid entering a new tcx in `layout_of`

Fixes #49735
Updates #48685

The `layout_of` query needs to check whether it overflows the depth limit, and the current implementation needs to create a new `ImplicitCtxt` inside `layout_of`. However, `start_query` will already create a new `ImplicitCtxt`, so we can check the depth limit in `start_query`.

We can tell whether we need to check the depth limit simply by whether the return value of `to_debug_str` of the query is `layout_of`. But I think adding the `depth_limit` field in `QueryVTable` may be more elegant and more scalable.
2022-08-25 21:27:38 +00:00
bors
76531befc4 Auto merge of #100436 - jyn514:macro-query-system, r=cjgillot
try and simplify some things in the query system
2022-08-25 05:35:27 +00:00
Joshua Nelson
b061550ed3 Remove the $tcx:tt parameter from rustc_query_description
It's unnecessary.
2022-08-24 00:37:49 -05:00
Joshua Nelson
7b8e2a52ff Simplify the syntax for macros generated by rustc_queries
- Disallow multiple macros callbacks in the same invocation. In practice, this was never used.
- Remove the `[]` brackets around the macro name
- Require an `ident`, not an arbitrary `tt`
2022-08-24 00:37:49 -05:00
Joshua Nelson
0bedd354ca Move most of make_query into a generic function, away from the macro
This should both make the code easier to read and also greatly reduce the amount of codegen
the compiler has to do, since it only needs to monomorphize `create_query_frame` for each
new key and not for each query.
2022-08-23 21:52:29 -05:00
Joshua Nelson
1de08b19d1 Get rid of some usages of query_keys
Rustdoc documents these with the name of the type alias instead of normalizing them to the underlying type.
Use associated types instead so that the generated docs for nightly-rustc are easier to read.
2022-08-23 21:33:52 -05:00
Joshua Nelson
b53761969f Remove $tcx metavariable from rustc_query_append
It's not actually necessary and it makes the code harder to read.
2022-08-23 21:33:19 -05:00
SparrowLii
cbc6bd2019 add depth_limit in QueryVTable 2022-08-24 09:42:12 +08:00
klensy
f6329485a8 rmeta/query cache: don't write string values of preinterned symbols 2022-08-20 15:39:21 +03:00
Matthias Krüger
84f81e7974
Rollup merge of #100723 - 5225225:the-easy-ones, r=compiler-errors
Add the diagnostic translation lints to crates that don't emit them

Some of these have a note saying that they should build on a stable compiler, does that mean they shouldn't get these lints? Or can we cfg them out on those?
2022-08-20 07:09:03 +02:00
bors
71ecf5d359 Auto merge of #98851 - klensy:encode_symbols, r=cjgillot
rustc_metadata: dedupe strings to prevent multiple copies in rmeta/query cache blow file size

r? `@cjgillot`

Encodes strings in rmeta/query cache so duplicated ones will be encoded as offsets to first strings, reducing file size.
2022-08-18 23:53:22 +00:00
5225225
09ea9f0a87 Add diagnostic translation lints to crates that don't emit them 2022-08-18 19:29:02 +01:00
Matthias Krüger
1948288615
Rollup merge of #100389 - compiler-errors:return-type-suggestion-cycle, r=cjgillot
Do not report cycle error when inferring return type for suggestion

The UI test is a good example of a case where this happens. The cycle is due to needing the value of the return type `-> _` to compute the variances of items in the crate, but then needing the variances of the items in the crate to do typechecking to infer what `-> _`'s real type is.

Since we're already gonna emit an error in astconv, just delay the cycle bug as an error.
2022-08-17 12:32:51 +02:00
Michael Goulet
5309375d2c Do not report cycle error when inferring return type for suggestion 2022-08-16 03:00:32 +00:00
Miguel Guarniz
3a37f0b7ae Remove usages of opt_remap_env_constness
Signed-off-by: Miguel Guarniz <mi9uel9@gmail.com>
2022-08-15 21:26:31 -04:00
Miguel Guarniz
ffb925c0f0 Remove opt_remap_env_constness from rustc_query_impl
Signed-off-by: Miguel Guarniz <mi9uel9@gmail.com>
2022-08-15 13:49:39 -04:00
klensy
adba4691f6 cache strings while encoding/decoding to compiler artifacts 2022-08-15 17:56:37 +03:00
Cameron Steffen
cf2433a74f Use LocalDefId for closures more 2022-07-30 15:59:17 -05:00
Ralf Jung
3dad266f40 consistently use VTable over Vtable (matching stable stdlib API RawWakerVTable) 2022-07-20 17:12:07 -04:00
5225225
27412d1e3e Use constant eval to do strict validity checks 2022-07-14 22:55:17 +01:00
kadmin
e612e2603c Move abstract const to rustc_middle::ty 2022-07-12 02:21:31 +00:00
Camille GILLOT
e912c8dfe0 Use a dedicated DepKind for the forever-red node. 2022-07-06 23:20:12 +02:00
Camille GILLOT
15530a1c84 Create a forever red node and use it to force side effects. 2022-07-06 23:11:44 +02:00
Camille GILLOT
43bb31b954 Allow to create definitions inside the query system. 2022-07-06 22:50:55 +02:00
Dylan DPC
707c0d9a2d
Rollup merge of #98881 - cjgillot:q-def-kind, r=fee1-dead
Only compute DefKind through the query.
2022-07-06 14:49:08 +05:30
Camille GILLOT
974b1e3e51 Clarify the behaviour from inside the query system. 2022-07-05 22:52:21 +02:00
Camille GILLOT
e3d63203a3 Only compute DefKind through the query. 2022-07-04 10:42:23 +02:00
SparrowLii
fbca21edd2 get rid of tcx in deadlock handler when parallel compilation 2022-06-29 10:02:30 +08:00
bors
3a8b0144c8 Auto merge of #98106 - cjgillot:split-definitions, r=michaelwoerister
Split up `Definitions` and `ResolverAstLowering`.

Split off https://github.com/rust-lang/rust/pull/95573

r? `@michaelwoerister`
2022-06-17 10:00:11 +00:00
Nicholas Nethercote
bb02cc47c4 Move finish out of the Encoder trait.
This simplifies things, but requires making `CacheEncoder` non-generic.

(This was previously merged as commit 4 in #94732 and then was reverted
in #97905 because it caused a perf regression.)
2022-06-16 16:20:32 +10:00
Yuki Okushi
97b9347c93
Rollup merge of #98083 - nnethercote:rename-Encoder, r=bjorn3
Rename rustc_serialize::opaque::Encoder as MemEncoder.

This avoids the name clash with `rustc_serialize::Encoder` (a trait),
and allows lots qualifiers to be removed and imports to be simplified
(e.g. fewer `as` imports).

(This was previously merged as commit 5 in #94732 and then was reverted
in #97905 because of a perf regression caused by commit 4 in #94732.)

r? ```@bjorn3```
2022-06-15 12:02:04 +09:00
Yuki Okushi
bb4805118a
Rollup merge of #98067 - klensy:compiler-deps2, r=Dylan-DPC
compiler: remove unused deps

Removed unused dependencies in compiler crates and moves few `libc` under `target.cfg(unix)` .
2022-06-15 12:02:02 +09:00
Camille GILLOT
34e4d72929 Separate source_span and expn_that_defined from Definitions. 2022-06-14 22:45:51 +02:00
Nicholas Nethercote
abe45a9ffa Rename rustc_serialize::opaque::Encoder as MemEncoder.
This avoids the name clash with `rustc_serialize::Encoder` (a trait),
and allows lots qualifiers to be removed and imports to be simplified
(e.g. fewer `as` imports).

(This was previously merged as commit 5 in #94732 and then was reverted
in #97905 because of a perf regression caused by commit 4 in #94732.)
2022-06-14 14:52:01 +10:00
klensy
4ea4e2e76d remove currently unused deps 2022-06-13 22:20:51 +03:00
Eduard-Mihai Burtescu
d76573abd1 Integrate measureme's hardware performance counter support. 2022-06-13 07:56:47 +00:00
Nicholas Nethercote
3186e311e5 Revert dc08bc51f2. 2022-06-10 11:58:29 +10:00
Nicholas Nethercote
7f51a1b976 Revert b983e42936. 2022-06-10 08:35:03 +10:00
Nicholas Nethercote
b983e42936 Rename rustc_serialize::opaque::Encoder as MemEncoder.
This avoids the name clash with `rustc_serialize::Encoder` (a trait),
and allows lots qualifiers to be removed and imports to be simplified
(e.g. fewer `as` imports).
2022-06-08 09:50:44 +10:00
Nicholas Nethercote
dc08bc51f2 Move finish out of the Encoder trait.
This simplifies things, but requires making `CacheEncoder` non-generic.
2022-06-08 09:21:05 +10:00
Nicholas Nethercote
1acbe7573d Use delayed error handling for Encodable and Encoder infallible.
There are two impls of the `Encoder` trait: `opaque::Encoder` and
`opaque::FileEncoder`. The former encodes into memory and is infallible, the
latter writes to file and is fallible.

Currently, standard `Result`/`?`/`unwrap` error handling is used, but this is a
bit verbose and has non-trivial cost, which is annoying given how rare failures
are (especially in the infallible `opaque::Encoder` case).

This commit changes how `Encoder` fallibility is handled. All the `emit_*`
methods are now infallible. `opaque::Encoder` requires no great changes for
this. `opaque::FileEncoder` now implements a delayed error handling strategy.
If a failure occurs, it records this via the `res` field, and all subsequent
encoding operations are skipped if `res` indicates an error has occurred. Once
encoding is complete, the new `finish` method is called, which returns a
`Result`. In other words, there is now a single `Result`-producing method
instead of many of them.

This has very little effect on how any file errors are reported if
`opaque::FileEncoder` has any failures.

Much of this commit is boring mechanical changes, removing `Result` return
values and `?` or `unwrap` from expressions. The more interesting parts are as
follows.
- serialize.rs: The `Encoder` trait gains an `Ok` associated type. The
  `into_inner` method is changed into `finish`, which returns
  `Result<Vec<u8>, !>`.
- opaque.rs: The `FileEncoder` adopts the delayed error handling
  strategy. Its `Ok` type is a `usize`, returning the number of bytes
  written, replacing previous uses of `FileEncoder::position`.
- Various methods that take an encoder now consume it, rather than being
  passed a mutable reference, e.g. `serialize_query_result_cache`.
2022-06-08 07:01:26 +10:00
Jack Huey
410dcc9674 Fully stabilize NLL 2022-06-03 17:16:41 -04:00
bjorn3
7381ea019c Remove emit_unit
It doesn't do anything for all encoders
2022-06-03 17:02:14 +00:00
Nicholas Nethercote
0b81d7cdc6 Lazify SourceFile::lines.
`SourceFile::lines` is a big part of metadata. It's stored in a compressed form
(a difference list) to save disk space. Decoding it is a big fraction of
compile time for very small crates/programs.

This commit introduces a new type `SourceFileLines` which has a `Lines`
form and a `Diffs` form. The latter is used when the metadata is first
read, and it is only decoded into the `Lines` form when line data is
actually needed. This avoids the decoding cost for many files,
especially in `std`. It's a performance win of up to 15% for tiny
crates/programs where metadata decoding is a high part of compilation
costs.

A `Lock` is needed because the methods that access lines data (which can
trigger decoding) take `&self` rather than `&mut self`. To allow for this,
`SourceFile::lines` now takes a `FnMut` that operates on the lines slice rather
than returning the lines slice.
2022-06-01 10:36:39 +10:00
bors
0f06824013 Auto merge of #97287 - compiler-errors:type-interner, r=jackh726,oli-obk
Move things to `rustc_type_ir`

Finishes some work proposed in https://github.com/rust-lang/compiler-team/issues/341.

r? `@ghost`
2022-05-29 08:20:13 +00:00
Michael Goulet
4638915940 Make TyCtxt implement Interner, make HashStable generic and move to rustc_type_ir 2022-05-28 12:16:05 -07:00
Michael Goulet
a056a953f0 Initial fixes on top of type interner commit 2022-05-28 11:38:22 -07:00
Wilco Kusee
a7015fe816 Move things to rustc_type_ir 2022-05-28 11:38:22 -07:00
Josh Stone
ab57e36268 Update to rebased rustc-rayon 0.4 2022-05-27 20:20:41 -07:00
bors
4f372b14de Auto merge of #97239 - jhpratt:remove-crate-vis, r=joshtriplett
Remove `crate` visibility modifier

FCP to remove this syntax is just about complete in #53120. Once it completes, this should be merged ASAP to avoid merge conflicts.

The first two commits remove usage of the feature in this repository, while the last removes the feature itself.
2022-05-21 06:38:49 +00:00
Jacob Pratt
49c82f31a8
Remove crate visibility usage in compiler 2022-05-20 20:04:54 -04:00
Camille GILLOT
9900ea352b Cache more queries on disk. 2022-05-13 08:06:48 +02:00
xFrednet
2c5e85249f
Move lint expectation checking into a separate query (RFC 2383) 2022-05-08 14:37:14 +02:00
Oli Scherer
0d5a738b8b Enable tracing for all queryies 2022-05-04 16:15:26 +00:00
b-naber
28af967bb9 implement (as of now still unused) query for valtree -> constvalue conversion 2022-04-21 16:37:24 +02:00
Camille GILLOT
07ee031763 Stop using CRATE_DEF_INDEX.
`CRATE_DEF_ID` and `CrateNum::as_def_id` are almost always what we want.
2022-04-17 12:14:42 +02:00
lcnr
bef6f3e895 rework implementation for inherent impls for builtin types 2022-03-30 11:23:58 +02:00
klensy
008fc79dcd Propagate parallel_compiler feature through rustc crates. Turned off feature gives change of builded crates: 238 -> 224. 2022-03-28 08:41:12 +03:00
bors
3b1fe7e7c9 Auto merge of #94084 - Mark-Simulacrum:drop-sharded, r=cjgillot
Avoid query cache sharding code in single-threaded mode

In non-parallel compilers, this is just adding needless overhead at compilation time (since there is only one shard statically anyway). This amounts to roughly ~10 seconds reduction in bootstrap time, with overall neutral (some wins, some losses) performance results.

Parallel compiler performance should be largely unaffected by this PR; sharding is kept there.
2022-02-27 14:04:07 +00:00
Mark Rousskov
22c3a71de1 Switch bootstrap cfgs 2022-02-25 08:00:52 -05:00
bors
026d8ce7f5 Auto merge of #94066 - Mark-Simulacrum:factor-out-simple-def-kind, r=davidtwco
Remove SimpleDefKind

Now that rustc_query_system depends on rustc_hir, we can just directly make use of the regular DefKind.
2022-02-21 03:36:55 +00:00
Mark Rousskov
75ef068920 Delete QueryLookup
This was largely just caching the shard value at this point, which is not
particularly useful -- in the use sites the key was being hashed nearby anyway.
2022-02-20 12:11:28 -05:00
Mark Rousskov
9deed6f74e Move Sharded maps into each QueryCache impl 2022-02-20 12:10:46 -05:00
Mark Rousskov
ddda851fd5 Remove SimpleDefKind 2022-02-17 18:08:45 -05:00
Mark Rousskov
9763486034 Move ty::print methods to Drop-based scope guards 2022-02-16 17:24:23 -05:00
Nicholas Nethercote
a95fb8b150 Overhaul Const.
Specifically, rename the `Const` struct as `ConstS` and re-introduce `Const` as
this:
```
pub struct Const<'tcx>(&'tcx Interned<ConstS>);
```
This now matches `Ty` and `Predicate` more closely, including using
pointer-based `eq` and `hash`.

Notable changes:
- `mk_const` now takes a `ConstS`.
- `Const` was copy, despite being 48 bytes. Now `ConstS` is not, so need a
  we need separate arena for it, because we can't use the `Dropless` one any
  more.
- Many `&'tcx Const<'tcx>`/`&Const<'tcx>` to `Const<'tcx>` changes
- Many `ct.ty` to `ct.ty()` and `ct.val` to `ct.val()` changes.
- Lots of tedious sigil fiddling.
2022-02-15 16:19:59 +11:00
Nicholas Nethercote
e9a0c429c5 Overhaul TyS and Ty.
Specifically, change `Ty` from this:
```
pub type Ty<'tcx> = &'tcx TyS<'tcx>;
```
to this
```
pub struct Ty<'tcx>(Interned<'tcx, TyS<'tcx>>);
```
There are two benefits to this.
- It's now a first class type, so we can define methods on it. This
  means we can move a lot of methods away from `TyS`, leaving `TyS` as a
  barely-used type, which is appropriate given that it's not meant to
  be used directly.
- The uniqueness requirement is now explicit, via the `Interned` type.
  E.g. the pointer-based `Eq` and `Hash` comes from `Interned`, rather
  than via `TyS`, which wasn't obvious at all.

Much of this commit is boring churn. The interesting changes are in
these files:
- compiler/rustc_middle/src/arena.rs
- compiler/rustc_middle/src/mir/visit.rs
- compiler/rustc_middle/src/ty/context.rs
- compiler/rustc_middle/src/ty/mod.rs

Specifically:
- Most mentions of `TyS` are removed. It's very much a dumb struct now;
  `Ty` has all the smarts.
- `TyS` now has `crate` visibility instead of `pub`.
- `TyS::make_for_test` is removed in favour of the static `BOOL_TY`,
  which just works better with the new structure.
- The `Eq`/`Ord`/`Hash` impls are removed from `TyS`. `Interned`s impls
  of `Eq`/`Hash` now suffice. `Ord` is now partly on `Interned`
  (pointer-based, for the `Equal` case) and partly on `TyS`
  (contents-based, for the other cases).
- There are many tedious sigil adjustments, i.e. adding or removing `*`
  or `&`. They seem to be unavoidable.
2022-02-15 16:03:24 +11:00
bors
e7aca89598 Auto merge of #93741 - Mark-Simulacrum:global-job-id, r=cjgillot
Refactor query system to maintain a global job id counter

This replaces the per-shard counters with a single global counter, simplifying
the JobId struct down to just a u64 and removing the need to pipe a DepKind
generic through a bunch of code. The performance implications on non-parallel
compilers are likely minimal (this switches to `Cell<u64>` as the backing
storage over a `u64`, but the latter was already inside a `RefCell` so it's not
really a significance divergence). On parallel compilers, the cost of a single
global u64 counter may be more significant: it adds a serialization point in
theory. On the other hand, we can imagine changing the counter to have a
thread-local component if it becomes worrisome or some similar structure.

The new design is sufficiently simpler that it warrants the potential for slight
changes down the line if/when we get parallel compilation to be more of a
default.

A u64 counter, instead of u32 (the old per-shard width), is chosen to avoid
possibly overflowing it and causing problems; it is effectively impossible that
we would overflow a u64 counter in this context.
2022-02-09 18:54:30 +00:00
bors
9747ee4755 Auto merge of #93724 - Mark-Simulacrum:drop-query-stats, r=michaelwoerister
Delete -Zquery-stats infrastructure

These statistics are computable from the self-profile data and/or ad-hoc collectable as needed, and in the meantime contribute to rustc bootstrap times -- locally, this PR shaves ~2.5% from rustc_query_impl builds in instruction counts.

If this does lose some functionality we want to keep, I think we should migrate it to self-profile (or a similar interface) rather than this ad-hoc reporting.
2022-02-09 15:53:10 +00:00
Mark Rousskov
e240783a4d Switch QueryJobId to a single global counter
This replaces the per-shard counters with a single global counter, simplifying
the JobId struct down to just a u64 and removing the need to pipe a DepKind
generic through a bunch of code. The performance implications on non-parallel
compilers are likely minimal (this switches to `Cell<u64>` as the backing
storage over a `u64`, but the latter was already inside a `RefCell` so it's not
really a significance divergence). On parallel compilers, the cost of a single
global u64 counter may be more significant: it adds a serialization point in
theory. On the other hand, we can imagine changing the counter to have a
thread-local component if it becomes worrisome or some similar structure.

The new design is sufficiently simpler that it warrants the potential for slight
changes down the line if/when we get parallel compilation to be more of a
default.

A u64 counter, instead of u32 (the old per-shard width), is chosen to avoid
possibly overflowing it and causing problems; it is effectively impossible that
we would overflow a u64 counter in this context.
2022-02-08 18:49:55 -05:00
klensy
eb3b29fd09 14956 -> 14952 exports 2022-02-08 00:13:31 +03:00
klensy
7a75ebed09 15221 -> 14956 exports 2022-02-07 22:45:29 +03:00
Mark Rousskov
257839bd88 Delete query stats
These statistics are computable from the self-profile data and/or ad-hoc
collectable as needed, and in the meantime contribute to rustc bootstrap times.
2022-02-06 21:35:00 -05:00
lcnr
a1a30f7548 add a rustc::query_stability lint 2022-02-01 10:15:59 +01:00
Nicholas Nethercote
416399dc10 Make Decodable and Decoder infallible.
`Decoder` has two impls:
- opaque: this impl is already partly infallible, i.e. in some places it
  currently panics on failure (e.g. if the input is too short, or on a
  bad `Result` discriminant), and in some places it returns an error
  (e.g. on a bad `Option` discriminant). The number of places where
  either happens is surprisingly small, just because the binary
  representation has very little redundancy and a lot of input reading
  can occur even on malformed data.
- json: this impl is fully fallible, but it's only used (a) for the
  `.rlink` file production, and there's a `FIXME` comment suggesting it
  should change to a binary format, and (b) in a few tests in
  non-fundamental ways. Indeed #85993 is open to remove it entirely.

And the top-level places in the compiler that call into decoding just
abort on error anyway. So the fallibility is providing little value, and
getting rid of it leads to some non-trivial performance improvements.

Much of this commit is pretty boring and mechanical. Some notes about
a few interesting parts:
- The commit removes `Decoder::{Error,error}`.
- `InternIteratorElement::intern_with`: the impl for `T` now has the same
  optimization for small counts that the impl for `Result<T, E>` has,
  because it's now much hotter.
- Decodable impls for SmallVec, LinkedList, VecDeque now all use
  `collect`, which is nice; the one for `Vec` uses unsafe code, because
  that gave better perf on some benchmarks.
2022-01-22 10:38:31 +11:00
Aaron Hill
70d36a05bc
Show a more informative panic message when DefPathHash does not exist
This should hopefully make it easier to debug incremental compilation
bugs like #93096 without affecting performance.
2022-01-19 17:36:44 -05:00
Josh Stone
f3b8812f24 Update rayon and rustc-rayon 2022-01-10 11:34:07 -08:00
Aaron Hill
d9220924dc
Import SourceFiles from crate before decoding foreign Span
Fixes #92163
Fixes #92014

When writing to the incremental cache, we encode all `Span`s
we encounter, regardless of whether or not their `SourceFile`
comes from the local crate, or from a foreign crate.

When we decode a `Span`, we use the `StableSourceFileId` we encoded
to locate the matching `SourceFile` in the current session. If this
id corresponds to a `SourceFile` from another crate, then we need to
have already imported that `SourceFile` into our current session.

This usually happens automatically during resolution / macro expansion,
when we try to resolve definitions from other crates. In certain cases,
however, we may try to load a `Span` from a transitive dependency
without having ever imported the `SourceFile`s from that crate, leading
to an ICE.

This PR fixes the issue by calling `imported_source_files()`
when we encounter a `SourceFile` with a foreign `CrateNum`.
This ensures that all `SourceFile`s from that crate are imported
into the current session.
2021-12-23 12:56:12 -05:00
bors
a41a6925ba Auto merge of #91957 - nnethercote:rm-SymbolStr, r=oli-obk
Remove `SymbolStr`

This was originally proposed in https://github.com/rust-lang/rust/pull/74554#discussion_r466203544. As well as removing the icky `SymbolStr` type, it allows the removal of a lot of `&` and `*` occurrences.

Best reviewed one commit at a time.

r? `@oli-obk`
2021-12-19 09:31:37 +00:00
Nicholas Nethercote
8cddcd39ba Remove SymbolStr.
By changing `as_str()` to take `&self` instead of `self`, we can just
return `&str`. We're still lying about lifetimes, but it's a smaller lie
than before, where `SymbolStr` contained a (fake) `&'static str`!
2021-12-15 13:30:26 +11:00
LegionMammal978
77a0c65264 Remove in_band_lifetimes from rustc_query_impl
See #91867 for more information.
2021-12-14 12:13:07 -05:00
Deadbeef
42963f4d50
Query modifier 2021-12-12 12:35:00 +08:00
est31
15de4cbc4b Remove redundant [..]s 2021-12-09 00:01:29 +01:00
Mark Rousskov
3215eeb99f
Revert "Add rustc lint, warning when iterating over hashmaps" 2021-10-28 11:01:42 -04:00
bjorn3
f5c3e83013 Avoid a branch on key being local for queries that use the same local and extern providers 2021-10-25 13:36:23 +02:00
bors
28d0e75269 Auto merge of #90210 - cjgillot:qarray2, r=Mark-Simulacrum
Build the query vtable directly.

Continuation of https://github.com/rust-lang/rust/pull/89978.

This shrinks the query interface and attempts to reduce the amount of function pointer calls.
2021-10-25 01:10:50 +00:00
Matthias Krüger
87822b27ee
Rollup merge of #89558 - lcnr:query-stable-lint, r=estebank
Add rustc lint, warning when iterating over hashmaps

r? rust-lang/wg-incr-comp
2021-10-24 15:48:42 +02:00
Camille GILLOT
138e96b719 Do not require QueryCtxt for cache_on_disk. 2021-10-23 18:12:43 +02:00
Camille GILLOT
7c0920f5fb Build the query vtable directly. 2021-10-23 16:59:19 +02:00
Camille GILLOT
0a5666b838 Do not depend on the stored value when trying to cache on disk. 2021-10-21 20:00:45 +02:00
Camille GILLOT
bd5c107672 Build jump table at runtime. 2021-10-20 18:32:29 +02:00
Camille GILLOT
602d3cbce3 Invoke callbacks from rustc_middle. 2021-10-20 18:29:33 +02:00
Camille GILLOT
b09de95fab Merge two query callbacks arrays. 2021-10-20 18:29:27 +02:00
Camille GILLOT
aa404c24dd Make hash_result an Option. 2021-10-20 18:29:18 +02:00
Camille GILLOT
e53404cca6 Move def_path_hash_to_def_id to rustc_middle. 2021-10-20 18:28:54 +02:00
Yuki Okushi
3d95330230
Rollup merge of #87404 - rylev:artifact-size-profiling, r=wesleywiser
Add support for artifact size profiling

This adds support for profiling artifact file sizes (incremental compilation artifacts and query cache to begin with).

Eventually we want to track this in perf.rlo so we can ensure that file sizes do not change dramatically on each pull request.

This relies on support in measureme: https://github.com/rust-lang/measureme/pull/169. Once that lands we can update this PR to not point to a git dependency.

This was worked on together with `@michaelwoerister.`

r? `@wesleywiser`
2021-10-20 04:35:11 +09:00