Commit Graph

626 Commits

Author SHA1 Message Date
bors
5926e82dd1 Auto merge of #124780 - Mark-Simulacrum:lockless-cache, r=lcnr
Improve VecCache under parallel frontend

This replaces the single Vec allocation with a series of progressively larger buckets. With the cfg for parallel enabled but with -Zthreads=1, this looks like a slight regression in i-count and cycle counts (~1%).

With the parallel frontend at -Zthreads=4, this is an improvement (-5% wall-time from 5.788 to 5.4688 on libcore) than our current Lock-based approach, likely due to reducing the bouncing of the cache line holding the lock. At -Zthreads=32 it's a huge improvement (-46%: 8.829 -> 4.7319 seconds).

try-job: i686-gnu-nopt
try-job: dist-x86_64-linux
2024-11-19 02:07:48 +00:00
Mark Rousskov
da58efb11d Improve VecCache under parallel frontend
This replaces the single Vec allocation with a series of progressively
larger buckets. With the cfg for parallel enabled but with -Zthreads=1,
this looks like a slight regression in i-count and cycle counts (<0.1%).

With the parallel frontend at -Zthreads=4, this is an improvement (-5%
wall-time from 5.788 to 5.4688 on libcore) than our current Lock-based
approach, likely due to reducing the bouncing of the cache line holding
the lock. At -Zthreads=32 it's a huge improvement (-46%: 8.829 -> 4.7319
seconds).
2024-11-15 18:20:32 -05:00
bors
6503543d11 Auto merge of #132282 - Noratrieb:it-is-the-end-of-serial, r=cjgillot
Delete the `cfg(not(parallel))` serial compiler

Since it's inception a long time ago, the parallel compiler and its cfgs have been a maintenance burden. This was a necessary evil the allow iteration while not degrading performance because of synchronization overhead.

But this time is over. Thanks to the amazing work by the parallel working group (and the dyn sync crimes), the parallel compiler has now been fast enough to be shipped by default in nightly for quite a while now.
Stable and beta have still been on the serial compiler, because they can't use `-Zthreads` anyways.
But this is quite suboptimal:
- the maintenance burden still sucks
- we're not testing the serial compiler in nightly

Because of these reasons, it's time to end it. The serial compiler has served us well in the years since it was split from the parallel one, but it's over now.

Let the knight slay one head of the two-headed dragon!

#113349

Note that the default is still 1 thread, as more than 1 thread is still fairly broken.

cc `@onur-ozkan` to see if i did the bootstrap field removal correctly, `@SparrowLii` on the sync parts
2024-11-12 15:14:56 +00:00
Noratrieb
505b8e1332 Delete the cfg(not(parallel)) serial compiler
Since it's inception a long time ago, the parallel compiler and its cfgs
have been a maintenance burden. This was a necessary evil the allow
iteration while not degrading performance because of synchronization
overhead.

But this time is over. Thanks to the amazing work by the parallel
working group (and the dyn sync crimes), the parallel compiler has now
been fast enough to be shipped by default in nightly for quite a while
now.
Stable and beta have still been on the serial compiler, because they
can't use `-Zthreads` anyways.
But this is quite suboptimal:
- the maintenance burden still sucks
- we're not testing the serial compiler in nightly

Because of these reasons, it's time to end it. The serial compiler has
served us well in the years since it was split from the parallel one,
but it's over now.

Let the knight slay one head of the two-headed dragon!
2024-11-12 13:38:58 +00:00
klensy
be3a635fe7 replace manual time convertions with std ones 2024-11-03 15:51:39 +03:00
klensy
746b675c5a fix clippy::clone_on_ref_ptr for compiler 2024-10-28 18:05:08 +03:00
许杰友 Jieyou Xu (Joe)
3528149f73 Introduce Enabled{Lang,Lib}Feature
Instead of passing around random n-tuples of e.g. `(gate_name, attr_sp,
since)`.
2024-10-25 10:30:37 +08:00
Ralf Jung
03048096f6 stop hashing compile-time constant 2024-10-23 19:46:52 +02:00
Ralf Jung
44638853f5 rename lang feature lists to include LANG 2024-10-23 09:14:43 +01:00
Ralf Jung
e82bca6f32 remove no longer needd UnstableFeature type 2024-10-23 09:14:43 +01:00
Ralf Jung
ad3991d303 nightly feature tracking: get rid of the per-feature bool fields 2024-10-23 09:14:41 +01:00
Ralf Jung
1381773e01 make some rustc_feature internals private, and ensure invariants with debug assertions 2024-10-22 07:37:55 +01:00
Ralf Jung
46ce5cbf33 terminology: #[feature] *enables* a feature (instead of "declaring" or "activating" it) 2024-10-22 07:37:54 +01:00
Jacob Kiesel
bb5a8276be add unstable support for outputting file checksums for use in cargo 2024-10-01 21:23:20 -06:00
Michael Goulet
c682aa162b Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
Alexander Cyon
00de006f22
chore: Fix typos in 'compiler' (batch 2) 2024-09-02 07:50:22 +02:00
Nicholas Nethercote
afc58beebe Add warn(unreachable_pub) to rustc_query_system. 2024-08-29 20:18:44 +10:00
c8ef
2575196152 fix: simple typo in compiler directory 2024-08-20 20:50:32 +08:00
Felix Rath
2bf2455925 Prevent double panic in query system, improve diagnostics 2024-08-19 17:42:42 +02:00
León Orell Valerian Liehr
c4c518d2d4
Use more slice patterns inside the compiler 2024-08-07 13:37:52 +02:00
Yuri Astrakhan
84e261e5cb chore: use shorthand initializer 2024-08-02 13:22:28 -04:00
Nicholas Nethercote
84ac80f192 Reformat use declarations.
The previous commit updated `rustfmt.toml` appropriately. This commit is
the outcome of running `x fmt --all` with the new formatting options.
2024-07-29 08:26:52 +10:00
Esteban Küber
2d7795dfb9 Be more accurate about calculating display_col from a BytePos
No longer track "zero-width" chars in `SourceMap`, read directly from the line when calculating the `display_col` of a `BytePos`. Move `char_width` to `rustc_span` and use it from the emitter.

This change allows the following to properly align in terminals (depending on the font, the replaced control codepoints are rendered as 1 or 2 width, on my terminal they are rendered as 1, on VSCode text they are rendered as 2):

```
error: this file contains an unclosed delimiter
  --> $DIR/issue-68629.rs:5:17
   |
LL | ␜␟ts␀![{i
   |       -- unclosed delimiter
   |       |
   |       unclosed delimiter
LL | ␀␀  fn rݻoa>rݻm
   |                ^
```
2024-07-18 20:08:38 +00:00
Michael Goulet
247ad3385c Use dep: for crate dependencies 2024-07-15 12:40:10 -04:00
lcnr
15f770b143 enable fuzzing of SearchGraph
fully move it into `rustc_type_ir` and make it
independent of `Interner`.
2024-07-12 06:30:19 -04:00
bors
c290e9de32 Auto merge of #126326 - eggyal:ununsafe-StableOrd, r=michaelwoerister
Un-unsafe the `StableOrd` trait

Whilst incorrect implementations of this trait can cause miscompilation, they cannot cause memory unsafety in rustc.

[Discussed on Zulip](https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/Policy.20of.20.60unsafe.60.20within.20the.20compiler).

cc [MCP 533](https://github.com/rust-lang/compiler-team/issues/533), #105175, `@michaelwoerister`

r? `@Nilstrieb`
2024-06-25 15:51:35 +00:00
Alan Egerton
0e73e7095a
Ensure careful consideration is given by impls
Added an associated `const THIS_IMPLEMENTATION_HAS_BEEN_TRIPLE_CHECKED`
to the `StableOrd` trait to ensure that implementors carefully consider
whether the trait's contract is upheld, as incorrect implementations can
cause miscompilations.
2024-06-22 07:17:02 +01:00
Michael Baikov
db5ed4bd79 Allow for try_force_from_dep_node to fail
The way it is implemented currently try_force_from_dep_node returns true
as long as there's a function to force the query. It wasn't this way
from the beginning, earlier version was producing forcing result and it
was changed in https://github.com/rust-lang/rust/pull/89978, I couldn't
find any comments addressing this change.

One way it can fail is by failing to recover the query in
DepNodeParams::recover - when we are trying to query something that no
longer exists in the current environment
2024-06-19 07:21:41 -04:00
Oli Scherer
7ba82d61eb Use a dedicated type instead of a reference for the diagnostic context
This paves the way for tracking more state (e.g. error tainting) in the diagnostic context handle
2024-06-18 15:42:11 +00:00
Alan Egerton
114dd2061e
Un-unsafe the StableOrd trait
Whilst incorrect implementations of this trait can cause miscompilation,
they cannot cause memory unsafety in rustc.
2024-06-12 13:01:22 +01:00
Nicholas Nethercote
75b164d836 Use tidy to sort crate attributes for all compiler crates.
We already do this for a number of crates, e.g. `rustc_middle`,
`rustc_span`, `rustc_metadata`, `rustc_span`, `rustc_errors`.

For the ones we don't, in many cases the attributes are a mess.
- There is no consistency about order of attribute kinds (e.g.
  `allow`/`deny`/`feature`).
- Within attribute kind groups (e.g. the `feature` attributes),
  sometimes the order is alphabetical, and sometimes there is no
  particular order.
- Sometimes the attributes of a particular kind aren't even grouped
  all together, e.g. there might be a `feature`, then an `allow`, then
  another `feature`.

This commit extends the existing sorting to all compiler crates,
increasing consistency. If any new attribute line is added there is now
only one place it can go -- no need for arbitrary decisions.

Exceptions:
- `rustc_log`, `rustc_next_trait_solver` and `rustc_type_ir_macros`,
  because they have no crate attributes.
- `rustc_codegen_gcc`, because it's quasi-external to rustc (e.g. it's
  ignored in `rustfmt.toml`).
2024-06-12 15:49:10 +10:00
Oli Scherer
c7ced1ba53 Make the panic info more useful 2024-06-06 09:47:10 +00:00
Ben Kimock
95150d7246 Add a footer in FileEncoder and check for it in MemDecoder 2024-05-21 20:12:29 -04:00
Nicholas Nethercote
9b3ef5404c Remove some unneeded Cargo.toml dependencies.
I found these with a hacky shell script.
2024-05-03 15:33:52 +10:00
Nicholas Nethercote
6341935a13 Remove extern crate tracing from numerous crates. 2024-04-30 16:47:49 +10:00
Nicholas Nethercote
f3e05d1609 Remove extern crate rustc_data_structures from rustc_query_system. 2024-04-29 15:40:29 +10:00
Nicholas Nethercote
4814fd0a4b Remove extern crate rustc_macros from numerous crates. 2024-04-29 10:21:54 +10:00
Guillaume Gomez
a60ccc1876
Rollup merge of #124252 - michaelwoerister:better-forbidden-read-ice, r=oli-obk
Improve ICE message for forbidden dep-graph reads.

The new message mentions the main context that the ICE might occur in and it mentions the query/dep-node that is being read.

cc https://github.com/rust-lang/rust/pull/123781, where this would have been helpful.
2024-04-22 20:26:00 +02:00
Markus Reiter
33e68aadc9
Stabilize generic NonZero. 2024-04-22 18:48:47 +02:00
Michael Woerister
6146a51f17 Add more context to the forbidden dep-graph read ICE error message. 2024-04-22 14:54:28 +02:00
Michael Woerister
c373ec07c4 Improve ICE message for forbidden dep-graph reads. 2024-04-22 12:11:07 +02:00
Matthias Krüger
f9ca213510 remove some things that do not need to be 2024-04-11 21:09:52 +02:00
Mark Rousskov
668b3188ab Remove sharding for VecCache
This sharding is never used (per the comment in code). If we re-add
sharding at some point in the future this is cheap to restore, but for
now no need for the extra complexity.
2024-04-06 10:49:31 -04:00
bors
4563f70c3b Auto merge of #122070 - Zoxc:dep-edges-from-previous, r=cjgillot
Encode dep graph edges directly from the previous graph when promoting

This encodes dep graph edges directly from the previous graph when promoting nodes from a previous session, avoiding allocations / copies.

~~Based on https://github.com/rust-lang/rust/pull/122064 and https://github.com/rust-lang/rust/pull/116375.~~

<table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th></tr><tr><td>🟣 <b>clap</b>:check:unchanged</td><td align="right">0.4177s</td><td align="right">0.4072s</td><td align="right">💚  -2.52%</td></tr><tr><td>🟣 <b>hyper</b>:check:unchanged</td><td align="right">0.1430s</td><td align="right">0.1420s</td><td align="right"> -0.69%</td></tr><tr><td>🟣 <b>regex</b>:check:unchanged</td><td align="right">0.3106s</td><td align="right">0.3038s</td><td align="right">💚  -2.19%</td></tr><tr><td>🟣 <b>syn</b>:check:unchanged</td><td align="right">0.5823s</td><td align="right">0.5688s</td><td align="right">💚  -2.33%</td></tr><tr><td>🟣 <b>syntex_syntax</b>:check:unchanged</td><td align="right">1.3992s</td><td align="right">1.3692s</td><td align="right">💚  -2.14%</td></tr><tr><td>Total</td><td align="right">2.8528s</td><td align="right">2.7910s</td><td align="right">💚  -2.17%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">0.9803s</td><td align="right">💚  -1.97%</td></tr></table>
2024-04-05 11:11:17 +00:00
Vadim Petrochenkov
b40ea03f8a rustc_index: Add a ZERO constant to index types
It is commonly used.
2024-04-03 19:06:22 +03:00
Oli Scherer
3b94f33c23 Remove CacheSelector trait now that we can use GATs 2024-03-26 11:03:23 +00:00
John Kåre Alsaker
aa9c9a36c0 Add some comments and do some renames 2024-03-23 20:23:25 +01:00
John Kåre Alsaker
6119763e19 Encode dep graph edges directly from the previous graph when promoting 2024-03-23 20:03:55 +01:00
Jubilee
f54350a9ba
Rollup merge of #122245 - saethlin:check-dep-graph-size, r=petrochenkov
Detect truncated DepGraph files

I suspect that the following issues are caused by truncated incr comp files:

* https://github.com/rust-lang/rust/issues/120582
* https://github.com/rust-lang/rust/issues/121499
* https://github.com/rust-lang/rust/issues/122210

We fail with an allocation failure or capacity overflow in this case because we assume that the ending bytes of an DepGraph file are the lengths of arrays. If the file has somehow been truncated then the ending bytes are probably some of our varint encoding, which tries to eliminate zero bytes, so interpreting a random 8 bytes as an array length has a very high chance of producing a byte capacity over `isize::MAX`.

Now theoretically since https://github.com/rust-lang/rust/pull/119510 merged I have fixed the out-of-disk issues and yet in https://github.com/rust-lang/rust/issues/120894#issuecomment-1945126700 I still see some decoding failures that look like out-of-disk ICEs, for example https://crater-reports.s3.amazonaws.com/beta-1.77-1/beta-2024-02-10/gh/scottfones.aoc_2022/log.txt

So this PR should ensure that we get an ICE that clearly identifies if the file in question is truncated.
2024-03-12 09:04:00 -07:00
bors
094a6204f5 Auto merge of #122064 - Zoxc:dep-graph-encode-tweaks, r=cjgillot
Dep node encoding cleanups

This does some cleanups around dep node encoding.

Performance change with `-Zthreads=2`:

<table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th><td align="right">Memory</td><td align="right">Memory</td><td align="right">%</th></tr><tr><td>🟣 <b>clap</b>:check:unchanged</td><td align="right">0.4337s</td><td align="right">0.4306s</td><td align="right"> -0.72%</td><td align="right">88.90 MiB</td><td align="right">89.04 MiB</td><td align="right"> 0.15%</td></tr><tr><td>🟣 <b>hyper</b>:check:unchanged</td><td align="right">0.1541s</td><td align="right">0.1528s</td><td align="right"> -0.86%</td><td align="right">51.99 MiB</td><td align="right">52.03 MiB</td><td align="right"> 0.07%</td></tr><tr><td>🟣 <b>regex</b>:check:unchanged</td><td align="right">0.3286s</td><td align="right">0.3248s</td><td align="right">💚  -1.15%</td><td align="right">71.89 MiB</td><td align="right">71.74 MiB</td><td align="right"> -0.21%</td></tr><tr><td>🟣 <b>syn</b>:check:unchanged</td><td align="right">0.6118s</td><td align="right">0.6057s</td><td align="right">💚  -1.01%</td><td align="right">106.59 MiB</td><td align="right">106.66 MiB</td><td align="right"> 0.06%</td></tr><tr><td>🟣 <b>syntex_syntax</b>:check:unchanged</td><td align="right">1.4570s</td><td align="right">1.4463s</td><td align="right"> -0.74%</td><td align="right">197.29 MiB</td><td align="right">197.33 MiB</td><td align="right"> 0.02%</td></tr><tr><td>Total</td><td align="right">2.9852s</td><td align="right">2.9601s</td><td align="right"> -0.84%</td><td align="right">516.66 MiB</td><td align="right">516.80 MiB</td><td align="right"> 0.03%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">0.9911s</td><td align="right"> -0.89%</td><td align="right">1 byte</td><td align="right">1.00 bytes</td><td align="right"> 0.02%</td></tr></table>

r? `@cjgillot`
2024-03-10 06:27:41 +00:00