Commit Graph

462 Commits

Author SHA1 Message Date
Cameron Steffen
10a5e75537 Add track_caller to some Lock methods 2022-11-01 12:28:14 -05:00
Michael Woerister
9117ea9758 Introduce UnordMap, UnordSet, and UnordBag (see MCP 533)
MCP 533: https://github.com/rust-lang/compiler-team/issues/533

Also, as an example, substitute UnordMap for FxHashMap in
used_trait_imports query result.
2022-10-27 13:23:26 +00:00
Rageking8
62a2a1d257 fix some typos 2022-10-21 18:04:00 +08:00
bors
4b8f431995 Auto merge of #103214 - Nilstrieb:set-theory, r=petrochenkov
Use Set instead of Vec in transitive_relation

Helps with #103195. It doesn't fix the underlying quadraticness but it makes it _a lot_ faster to an extent where even doubling the amount of nested references still takes less than two seconds (50s on nightly).

I want to see whether this causes regressions (because the vec was usually quite small) or improvements (as lookup for bigger sets is now much faster) in real code.
2022-10-19 13:53:06 +00:00
nils
d45f025c90
Use Set instead of Vec in transitive_relation 2022-10-19 12:50:28 +02:00
bors
98a5ac269c Auto merge of #102543 - daym:patch-1, r=joshtriplett
Remove "execute" bit from lock file permissions

Previously, flock would set the "execute" bit on Rust lock files. That makes no sense.

This patch clears the "execute" bit on Rust lock files.

See issue #102531.
2022-10-18 02:04:24 +00:00
Danny Milosavljevic
e07e5104fe Remove execute bit from lock file permissions 2022-10-17 21:10:46 +02:00
Rageking8
7122abaddf more dupe word typos 2022-10-14 12:57:56 +08:00
Matthias Krüger
42df0a580f
Rollup merge of #102725 - nnethercote:rm-Z-time, r=davidtwco
Remove `-Ztime`

Because it has a lot of overlap with `-Ztime-passes` but is generally less useful. Plus some related cleanups.

Best reviewed one commit at a time.

r? `@davidtwco`
2022-10-06 16:29:45 +02:00
Nicholas Nethercote
4e8faff3a1 Be consistent about deciding whether to print pass data.
`print_time_passes_entry` unconditionally prints data about a pass. The
most commonly used call site, in `VerboseTimingGuard::drop`, guards it
with a `should_print_passes` test. But there are a couple of other call
sites that don't do that test.

This commit moves the `should_print_passes` test within
`print_time_passes_entry` so that all passes are treated equally.
2022-10-06 15:50:10 +11:00
Nicholas Nethercote
9110d925d0 Remove -Ztime option.
The compiler currently has `-Ztime` and `-Ztime-passes`. I've used
`-Ztime-passes` for years but only recently learned about `-Ztime`.

What's the difference? Let's look at the `-Zhelp` output:
```
  -Z        time=val -- measure time of rustc processes (default: no)
  -Z time-passes=val -- measure time of each rustc pass (default: no)
```
The `-Ztime-passes` description is clear, but the `-Ztime` one is less so.
Sounds like it measures the time for the entire process?

No. The real difference is that `-Ztime-passes` prints out info about passes,
and `-Ztime` does the same, but only for a subset of those passes. More
specifically, there is a distinction in the profiling code between a "verbose
generic activity" and an "extra verbose generic activity". `-Ztime-passes`
prints both kinds, while `-Ztime` only prints the first one. (It took me
a close reading of the source code to determine this difference.)

In practice this distinction has low value. Perhaps in the past the "extra
verbose" output was more voluminous, but now that we only print stats for a
pass if it exceeds 5ms or alters the RSS, `-Ztime-passes` is less spammy. Also,
a lot of the "extra verbose" cases are for individual lint passes, and you need
to also use `-Zno-interleave-lints` to see those anyway.

Therefore, this commit removes `-Ztime` and the associated machinery. One thing
to note is that the existing "extra verbose" activities all have an extra
string argument, so the commit adds the ability to accept an extra argument to
the "verbose" activities.
2022-10-06 15:49:44 +11:00
Nicholas Nethercote
eea06de0c8 Fix some comments.
- It's `--print`, not `--prints`.
- `-Ztime` and `-Ztime-passes` print to stderr, not stdout.
2022-10-06 14:22:12 +11:00
bors
0152393048 Auto merge of #99324 - reez12g:issue-99144, r=jyn514
Enable doctests in compiler/ crates

Helps with https://github.com/rust-lang/rust/issues/99144
2022-10-06 03:01:57 +00:00
Camille GILLOT
9f2ab5b9ad Use a SortedMap instead of a VecMap. 2022-10-01 16:24:30 +02:00
Camille GILLOT
d08669c4fa Compute by owner instead of HirId. 2022-10-01 16:22:40 +02:00
reez12g
9a4c5abe45 Remove from compiler/ crates 2022-09-29 16:49:04 +09:00
Pietro Albini
3975d55d98
remove cfg(bootstrap) 2022-09-26 10:14:45 +02:00
Dylan DPC
d5ae6737bf
Rollup merge of #102037 - jyn514:normalize-docs, r=lcnr
Make cycle errors recoverable

In particular, this allows rustdoc to recover from cycle errors when normalizing associated types for documentation.

In the past, ```@jackh726``` has said we need to be careful about overflow errors: https://github.com/rust-lang/rust/pull/91430#issuecomment-983997013

> Off the top of my head, we definitely should be careful about treating overflow errors the same as
"not implemented for some reason" errors. Otherwise, you could end up with behavior that is
different depending on recursion depth. But, that might be context-dependent.

But cycle errors should be safe to unconditionally report; they don't depend on the recursion depth, they will always be an error whenever they're encountered.

Helps with https://github.com/rust-lang/rust/issues/81091.

r? ```@lcnr``` cc ```@matthewjasper```
2022-09-22 18:25:53 +05:30
Joshua Nelson
1512ce5925 Make cycle errors recoverable
In particular, this allows rustdoc to recover from cycle errors when normalizing associated types for documentation.

In the past, `@jackh726` has said we need to be careful about overflow errors:

> Off the top of my head, we definitely should be careful about treating overflow errors the same as
"not implemented for some reason" errors. Otherwise, you could end up with behavior that is
different depending on recursion depth. But, that might be context-dependent.

But cycle errors should be safe to unconditionally report; they don't depend on the recursion depth, they will always be an error whenever they're encountered.
2022-09-19 22:14:40 -05:00
Joshua Nelson
749dec6451 Make OUT an associated type instead of a generic parameter
This avoids toil when changing other functions in `ObligationForest` to take an `OUT` parameter.
2022-09-19 22:14:39 -05:00
est31
173eb6f407 Only enable the let_else feature on bootstrap
On later stages, the feature is already stable.

Result of running:

rg -l "feature.let_else" compiler/ src/librustdoc/ library/ | xargs sed -s -i "s#\\[feature.let_else#\\[cfg_attr\\(bootstrap, feature\\(let_else\\)#"
2022-09-15 21:06:45 +02:00
bors
52e003a6e9 Auto merge of #99334 - NiklasJonsson:84447/error-privacy, r=oli-obk
rustc_error, rustc_private: Switch to stable hash containers

Relates https://github.com/rust-lang/rust/issues/84447
2022-09-12 15:57:37 +00:00
Dylan DPC
b0455e9ab4
Rollup merge of #101635 - jyn514:queries-new-derived, r=cjgillot
Move `Queries::new` out of the macro

Split out from https://github.com/rust-lang/rust/pull/101178 to make sure it's not contributing to the perf impact.

r? `@cjgillot`
2022-09-10 18:56:10 +05:30
Niklas Jonsson
8d3c30c004 rustc_error, rustc_private, rustc_ast: Switch to stable hash containers 2022-09-10 11:49:12 +02:00
Michael Benfield
d7a750b504 Use niche-filling optimization even when multiple variants have data.
Fixes #46213
2022-09-07 20:12:45 +00:00
Joshua Nelson
d2c53caee2 Move Queries::new out of the macro 2022-09-06 21:41:01 -05:00
bors
eac6c33bc6 Auto merge of #100869 - nnethercote:replace-ThinVec, r=spastorino
Replace `rustc_data_structures::thin_vec::ThinVec` with `thin_vec::ThinVec`

`rustc_data_structures::thin_vec::ThinVec` looks like this:
```
pub struct ThinVec<T>(Option<Box<Vec<T>>>);
```
It's just a zero word if the vector is empty, but requires two
allocations if it is non-empty. So it's only usable in cases where the
vector is empty most of the time.

This commit removes it in favour of `thin_vec::ThinVec`, which is also
word-sized, but stores the length and capacity in the same allocation as
the elements. It's good in a wider variety of situation, e.g. in enum
variants where the vector is usually/always non-empty.

The commit also:
- Sorts some `Cargo.toml` dependency lists, to make additions easier.
- Sorts some `use` item lists, to make additions easier.
- Changes `clean_trait_ref_with_bindings` to take a
  `ThinVec<TypeBinding>` rather than a `&[TypeBinding]`, because this
  avoid some unnecessary allocations.

r? `@spastorino`
2022-09-01 08:01:06 +00:00
bors
b32223fec1 Auto merge of #100707 - dzvon:fix-typo, r=davidtwco
Fix a bunch of typo

This PR will fix some typos detected by [typos].

I only picked the ones I was sure were spelling errors to fix, mostly in
the comments.

[typos]: https://github.com/crate-ci/typos
2022-09-01 05:39:58 +00:00
Dezhi Wu
b1430fb7ca Fix a bunch of typo
This PR will fix some typos detected by [typos].

I only picked the ones I was sure were spelling errors to fix, mostly in
the comments.

[typos]: https://github.com/crate-ci/typos
2022-08-31 18:24:55 +08:00
Donough Liu
97b1a6146c Use more into_iter rather than drain(..) 2022-08-30 04:42:03 +01:00
Nicholas Nethercote
b38106b6d8 Replace rustc_data_structures::thin_vec::ThinVec with thin_vec::ThinVec.
`rustc_data_structures::thin_vec::ThinVec` looks like this:
```
pub struct ThinVec<T>(Option<Box<Vec<T>>>);
```
It's just a zero word if the vector is empty, but requires two
allocations if it is non-empty. So it's only usable in cases where the
vector is empty most of the time.

This commit removes it in favour of `thin_vec::ThinVec`, which is also
word-sized, but stores the length and capacity in the same allocation as
the elements. It's good in a wider variety of situation, e.g. in enum
variants where the vector is usually/always non-empty.

The commit also:
- Sorts some `Cargo.toml` dependency lists, to make additions easier.
- Sorts some `use` item lists, to make additions easier.
- Changes `clean_trait_ref_with_bindings` to take a
  `ThinVec<TypeBinding>` rather than a `&[TypeBinding]`, because this
  avoid some unnecessary allocations.
2022-08-29 15:42:13 +10:00
SparrowLii
5d9e4d07fc get rid of RefCell in TransitiveRelation 2022-08-22 18:08:46 +08:00
Nicholas Nethercote
619b8abaa6 Use AttrVec in more places.
In some places we use `Vec<Attribute>` and some places we use
`ThinVec<Attribute>` (a.k.a. `AttrVec`). This results in various points
where we have to convert between `Vec` and `ThinVec`.

This commit changes the places that use `Vec<Attribute>` to use
`AttrVec`. A lot of this is mechanical and boring, but there are
some interesting parts:
- It adds a few new methods to `ThinVec`.
- It implements `MapInPlace` for `ThinVec`, and introduces a macro to
  avoid the repetition of this trait for `Vec`, `SmallVec`, and
  `ThinVec`.

Overall, it makes the code a little nicer, and has little effect on
performance. But it is a precursor to removing
`rustc_data_structures::thin_vec::ThinVec` and replacing it with
`thin_vec::ThinVec`, which is implemented more efficiently.
2022-08-22 07:35:33 +10:00
5225225
09ea9f0a87 Add diagnostic translation lints to crates that don't emit them 2022-08-18 19:29:02 +01:00
Tshepang Mbambo
3f379f6df5 needless separation of impl blocks 2022-08-16 03:46:45 +02:00
lcnr
d3ad264a91 remove some manual hash stable impls 2022-07-29 12:48:28 +02:00
bors
4dbc89de3f Auto merge of #99251 - cuviper:hashbrown-0.12, r=Mark-Simulacrum
Upgrade indexmap and thorin-dwp to use hashbrown 0.12

This removes the last dependencies on hashbrown 0.11.

This also upgrades to hashbrown 0.12.3 to fix a double-free (#99372).
2022-07-24 04:03:29 +00:00
Michael Woerister
88f6c6d8a0 Remove unused StableMap and StableSet types from rustc_data_structures 2022-07-20 13:11:39 +02:00
SparrowLii
e2ecb68a0e use par_for_each_in in par_body_owners and collect_crate_mono_items 2022-07-19 17:00:51 +08:00
Josh Stone
daa7c14a44 Upgrade indexmap and thorin-dwp to use hashbrown 0.12
This removes the last dependencies on hashbrown 0.11.
2022-07-17 07:05:58 -07:00
bors
1ba1fec234 Auto merge of #96544 - m-ysk:feature/issue-96358, r=cjgillot
Stop keeping metadata in memory before writing it to disk

Fixes #96358

I created this PR according with the instruction given in the issue except for the following points:

- While the issue says "Write metadata into the temporary file in `encode_and_write_metadata` even if `!need_metadata_file`", I could not do that. That is because though I tried to do that and run `x.py test`, I got a lot of test failures as follows.

<details>
<summary>List of failed tests</summary>
<pre>
<code>
failures:
    [ui] src/test/ui/json-multiple.rs
    [ui] src/test/ui/json-options.rs
    [ui] src/test/ui/rmeta/rmeta-rpass.rs
    [ui] src/test/ui/save-analysis/emit-notifications.rs
    [ui] src/test/ui/svh/changing-crates.rs
    [ui] src/test/ui/svh/svh-change-lit.rs
    [ui] src/test/ui/svh/svh-change-significant-cfg.rs
    [ui] src/test/ui/svh/svh-change-trait-bound.rs
    [ui] src/test/ui/svh/svh-change-type-arg.rs
    [ui] src/test/ui/svh/svh-change-type-ret.rs
    [ui] src/test/ui/svh/svh-change-type-static.rs
    [ui] src/test/ui/svh/svh-use-trait.rs

test result: FAILED. 12915 passed; 12 failed; 100 ignored; 0 measured; 0 filtered out; finished in 71.41s

Some tests failed in compiletest suite=ui mode=ui host=x86_64-unknown-linux-gnu target=x86_64-unknown-linux-gnu
Build completed unsuccessfully in 0:01:58
</code>
</pre>
</details>

- I could not resolve the extra tasks about `create_rmeta_file` and `create_compressed_metadata_file` for my lack of ability.
2022-07-14 21:50:14 +00:00
Camille GILLOT
43bb31b954 Allow to create definitions inside the query system. 2022-07-06 22:50:55 +02:00
Yoshiki Matsuda
c57d778872 define MmapMut and use it in Decodable impl 2022-07-02 22:54:37 +09:00
bors
66c83ffca1 Auto merge of #98558 - nnethercote:smallvec-1.8.1, r=lqd
Update `smallvec` to 1.8.1.

This pulls in https://github.com/servo/rust-smallvec/pull/282, which
gives some small wins for rustc.

r? `@lqd`
2022-06-29 09:11:29 +00:00
Dylan DPC
56b7786914
Rollup merge of #98384 - rdzhaafar:fix-macos-rss-reporting, r=davidtwco,michaelwoerister
Fix RSS reporting on macOS

> NOTE: This is a duplicate of #98164, which I closed because I borked my rustc fork

Currently, `rustc_data_structures::profiling::get_resident_set_size()` always returns `None` on macOS. This is because
macOS does not implement procfs used in the unix version of the function:

```rust
...
else if #[cfg(unix)] {
        pub fn get_resident_set_size() -> Option<usize> {
            let field = 1;
            let contents = fs::read("/proc/self/statm").ok()?;
            let contents = String::from_utf8(contents).ok()?;
            let s = contents.split_whitespace().nth(field)?;
            let npages = s.parse::<usize>().ok()?;
            Some(npages * 4096)
        }
...
```

The proposed solution uses libproc, and more specifically `proc_pidinfo`, which has been available on macOS since 10.5 if the function signature inside libproc.h is to be believed:

```c
int proc_pidinfo(int pid, int flavor, uint64_t arg, void *buffer, int buffersize) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
```
2022-06-28 15:30:03 +05:30
SparrowLii
ec137f29c5 catch unwind of every iter in parallel mode during wfcheck 2022-06-27 16:39:10 +08:00
Nicholas Nethercote
7c40661ddb Update smallvec to 1.8.1.
This pulls in https://github.com/servo/rust-smallvec/pull/282, which
gives some small wins for rustc.
2022-06-27 08:48:55 +10:00
Rida Dzhaafar
c41630735c Fixed RSS reporting on macOS 2022-06-22 15:24:54 +03:00
Yuki Okushi
8f861dae7a
Rollup merge of #97895 - nbdd0121:unlikely, r=estebank
Simplify `likely!` and `unlikely!` macro

The corresponding intrinsics have long been safe-to-call, so the unsafe block is no longer needed.
2022-06-22 15:16:13 +09:00
bors
1d6010816c Auto merge of #97674 - nnethercote:oblig-forest-tweaks, r=nikomatsakis
Obligation forest tweaks

A few minor improvements to the code.

r? `@nikomatsakis`
2022-06-20 10:58:56 +00:00
Gary Guo
8b7299dd12 Remove likely! and unlikely! macro from compiler 2022-06-18 04:52:11 +01:00
Maybe Waffle
c1a2db3372 Move/rename lazy::Sync{OnceCell,Lazy} to sync::{Once,Lazy}Lock 2022-06-16 19:54:42 +04:00
Maybe Waffle
7c360dc117 Move/rename lazy::{OnceCell, Lazy} to cell::{OnceCell, LazyCell} 2022-06-16 19:53:59 +04:00
Yuki Okushi
97b9347c93
Rollup merge of #98083 - nnethercote:rename-Encoder, r=bjorn3
Rename rustc_serialize::opaque::Encoder as MemEncoder.

This avoids the name clash with `rustc_serialize::Encoder` (a trait),
and allows lots qualifiers to be removed and imports to be simplified
(e.g. fewer `as` imports).

(This was previously merged as commit 5 in #94732 and then was reverted
in #97905 because of a perf regression caused by commit 4 in #94732.)

r? ```@bjorn3```
2022-06-15 12:02:04 +09:00
Nicholas Nethercote
abe45a9ffa Rename rustc_serialize::opaque::Encoder as MemEncoder.
This avoids the name clash with `rustc_serialize::Encoder` (a trait),
and allows lots qualifiers to be removed and imports to be simplified
(e.g. fewer `as` imports).

(This was previously merged as commit 5 in #94732 and then was reverted
in #97905 because of a perf regression caused by commit 4 in #94732.)
2022-06-14 14:52:01 +10:00
Eduard-Mihai Burtescu
d76573abd1 Integrate measureme's hardware performance counter support. 2022-06-13 07:56:47 +00:00
Nicholas Nethercote
7f51a1b976 Revert b983e42936. 2022-06-10 08:35:03 +10:00
Alan Egerton
9208c08a77
Use liballoc's specialised in-place vec collection
liballoc already specialises in-place vector collection, so manually
reimplementing it in `IdFunctor::try_map_id` was superfluous.
2022-06-08 12:14:49 +01:00
Nicholas Nethercote
b983e42936 Rename rustc_serialize::opaque::Encoder as MemEncoder.
This avoids the name clash with `rustc_serialize::Encoder` (a trait),
and allows lots qualifiers to be removed and imports to be simplified
(e.g. fewer `as` imports).
2022-06-08 09:50:44 +10:00
Nicholas Nethercote
1acbe7573d Use delayed error handling for Encodable and Encoder infallible.
There are two impls of the `Encoder` trait: `opaque::Encoder` and
`opaque::FileEncoder`. The former encodes into memory and is infallible, the
latter writes to file and is fallible.

Currently, standard `Result`/`?`/`unwrap` error handling is used, but this is a
bit verbose and has non-trivial cost, which is annoying given how rare failures
are (especially in the infallible `opaque::Encoder` case).

This commit changes how `Encoder` fallibility is handled. All the `emit_*`
methods are now infallible. `opaque::Encoder` requires no great changes for
this. `opaque::FileEncoder` now implements a delayed error handling strategy.
If a failure occurs, it records this via the `res` field, and all subsequent
encoding operations are skipped if `res` indicates an error has occurred. Once
encoding is complete, the new `finish` method is called, which returns a
`Result`. In other words, there is now a single `Result`-producing method
instead of many of them.

This has very little effect on how any file errors are reported if
`opaque::FileEncoder` has any failures.

Much of this commit is boring mechanical changes, removing `Result` return
values and `?` or `unwrap` from expressions. The more interesting parts are as
follows.
- serialize.rs: The `Encoder` trait gains an `Ok` associated type. The
  `into_inner` method is changed into `finish`, which returns
  `Result<Vec<u8>, !>`.
- opaque.rs: The `FileEncoder` adopts the delayed error handling
  strategy. Its `Ok` type is a `usize`, returning the number of bytes
  written, replacing previous uses of `FileEncoder::position`.
- Various methods that take an encoder now consume it, rather than being
  passed a mutable reference, e.g. `serialize_query_result_cache`.
2022-06-08 07:01:26 +10:00
Nicholas Nethercote
32741d5d16 Split process_obligation in two.
Because it really has two halves:
- A read-only part that checks if further work is needed.
- The further work part, which is much less hot.

This makes things a bit clearer and nicer.
2022-06-06 08:47:49 +10:00
Nicholas Nethercote
281229a6d3 Handle stalling within ObligationForest.
It is simpler if `ObligationForest` does this itself, rather than the
caller having to manage it.
2022-06-06 08:47:49 +10:00
Nicholas Nethercote
cdb446fec3 Streamline active_cache to done_cache transfer. 2022-06-06 08:47:49 +10:00
Nilstrieb
7e3bee6d8e Fix stacked borrows invalidation in rustc_data_structures sip128
It creates the src pointer first, which is then invalidated by a
unique borrow of the destination pointer. Swap the borrows around
to fix this. Found with miri.
2022-06-04 17:46:36 +02:00
Nilstrieb
fc8b13cb96 Adapt rustc_data_structures tests to run in strict miri
Some tests took too long and owning_ref is fundamentally flawed,
so don't run these tests or run them with a shorter N. This makes
miri with `-Zmiri-strict-provenance` usable to find UB.
2022-06-04 17:46:29 +02:00
Ariel Davis
b02146a370 Tweak insert docs 2022-05-31 22:08:14 -07:00
Tobias Stoeckmann
56662bcdff Fix typos in comment 2022-05-30 21:21:32 +02:00
Josh Stone
ab57e36268 Update to rebased rustc-rayon 0.4 2022-05-27 20:20:41 -07:00
Vadim Petrochenkov
5bf23f64cc libcore: Add iter::from_generator which is like iter::from_fn, but for coroutines instead of functions 2022-05-27 01:51:31 +03:00
bors
574830f573 Auto merge of #96094 - Elliot-Roberts:fix_doctests, r=compiler-errors
Begin fixing all the broken doctests in `compiler/`

Begins to fix #95994.
All of them pass now but 24 of them I've marked with `ignore HELP (<explanation>)` (asking for help) as I'm unsure how to get them to work / if we should leave them as they are.
There are also a few that I marked `ignore` that could maybe be made to work but seem less important.
Each `ignore` has a rough "reason" for ignoring after it parentheses, with

- `(pseudo-rust)` meaning "mostly rust-like but contains foreign syntax"
- `(illustrative)` a somewhat catchall for either a fragment of rust that doesn't stand on its own (like a lone type), or abbreviated rust with ellipses and undeclared types that would get too cluttered if made compile-worthy.
- `(not-rust)` stuff that isn't rust but benefits from the syntax highlighting, like MIR.
- `(internal)` uses `rustc_*` code which would be difficult to make work with the testing setup.

Those reason notes are a bit inconsistently applied and messy though. If that's important I can go through them again and try a more principled approach. When I run `rg '```ignore \(' .` on the repo, there look to be lots of different conventions other people have used for this sort of thing. I could try unifying them all if that would be helpful.

I'm not sure if there was a better existing way to do this but I wrote my own script to help me run all the doctests and wade through the output. If that would be useful to anyone else, I put it here: https://github.com/Elliot-Roberts/rust_doctest_fixing_tool
2022-05-07 06:30:29 +00:00
bors
d60b4f52c9 Auto merge of #95454 - randomicon00:fix95444, r=wesleywiser
Fixing #95444 by only displaying passes that take more than 5 millise…

As discussed in #95444, I have added the code to test and only display prints that are greater than 5 milliseconds.

r? `@jyn514`
2022-05-06 17:52:47 +00:00
bors
8c4fc9d9a4 Auto merge of #94598 - scottmcm:prefix-free-hasher-methods, r=Amanieu
Add a dedicated length-prefixing method to `Hasher`

This accomplishes two main goals:
- Make it clear who is responsible for prefix-freedom, including how they should do it
- Make it feasible for a `Hasher` that *doesn't* care about Hash-DoS resistance to get better performance by not hashing lengths

This does not change rustc-hash, since that's in an external crate, but that could potentially use it in future.

Fixes #94026

r? rust-lang/libs

---

The core of this change is the following two new methods on `Hasher`:

```rust
pub trait Hasher {
    /// Writes a length prefix into this hasher, as part of being prefix-free.
    ///
    /// If you're implementing [`Hash`] for a custom collection, call this before
    /// writing its contents to this `Hasher`.  That way
    /// `(collection![1, 2, 3], collection![4, 5])` and
    /// `(collection![1, 2], collection![3, 4, 5])` will provide different
    /// sequences of values to the `Hasher`
    ///
    /// The `impl<T> Hash for [T]` includes a call to this method, so if you're
    /// hashing a slice (or array or vector) via its `Hash::hash` method,
    /// you should **not** call this yourself.
    ///
    /// This method is only for providing domain separation.  If you want to
    /// hash a `usize` that represents part of the *data*, then it's important
    /// that you pass it to [`Hasher::write_usize`] instead of to this method.
    ///
    /// # Examples
    ///
    /// ```
    /// #![feature(hasher_prefixfree_extras)]
    /// # // Stubs to make the `impl` below pass the compiler
    /// # struct MyCollection<T>(Option<T>);
    /// # impl<T> MyCollection<T> {
    /// #     fn len(&self) -> usize { todo!() }
    /// # }
    /// # impl<'a, T> IntoIterator for &'a MyCollection<T> {
    /// #     type Item = T;
    /// #     type IntoIter = std::iter::Empty<T>;
    /// #     fn into_iter(self) -> Self::IntoIter { todo!() }
    /// # }
    ///
    /// use std:#️⃣:{Hash, Hasher};
    /// impl<T: Hash> Hash for MyCollection<T> {
    ///     fn hash<H: Hasher>(&self, state: &mut H) {
    ///         state.write_length_prefix(self.len());
    ///         for elt in self {
    ///             elt.hash(state);
    ///         }
    ///     }
    /// }
    /// ```
    ///
    /// # Note to Implementers
    ///
    /// If you've decided that your `Hasher` is willing to be susceptible to
    /// Hash-DoS attacks, then you might consider skipping hashing some or all
    /// of the `len` provided in the name of increased performance.
    #[inline]
    #[unstable(feature = "hasher_prefixfree_extras", issue = "88888888")]
    fn write_length_prefix(&mut self, len: usize) {
        self.write_usize(len);
    }

    /// Writes a single `str` into this hasher.
    ///
    /// If you're implementing [`Hash`], you generally do not need to call this,
    /// as the `impl Hash for str` does, so you can just use that.
    ///
    /// This includes the domain separator for prefix-freedom, so you should
    /// **not** call `Self::write_length_prefix` before calling this.
    ///
    /// # Note to Implementers
    ///
    /// The default implementation of this method includes a call to
    /// [`Self::write_length_prefix`], so if your implementation of `Hasher`
    /// doesn't care about prefix-freedom and you've thus overridden
    /// that method to do nothing, there's no need to override this one.
    ///
    /// This method is available to be overridden separately from the others
    /// as `str` being UTF-8 means that it never contains `0xFF` bytes, which
    /// can be used to provide prefix-freedom cheaper than hashing a length.
    ///
    /// For example, if your `Hasher` works byte-by-byte (perhaps by accumulating
    /// them into a buffer), then you can hash the bytes of the `str` followed
    /// by a single `0xFF` byte.
    ///
    /// If your `Hasher` works in chunks, you can also do this by being careful
    /// about how you pad partial chunks.  If the chunks are padded with `0x00`
    /// bytes then just hashing an extra `0xFF` byte doesn't necessarily
    /// provide prefix-freedom, as `"ab"` and `"ab\u{0}"` would likely hash
    /// the same sequence of chunks.  But if you pad with `0xFF` bytes instead,
    /// ensuring at least one padding byte, then it can often provide
    /// prefix-freedom cheaper than hashing the length would.
    #[inline]
    #[unstable(feature = "hasher_prefixfree_extras", issue = "88888888")]
    fn write_str(&mut self, s: &str) {
        self.write_length_prefix(s.len());
        self.write(s.as_bytes());
    }
}
```

With updates to the `Hash` implementations for slices and containers to call `write_length_prefix` instead of `write_usize`.

`write_str` defaults to using `write_length_prefix` since, as was pointed out in the issue, the `write_u8(0xFF)` approach is insufficient for hashers that work in chunks, as those would hash `"a\u{0}"` and `"a"` to the same thing.  But since `SipHash` works byte-wise (there's an internal buffer to accumulate bytes until a full chunk is available) it overrides `write_str` to continue to use the add-non-UTF-8-byte approach.

---

Compatibility:

Because the default implementation of `write_length_prefix` calls `write_usize`, the changed hash implementation for slices will do the same thing the old one did on existing `Hasher`s.
2022-05-06 09:43:57 +00:00
Scott McMurray
98054377ee Add a dedicated length-prefixing method to Hasher
This accomplishes two main goals:
- Make it clear who is responsible for prefix-freedom, including how they should do it
- Make it feasible for a `Hasher` that *doesn't* care about Hash-DoS resistance to get better performance by not hashing lengths

This does not change rustc-hash, since that's in an external crate, but that could potentially use it in future.
2022-05-06 00:03:38 -07:00
Peh
e79ba76ec4 Fixing #95444 by only displaying passes that take more than 5 milliseconds
95444: Adding passes that include memory increase

Fix95444: Change the substraction with the abs_diff() method

Fix95444: Change the substraction with abs_diff() method
2022-05-05 23:56:40 +00:00
Josh Triplett
0fc5c524f5 Stabilize bool::then_some 2022-05-04 13:22:08 +02:00
Elliot Roberts
7907385999 fix most compiler/ doctests 2022-05-02 17:40:30 -07:00
Michael Woerister
c0be619724 incr. comp.: Don't export impl_stable_hash_via_hash!() and warn about using it. 2022-04-19 10:43:20 +02:00
bors
563ef23529 Auto merge of #95899 - petrochenkov:modchild2, r=cjgillot
rustc_metadata: Do not encode unnecessary module children

This should remove the syntax context shift and the special case for `ExternCrate` in decoder in https://github.com/rust-lang/rust/pull/95880.

This PR also shifts some work from decoding to encoding, which is typically useful for performance (but probably not much in this case).
r? `@cjgillot`
2022-04-16 22:04:10 +00:00
Dylan DPC
9905774762
Rollup merge of #96058 - euclio:flock-impls, r=nagisa
separate flock implementations into separate modules

The main benefit of doing this is that rustfmt will now format each of these modules.
2022-04-16 19:42:05 +02:00
bors
febce1fc31 Auto merge of #95689 - lqd:self-profiler, r=wesleywiser
Allow self-profiler to only record potentially costly arguments when argument recording is turned on

As discussed [on zulip](https://rust-lang.zulipchat.com/#narrow/stream/247081-t-compiler.2Fperformance/topic/Identifying.20proc-macro.20slowdowns/near/277304909) with `@wesleywiser,` I'd like to record proc-macro expansions in the self-profiler, with some detailed data (per-expansion spans for example, to follow #95473).

At the same time, I'd also like to avoid doing expensive things when tracking a generic activity's arguments, if they were not specifically opted into the event filter mask, to allow the self-profiler to be used in hotter contexts.

This PR tries to offer:
- a way to ensure a closure to record arguments will only be called in that situation, so that potentially costly arguments can still be recorded when needed. With the additional requirement that, if possible, it would offer a way to record non-owned data without adding many `generic_activity_with_arg_{...}`-style methods. This lead to the `generic_activity_with_arg_recorder` single entry-point, and the closure parameter would offer the new methods, able to be executed in a context where costly argument could be created without disturbing the profiled piece of code.
- some facilities/patterns allowing to record more rustc specific data in this situation, without making `rustc_data_structures`  where the self-profiler is defined, depend on other rustc crates (causing circular dependencies): in particular, spans. They are quite tricky to turn into strings (if the default `Debug` impl output does not match the context one needs them for), and since I'd also like to avoid the allocation there when arg recording is turned off today, that has turned into another flexibility requirement for the API in this PR (separating the span-specific recording into an extension trait). **edit**: I've removed this from the PR so that it's easier to review, and opened https://github.com/rust-lang/rust/pull/95739.
- allow for extensibility in the future: other ways to record arguments, or additional data attached to them could be added in the future (e.g. recording the argument's name as well as its data).

Some areas where I'd love feedback:
- the API and names: the `EventArgRecorder` and its method for example. As well as the verbosity that comes from the increased flexibility.
- if I should convert the existing `generic_activity_with_arg{s}` to just forward to `generic_activity_with_arg_recorder` + `recorder.record_arg` (or remove them altogether ? Probably not): I've used the new API in the simple case I could find of allocating for an arg that may not be recorded, and the rest don't seem costly.
- [x] whether this API should panic if no arguments were recorded by the user-provided closure (like this PR currently does: it seems like an error to use an API dedicated to record arguments but not call the methods to then do so) or if this should just record a generic activity without arguments ?
- whether the `record_arg` function should be `#[inline(always)]`, like the `generic_activity_*` functions ?

As mentioned, r? `@wesleywiser` following our recent discussion.
2022-04-16 11:43:28 +00:00
Dylan DPC
49a31cdc1d
Rollup merge of #95372 - RalfJung:unaligned_references, r=oli-obk
make unaligned_references lint deny-by-default

This lint has been warn-by-default for a year now (since https://github.com/rust-lang/rust/pull/82525), so I think it is time to crank it up a bit. Code that triggers the lint causes UB (without `unsafe`) when executed, so we really don't want people to write code like this.
2022-04-16 07:12:43 +02:00
Ralf Jung
e30d6d9096 make unaligned_references lint deny-by-default 2022-04-14 21:16:42 -04:00
Andy Russell
219d81f19b
separate flock implementations into separate modules 2022-04-14 18:30:53 -04:00
Vadim Petrochenkov
233fa659f4 rustc_metadata: Do not encode unnecessary module children 2022-04-13 01:35:27 +03:00
Camille GILLOT
443333dc1f Remove NodeIdHashingMode. 2022-04-12 19:59:32 +02:00
bors
e980c62955 Auto merge of #95524 - oli-obk:cached_stable_hash_cleanups, r=nnethercote
Cached stable hash cleanups

r? `@nnethercote`

Add a sanity assertion in debug mode to check that the cached hashes are actually the ones we get if we compute the hash each time.

Add a new data structure that bundles all the hash-caching work to make it easier to re-use it for different interned data structures
2022-04-09 02:31:24 +00:00
bors
f4a7ce997a Auto merge of #95519 - oli-obk:tait_ub2, r=compiler-errors
Enforce well formedness for type alias impl trait's hidden type

fixes #84657

This was not an issue with return-position-impl-trait because the generic bounds of the function are the same as those of the opaque type, and the hidden type must already be well formed within the function.

With type-alias-impl-trait the hidden type could be defined in a function that has *more* lifetime bounds than the type alias. This is fine, but the hidden type must still be well formed without those additional bounds.
2022-04-08 20:45:16 +00:00
Rémy Rakic
7585269673 add generic_activity_with_arg_recorder to the self-profiler
This allows profiling costly arguments to be recorded only when `-Zself-profile-events=args` is on: using a closure that takes an `EventArgRecorder` and call its `record_arg` or `record_args` methods.
2022-04-07 15:47:20 +02:00
Rémy Rakic
1c4ae7aa4a turn exec comment into doc comment 2022-04-07 15:47:19 +02:00
Oli Scherer
2e0ef701c2 Document and rename the new wrapper type 2022-04-07 13:01:48 +00:00
Oli Scherer
b30bcfae38 Fix some fallout around type alias impl trait in associated types 2022-04-06 12:56:22 +00:00
bors
168a020900 Auto merge of #92686 - saethlin:unsafe-debug-asserts, r=Amanieu
Add debug assertions to some unsafe functions

As suggested by https://github.com/rust-lang/rust/issues/51713

~~Some similar code calls `abort()` instead of `panic!()` but aborting doesn't work in a `const fn`, and the intrinsic for doing dispatch based on whether execution is in a const is unstable.~~

This picked up some invalid uses of `get_unchecked` in the compiler, and fixes them.

I can confirm that they do in fact pick up invalid uses of `get_unchecked` in the wild, though the user experience is less-than-awesome:
```
     Running unittests (target/x86_64-unknown-linux-gnu/debug/deps/rle_decode_fast-04b7918da2001b50)

running 6 tests
error: test failed, to rerun pass '--lib'

Caused by:
  process didn't exit successfully: `/home/ben/rle-decode-helper/target/x86_64-unknown-linux-gnu/debug/deps/rle_decode_fast-04b7918da2001b50` (signal: 4, SIGILL: illegal instruction)
```

~~As best I can tell these changes produce a 6% regression in the runtime of `./x.py test` when `[rust] debug = true` is set.~~
Latest commit (6894d559bd) brings the additional overhead from this PR down to 0.5%, while also adding a few more assertions. I think this actually covers all the places in `core` that it is reasonable to check for safety requirements at runtime.

Thoughts?
2022-04-03 16:04:47 +00:00
Oli Scherer
6ffd654683 Check that the cached stable hash is the right one if debug assertions are enabled 2022-03-31 14:54:04 +00:00
Oli Scherer
00c24dd8ce Move stable hash from TyS into a datastructure that can be shared with other interned types. 2022-03-31 14:54:04 +00:00
bors
05142a7e44 Auto merge of #95466 - Dylan-DPC:rollup-g7ddr8y, r=Dylan-DPC
Rollup of 5 pull requests

Successful merges:

 - #95294 (Document Linux kernel handoff in std::io::copy and std::fs::copy)
 - #95443 (Clarify how `src/tools/x` searches for python)
 - #95452 (fix since field version for termination stabilization)
 - #95460 (Spellchecking compiler code)
 - #95461 (Spellchecking some comments)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
2022-03-30 07:45:42 +00:00
Dylan DPC
03b3993ae8
Rollup merge of #95461 - nyurik:spelling, r=lcnr
Spellchecking some comments

This PR attempts to clean up some minor spelling mistakes in comments
2022-03-30 09:10:07 +02:00
Yuri Astrakhan
7e8201ae0a Spellchecking some comments
This PR attempts to clean up some minor spelling mistakes in comments
2022-03-30 01:39:38 -04:00
bors
f132bcf3bd Auto merge of #94081 - oli-obk:lazy_tait_take_two, r=nikomatsakis
Lazy type-alias-impl-trait take two

### user visible change 1: RPIT inference from recursive call sites

Lazy TAIT has an insta-stable change. The following snippet now compiles, because opaque types can now have their hidden type set from wherever the opaque type is mentioned.

```rust
fn bar(b: bool) -> impl std::fmt::Debug {
    if b {
        return 42
    }
    let x: u32 = bar(false); // this errors on stable
    99
}
```

The return type of `bar` stays opaque, you can't do `bar(false) + 42`, you need to actually mention the hidden type.

### user visible change 2: divergence between RPIT and TAIT in return statements

Note that `return` statements and the trailing return expression are special with RPIT (but not TAIT). So

```rust
#![feature(type_alias_impl_trait)]
type Foo = impl std::fmt::Debug;

fn foo(b: bool) -> Foo {
    if b {
        return vec![42];
    }
    std::iter::empty().collect() //~ ERROR `Foo` cannot be built from an iterator
}

fn bar(b: bool) -> impl std::fmt::Debug {
    if b {
        return vec![42]
    }
    std::iter::empty().collect() // Works, magic (accidentally stabilized, not intended)
}
```

But when we are working with the return value of a recursive call, the behavior of RPIT and TAIT is the same:

```rust
type Foo = impl std::fmt::Debug;

fn foo(b: bool) -> Foo {
    if b {
        return vec![];
    }
    let mut x = foo(false);
    x = std::iter::empty().collect(); //~ ERROR `Foo` cannot be built from an iterator
    vec![]
}

fn bar(b: bool) -> impl std::fmt::Debug {
    if b {
        return vec![];
    }
    let mut x = bar(false);
    x = std::iter::empty().collect(); //~ ERROR `impl Debug` cannot be built from an iterator
    vec![]
}
```

### user visible change 3: TAIT does not merge types across branches

In contrast to RPIT, TAIT does not merge types across branches, so the following does not compile.

```rust
type Foo = impl std::fmt::Debug;

fn foo(b: bool) -> Foo {
    if b {
        vec![42_i32]
    } else {
        std::iter::empty().collect()
        //~^ ERROR `Foo` cannot be built from an iterator over elements of type `_`
    }
}
```

It is easy to support, but we should make an explicit decision to include the additional complexity in the implementation (it's not much, see a721052457cf513487fb4266e3ade65c29b272d2 which needs to be reverted to enable this).

### PR formalities

previous attempt: #92007

This PR also includes #92306 and #93783, as they were reverted along with #92007 in #93893

fixes #93411
fixes #88236
fixes #89312
fixes #87340
fixes #86800
fixes #86719
fixes #84073
fixes #83919
fixes #82139
fixes #77987
fixes #74282
fixes #67830
fixes #62742
fixes #54895
2022-03-30 05:04:45 +00:00
Ben Kimock
6e6d0cbf83 Add debug assertions to some unsafe functions
These debug assertions are all implemented only at runtime using
`const_eval_select`, and in the error path they execute
`intrinsics::abort` instead of being a normal debug assertion to
minimize the impact of these assertions on code size, when enabled.

Of all these changes, the bounds checks for unchecked indexing are
expected to be most impactful (case in point, they found a problem in
rustc).
2022-03-29 11:05:24 -04:00
Oli Scherer
264cd05b16 Revert "Auto merge of #93893 - oli-obk:sad_revert, r=oli-obk"
This reverts commit 6499c5e7fc, reversing
changes made to 78450d2d60.
2022-03-28 16:27:14 +00:00