Commit Graph

46 Commits

Author SHA1 Message Date
riverbl
a0fcc8ebc0 Add display method to OsStr
Add `display` method to `OsStr` for lossy display of an `OsStr` which may contain invalid unicode.

Invalid Unicode sequences are replaced with `U+FFFD REPLACEMENT CHARACTER`.

This change also makes the `std::ffi::os_str` module public.
2024-01-18 20:38:31 +00:00
Matthias Krüger
d5fd88cb85
Rollup merge of #118194 - notriddle:notriddle/tuple-unit, r=GuillaumeGomez
rustdoc: search for tuples and unit by type with `()`

This feature extends rustdoc to support the syntax that most users will naturally attempt to use to search for tuples. Part of https://github.com/rust-lang/rust/issues/60485

Function signature searches already support tuples and unit. The explicit name `primitive:tuple` and `primitive:unit` can be used to match a tuple or unit, while `()` will match either one. It also follows the direction set by the actual language for parens as a group, so `(u8,)` will only match a tuple, while `(u8)` will match a plain, unwrapped byte—thanks to loose search semantics, it will also match the tuple.

## Preview

* [`option<t>, option<u> -> (t, u)`](<https://notriddle.com/rustdoc-html-demo-5/tuple-unit/std/index.html?search=option%3Ct%3E%2C option%3Cu%3E -%3E (t%2C u)>)
* [`[t] -> (t,)`](<https://notriddle.com/rustdoc-html-demo-5/tuple-unit/std/index.html?search=[t] -%3E (t%2C)>)
* [`(ipaddr,) -> socketaddr`](<https://notriddle.com/rustdoc-html-demo-5/tuple-unit/std/index.html?search=(ipaddr%2C) -%3E socketaddr>)

## Motivation

When type-based search was first landed, it was directly [described as incomplete][a comment].

[a comment]: https://github.com/rust-lang/rust/pull/23289#issuecomment-79437386

Filling out the missing functionality is going to mean adding support for more of Rust's [type expression] syntax, such as tuples (in this PR), references, raw pointers, function pointers, and closures.

[type expression]: https://doc.rust-lang.org/reference/types.html#type-expressions

There does seem to be demand for this sort of thing, such as [this Discord message](https://discord.com/channels/442252698964721669/443150878111694848/1042145740065099796) expressing regret at rustdoc not supporting tuples in search queries.

## Reference description (from the Rustdoc book)

<table>
<thead>
  <tr>
    <th>Shorthand</th>
    <th>Explicit names</th>
  </tr>
</thead>
<tbody>
  <tr><td colspan="2">Before this PR</td></tr>
  <tr>
    <td><code>[]</code></td>
    <td><code>primitive:slice</code> and/or <code>primitive:array</code></td>
  </tr>
  <tr>
    <td><code>[T]</code></td>
    <td><code>primitive:slice&lt;T&gt;</code> and/or <code>primitive:array&lt;T&gt;</code></td>
  </tr>
  <tr>
    <td><code>!</code></td>
    <td><code>primitive:never</code></td>
  </tr>
  <tr><td colspan="2">After this PR</td></tr>
  <tr>
    <td><code>()</code></td>
    <td><code>primitive:unit</code> and/or <code>primitive:tuple</code></td>
  </tr>
  <tr>
    <td><code>(T)</code></td>
    <td><code>T</code></td>
  </tr>
  <tr>
    <td><code>(T,)</code></td>
    <td><code>primitive:tuple&lt;T&gt;</code></td>
  </tr>
</tbody>
</table>

A single type expression wrapped in parens is the same as that type expression, since parens act as the grouping operator. If they're empty, though, they will match both `unit` and `tuple`, and if there's more than one type (or a trailing or leading comma) it is the same as `primitive:tuple<...>`.

However, since items can be left out of the query, `(T)` will still return results for types that match tuples, even though it also matches the type on its own. That is, `(u32)` matches `(u32,)` for the exact same reason that it also matches `Result<u32, Error>`.

## Future direction

The [type expression grammar](https://doc.rust-lang.org/reference/types.html#type-expressions) from the Reference is given below:

<pre><code>Syntax
    Type :
        TypeNoBounds
        | <a href="https://doc.rust-lang.org/reference/types/impl-trait.html">ImplTraitType</a>
        | <a href="https://doc.rust-lang.org/reference/types/trait-object.html">TraitObjectType</a>
<br>
    TypeNoBounds :
        <a href="https://doc.rust-lang.org/reference/types.html#parenthesized-types">ParenthesizedType</a>
        | <a href="https://doc.rust-lang.org/reference/types/impl-trait.html">ImplTraitTypeOneBound</a>
        | <a href="https://doc.rust-lang.org/reference/types/trait-object.html">TraitObjectTypeOneBound</a>
        | <a href="https://doc.rust-lang.org/reference/paths.html#paths-in-types">TypePath</a>
        | <a href="https://doc.rust-lang.org/reference/types/tuple.html#tuple-types">TupleType</a>
        | <a href="https://doc.rust-lang.org/reference/types/never.html">NeverType</a>
        | <a href="https://doc.rust-lang.org/reference/types/pointer.html#raw-pointers-const-and-mut">RawPointerType</a>
        | <a href="https://doc.rust-lang.org/reference/types/pointer.html#shared-references-">ReferenceType</a>
        | <a href="https://doc.rust-lang.org/reference/types/array.html">ArrayType</a>
        | <a href="https://doc.rust-lang.org/reference/types/slice.html">SliceType</a>
        | <a href="https://doc.rust-lang.org/reference/types/inferred.html">InferredType</a>
        | <a href="https://doc.rust-lang.org/reference/paths.html#qualified-paths">QualifiedPathInType</a>
        | <a href="https://doc.rust-lang.org/reference/types/function-pointer.html">BareFunctionType</a>
        | <a href="https://doc.rust-lang.org/reference/macros.html#macro-invocation">MacroInvocation</a>
</code></pre>

ImplTraitType and TraitObjectType (and ImplTraitTypeOneBound and TraitObjectTypeOneBound) are not yet implemented. They would mostly desugar to `trait:`, similarly to how `!` desugars to `primitive:never`.

ParenthesizedType and TuplePath are added in this PR.

TypePath is already implemented (except const generics, which is not planned, and function-like trait syntax, which is planned as part of closure support).

NeverType is already implemented.

RawPointerType and ReferenceType require parsing and fixes to the search index to store this information, but otherwise their behavior seems simple enough. Just like tuples and slices, `&T` would be equivalent to `primitive:reference<T>`, `&mut T` would be equivalent to `primitive:reference<keyword:mut, T>`, `*T` would be equivalent to `primitive:pointer<T>`, `*mut T` would be equivalent to `primitive:pointer<keyword:mut, T>`, and `*const T` would be equivalent to `primitive:pointer<keyword:const, T>`. Lifetime generics support is not planned, because lifetime subtyping seems too complicated.

ArrayType is subsumed by SliceType right now. Implementing const generics is not planned, because it seems like it would require a lot of implementation complexity for not much gain.

InferredType isn't really covered right now. Its semantics in a search context are not obvious.

QualifiedPathInType is not implemented, and it is not planned. I would need a use case to justify it, and act as a guide for what the exact semantics should be.

BareFunctionType is not implemented. Along with function-like trait syntax, which is formally considered a TypePath, it's the biggest missing feature to be able to do structured searches over generic APIs like `Option`.

MacroInvocation is not parsed (macro names are, but they don't mean the same thing here at all). Those are gone by the time Rustdoc sees the source code.
2024-01-06 16:07:46 +01:00
Michael Howell
0ea58e2346 rustdoc-search: count path edits with separate edit limit
Since the two are counted separately elsewhere, they should get
their own limits, too. The biggest problem with combining them
is that paths are loosely checked by not requiring every component
to match, which means that if they are short and matched loosely,
they can easily find "drunk typist" matches that make no sense,
like this old result:

    std::collections::btree_map::itermut matching slice::itermut
    maxEditDistance = ("slice::itermut".length) / 3 = 14 / 3 = 4
    editDistance("std", "slice") = 4
    editDistance("itermut", "itermut") = 0
        4 + 0 <= 4 PASS

Of course, `slice::itermut` should not match stuff from btreemap.
`slice` should not match `std`.

The new result counts them separately:

    maxPathEditDistance = "slice".length / 3 = 5 / 3 = 1
    maxEditDistance = "itermut".length / 3 = 7 / 3 = 2
    editDistance("std", "slice") = 4
        4 <= 1 FAIL

Effectively, this makes path queries less "typo-resistant".
It's not zero, but it means `vec` won't match the `v1` prelude.

Queries without parent paths are unchanged.
2023-12-26 18:46:17 -07:00
Michael Howell
f6a045cc6b rustdoc: search for tuples and unit by type with () 2023-12-26 12:54:17 -07:00
bors
3f1e30a0a5 Auto merge of #118077 - calebzulawski:sync-portable-simd-2023-11-19, r=workingjubilee
Portable SIMD subtree update

Syncs nightly to the latest changes from rust-lang/portable-simd

r? `@rust-lang/libs`
2023-12-02 18:04:01 +00:00
Michael Howell
930cba8061 rustdoc-search: replace TAB/NL/LF with SP first
This way, most of the parsing code doesn't need to be designed to handle
it, since they should always be treated exactly the same anyhow.
2023-11-29 11:02:56 -07:00
Michael Howell
93f17117ed rustdoc-search: removed dead parser code
This is already covered by the normal unexpected char path.
2023-11-29 11:02:56 -07:00
Michael Howell
c28de27a73 rustdoc-search: allow :: and ::
This restriction made sense back when spaces separated function
parameters, but now that they separate path components, there's
no real ambiguity any more.

Additionally, the Rust language allows it.
2023-11-29 11:02:50 -07:00
Caleb Zulawski
4d9607869a Update std::simd usage and test outputs 2023-11-26 09:02:25 -05:00
Michael Howell
28f17d97a9 rustdoc-search: make primitives and keywords less special
The search sorting code already sorts by item type discriminant,
putting things with smaller discriminants first. There was
also a special case for sorting keywords and primitives earlier,
and this commit removes it by giving them lower discriminants.

The sorting code has another criteria where items with descriptions
appear earlier than items without, and that criteria has higher
priority than the item type. This shouldn't matter, though,
because primitives and keywords normally only appear in the
standard library, and it always gives them descriptions.
2023-11-21 13:59:26 -07:00
Michael Howell
63c50712f4 rustdoc-search: add support for associated types 2023-11-19 18:54:36 -07:00
Michael Howell
3583e86674 rustdoc: update test cases for changes to the printing style
This whole thing changes it so that the JS and the UI both use
rustc's own path printing to handle the impl IDs. This results in
the format changing a little bit; full paths are used in spots
where they aren't strictly necessary, and the path sometimes uses
generics where the old system used the trait's own name, but it
shouldn't matter since the orphan rules will prevent it anyway.
2023-09-21 15:16:44 -07:00
Michael Howell
3fbfe2bca5 rustdoc-search: add impl disambiguator to duplicate assoc items
Helps with #90929

This changes the search results, specifically, when there's more than
one impl with an associated item with the same name. For example,
the search queries `simd<i8> -> simd<i8>` and `simd<i64> -> simd<i64>`
don't link to the same function, but most of the functions have the
same names.

This change should probably be FCP-ed, especially since it adds a new
anchor link format for `main.js` to handle, so that URLs like
`struct.Vec.html#impl-AsMut<[T]>-for-Vec<T,+A>/method.as_mut` redirect
to `struct.Vec.html#method.as_mut-2`. It's a strange design, but there
are a few reasons for it:

* I'd like to avoid making the HTML bigger. Obviously, fixing this bug
  is going to add at least a little more data to the search index, but
  adding more HTML penalises viewers for the benefit of searchers.

* Breaking `struct.Vec.html#method.len` would also be a disappointment.

On the other hand:

* The path-style anchors might be less prone to link rot than the numbered
  anchors. It's definitely less likely to have URLs that appear to "work",
  but silently point at the wrong thing.

* This commit arranges the path-style anchor to redirect to the numbered
  anchor. Nothing stops rustdoc from doing the opposite, making path-style
  anchors the default and redirecting the "legacy" numbered ones.
2023-09-21 15:16:44 -07:00
Michael Howell
269cb57947 rustdoc-search: fix bugs when unboxing and reordering combine 2023-09-09 16:58:37 -07:00
Michael Howell
89a4c7f552 rustdoc: bug fix for -> option<t> 2023-09-03 13:06:07 -07:00
Michael Howell
0b3c617ec0 rustdoc-search: add support for type parameters
When writing a type-driven search query in rustdoc, specifically one
with more than one query element, non-existent types become generic
parameters instead of auto-correcting (which is currently only done
for single-element queries) or giving no result. You can also force a
generic type parameter by writing `generic:T` (and can force it to not
use a generic type parameter with something like `struct:T` or whatever,
though if this happens it means the thing you're looking for doesn't
exist and will give you no results).

There is no syntax provided for specifying type constraints
for generic type parameters.

When you have a generic type parameter in a search query, it will only
match up with generic type parameters in the actual function, not
concrete types that match, not concrete types that implement a trait.
It also strictly matches based on when they're the same or different,
so `option<T>, option<U> -> option<U>` matches `Option::and`, but not
`Option::or`. Similarly, `option<T>, option<T> -> option<T>`` matches
`Option::or`, but not `Option::and`.
2023-09-03 13:06:06 -07:00
Guillaume Gomez
05eda41658 Add tests for type-based search 2023-09-01 15:16:11 +02:00
bors
bf0e22b298 Auto merge of #108537 - GuillaumeGomez:rustdoc-search-whitespace-as-separator, r=notriddle
rustdoc: Allow whitespace as path separator like double colon

Fixes https://github.com/rust-lang/rust/issues/108447.

I think it makes sense since it allows more common cases, however it also makes the syntax heavier. Not sure what the rest of the team thinks about it. In any case we'll need to go through FCP.

Full explanation for the changes is available [here](https://github.com/rust-lang/rust/pull/108537#issuecomment-1589480564).

r? `@notriddle`
2023-07-02 18:49:29 +00:00
bors
314c39d2ea Auto merge of #112233 - notriddle:notriddle/search-unify, r=GuillaumeGomez
rustdoc-search: clean up type unification and "unboxing"

This PR redesigns parameter matching, return matching, and generics matching to use a single function that compares two lists of types.

It also makes the algorithms more consistent, so the "unboxing" behavior where `Vec<i32>` is considered a match for `i32` works inside generics, and not just at the top level.
2023-06-15 03:04:46 +00:00
Guillaume Gomez
e4ee329865 Fix eBNF and handling of whitespace characters when not in a path 2023-06-14 14:22:17 +02:00
Guillaume Gomez
7dc684f173 Add "vec new" test 2023-06-14 14:22:17 +02:00
Guillaume Gomez
b5f8258497 Update rustdoc-js-std tests 2023-06-14 14:22:17 +02:00
Michael Howell
db277f5284 rustdoc-search: search never type with !
This feature extends rustdoc to support the syntax that most users will
naturally attempt to use to search for diverging functions.
Part of #60485

It's already possible to do this search with `primitive:never`, but
that's not what the Rust language itself uses, so nobody will try it if
they aren't told or helped along.
2023-06-12 17:30:23 -07:00
Michael Howell
94badbe599 rustdoc-search: fix order-independence bug 2023-06-11 18:57:33 -07:00
Michael Howell
c897deddb8 rustdoc-search: add test case for bufread -> result<u8> 2023-06-11 18:19:41 -07:00
Michael Howell
d3a4cd6813 rustdoc: add note about slice/array searches to help popup 2023-06-10 14:08:26 -07:00
Michael Howell
2e569274d3 rustdoc: search for slices and arrays by type with []
Part of #60485
2023-06-10 13:52:54 -07:00
Michael Howell
3ed4c17d90 rustdoc: add test case for OsString::into_string 2023-06-10 13:50:40 -07:00
Guillaume Gomez
9803651ee8 Update rustdoc-js* format 2023-06-09 17:00:47 +02:00
Michael Howell
afee2411e3 rustdoc-search: add support for nested generics 2023-04-14 14:55:45 -07:00
Michael Howell
e600c0ba0e rustdoc: add support for type filters in arguments and generics
This makes sense, since the search index has the information in it,
and it's more useful for function signature searches since a
function signature search's item type is, by definition, some type
of function (there's more than one, but not very many).
2023-03-20 22:41:57 -07:00
Matthias Krüger
aa881f16ec
Rollup merge of #108875 - notriddle:notriddle/return-trait, r=GuillaumeGomez
rustdoc: fix type search for `Option` combinators
2023-03-16 08:57:05 +01:00
Michael Howell
dfd9e5e3fa rustdoc: use restricted Damerau-Levenshtein distance for search
Based on https://github.com/rust-lang/rust/pull/108200, for the same
rationale.

> This replaces the existing Levenshtein algorithm with the
> Damerau-Levenshtein algorithm. This means that "ab" to "ba" is one change
> (a transposition) instead of two (a deletion and insertion). More
> specifically, this is a restricted implementation, in that "ca" to "abc"
> cannot be performed as "ca" → "ac" → "abc", as there is an insertion in the
> middle of a transposition. I believe that errors like that are sufficiently
> rare that it's not worth taking into account.

Before this change, searching `prinltn!` listed `print!` first, followed
by `println!`. With this change, `println!` matches more closely.
2023-03-10 19:47:08 -07:00
Michael Howell
44813e038c rustdoc: fix type search when more than one where clause applies 2023-03-07 11:37:04 -07:00
Matthias Krüger
9cabc40ab1
Rollup merge of #108723 - notriddle:notriddle/where-clause, r=GuillaumeGomez
rustdoc: function signature search with traits in `where` clause

## Before

![image](https://user-images.githubusercontent.com/1593513/222873534-a640a72a-c654-4702-9f3b-175129d9591d.png)

## After

![image](https://user-images.githubusercontent.com/1593513/222873544-fdfc431d-2b65-4b56-bede-0302ea9f153a.png)
2023-03-04 20:48:18 +01:00
Michael Howell
9d27028391 rustdoc: function signature search with traits in where clause 2023-03-04 09:05:57 -07:00
Guillaume Gomez
823671589f Add test for unclosed generic 2023-03-03 20:49:25 +01:00
Guillaume Gomez
cfb4af87e3 Add GUI test for rustdoc search errors background 2023-03-02 12:59:04 +01:00
Dylan DPC
f03e5345aa
Rollup merge of #108143 - notriddle:notriddle/filter-exclamation-macro, r=GuillaumeGomez
rustdoc: search by macro when query ends with `!`

Related to #96399

Note: the `never` type alias is tested in [`/tests/rustdoc-js-std/alias-3.js`](08ad401633/tests/rustdoc-js-std/alias-3.js)

## Before

![image](https://user-images.githubusercontent.com/1593513/219504192-54cc0753-ff97-4a37-ad4a-8ae915181325.png)

## After

![image](https://user-images.githubusercontent.com/1593513/219504251-589a7e11-1e7b-4b7b-879d-1b564080017c.png)
2023-03-01 23:40:18 +05:30
Michael Howell
4de9c6d491 rustdoc: search by macro when query ends with !
Related to #96399
2023-02-16 18:16:09 -07:00
Michael Howell
5eba3f688c rustdoc: hide reference methods in search index 2023-02-16 17:21:57 -07:00
Michael Howell
616a0db7d6 rustdoc: add test case based on #103357 2023-01-24 09:49:33 -07:00
Michael Howell
39fd4bb476 rustdoc: update test cases to match with stricter match criteria 2023-01-21 00:11:39 -07:00
Michael Howell
e09e6df787 rustdoc: compute maximum Levenshtein distance based on the query
The heuristic is pretty close to the name resolver.

Fixes #103357
2023-01-21 00:11:39 -07:00
Michael Howell
db558b4686 rustdoc: update search test cases 2023-01-14 12:04:12 -07:00
Albert Larsan
cf2dff2b1e
Move /src/test to /tests 2023-01-11 09:32:08 +00:00