nordic-dev.net/rust
nordic-dev.net/rust
2
0
Fork 0
mirror of https://github.com/rust-lang/rust.git synced 2025-02-21 11:23:03 +00:00
Code Issues Packages Projects Releases Wiki Activity
Empowering everyone to build reliable and efficient software.
28,246 Commits 6 Branches 143 Tags 1.9 GiB
Rust 95.6%
Shell 1%
RenderScript 0.7%
JavaScript 0.6%
Fluent 0.4%
Other 1.5%
e64610dbbe
Go to file
bors e64610dbbe Auto merge of #17037 - davidsemakula:token-set-collisions, r=Veykril 
internal: improve `TokenSet` implementation and add reserved keywords

The current `TokenSet` type represents "A bit-set of `SyntaxKind`s" as a newtype `u128`.
Internally, the flag for each `SyntaxKind` variant in the bit-set is set as the n-th LSB (least significant bit) via a bit-wise left shift operation, where n is the discriminant.

Edit: This is problematic because there's currently ~121 token `SyntaxKind`s, so adding new token kinds for missing reserved keywords increases the number of token `SyntaxKind`s above 128, thus making this ["mask"](7a8374c162/crates/parser/src/token_set.rs (L31-L33)) operation overflow.
~~This is problematic because there's currently 266 SyntaxKinds, so this ["mask"](7a8374c162/crates/parser/src/token_set.rs (L31-L33)) operation silently overflows in release mode.~~
~~This leads to a single flag/bit in the bit-set being shared by multiple `SyntaxKind`s~~.

This PR:
- Changes the wrapped type for `TokenSet` from `u128` to `[u64; 3]` ~~`[u*; N]` (currently `[u16; 17]`) where `u*` can be any desirable unsigned integer type and `N` is the minimum array length needed to represent all token `SyntaxKind`s without any collisions~~.
- Edit: Add assertion that `TokenSet`s only include token `SyntaxKind`s
- Edit: Add ~7 missing [reserved keywords](https://doc.rust-lang.org/stable/reference/keywords.html#reserved-keywords)
- ~~Moves the definition of the `TokenSet` type to grammar codegen in xtask, so that `N` is adjusted automatically (depending on the chosen `u*` "base" type) when new `SyntaxKind`s are added~~.
- ~~Updates the `token_set_works_for_tokens` unit test to include the `__LAST` `SyntaxKind` as a way of catching overflows in tests.~~

~~Currently `u16` is arbitrarily chosen as the `u*` "base" type mostly because it strikes a good balance (IMO) between unused bits and readability of the generated `TokenSet` code (especially the [`union` method](7a8374c162/crates/parser/src/token_set.rs (L26-L28))), but I'm open to other suggestions or a better methodology for choosing `u*` type.~~

~~I considered using a third-party crate for the bit-set, but a direct implementation seems simple enough without adding any new dependencies. I'm not strongly opposed to using a third-party crate though, if that's preferred.~~

~~Finally, I haven't had the chance to review issues, to figure out if there are any parser issues caused by collisions due the current implementation that may be fixed by this PR - I just stumbled upon the issue while adding "new" keywords to solve #16858~~

Edit: fixes #16858
2024-04-16 07:00:12 +00:00
.cargo internal: Move ide-assists codegen tests into an xtask codegen command 2024-02-28 16:54:44 +01:00
.github Run Windows tests on PRs too 2024-04-08 11:36:09 -07:00
.vscode Spelling 2023-04-19 09:45:55 -04:00
assets Automatically change text color in logo based on dark mode 2022-03-06 23:06:53 +11:00
bench_data Spelling 2023-04-19 09:45:55 -04:00
crates Auto merge of #17037 - davidsemakula:token-set-collisions, r=Veykril 2024-04-16 07:00:12 +00:00
docs Auto merge of #17064 - Veykril:inlay-hints-fix, r=Veykril 2024-04-14 10:27:40 +00:00
editors/code Adjust package.json semantic highlighting items 2024-04-15 17:00:03 +02:00
lib Move lint definition generation into xtask/codegen 2024-03-04 17:14:14 +01:00
xtask internal: add reserved keywords 2024-04-15 15:06:26 +03:00
.editorconfig add max_line_length to .editorconfig 2024-01-20 17:14:00 +03:00
.git-blame-ignore-revs add prettier format commit to ignored revisions 2022-05-17 18:15:51 +01:00
.gitattributes Fix .gitattributes for test_data 2022-07-24 14:05:35 +02:00
.gitignore internal: add "Shuffle Crate Graph" command 2021-12-07 16:37:19 +01:00
.typos.toml remove ignore-hidden = false 2024-03-05 20:16:32 +08:00
Cargo.lock Use Edition::CURRENT 2024-04-14 15:30:29 +02:00
Cargo.toml fix: Fix impl Trait<Self> causing stackoverflows 2024-04-15 15:41:20 +02:00
clippy.toml Lint debug prints and disallowed types with clippy 2024-02-01 17:57:27 +01:00
LICENSE-APACHE Licenses 2018-01-10 22:47:04 +03:00
LICENSE-MIT Licenses 2018-01-10 22:47:04 +03:00
PRIVACY.md Update privacy note 2021-12-23 14:04:15 +02:00
README.md Fix r-a spelling in some places 2022-08-01 13:47:09 +02:00
rust-bors.toml Prepare for rust-bors 2023-10-05 15:26:09 +03:00
rustfmt.toml Shuffle hir-expand things around 2024-01-26 19:28:39 +01:00
triagebot.toml triagebot exclude_labels -> exclude_titles 2023-09-16 10:58:53 -06:00

README.md

rust-analyzer logo

rust-analyzer is a modular compiler frontend for the Rust language. It is a part of a larger rls-2.0 effort to create excellent IDE support for Rust.

Quick Start

https://rust-analyzer.github.io/manual.html#installation

Documentation

If you want to contribute to rust-analyzer or are just curious about how things work under the hood, check the ./docs/dev folder.

If you want to use rust-analyzer's language server with your editor of choice, check the manual folder. It also contains some tips & tricks to help you be more productive when using rust-analyzer.

Security and Privacy

See the corresponding sections of the manual.

Communication

For usage and troubleshooting requests, please use "IDEs and Editors" category of the Rust forum:

https://users.rust-lang.org/c/ide/14

For questions about development and implementation, join rust-analyzer working group on Zulip:

https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Frust-analyzer

Quick Links

License

rust-analyzer is primarily distributed under the terms of both the MIT license and the Apache License (Version 2.0).

See LICENSE-APACHE and LICENSE-MIT for details.