Julian Wollersberger
1f147a2ed7
Replace nth_char(0)
with next()
in cursor.first()
...
and optimize the iterator returned by `tokenize().
This improves lexer performance by 35%
2021-12-01 19:14:10 +01:00
Esteban Kuber
38979a3ba1
udpate comment to be more accurate
2021-11-23 20:37:23 +00:00
Esteban Kuber
5a68abb094
Tokenize emoji as if they were valid indentifiers
...
In the lexer, consider emojis to be valid identifiers and reject
them later to avoid knock down parse errors.
2021-11-23 20:35:07 +00:00
Matthias Krüger
0a5640b55f
use matches!() macro in more places
2021-11-06 16:13:14 +01:00
Matthias Krüger
4457014398
Revert "Auto merge of #89709 - clemenswasser:apply_clippy_suggestions_2, r=petrochenkov"
...
The PR had some unforseen perf regressions that are not as easy to find.
Revert the PR for now.
This reverts commit 6ae8912a3e
, reversing
changes made to 86d6d2b738
.
2021-10-15 11:28:23 +02:00
Clemens Wasser
71dd0b928b
Apply clippy suggestions
2021-10-10 15:38:19 +02:00
Mark Rousskov
c746be2219
Migrate to 2021
2021-09-20 22:21:42 -04:00
Frank Steffahn
2396fad095
Fix more “a”/“an” typos
2021-08-22 17:27:18 +02:00
Anton Golov
07aacf53c5
Renamed variable str -> tail for clarity
2021-08-11 13:57:28 +02:00
Anton Golov
a03fbfe2ff
Warn when an escaped newline skips multiple lines
2021-08-11 11:35:08 +02:00
bors
b53a93db2d
Auto merge of #87535 - lf-:authors, r=Mark-Simulacrum
...
rfc3052 followup: Remove authors field from Cargo manifests
Since RFC 3052 soft deprecated the authors field, hiding it from
crates.io, docs.rs, and making Cargo not add it by default, and it is
not generally up to date/useful information for contributors, we may as well
remove it from crates in this repo.
2021-08-02 05:49:17 +00:00
Anton Golov
5d59b4412e
Add warning when whitespace is not skipped after an escaped newline.
2021-07-30 16:26:39 +02:00
Jade
3cf820e17d
rfc3052: Remove authors field from Cargo manifests
...
Since RFC 3052 soft deprecated the authors field anyway, hiding it from
crates.io, docs.rs, and making Cargo not add it by default, and it is
not generally up to date/useful information, we should remove it from
crates in this repo.
2021-07-29 14:56:05 -07:00
Ibraheem Ahmed
a397fdcc38
Remove ASCII fast path from rustc_lexer::{is_id_continue, is_id_start}
2021-07-26 20:17:28 -04:00
Mara Bos
0eeeebc990
Rename 'bad prefix' to 'unknown prefix'.
2021-06-26 23:11:14 +08:00
Mara Bos
6adce70a58
Improve comments for reserved prefixes.
...
Co-authored-by: Niko Matsakis <niko@alum.mit.edu>
2021-06-26 23:11:13 +08:00
lrh2000
8dee9bc8fc
Reserve prefixed identifiers and string literals (RFC 3101)
...
This commit denies any identifiers immediately followed by
one of three tokens `"`, `'` or `#`, which is stricter than
the requirements of RFC 3101 but may be necessary according
to the discussion at [Zulip].
[Zulip]: https://rust-lang.zulipchat.com/#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/238470099
2021-06-26 23:09:43 +08:00
pierwill
0019ca9141
Fix outdated crate names in compiler docs
...
Changes `librustc_X` to `rustc_X`, only in documentation comments.
Plain code comments are left unchanged.
Also fix incorrect file paths.
2021-04-08 11:12:14 -05:00
Hanzhen Liang
f942c3cbf4
Return EOF_CHAR constant instead of magic char.
2021-01-07 13:20:04 +01:00
Hirochika Matsumoto
56530a2f25
Fix typo
2020-12-18 22:13:25 +09:00
Joshua Nelson
0ad3dce83a
Fix some clippy lints
2020-12-03 17:08:19 -05:00
Joshua Nelson
5339bd1ebe
Add back missing comments
2020-10-30 10:13:41 -04:00
Joshua Nelson
57c6ed0c07
Fix even more clippy warnings
2020-10-30 10:13:39 -04:00
Julian Wollersberger
bd49ded308
Noticed a potential bug in eat_while()
: it doesn't account for number of UTF8 bytes.
...
Fixed it by inlining it in the two places where the count is used and simplified the logic there.
2020-10-09 11:12:54 +02:00
LingMan
fc20b7841c
Fix typo in rustc_lexer docs
...
Also add an Oxford comma while we're editing that line.
2020-09-21 05:43:39 +02:00
Vadim Petrochenkov
b1491eacfc
lexer: Tiny improvement to shebang detection
...
Lexer now discerns between regular comments and doc comments, so use that.
The change only affects the choice of reported errors.
2020-09-02 00:40:19 +03:00
Aleksey Kladov
5716c3e18d
Update expect-test to 1.0
...
The only change is that `expect_file` now uses path relative to the
current file (same as `include!`). Before, it used paths relative to
the workspace root, which makes no sense.
2020-08-31 21:04:09 +02:00
Aleksey Kladov
7d2460e5c4
Rollup merge of #76050 - matklad:pos, r=petrochenkov
...
Remove unused function
2020-08-31 15:22:40 +02:00
Vadim Petrochenkov
7dfbf59f10
cleanup: Remove duplicate library names from Cargo.toml
s
2020-08-30 22:57:54 +03:00
Aleksey Kladov
ccffea5b6b
Move lexer unit tests to rustc_lexer
...
StringReader is an intornal abstraction which at the moment changes a
lot, so these unit tests cause quite a bit of friction.
Moving them to rustc_lexer and more ingerated-testing style should
make them much less annoying, hopefully without decreasing their
usefulness much.
Note that coloncolon tests are removed (it's unclear what those are
testing).
\r\n tests are removed as well, as we normalize line endings even
before lexing.
2020-08-30 19:53:36 +02:00
mark
9e5f7d5631
mv compiler to compiler/
2020-08-30 18:45:07 +03:00