105 Commits

Author	SHA1	Message	Date
Julian Wollersberger	bd49ded308	Noticed a potential bug in eat_while(): it doesn't account for number of UTF8 bytes. ... Fixed it by inlining it in the two places where the count is used and simplified the logic there.	2020-10-09 11:12:54 +02:00
LingMan	fc20b7841c	Fix typo in rustc_lexer docs ... Also add an Oxford comma while we're editing that line.	2020-09-21 05:43:39 +02:00
Vadim Petrochenkov	b1491eacfc	lexer: Tiny improvement to shebang detection ... Lexer now discerns between regular comments and doc comments, so use that. The change only affects the choice of reported errors.	2020-09-02 00:40:19 +03:00
Aleksey Kladov	ccffea5b6b	Move lexer unit tests to rustc_lexer ... StringReader is an intornal abstraction which at the moment changes a lot, so these unit tests cause quite a bit of friction. Moving them to rustc_lexer and more ingerated-testing style should make them much less annoying, hopefully without decreasing their usefulness much. Note that coloncolon tests are removed (it's unclear what those are testing). \r\n tests are removed as well, as we normalize line endings even before lexing.	2020-08-30 19:53:36 +02:00
mark	9e5f7d5631	mv compiler to compiler/	2020-08-30 18:45:07 +03:00