2 Commits

Author	SHA1	Message	Date
topecongiro	13b41000ea	Add lib section to rustc_lexer's Cargo.toml	2019-07-27 17:21:42 +09:00
Aleksey Kladov	395ee0b79f	Introduce rustc_lexer ... The idea here is to make a reusable library out of the existing rust-lexer, by separating out pure lexing and rustc-specific concerns, like spans, error reporting an interning. So, rustc_lexer operates directly on `&str`, produces simple tokens which are a pair of type-tag and a bit of original text, and does not report errors, instead storing them as flags on the token.	2019-07-20 21:12:34 +03:00