8 commits

Author	SHA1	Message	Date
Jonas Schievink	1bc3305d95	Split float literal tokens at the .	2022-05-05 16:28:58 +02:00
Jonas Schievink	1f50e19eb2	Add a Converter type for token conversion	2022-05-02 17:47:12 +02:00
Aleksey Kladov	f4cb0ff9be	internal: move ws attachment logic to the parser crate ... This has to re-introduce the `sink` pattern, because doing this purely with iterators is awkward :( Maaaybe the event vector was a false start? But, anyway, I like the current factoring more -- it sort-of obvious that we do want to keep ws-attachment business in the parser, and that we also don't want that to depend on the particular tree structure. I think `shortcuts` module achieves that.	2021-12-26 16:47:10 +03:00
Aleksey Kladov	74de79b1da	internal: rename	2021-12-25 22:02:26 +03:00
Aleksey Kladov	92dad471bc	Update crates/parser/src/lexed_str.rs ... Co-authored-by: bjorn3 <bjorn3@users.noreply.github.com>	2021-12-18 17:34:55 +03:00
Aleksey Kladov	a022ad68c9	internal: move all the lexing to the parser crate	2021-12-18 17:20:38 +03:00
Aleksey Kladov	78926027e3	converting lexed str to tokens	2021-12-18 15:36:21 +03:00
Aleksey Kladov	8b9d145dea	soa all the things	2021-12-18 15:31:50 +03:00

Renamed from crates/parser/src/lexer_token.rs (Browse further)