TokenMap -> SpanMap rewrite
Opening early so I can have an overview over the full diff more easily, still very unfinished and lots of work to be done.
The gist of what this PR does is move away from assigning IDs to tokens in arguments and expansions and instead gives the subtrees the text ranges they are sourced from (made relative to some item for incrementality). This means we now only have a single map per expension, opposed to map for expansion and arguments.
A few of the things that are not done yet (in arbitrary order):
- [x] generally clean up the current mess
- [x] proc-macros, have been completely ignored so far
- [x] syntax fixups, has been commented out for the time being needs to be rewritten on top of some marker SyntaxContextId
- [x] macro invocation syntax contexts are not properly passed around yet, so $crate hygiene does not work in all cases (but most)
- [x] builtin macros do not set spans properly, $crate basically does not work with them rn (which we use)
~~- [ ] remove all uses of dummy spans (or if that does not work, change the dummy entries for dummy spans so that tests will not silently pass due to havin a file id for the dummy file)~~
- [x] de-queryfy `macro_expand`, the sole caller of it is `parse_macro_expansion`, and both of these are lru-cached with the same limit so having it be a query is pointless
- [x] docs and more docs
- [x] fix eager macro spans and other stuff
- [x] simplify include! handling
- [x] Figure out how to undo the sudden `()` expression wrapping in expansions / alternatively prioritize getting invisible delimiters working again
- [x] Simplify InFile stuff and HirFIleId extensions
~~- [ ] span crate containing all the file ids, span stuff, ast ids. Then remove the dependency injection generics from tt and mbe~~
Fixes https://github.com/rust-lang/rust-analyzer/issues/10300
Fixes https://github.com/rust-lang/rust-analyzer/issues/15685
Sered no longer uses blobs as of
https://github.com/serde-rs/serde/pull/2590
As such, there's no longer need for us to pin it.
Note that this doesn't upgrade serde version we use: I am fairly
confident that the blobs are already there are fine, and now I am fairly
confident that all future versions of serde will be fine as well.
serde 1.0.172 and up rely on opaque non-reproducible binary blobs to
function, explicitly not providing a library-level opt-out.
This is problematic for two reasons:
- directly, unauditable binary blobs are a security issue.
- indirectly, it becomes much harder to predict future behaviors of the
crate.
As such, I am willing to go on a limb here and forbid building
rust-analyzer with those versions of serde. Normally, my philosophy is
to defer the choice to the end user, but it's also a design constraint
of rust-analyzer that we don't run random binaries downloaded from the
internet without explicit user's concent.
Concretely, this upper-bounds serde for both rust-analyzer workspace, as
well as the lsp-server lib.
See https://github.com/serde-rs/serde/issues/2538 for wider context.
