TokenMap -> SpanMap rewrite
Opening early so I can have an overview over the full diff more easily, still very unfinished and lots of work to be done.
The gist of what this PR does is move away from assigning IDs to tokens in arguments and expansions and instead gives the subtrees the text ranges they are sourced from (made relative to some item for incrementality). This means we now only have a single map per expension, opposed to map for expansion and arguments.
A few of the things that are not done yet (in arbitrary order):
- [x] generally clean up the current mess
- [x] proc-macros, have been completely ignored so far
- [x] syntax fixups, has been commented out for the time being needs to be rewritten on top of some marker SyntaxContextId
- [x] macro invocation syntax contexts are not properly passed around yet, so $crate hygiene does not work in all cases (but most)
- [x] builtin macros do not set spans properly, $crate basically does not work with them rn (which we use)
~~- [ ] remove all uses of dummy spans (or if that does not work, change the dummy entries for dummy spans so that tests will not silently pass due to havin a file id for the dummy file)~~
- [x] de-queryfy `macro_expand`, the sole caller of it is `parse_macro_expansion`, and both of these are lru-cached with the same limit so having it be a query is pointless
- [x] docs and more docs
- [x] fix eager macro spans and other stuff
- [x] simplify include! handling
- [x] Figure out how to undo the sudden `()` expression wrapping in expansions / alternatively prioritize getting invisible delimiters working again
- [x] Simplify InFile stuff and HirFIleId extensions
~~- [ ] span crate containing all the file ids, span stuff, ast ids. Then remove the dependency injection generics from tt and mbe~~
Fixes https://github.com/rust-lang/rust-analyzer/issues/10300
Fixes https://github.com/rust-lang/rust-analyzer/issues/15685
Resolve inlay hint data
Part of https://github.com/rust-lang/rust-analyzer/issues/13962
Support https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#inlayHint_resolve better, by omitting all inlay hint fields specified in the client hint resolve capabilities.
Current list of all capabilities possible to resolve later:
```
"textEdits"
"tooltip"
"label.tooltip"
"label.location"
"label.command"
```
and every one specified in the client capabilities is now resolved by r-a, being omitted in the initial response.
--------------
When editing `inlay_hints.rs` file around line `457` with no resolve capabilities, I get
<details>
<summary>resolved json, 10803 characters</summary>
```json
{"jsonrpc":"2.0","id":55,"result":[{"position":{"line":477,"character":1},"label":[{"value":"fn inlay_hints","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/ide/src/inlay_hints.rs","range":{"start":{"line":445,"character":14},"end":{"line":445,"character":25}}}}],"paddingLeft":true,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":451,"character":10},"label":[{"value":": "},{"value":"ProfileSpan","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/profile/src/hprof.rs","range":{"start":{"line":85,"character":11},"end":{"line":85,"character":22}}}},{"value":""}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":451,"character":27},"label":[{"value":"label:","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/profile/src/hprof.rs","range":{"start":{"line":60,"character":12},"end":{"line":60,"character":17}}}}],"kind":2,"paddingLeft":false,"paddingRight":true,"data":{"file_id":0}},{"position":{"line":452,"character":12},"label":[{"value":": "},{"value":"Semantics","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/hir/src/semantics.rs","range":{"start":{"line":108,"character":11},"end":{"line":108,"character":20}}}},{"value":"<'_, "},{"value":"RootDatabase","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/ide-db/src/lib.rs","range":{"start":{"line":75,"character":11},"end":{"line":75,"character":23}}}},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":453,"character":12},"label":[{"value":": "},{"value":"SourceFile","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/syntax/src/ast/generated/nodes.rs","range":{"start":{"line":223,"character":11},"end":{"line":223,"character":21}}}},{"value":""}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":454,"character":12},"label":[{"value":": &"},{"value":"SyntaxNode","location":{"uri":"file:///Users/someonetoignore/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rowan-0.15.11/src/api.rs","range":{"start":{"line":15,"character":11},"end":{"line":15,"character":21}}}},{"value":"<"},{"value":"RustLanguage","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/syntax/src/syntax_node.rs","range":{"start":{"line":15,"character":9},"end":{"line":15,"character":21}}}},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":456,"character":12},"label":": i32","kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":458,"character":15},"label":[{"value":": "},{"value":"Vec","location":{"uri":"file:///Users/someonetoignore/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/vec/mod.rs","range":{"start":{"line":395,"character":11},"end":{"line":395,"character":14}}}},{"value":"<"},{"value":"InlayHint","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/ide/src/inlay_hints.rs","range":{"start":{"line":149,"character":11},"end":{"line":149,"character":20}}}},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":460,"character":21},"label":[{"value":": "},{"value":"SemanticsScope","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/hir/src/semantics.rs","range":{"start":{"line":1651,"character":11},"end":{"line":1651,"character":25}}}},{"value":"<'_>"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":460,"character":36},"label":[{"value":"node:","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/hir/src/semantics.rs","range":{"start":{"line":482,"character":24},"end":{"line":482,"character":28}}}}],"kind":2,"paddingLeft":false,"paddingRight":true,"data":{"file_id":0}},{"position":{"line":461,"character":23},"label":[{"value":": "},{"value":"FamousDefs","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/ide-db/src/famous_defs.rs","range":{"start":{"line":20,"character":11},"end":{"line":20,"character":21}}}},{"value":"<'_, '_>"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":463,"character":17},"label":[{"value":": impl FnMut("},{"value":"SyntaxNode","location":{"uri":"file:///Users/someonetoignore/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rowan-0.15.11/src/api.rs","range":{"start":{"line":15,"character":11},"end":{"line":15,"character":21}}}},{"value":"<"},{"value":"RustLanguage","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/syntax/src/syntax_node.rs","range":{"start":{"line":15,"character":9},"end":{"line":15,"character":21}}}},{"value":">)"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":463,"character":25},"label":[{"value":": "},{"value":"SyntaxNode","location":{"uri":"file:///Users/someonetoignore/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rowan-0.15.11/src/api.rs","range":{"start":{"line":15,"character":11},"end":{"line":15,"character":21}}}},{"value":"<"},{"value":"RustLanguage","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/syntax/src/syntax_node.rs","range":{"start":{"line":15,"character":9},"end":{"line":15,"character":21}}}},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":463,"character":33},"label":[{"value":"hints:","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/ide/src/inlay_hints.rs","range":{"start":{"line":480,"character":4},"end":{"line":480,"character":9}}}}],"kind":2,"paddingLeft":false,"paddingRight":true,"data":{"file_id":0}},{"position":{"line":465,"character":22},"label":[{"value":": "},{"value":"TextRange","location":{"uri":"file:///Users/someonetoignore/.cargo/registry/src/index.crates.io-6f17d22bba15001f/text-size-1.1.0/src/range.rs","range":{"start":{"line":14,"character":11},"end":{"line":14,"character":20}}}},{"value":""}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":467,"character":35},"label":[{"value":": "},{"value":"SyntaxNode","location":{"uri":"file:///Users/someonetoignore/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rowan-0.15.11/src/api.rs","range":{"start":{"line":15,"character":11},"end":{"line":15,"character":21}}}},{"value":"<"},{"value":"RustLanguage","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/syntax/src/syntax_node.rs","range":{"start":{"line":15,"character":9},"end":{"line":15,"character":21}}}},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":469,"character":92},"label":[{"value":"impl "},{"value":"Iterator","location":{"uri":"file:///Users/someonetoignore/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs","range":{"start":{"line":72,"character":10},"end":{"line":72,"character":18}}}},{"value":"<"},{"value":"Item","location":{"uri":"file:///Users/someonetoignore/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs","range":{"start":{"line":76,"character":9},"end":{"line":76,"character":13}}}},{"value":" = "},{"value":"SyntaxNode","location":{"uri":"file:///Users/someonetoignore/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rowan-0.15.11/src/api.rs","range":{"start":{"line":15,"character":11},"end":{"line":15,"character":21}}}},{"value":"<"},{"value":"RustLanguage","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/syntax/src/syntax_node.rs","range":{"start":{"line":15,"character":9},"end":{"line":15,"character":21}}}},{"value":">>"}],"kind":1,"paddingLeft":true,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":468,"character":34},"label":[{"value":"impl "},{"value":"Iterator","location":{"uri":"file:///Users/someonetoignore/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs","range":{"start":{"line":72,"character":10},"end":{"line":72,"character":18}}}},{"value":"<"},{"value":"Item","location":{"uri":"file:///Users/someonetoignore/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs","range":{"start":{"line":76,"character":9},"end":{"line":76,"character":13}}}},{"value":" = "},{"value":"SyntaxNode","location":{"uri":"file:///Users/someonetoignore/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rowan-0.15.11/src/api.rs","range":{"start":{"line":15,"character":11},"end":{"line":15,"character":21}}}},{"value":"<"},{"value":"RustLanguage","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/syntax/src/syntax_node.rs","range":{"start":{"line":15,"character":9},"end":{"line":15,"character":21}}}},{"value":">>"},{"value":""}],"kind":1,"paddingLeft":true,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":467,"character":41},"label":[{"value":""},{"value":"SyntaxNode","location":{"uri":"file:///Users/someonetoignore/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rowan-0.15.11/src/api.rs","range":{"start":{"line":15,"character":11},"end":{"line":15,"character":21}}}},{"value":"<"},{"value":"RustLanguage","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/syntax/src/syntax_node.rs","range":{"start":{"line":15,"character":9},"end":{"line":15,"character":21}}}},{"value":">"}],"kind":1,"paddingLeft":true,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":469,"character":40},"label":" -> bool","kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":469,"character":39},"label":[{"value":": &"},{"value":"SyntaxNode","location":{"uri":"file:///Users/someonetoignore/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rowan-0.15.11/src/api.rs","range":{"start":{"line":15,"character":11},"end":{"line":15,"character":21}}}},{"value":"<"},{"value":"RustLanguage","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/syntax/src/syntax_node.rs","range":{"start":{"line":15,"character":9},"end":{"line":15,"character":21}}}},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}}]}
```
</details>
for the visible editor range alone, pretty much repeated on every consequent edit.
With this patch and all inlay hint resolve capabilities enabled, for the same example I observe quite a footprint reduction:
<details>
<summary>unresolved json, 4142 characters</summary>
```json
{"jsonrpc":"2.0","id":49,"result":[{"position":{"line":477,"character":1},"label":[{"value":"fn inlay_hints"}],"paddingLeft":true,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":451,"character":10},"label":[{"value":": "},{"value":"ProfileSpan"},{"value":""}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":451,"character":27},"label":[{"value":"label:"}],"kind":2,"paddingLeft":false,"paddingRight":true,"data":{"file_id":0}},{"position":{"line":452,"character":12},"label":[{"value":": "},{"value":"Semantics"},{"value":"<'_, "},{"value":"RootDatabase"},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":453,"character":12},"label":[{"value":": "},{"value":"SourceFile"},{"value":""}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":454,"character":12},"label":[{"value":": &"},{"value":"SyntaxNode"},{"value":"<"},{"value":"RustLanguage"},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":456,"character":12},"label":": i32","kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":458,"character":15},"label":[{"value":": "},{"value":"Vec"},{"value":"<"},{"value":"InlayHint"},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":460,"character":21},"label":[{"value":": "},{"value":"SemanticsScope"},{"value":"<'_>"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":460,"character":36},"label":[{"value":"node:"}],"kind":2,"paddingLeft":false,"paddingRight":true,"data":{"file_id":0}},{"position":{"line":461,"character":23},"label":[{"value":": "},{"value":"FamousDefs"},{"value":"<'_, '_>"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":463,"character":17},"label":[{"value":": impl FnMut("},{"value":"SyntaxNode"},{"value":"<"},{"value":"RustLanguage"},{"value":">)"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":463,"character":25},"label":[{"value":": "},{"value":"SyntaxNode"},{"value":"<"},{"value":"RustLanguage"},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":463,"character":33},"label":[{"value":"hints:"}],"kind":2,"paddingLeft":false,"paddingRight":true,"data":{"file_id":0}},{"position":{"line":465,"character":22},"label":[{"value":": "},{"value":"TextRange"},{"value":""}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":467,"character":35},"label":[{"value":": "},{"value":"SyntaxNode"},{"value":"<"},{"value":"RustLanguage"},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":469,"character":92},"label":[{"value":"impl "},{"value":"Iterator"},{"value":"<"},{"value":"Item"},{"value":" = "},{"value":"SyntaxNode"},{"value":"<"},{"value":"RustLanguage"},{"value":">>"}],"kind":1,"paddingLeft":true,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":468,"character":34},"label":[{"value":"impl "},{"value":"Iterator"},{"value":"<"},{"value":"Item"},{"value":" = "},{"value":"SyntaxNode"},{"value":"<"},{"value":"RustLanguage"},{"value":">>"},{"value":""}],"kind":1,"paddingLeft":true,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":467,"character":41},"label":[{"value":""},{"value":"SyntaxNode"},{"value":"<"},{"value":"RustLanguage"},{"value":">"}],"kind":1,"paddingLeft":true,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":469,"character":40},"label":" -> bool","kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":469,"character":39},"label":[{"value":": &"},{"value":"SyntaxNode"},{"value":"<"},{"value":"RustLanguage"},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}}]}
```
</details>
with all unresolved parts needing only for navigation, hover or applying the hint edit — dynamic parts that are made after mouse hover or similar events, that resolve the hint data.
