internal: improve `TokenSet` implementation and add reserved keywords
The current `TokenSet` type represents "A bit-set of `SyntaxKind`s" as a newtype `u128`.
Internally, the flag for each `SyntaxKind` variant in the bit-set is set as the n-th LSB (least significant bit) via a bit-wise left shift operation, where n is the discriminant.
Edit: This is problematic because there's currently ~121 token `SyntaxKind`s, so adding new token kinds for missing reserved keywords increases the number of token `SyntaxKind`s above 128, thus making this ["mask"](7a8374c162/crates/parser/src/token_set.rs (L31-L33)) operation overflow.
~~This is problematic because there's currently 266 SyntaxKinds, so this ["mask"](7a8374c162/crates/parser/src/token_set.rs (L31-L33)) operation silently overflows in release mode.~~
~~This leads to a single flag/bit in the bit-set being shared by multiple `SyntaxKind`s~~.
This PR:
- Changes the wrapped type for `TokenSet` from `u128` to `[u64; 3]` ~~`[u*; N]` (currently `[u16; 17]`) where `u*` can be any desirable unsigned integer type and `N` is the minimum array length needed to represent all token `SyntaxKind`s without any collisions~~.
- Edit: Add assertion that `TokenSet`s only include token `SyntaxKind`s
- Edit: Add ~7 missing [reserved keywords](https://doc.rust-lang.org/stable/reference/keywords.html#reserved-keywords)
- ~~Moves the definition of the `TokenSet` type to grammar codegen in xtask, so that `N` is adjusted automatically (depending on the chosen `u*` "base" type) when new `SyntaxKind`s are added~~.
- ~~Updates the `token_set_works_for_tokens` unit test to include the `__LAST` `SyntaxKind` as a way of catching overflows in tests.~~
~~Currently `u16` is arbitrarily chosen as the `u*` "base" type mostly because it strikes a good balance (IMO) between unused bits and readability of the generated `TokenSet` code (especially the [`union` method](7a8374c162/crates/parser/src/token_set.rs (L26-L28))), but I'm open to other suggestions or a better methodology for choosing `u*` type.~~
~~I considered using a third-party crate for the bit-set, but a direct implementation seems simple enough without adding any new dependencies. I'm not strongly opposed to using a third-party crate though, if that's preferred.~~
~~Finally, I haven't had the chance to review issues, to figure out if there are any parser issues caused by collisions due the current implementation that may be fixed by this PR - I just stumbled upon the issue while adding "new" keywords to solve #16858~~
Edit: fixes#16858
Add more methods for resolving definitions from AST to their corresponding HIR types
In order to be able to add these methods with consistent naming I had to also rename two existing methods that would otherwise be conflicting/confusing:
`Semantics::to_module_def(&self, file: FileId) -> Option<Module>` (before)
`Semantics::file_to_module_def(&self, file: FileId) -> Option<Module>` (after)
`Semantics::to_module_defs(&self, file: FileId) -> impl Iterator<Item = Module>` (before)
`Semantics::file_to_module_defs(&self, file: FileId) -> impl Iterator<Item = Module>` (after)
(the PR is motivated by an outside use of the `ra_ap_hir` crate that would benefit from being able to walk a `hir::Function`'s AST, resolving its exprs/stmts/items to their HIR equivalents)
fix: use 4 spaces for indentation in macro expansion
Partial fix for #16471.
In the previous code, the indentation produced by macro expansion was set to 2 spaces. This PR modifies it to 4 spaces for the sake of consistency.
Separate into create and apply edit
Rename usages
Hacky name map
Add more tests
Handle non-exhaustive
Add some more TODOs
Private fields
Use todo
Nesting
Improve rest token generation
Cleanup
Doc -> regular comment
Support mut
feat: Add "make tuple" tactic to term search
Follow up to https://github.com/rust-lang/rust-analyzer/pull/16092
Now term search also supports tuples.
```rust
let a: i32 = 1;
let b: f64 = 0.0;
let c: (i32, (f64, i32)) = todo!(); // Finds (a, (b, a))
```
In addition to new tactic that handles tuples I changed how the generics are handled.
Previously it tried all possible options from types we had in scope but now it only tries useful ones that help us directly towards the goal or at least towards calling some other function.
This changes O(2^n) to O(n^2) where n is amount of rounds which in practice allows using types that take generics for multiple rounds (previously limited to 1). Average case that also used to be exponential is now roughly linear.
This means that deeply nested generics also work.
````rust
// Finds all valid combos, including `Some(Some(Some(...)))`
let a: Option<Option<Option<bool>>> = todo!();
````
_Note that although the complexity is smaller allowing more types with generics the search overall slows down considerably. I hope it's fine tho as the autocomplete is disabled by default and for code actions it's not super slow. Might have to tweak the depth hyper parameter tho_
This resulted in a huge increase of results found (benchmarks on `ripgrep` crate):
Before
````
Tail Expr syntactic hits: 149/1692 (8%)
Tail Exprs found: 749/1692 (44%)
Term search avg time: 18ms
```
After
```
Tail Expr syntactic hits: 291/1692 (17%)
Tail Exprs found: 1253/1692 (74%)
Term search avg time: 139ms
````
Most changes are local to term search except some tuple related stuff on `hir::Type`.
Abstract more over ItemTreeLoc-like structs
Allows reducing some code duplication by using functions generic over said structs. The diff isn't negative due to me adding some additional impls for completeness.
This commit also adds `tracing` to NotificationDispatcher/RequestDispatcher,
bumps `rust-analyzer-salsa` to 0.17.0-pre.6, `always-assert` to 0.2, and
removes the homegrown `hprof` implementation in favor of a vendored
tracing-span-tree.
fix panic with reference in macro
it panic at `builder.make_mut(segment)`, where segment is from macro expand. And the usage reference in orginal macro call isn't a `PathSegment` so we can't update it in `apply_references`, I can't find a way to deal with it properly so here just filter out the reference in macro. LMK if there are better way to fix this
try to close https://github.com/rust-lang/rust-analyzer/issues/16328
`cargo clippy --fix`
This PR is the result of running `cargo clippy --fix && cargo fmt` in the root of the repository. I did not manually review all the changes, but just skimmed through a few of them. The tests still pass, so it seems fine.
fix: better handling of SelfParam in assist 'inline_call'
fix#15470.
The current `inline_call` directly translates `&self` into `let ref this = ...;` and `&mut self` into `let ref mut this = ...;`. However, it does not handle some complex scenarios.
This PR addresses the following transformations (assuming the receiving object is `obj`):
- `self`: `let this = obj`
- `mut self`: `let mut this = obj`
- `&self`: `let this = &obj`
- `&mut self`
+ If `obj` is `let mut obj = ...`, use a mutable reference: `let this = &mut obj`
+ If `obj` is `let obj = &mut ...;`, perform a reborrow: `let this = &mut *obj`
internal: Follow rustfmt's algorithm for ordering imports when ordering and merging use trees
Updates use tree ordering and merging utilities to follow rustfmt's algorithm for ordering imports.
The [rustfmt implementation](6356fca675/src/imports.rs) was used as reference.
fix: Acknowledge `pub(crate)` imports in import suggestions
rust-analyzer has logic that discounts suggesting `use`s for private imports, but that logic is unnecessarily strict - for instance given this code:
```rust
mod foo {
pub struct Foo;
}
pub(crate) use self::foo::*;
mod bar {
fn main() {
Foo$0;
}
}
```
... RA will suggest to add `use crate::foo::Foo;`, which not only makes the code overly verbose (especially in larger code bases), but also is disjoint with what rustc itself suggests.
This commit adjusts the logic, so that `pub(crate)` imports are taken into account when generating the suggestions; considering rustc's behavior, I think this change doesn't warrant any extra configuration flag.
Note that this is my first commit to RA, so I guess the approach taken here might be suboptimal - certainly feels somewhat hacky, maybe there's some better way of finding out the optimal import path 😅
rust-analyzer has logic that discounts suggesting `use`s for private
imports, but that logic is unnecessarily strict - for instance given
this code:
```rust
mod foo {
pub struct Foo;
}
pub(crate) use self::foo::*;
mod bar {
fn main() {
Foo$0;
}
}
```
... RA will suggest to add `use crate::foo::Foo;`, which not only makes
the code overly verbose (especially in larger code bases), but also is
disjoint with what rustc itself suggests.
This commit adjusts the logic, so that `pub(crate)` imports are taken
into account when generating the suggestions; considering rustc's
behavior, I think this change doesn't warrant any extra configuration
flag.
Note that this is my first commit to RA, so I guess the approach taken
here might be suboptimal - certainly feels somewhat hacky, maybe there's
some better way of finding out the optimal import path 😅
Resolve panic in `generate_delegate_methods`
Fixes#16276
This PR addresses two issues:
1. When using `PathTransform`, it searches for the node corresponding to the `path` in the `source_scope` during `make::fn_`. Therefore, we need to perform the transform before `make::fn_` (similar to the problem in issue #15804). Otherwise, even though the tokens are the same, their offsets (i.e., `span`) differ, resulting in the error "Can't find CONST_ARG@xxx."
2. As mentioned in the first point, `PathTransform` searches for the node corresponding to the `path` in the `source_scope`. Thus, when transforming paths, we should update nodes from right to left (i.e., use **reverse of preorder** (right -> left -> root) instead of **postorder** (left -> right -> root)). Reasons are as follows:
In the red-green tree (rowan), we do not store absolute ranges but instead store the length of each node and dynamically calculate offsets (spans). Therefore, when modifying the left-side node (such as nodes are inserted or deleted), it causes all right-side nodes' spans to change. This, in turn, leads to PathTransform being unable to find nodes with the same paths (due to different spans), resulting in errors.
fix: Fix `ast::Path::segments` implementation
calling `ast::Path::segments` on a qualifier currently returns all the segments of the top path instead of just the segments of the qualifier.
The issue can be summarized by the simple failing test below:
```rust
#[test]
fn path_segments() {
//use ra_ap_syntax::ast;
let path: ast::Path = ...; // e.g. `ast::Path` for "foo::bar::item".
let path_segments: Vec<_> = path.segments().collect();
let qualifier_segments: Vec<_> = path.qualifier().unwrap().segments().collect();
assert_eq!(path_segments.len(), qualifier_segments.len() + 1); // Fails because `LHS = RHS`.
}
```
This PR:
- Fixes the implementation of `ast::Path::segments`
- Fixes `ast::Path::segments` callers that either implicitly relied on behavior of previous implementation or exhibited other "wrong" behavior directly related to the result of `ast::Path::segments` (all callers have been reviewed, only one required modification)
- Removes unnecessary (and now unused) `ast::Path::segments` alternatives
