Fix OOM caused by term search
The issue came from multi Cartesian product for exprs with many (25+) arguments, each having multiple options.
The solution is two fold:
### Avoid blowing up in Cartesian product
**Before the logic was:**
1. Find expressions for each argument/param - there may be many
2. Take the Cartesian product (which blows up in some cases)
4. If there are more than 2 options throw them away by squashing them to `Many`
**Now the logic is:**
1. Find expressions for each argument/param and squash them to `Many` if there are more than 2 as otherwise we are guaranteed to also have more than 2 after taking the product which means squashing them anyway.
2. Take the Cartesian product on iterator
3. Start consuming it one by one
4. If there are more than 2 options throw them away by squashing them to `Many` (same as before)
This is also why I had to update some tests as the expressions get squashed to many more eagerly.
### Use fuel to avoid long search times and high memory usage
Now all the tactics use `should_continue: Fn() -> bool` to chech if they should keep iterating _(Similarly to chalk)_.
This reduces the search times by a magnitude, for example from ~139ms/hole to ~14ms/hole for `ripgrep` crate.
There are slightly less expressions found, but I think speed gain worth it for usability.
Also note that syntactic hits decreases more because of squashing so you simple need to run search multiple times to get full terms.
Also the worst case time (For example `nalgebra` crate cus it has tons of generics) has search times mostly under 200ms.
Benchmarks on `ripgrep` crate
Before:
```
Tail Expr syntactic hits: 291/1692 (17%)
Tail Exprs found: 1253/1692 (74%)
Term search avg time: 139ms
````
After:
```
Tail Expr syntactic hits: 239/1692 (14%)
Tail Exprs found: 1226/1692 (72%)
Term search avg time: 14ms
```
Before this commit `UseTree::remove_unnecessary_braces` removed the braces
around `{self}` in `use x::y::{self};` but `use x::y::self;` is not valid
rust.
feature: Make generate function assist generate a function as a constructor if the generated function has the name "new" and is an asscociated function.
close#17050
This PR makes `generate function assist` generate a function as a constructor if the generated function has the name "new" and is an asscociated function.
If the asscociate type is a record struct, it generates the constructor like this.
```rust
impl Foo {
fn new() -> Self {
Self { field_1: todo!(), field_2: todo!() }
}
}
```
If the asscociate type is a tuple struct, it generates the constructor like this.
```rust
impl Foo {
fn new() -> Self {
Self(todo!(), todo!())
}
}
```
If the asscociate type is a unit struct, it generates the constructor like this.
```rust
impl Foo {
fn new() -> Self {
Self
}
}
```
If the asscociate type is another adt, it generates the constructor like this.
```rust
impl Foo {
fn new() -> Self {
todo!()
}
}
```
It is bitset semantically --- many categorical things can be true about
a reference at the same time.
In parciular, a reference can be a "test" and a "write" at the same
time.
internal: improve `TokenSet` implementation and add reserved keywords
The current `TokenSet` type represents "A bit-set of `SyntaxKind`s" as a newtype `u128`.
Internally, the flag for each `SyntaxKind` variant in the bit-set is set as the n-th LSB (least significant bit) via a bit-wise left shift operation, where n is the discriminant.
Edit: This is problematic because there's currently ~121 token `SyntaxKind`s, so adding new token kinds for missing reserved keywords increases the number of token `SyntaxKind`s above 128, thus making this ["mask"](7a8374c162/crates/parser/src/token_set.rs (L31-L33)) operation overflow.
~~This is problematic because there's currently 266 SyntaxKinds, so this ["mask"](7a8374c162/crates/parser/src/token_set.rs (L31-L33)) operation silently overflows in release mode.~~
~~This leads to a single flag/bit in the bit-set being shared by multiple `SyntaxKind`s~~.
This PR:
- Changes the wrapped type for `TokenSet` from `u128` to `[u64; 3]` ~~`[u*; N]` (currently `[u16; 17]`) where `u*` can be any desirable unsigned integer type and `N` is the minimum array length needed to represent all token `SyntaxKind`s without any collisions~~.
- Edit: Add assertion that `TokenSet`s only include token `SyntaxKind`s
- Edit: Add ~7 missing [reserved keywords](https://doc.rust-lang.org/stable/reference/keywords.html#reserved-keywords)
- ~~Moves the definition of the `TokenSet` type to grammar codegen in xtask, so that `N` is adjusted automatically (depending on the chosen `u*` "base" type) when new `SyntaxKind`s are added~~.
- ~~Updates the `token_set_works_for_tokens` unit test to include the `__LAST` `SyntaxKind` as a way of catching overflows in tests.~~
~~Currently `u16` is arbitrarily chosen as the `u*` "base" type mostly because it strikes a good balance (IMO) between unused bits and readability of the generated `TokenSet` code (especially the [`union` method](7a8374c162/crates/parser/src/token_set.rs (L26-L28))), but I'm open to other suggestions or a better methodology for choosing `u*` type.~~
~~I considered using a third-party crate for the bit-set, but a direct implementation seems simple enough without adding any new dependencies. I'm not strongly opposed to using a third-party crate though, if that's preferred.~~
~~Finally, I haven't had the chance to review issues, to figure out if there are any parser issues caused by collisions due the current implementation that may be fixed by this PR - I just stumbled upon the issue while adding "new" keywords to solve #16858~~
Edit: fixes#16858
Add more methods for resolving definitions from AST to their corresponding HIR types
In order to be able to add these methods with consistent naming I had to also rename two existing methods that would otherwise be conflicting/confusing:
`Semantics::to_module_def(&self, file: FileId) -> Option<Module>` (before)
`Semantics::file_to_module_def(&self, file: FileId) -> Option<Module>` (after)
`Semantics::to_module_defs(&self, file: FileId) -> impl Iterator<Item = Module>` (before)
`Semantics::file_to_module_defs(&self, file: FileId) -> impl Iterator<Item = Module>` (after)
(the PR is motivated by an outside use of the `ra_ap_hir` crate that would benefit from being able to walk a `hir::Function`'s AST, resolving its exprs/stmts/items to their HIR equivalents)
fix: use 4 spaces for indentation in macro expansion
Partial fix for #16471.
In the previous code, the indentation produced by macro expansion was set to 2 spaces. This PR modifies it to 4 spaces for the sake of consistency.
Separate into create and apply edit
Rename usages
Hacky name map
Add more tests
Handle non-exhaustive
Add some more TODOs
Private fields
Use todo
Nesting
Improve rest token generation
Cleanup
Doc -> regular comment
Support mut
feat: Add "make tuple" tactic to term search
Follow up to https://github.com/rust-lang/rust-analyzer/pull/16092
Now term search also supports tuples.
```rust
let a: i32 = 1;
let b: f64 = 0.0;
let c: (i32, (f64, i32)) = todo!(); // Finds (a, (b, a))
```
In addition to new tactic that handles tuples I changed how the generics are handled.
Previously it tried all possible options from types we had in scope but now it only tries useful ones that help us directly towards the goal or at least towards calling some other function.
This changes O(2^n) to O(n^2) where n is amount of rounds which in practice allows using types that take generics for multiple rounds (previously limited to 1). Average case that also used to be exponential is now roughly linear.
This means that deeply nested generics also work.
````rust
// Finds all valid combos, including `Some(Some(Some(...)))`
let a: Option<Option<Option<bool>>> = todo!();
````
_Note that although the complexity is smaller allowing more types with generics the search overall slows down considerably. I hope it's fine tho as the autocomplete is disabled by default and for code actions it's not super slow. Might have to tweak the depth hyper parameter tho_
This resulted in a huge increase of results found (benchmarks on `ripgrep` crate):
Before
````
Tail Expr syntactic hits: 149/1692 (8%)
Tail Exprs found: 749/1692 (44%)
Term search avg time: 18ms
```
After
```
Tail Expr syntactic hits: 291/1692 (17%)
Tail Exprs found: 1253/1692 (74%)
Term search avg time: 139ms
````
Most changes are local to term search except some tuple related stuff on `hir::Type`.
Abstract more over ItemTreeLoc-like structs
Allows reducing some code duplication by using functions generic over said structs. The diff isn't negative due to me adding some additional impls for completeness.
This commit also adds `tracing` to NotificationDispatcher/RequestDispatcher,
bumps `rust-analyzer-salsa` to 0.17.0-pre.6, `always-assert` to 0.2, and
removes the homegrown `hprof` implementation in favor of a vendored
tracing-span-tree.
fix panic with reference in macro
it panic at `builder.make_mut(segment)`, where segment is from macro expand. And the usage reference in orginal macro call isn't a `PathSegment` so we can't update it in `apply_references`, I can't find a way to deal with it properly so here just filter out the reference in macro. LMK if there are better way to fix this
try to close https://github.com/rust-lang/rust-analyzer/issues/16328
`cargo clippy --fix`
This PR is the result of running `cargo clippy --fix && cargo fmt` in the root of the repository. I did not manually review all the changes, but just skimmed through a few of them. The tests still pass, so it seems fine.
fix: better handling of SelfParam in assist 'inline_call'
fix#15470.
The current `inline_call` directly translates `&self` into `let ref this = ...;` and `&mut self` into `let ref mut this = ...;`. However, it does not handle some complex scenarios.
This PR addresses the following transformations (assuming the receiving object is `obj`):
- `self`: `let this = obj`
- `mut self`: `let mut this = obj`
- `&self`: `let this = &obj`
- `&mut self`
+ If `obj` is `let mut obj = ...`, use a mutable reference: `let this = &mut obj`
+ If `obj` is `let obj = &mut ...;`, perform a reborrow: `let this = &mut *obj`
internal: Follow rustfmt's algorithm for ordering imports when ordering and merging use trees
Updates use tree ordering and merging utilities to follow rustfmt's algorithm for ordering imports.
The [rustfmt implementation](6356fca675/src/imports.rs) was used as reference.
fix: Acknowledge `pub(crate)` imports in import suggestions
rust-analyzer has logic that discounts suggesting `use`s for private imports, but that logic is unnecessarily strict - for instance given this code:
```rust
mod foo {
pub struct Foo;
}
pub(crate) use self::foo::*;
mod bar {
fn main() {
Foo$0;
}
}
```
... RA will suggest to add `use crate::foo::Foo;`, which not only makes the code overly verbose (especially in larger code bases), but also is disjoint with what rustc itself suggests.
This commit adjusts the logic, so that `pub(crate)` imports are taken into account when generating the suggestions; considering rustc's behavior, I think this change doesn't warrant any extra configuration flag.
Note that this is my first commit to RA, so I guess the approach taken here might be suboptimal - certainly feels somewhat hacky, maybe there's some better way of finding out the optimal import path 😅
rust-analyzer has logic that discounts suggesting `use`s for private
imports, but that logic is unnecessarily strict - for instance given
this code:
```rust
mod foo {
pub struct Foo;
}
pub(crate) use self::foo::*;
mod bar {
fn main() {
Foo$0;
}
}
```
... RA will suggest to add `use crate::foo::Foo;`, which not only makes
the code overly verbose (especially in larger code bases), but also is
disjoint with what rustc itself suggests.
This commit adjusts the logic, so that `pub(crate)` imports are taken
into account when generating the suggestions; considering rustc's
behavior, I think this change doesn't warrant any extra configuration
flag.
Note that this is my first commit to RA, so I guess the approach taken
here might be suboptimal - certainly feels somewhat hacky, maybe there's
some better way of finding out the optimal import path 😅
Resolve panic in `generate_delegate_methods`
Fixes#16276
This PR addresses two issues:
1. When using `PathTransform`, it searches for the node corresponding to the `path` in the `source_scope` during `make::fn_`. Therefore, we need to perform the transform before `make::fn_` (similar to the problem in issue #15804). Otherwise, even though the tokens are the same, their offsets (i.e., `span`) differ, resulting in the error "Can't find CONST_ARG@xxx."
2. As mentioned in the first point, `PathTransform` searches for the node corresponding to the `path` in the `source_scope`. Thus, when transforming paths, we should update nodes from right to left (i.e., use **reverse of preorder** (right -> left -> root) instead of **postorder** (left -> right -> root)). Reasons are as follows:
In the red-green tree (rowan), we do not store absolute ranges but instead store the length of each node and dynamically calculate offsets (spans). Therefore, when modifying the left-side node (such as nodes are inserted or deleted), it causes all right-side nodes' spans to change. This, in turn, leads to PathTransform being unable to find nodes with the same paths (due to different spans), resulting in errors.
fix: Fix `ast::Path::segments` implementation
calling `ast::Path::segments` on a qualifier currently returns all the segments of the top path instead of just the segments of the qualifier.
The issue can be summarized by the simple failing test below:
```rust
#[test]
fn path_segments() {
//use ra_ap_syntax::ast;
let path: ast::Path = ...; // e.g. `ast::Path` for "foo::bar::item".
let path_segments: Vec<_> = path.segments().collect();
let qualifier_segments: Vec<_> = path.qualifier().unwrap().segments().collect();
assert_eq!(path_segments.len(), qualifier_segments.len() + 1); // Fails because `LHS = RHS`.
}
```
This PR:
- Fixes the implementation of `ast::Path::segments`
- Fixes `ast::Path::segments` callers that either implicitly relied on behavior of previous implementation or exhibited other "wrong" behavior directly related to the result of `ast::Path::segments` (all callers have been reviewed, only one required modification)
- Removes unnecessary (and now unused) `ast::Path::segments` alternatives
internal: clean and enhance readability for `generate_delegate_trait`
Continue from #16112
This PR primarily involves some cleanup and simple refactoring work, including:
- Adding numerous comments to layer the code and explain the behavior of each step.
- Renaming some variables to make them more sensible.
- Simplify certain operations using a more elegant approach.
The goal is to make this intricate implementation clearer and facilitate future maintenance.
In addition to this, the PR also removes redundant `path_transform` operations for `type_gen_args`.
Taking the example of `impl Trait<T1> for S<S1>`, where `S1` is considered. The struct `S` must be in the file where the user triggers code actions, so there's no need for the `path_transform`. Furthermore, before performing the transform, we've already renamed `S1`, ensuring it won't clash with existing generics parameters. Therefore, there's no need to transform it.
* Extracted the function `for_unique_generic_name` that handling generics with identical names for reusability.
* Renamed `for_generic_params` to `for_impl_trait_as_generic` for clarity
* Added documentations for `for_impl_trait_as_generic` and `for_unique_generic_name`
fix: rewrite code_action `generate_delegate_trait`
I've made substantial enhancements to the "generate delegate trait" code action in rust-analyzer. Here's a summary of the changes:
#### Resolved the "Can’t find CONST_ARG@158..159 in AstIdMap" error
Fix#15804, fix#15968, fix#15108
The issue stemmed from an incorrect application of PathTransform in the original code. Previously, a new 'impl' was generated first and then transformed, causing PathTransform to fail in locating the correct AST node, resulting in an error. I rectified this by performing the transformation before generating the new 'impl' (using make::impl_trait), ensuring a step-by-step transformation of associated items.
#### Rectified generation of `Self` type
`generate_delegate_trait` is unable to properly handle trait with `Self` type.
Let's take the following code as an example:
```rust
trait Trait {
fn f() -> Self;
}
struct B {}
impl Trait for B {
fn f() -> B { B{} }
}
struct S {
b: B,
}
```
Here, if we implement `Trait` for `S`, the type of `f` should be `() -> Self`, i.e. `() -> S`. However we cannot automatically generate a function that constructs `S`.
To ensure that the code action doesn't generate delegate traits for traits with Self types, I add a function named `has_self_type` to handle it.
#### Extended support for generics in structs and fields within this code action
The former version of `generate_delegate_trait` cannot handle structs with generics properly. Here's an example:
```rust
struct B<T> {
a: T
}
trait Trait<T> {
fn f(a: T);
}
impl<T1, T2> Trait<T1> for B<T2> {
fn f(a: T1) -> T2 { self.a }
}
struct A {}
struct S {
b$0 : B<A>,
}
```
The former version will generates improper code:
```rust
impl<T1, T2> Trait<T1, T2> for S {
fn f(&self, a: T1) -> T1 {
<B as Trait<T1, T2>>::f( &self.b , a)
}
}
```
The rewritten version can handle generics properly:
```rust
impl<T1> Trait<T1> for S {
fn f(&self, a: T1) -> T1 {
<B<A> as Trait<T1>>::f(&self.b, a)
}
}
```
See more examples in added unit tests.
I enabled support for generic structs in `generate_delegate_trait` through the following steps (using the code example provided):
1. Initially, to prevent conflicts between the generic parameters in struct `S` and the ones in the impl of `B`, I renamed the generic parameters of `S`.
2. Then, since `B`'s parameters are instantiated within `S`, the original generic parameters of `B` needed removal within `S` (to avoid errors from redundant parameters). An important consideration here arises when Trait and B share parameters in `B`'s impl. In such cases, these shared generic parameters cannot be removed.
3. Next, I addressed the matching of types between `B`'s type in `S` and its type in the impl. Given that some generic parameters in the impl are instantiated in `B`, I replaced these parameters with their instantiated results using PathTransform. For instance, in the example provided, matching `B<A>` and `B<T2>`, where `T2` is instantiated as `A`, I replaced all occurrences of `T2` in the impl with `A` (i.e. apply the instantiated generic arguments to the params).
4. Finally, I performed transformations on each assoc item (also to prevent the initial issue) and handled redundant where clauses.
For a more detailed explanation, please refer to the code and comments. I welcome suggestions and any further questions!
fix: self type replacement in inline-function
Fix#16113, fix#16091
The problem described in this issue actually involves three bugs.
Firstly, when using `ted` to modify the syntax tree, the offset of nodes on the tree changes, which causes the syntax range information from `hir` to become invalid. Therefore, we need to edit the AST after the last usage for `usages_for_locals`.
The second issue is that when inserting nodes, it's necessary to use `clone_subtree` for duplication because the `ted::replace` operation essentially moves a node.
The third issue is that we should use `ancestors_with_macros` instead of `ancestors` to handle impl definition in macros.
I have fixed the three bugs mentioned above and added unit tests.
internal: Migrate assists to the structured snippet API, part 5
Continuing from #15874
Migrates the following assists:
- `extract_variable`
- `generate_function`
- `replace_is_some_with_if_let_some`
- `replace_is_ok_with_if_let_ok`
fix: no code action 'introduce_named_generic' for impl inside types
Fix#15734.
### Changes Made
- Find params in `ancestors` instead of just `parent`
- Added tests (`replace_impl_with_mut` and `replace_impl_inside`)
Fix incorrectly replacing references in macro invocation in "Convert to named struct" assist
Fixes#15630.
Complements #13647 (same assist but missed this one), #14920 (inverse action assist).
fix: resolve Self type references in delegate method assist
This PR makes the delegate method assist resolve any `Self` type references in the parameters or return type. It also works across macros such as the `uint_impl!` macro used for `saturating_mul` in the issue example.
Closes#14485
internal: Migrate assists to the structured snippet API, part 4
Continuing from #15260
Migrates the following assists:
- `add_turbo_fish`
- `add_type_ascription`
- `destructure_tuple_binding`
- `destructure_tuple_binding_in_subpattern`
I did this a while ago, but forgot to make a PR for the changes until now. 😅
Due to the way the current tree mutation api works, we need to collect
changes before we can apply them to the real syntax tree, and also can only
switch to a file once.
`destructure_tuple_binding_in_sub_pattern` also gets migrated even
though can't be used.
