Fix OOM caused by term search
The issue came from multi Cartesian product for exprs with many (25+) arguments, each having multiple options.
The solution is two fold:
### Avoid blowing up in Cartesian product
**Before the logic was:**
1. Find expressions for each argument/param - there may be many
2. Take the Cartesian product (which blows up in some cases)
4. If there are more than 2 options throw them away by squashing them to `Many`
**Now the logic is:**
1. Find expressions for each argument/param and squash them to `Many` if there are more than 2 as otherwise we are guaranteed to also have more than 2 after taking the product which means squashing them anyway.
2. Take the Cartesian product on iterator
3. Start consuming it one by one
4. If there are more than 2 options throw them away by squashing them to `Many` (same as before)
This is also why I had to update some tests as the expressions get squashed to many more eagerly.
### Use fuel to avoid long search times and high memory usage
Now all the tactics use `should_continue: Fn() -> bool` to chech if they should keep iterating _(Similarly to chalk)_.
This reduces the search times by a magnitude, for example from ~139ms/hole to ~14ms/hole for `ripgrep` crate.
There are slightly less expressions found, but I think speed gain worth it for usability.
Also note that syntactic hits decreases more because of squashing so you simple need to run search multiple times to get full terms.
Also the worst case time (For example `nalgebra` crate cus it has tons of generics) has search times mostly under 200ms.
Benchmarks on `ripgrep` crate
Before:
```
Tail Expr syntactic hits: 291/1692 (17%)
Tail Exprs found: 1253/1692 (74%)
Term search avg time: 139ms
````
After:
```
Tail Expr syntactic hits: 239/1692 (14%)
Tail Exprs found: 1226/1692 (72%)
Term search avg time: 14ms
```
If rust-analyzer receives a malformed LSP request, the IO thread
terminates with a meaningful error, but then closes the channel.
Once the channel has closed, the main_loop also terminates, but it
only has RecvError and can't show a meaningful error. As a result,
rust-analyzer would incorrectly claim that the client forgot to
shutdown.
```
$ buggy_lsp_client | rust-analyzer
Error: client exited without proper shutdown sequence
```
Instead, include both error messages when the server shuts down.
The documentation for `lens.run.enable` states that it only applies
when `lens.enable` is set. However, the config setting whether to show
the Run lens did not check `lens.enable`, so the Run lens would show
even though lenses were disabled.
fix: Tracing span names should match function names
When viewing traces, it's slightly confusing when the span name doesn't match the function name. Ensure the names are consistent.
(It might be worth moving most of these to use `#[tracing::instrument]` so the name can never go stale. `@davidbarsky` suggested that is marginally slower, so I've just done the simple change here.)
When viewing traces, it's slightly confusing when the span name doesn't
match the function name. Ensure the names are consistent.
(It might be worth moving most of these to use #[tracing::instrument]
so the name can never go stale. @davidbarsky suggested that is marginally
slower, so I've just done the simple change here.)
Support hovering limits for adts
Fix#17009
1. Currently, r-a supports limiting the number of struct fields displayed when hovering. This PR extends it to support enum variants and union fields. Since the display of these three (ADTs) is similar, this PR extends 'hover_show_structFields' to 'hover_show_adtFieldsOrVariants'.
2. This PR also resolved the problem that the layout of ADT was not restricted by display limitations when hovering on the Self type.
3. Additionally, this PR changes the default value of display limitations to `10` (instead of the original `null`), which helps users discover this feature.
Make `cargo run` always available for binaries
Previously, items for `cargo test` and `cargo check` would appear as in
the `Select Runnable` quick pick that appears when running
`rust-analyzer: Run`, but `run` would only appear as a runnable if a
`main`` function was selected in the editor. This change adds `cargo
run` as an always available runnable command for binary packages.
This makes it easier to develop cli / tui applications, as now users can
run application from anywhere in their codebase.
chore: add some `tracing` to project loading
I wanted to see what's happening during project loading and if it could be parallelized. I'm thinking maybe, but it's not this PR :)
* Added config `runnables.extraTestBinaryArgs` to control the args.
* The default is `--show-output` rather than `--nocapture` to prevent
unreadable output when 2 or more tests fail or print output at once.
* Renamed variables in `CargoTargetSpec::runnable_args()` for clarity.
Fixes <https://github.com/rust-lang/rust-analyzer/issues/12737>.
It is bitset semantically --- many categorical things can be true about
a reference at the same time.
In parciular, a reference can be a "test" and a "write" at the same
time.
internal : redesign rust-analyzer::config
This PR aims to cover the infrastructural requirements for the `rust-analyzer.toml` ( #13529 ) issue. This means, that
1. We no longer have a single config base. The once single `ConfigData` has been divided into 4 : A tree of `.ratoml` files, a set of configs coming from the client ( this is what was called before the `CrateData` except that now values do not default to anything when they are not defined) , a set of configs that will reflect what the contents of a `ratoml` file defined in user's config directory ( e.g `~/.config/rust-analyzer/.rust-analyzer.toml` and finally a tree root that is populated by default values only.
2. Configs have also been divided into 3 different blocks : `global` , `local` , `client`. The current status of a config may change until #13529 got merged.
Once again many thanks to `@cormacrelf` for doing all the serde work.
fix: Fix inlay hint resolution being broken
So, things broke because we now store a hash (u64) in the resolution payload, but javascript and hence JSON only support integers of up to 53 bits (anything beyond gets truncated in various ways) which caused almost all hashes to always differ when resolving them. This masks the hash to 53 bits to work around that.
Fixes https://github.com/rust-lang/rust-analyzer/issues/16962
On this file,
```rust
fn main() {
let 된장 = 1;
}
```
when using `"positionEncodings":["utf-16"]` I get an "unused variable" diagnostic on the variable
name (codepoint offset range `8..10`). So far so good.
When using `positionEncodings":["utf-8"]`, I expect to get the equivalent range in bytes (LSP:
"Character offsets count UTF-8 code units (e.g bytes)."), which is `8..14`, because both
characters are 3 bytes in UTF-8. However I actually get `10..14`.
Looks like this is because we accidentally treat a 1-based index as an offset value: when
converting from our internal char-indices to LSP byte offsets, we look at one character to many.
This causes wrong results if the extra character is a multi-byte one, such as when computing
the start coordinate of 된장.
Fix that by actually passing an offset. While at it, fix the variable name of the line number,
which is not an offset (yet).
Originally reported at https://github.com/kakoune-lsp/kakoune-lsp/issues/740
Fix crate IDs when multiple workspaces are loaded
Previously, we assumed that the crate numbers in a `rust-project.json` always matched the `CrateId` values in the crate graph. This isn't true when there are multiple workspaces, because the crate graphs are merged and the `CrateId` values in the merged graph are different.
This broke flycheck (see first commit), because we were unable to find the workspace when a file changed, so we every single flycheck, producing duplicate compilation errors.
Instead, use the crate root module path to look up the relevant flycheck. This makes `ProjectWorkspace::Json` consistenet with `ProjectWorkspace::Cargo`.
Also, define a separate JSON crate number type, to prevent bugs like this happening again.
feat: Add `rust-analyzer.cargo.allTargets` to configure passing `--all-targets` to cargo invocations
Closes#16859
## Unresolved question:
Should this be a setting for build scripts only ? All the other `--all-targets` I found where already covered by `checkOnSave.allTargets`
Previously, items for `cargo test` and `cargo check` would appear as in
the `Select Runnable` quick pick that appears when running
`rust-analyzer: Run`, but `run` would only appear as a runnable if a
`main`` function was selected in the editor. This change adds `cargo
run` as an always available runnable command for binary packages.
This makes it easier to develop cli / tui applications, as now users can
run application from anywhere in their codebase.
internal: Enforce utf8 paths
Cargo already requires this, and I highly doubt r-a works with non-utf8 paths generally either. This just makes dealing with paths a lot easier.
feat: Syntax highlighting improvements
Specifically
- Adds a new `constant` modifier, attached to keyword `const` (except for `*const ()` and `&raw const ()`), `const` items and `const` functions
- Adds (or rather reveals) `associated` modifier for associated items
- Fixes usage of the standard `static` modifier, now it acts like `associated` except being omitted for methods.
- Splits `SymbolKind::Function` into `Function` and `Method`. We already split other things like that (notable self param from params), so the split makes sense in general as a lot special cases around it anyways.
internal: Compress file text using LZ4
I haven't tested properly, but this roughly looks like:
```
1246 MB
59mb 4899 FileTextQuery
1008 MB
20mb 4899 CompressedFileTextQuery
555kb 1790 FileTextQuery
```
We might want to test on something more interesting, like `bevy`.
internal: Implement parent-child relation for `SourceRoot`s
This commit adds the said relation by keeping a map of type `FxHashMap<SourceRootId,Option<SourceRootId>>` inside the `GlobalState`. Its primary use case is reading `rust-analyzer.toml`(#13529) files that can be placed in every local source root. As a config will be found by traversing this "tree" we need the parent information for every local source root. This commit omits defining this relation for library source roots entirely.
This commit adds the said relation by keeping a map of type `FxHashMap<SourceRootId,Option<SourceRootId>>`
inside the `GlobalState`. Its primary use case is reading the rust-analyzer.toml files that can be
placed under every local source root. As a config will be found by traversing this "tree" we need the parent information
for every local source root. This commit omits defining this relation for library source roots entirely.
internal: Compute syntax validation errors on demand
The LRU cache causes us to re-parse trees quite often, yet we don't use the validation errors at all. With this we push calculating them off to the caller who is interested in them.
Commit 6a06f6f72 (Deduplicate reference search results, 2022-11-07) deduplicates references
within each definition.
There is an edge case when requesting references of a macro argument. Apparently, our
descend_into_macros() stanza in references.rs produces a cartesian product of
- references inside the macro times
- times references outside the macro.
Since the above deduplication only applies to the references within a single definition, we
return them all, leading to many redundant references.
Work around this by deduplicating definitions as well. Perhaps there is a better fix to not
produce this cartesian product in the first place; but I think at least for definitions the
problem would remain; a macro can contain multiple definitions of the same name, but since the
navigation target will be the unresolved location, it's the same for all of them.
We can't use unique() because we don't want to drop references that don't have a declaration
(though I dont' have an example for this case).
I discovered this working with the "bitflags" macro from the crate of the same name.
Fixes#16357
fix: Fix snippets being placed leftwards of where they should be
Snippet bits were being escaped before placing snippets, shifting snippets leftwards. Snippets were also being shifted leftwards on files with CRLF line endings since they were placed done after the Unix -> DOS line ending conversion.
Hoping this fixes all of the little bugs related to snippet rendering 😅
The eventual LSP representation looks like it will diverge from RA's representation of `SnippetTextEdit`s, so this'll make it easier to transition to the LSP representation later.
Previously, the documentation field was the same as the text shown to
users when they hover over that symbol. The documentation should
really just be the doc comment, and as of #16179 the signature is
already stored in the signatureDocumentation field.
fix: Fix build scripts not being rebuilt in some occasions
Also makes proc-macro changed flag setting async, we don't wanna block `process_changes` on the database as that is on the main thread!
Substitute $saved_file in custom check commands
If the custom command has a $saved_file placeholder, and we know the file being saved, replace the placeholder and run a check command.
If there's a placeholder and we don't know the saved file, do nothing.
This is a simplified version of #15381, which I hope is easier to review.
feat: Introduce term search to rust-analyzer
# Introduce term search to `rust-analyzer`
_I've marked this as draft as there might be some shortcomings, please point them out so I can fix them. Otherwise I think it is kind of ready as I think I'll rather introduce extra functionality in follow up PRs._
Term search (or I guess expression search for rust) is a technique to generate code by basically making the types match.
Consider the following program
```rust
fn wrap(arg: i32) -> Option<i32> {
todo!();
}
```
From the types of values in scope and constructors of `Option`, we can produce the expected result of wrapping the argument in `Option`
Dependently typed languages such as `Idris2` and `Agda` have similar tools to help with proofs, but this can be also used in everyday development as a "auto-complete".
# Demo videos
https://github.com/rust-lang/rust-analyzer/assets/19900308/7b68a1b7-7dba-4e31-9221-6c7485e77d88https://github.com/rust-lang/rust-analyzer/assets/19900308/0fae530a-aabb-4b28-af71-e19f8d3d64b2
# What does it currently do
- It works well with locals, free functions, type constructors and non-static impl methods that take items by value.
- Works with functions/methods that take shared references, but not with unique references (very conservative).
- Can handle projections to struct fields (eg. `foo.bar.baz`) but this might me more conservative than it has to be to avoid conflicting with borrow checker
- Should create only valid programs (no type / borrow checking errors). Tested with `rust-analyzer analysis-stats /path/to/ripgrep/Cargo.toml --run-term-search --validate-term-search` (basically running `cargo check` on all of the generated programs and only error seems to be due to type inference which is more of issue of testing method.
# Performace / fitness
```txt
ripgrep (latest)
Tail Expr syntactic hits: 130/1692 (7%)
Tail Exprs found: 523/1692 (30%)
Term search avg time: 9ms
Term search: 15.64s, 97ginstr, 8mb
rust-analyzer (on this branch)
Tail Expr syntactic hits: 804/13860 (5%)
Tail Exprs found: 6757/13860 (48%)
Term search avg time: 78ms
Term search: 1088.23s, 6765ginstr, 98mb
```
Highly generic code seems to blow up the search space so currently the amount of generics allowed is functions/methods is limited down to 0 (1 didn't give much improvement and 2 is already like 0.5+s search time)
# Plans for the future (not in this PR)
- ``~~Add impl methods that do not take `self` type (should be quite straight forward)~~ Done
- Be smarter (aka less restrictive) about borrow checking - this seems quite hard but since the current approach is rather naive I think some easy improvement is available.
- ``~~See if it works as a autocomplete while typing~~ Done
_Feel free to ask questions / point of shortcoming either here or on Zulip, I'll be happy to address them. I'm doing this as part of my MSc thesis so I'll be working on it till summer anyway 😄_
If the custom command has a $saved_file placeholder, and we know the
file being saved, replace the placeholder and then run a check command.
If there's a placeholder and we don't know the saved file, do nothing.
feature: Create `UnindexedProject` notification to be sent to the client
(Note that this branch contains commits from https://github.com/rust-lang/rust-analyzer/pull/15830, which I'll rebase atop of as needed.)
Based on the discussion in https://github.com/rust-lang/rust-analyzer/issues/15837, I've added a notification and off-by-default toggle to send that notification from `handle_did_open_text_document`. I'm happy to rename/tweak this as needed.
I've been using this for a little bit, and it does seem to cause a little bit more indexing/work in rust-analyzer, but it's something that I'll profile as needed, I think.
internal: `tracing` improvements and followups
Hi folks! Building on https://github.com/rust-lang/rust-analyzer/pull/16394, I've got a few small tweaks:
- Removed the accidental `mod.rs` usage that I introduced.
- Removed a panic in `pat_analysis.rs`.
- Recorded the event kind in `handle_event` to better distinguish what _kind_ of event is being handled.
- Did a small refactor of `hprof` to have somewhat more linear control flow, and more importantly, write the recorded fields to the output.
The end result is the following:
<img width="1530" alt="A screenshot of Visual Studio Code on a Mac. `hprof.rs` is open, with " src="https://github.com/rust-lang/rust-analyzer/assets/2067774/bd11dde5-b2da-4774-bc38-bcb4772d1192">
This commit also adds `tracing` to NotificationDispatcher/RequestDispatcher,
bumps `rust-analyzer-salsa` to 0.17.0-pre.6, `always-assert` to 0.2, and
removes the homegrown `hprof` implementation in favor of a vendored
tracing-span-tree.
`cargo clippy --fix`
This PR is the result of running `cargo clippy --fix && cargo fmt` in the root of the repository. I did not manually review all the changes, but just skimmed through a few of them. The tests still pass, so it seems fine.
Add a new config to allow renaming of non-local defs
With #15656 we started disallowing renaming of non-local items. Although this makes sense there are some false positives that impacted users' workflows. So this config aims to mitigate this by giving users the liberty to disable this feature.
The reason why this is a draft is that I saw one of the tests fail and I am not sure if the "got" result even syntactically makes sense
Test case is :
```rust
check(
"Baz",
r#"
//- /lib.rs crate:lib new_source_root:library
pub struct S;
//- /main.rs crate:main deps:lib new_source_root:local
use lib::S$0;
"#,
"use lib::Baz;"
);
```
```
Left:
use lib::Baz;
Right:
use lib::Baz;Baz
Diff:
use lib::Baz;Baz
```
With #15656 we started disallowing renaming of non-local items.
Although this makes sense there are some false positives that
impacted users' workflows. So this config aims to mitigate this
by giving users the liberty to disable this feature.
Show which roots are being scanned in progress messages
This changes the `Roots Scanned` message to include the directory being scanned.
Before: `Roots Scanned 206/210 (98%)`
After: `Roots Scanned 206/210: .direnv (98%)`
This makes it a lot easier to tell that `rust-analyzer` isn't crashed, it's just trying to scan a huge directory.
See: #12613
