Specify thread types using Quality of Service API
<details>
<summary>Some background (in case you haven’t heard of QoS before)</summary>
Heterogenous multi-core CPUs are increasingly found in laptops and desktops (e.g. Alder Lake, Snapdragon 8cx Gen 3, M1). To maximize efficiency on this kind of hardware, it is important to provide the operating system with more information so threads can be scheduled on different core types appropriately.
The approach that XNU (the kernel of macOS, iOS, etc) and Windows have taken is to provide a high-level semantic API – quality of service, or QoS – which informs the OS of the program’s intent. For instance, you might specify that a thread is running a render loop for a game. This makes the OS provide this thread with as large a share of the system’s resources as possible. Specifying a thread is running an unimportant background task, on the other hand, is cause for it to be scheduled exclusively on high-efficiency cores instead of high-performance cores.
QoS APIs allows for easy configuration of many different parameters at once; for instance, setting QoS on XNU affects scheduling, timer latency, I/O priorities, and of course what core type the thread in question should run on. I don’t know any details on how QoS works on Windows, but I would guess it’s similar.
Hypothetically, taking advantage of these APIs would improve power consumption, thermals, battery life if applicable, etc.
</details>
# Relevance to rust-analyzer
From what I can tell the philosophy behind both the XNU and Windows QoS APIs is that _user interfaces should never stutter under any circumstances._ You can see this in the array of QoS classes which are available: the highest QoS class in both APIs is one intended explicitly for UI render loops.
Imagine rust-analyzer is performing CPU-intensive background work – maybe you just invoked Find Usages on `usize` or opened a large project – in this scenario the editor’s render loop should absolutely get higher priority than rust-analyzer, no matter what. You could view it in terms of “realtime-ness”: flight control software is hard realtime, audio software is soft realtime, GUIs are softer realtime, and rust-analyzer is not realtime at all. Of course, maximizing responsiveness is important, but respecting the rest of the system is more important.
# Implementation
I’ve tried my best to unify thread creation in `stdx`, where the new API I’ve introduced _requires_ specifying a QoS class. Different points along the performance/efficiency curve can make a great difference; the M1’s e-cores use around three times less power than the p-cores, so putting in this effort is worthwhile IMO.
It’s worth mentioning that Linux does not [yet](https://youtu.be/RfgPWpTwTQo) have a QoS API. Maybe translating QoS into regular thread priorities would be acceptable? From what I can tell the only scheduling-related code in rust-analyzer is Windows-specific, so ignoring QoS entirely on Linux shouldn’t cause any new issues. Also, I haven’t implemented support for the Windows QoS APIs because I don’t have a Windows machine to test on, and because I’m completely unfamiliar with Windows APIs :)
I noticed that rust-analyzer handles some requests on the main thread (using `.on_sync()`) and others on a threadpool (using `.on()`). I think it would make sense to run the main thread at the User Initiated QoS and the threadpool at Utility, but only if all requests that are caused by typing use `.on_sync()` and all that don’t use `.on()`. I don’t understand how the `.on_sync()`/`.on()` split that’s currently present was chosen, so I’ve let this code be for the moment. Let me know if changing this to what I proposed makes any sense.
To avoid having to change everything back in case I’ve misunderstood something, I’ve left all threads at the Utility QoS for now. Of course, this isn’t what I hope the code will look like in the end, but I figured I have to start somewhere :P
# References
<ul>
<li><a href="https://developer.apple.com/library/archive/documentation/Performance/Conceptual/power_efficiency_guidelines_osx/PrioritizeWorkAtTheTaskLevel.html">Apple documentation related to QoS</a></li>
<li><a href="67e155c940/include/pthread/qos.h">pthread API for setting QoS on XNU</a></li>
<li><a href="https://learn.microsoft.com/en-us/windows/win32/procthread/quality-of-service">Windows’s QoS classes</a></li>
<li>
<details>
<summary>Full documentation of XNU QoS classes. This documentation is only available as a huge not-very-readable comment in a header file, so I’ve reformatted it and put it here for reference.</summary>
<ul>
<li><p><strong><code>QOS_CLASS_USER_INTERACTIVE</code>: A QOS class which indicates work performed by this thread is interactive with the user.</strong></p><p>Such work is requested to run at high priority relative to other work on the system. Specifying this QOS class is a request to run with nearly all available system CPU and I/O bandwidth even under contention. This is not an energy-efficient QOS class to use for large tasks. The use of this QOS class should be limited to critical interaction with the user such as handling events on the main event loop, view drawing, animation, etc.</p></li>
<li><p><strong><code>QOS_CLASS_USER_INITIATED</code>: A QOS class which indicates work performed by this thread was initiated by the user and that the user is likely waiting for the results.</strong></p><p>Such work is requested to run at a priority below critical user-interactive work, but relatively higher than other work on the system. This is not an energy-efficient QOS class to use for large tasks. Its use should be limited to operations of short enough duration that the user is unlikely to switch tasks while waiting for the results. Typical user-initiated work will have progress indicated by the display of placeholder content or modal user interface.</p></li>
<li><p><strong><code>QOS_CLASS_DEFAULT</code>: A default QOS class used by the system in cases where more specific QOS class information is not available.</strong></p><p>Such work is requested to run at a priority below critical user-interactive and user-initiated work, but relatively higher than utility and background tasks. Threads created by <code>pthread_create()</code> without an attribute specifying a QOS class will default to <code>QOS_CLASS_DEFAULT</code>. This QOS class value is not intended to be used as a work classification, it should only be set when propagating or restoring QOS class values provided by the system.</p></li>
<li><p><strong><code>QOS_CLASS_UTILITY</code>: A QOS class which indicates work performed by this thread may or may not be initiated by the user and that the user is unlikely to be immediately waiting for the results.</strong></p><p>Such work is requested to run at a priority below critical user-interactive and user-initiated work, but relatively higher than low-level system maintenance tasks. The use of this QOS class indicates the work should be run in an energy and thermally-efficient manner. The progress of utility work may or may not be indicated to the user, but the effect of such work is user-visible.</p></li>
<li><p><strong><code>QOS_CLASS_BACKGROUND</code>: A QOS class which indicates work performed by this thread was not initiated by the user and that the user may be unaware of the results.</strong></p><p>Such work is requested to run at a priority below other work. The use of this QOS class indicates the work should be run in the most energy and thermally-efficient manner.</p></li>
<li><p><strong><code>QOS_CLASS_UNSPECIFIED</code>: A QOS class value which indicates the absence or removal of QOS class information.</strong></p><p>As an API return value, may indicate that threads or pthread attributes were configured with legacy API incompatible or in conflict with the QOS class system.</p></li>
</ul>
</details>
</li>
</ul>
feat: Assist to replace generic with impl trait
This adds a new assist named "Replace named generic with impl". It is the inverse operation to the existing "Replace impl trait with generic" assist.
It allows to refactor the following statement:
```rust
// 👇 cursor
fn new<T$0: ToString>(input: T) -> Self {}
```
to be transformed into:
```rust
fn new(input: impl ToString) -> Self {}
```
* adds new helper function `impl_trait_type` to create AST node
* add method to remove an existing generic param type from param list
Closes#14626
This removes an existing generic param from the `GenericParamList`. It
also considers to remove the extra colon & whitespace to the previous
sibling.
* change order to get all param types first and mark them as mutable
before the first edit happens
* add helper function to remove a generic parameter
* fix test output
This adds a new assist named "replace named generic with impl" to move
the generic param type from the generic param list into the function
signature.
```rust
fn new<T: ToString>(input: T) -> Self {}
```
becomes
```rust
fn new(input: impl ToString) -> Self {}
```
The first step is to determine if the assist can be applied, there has
to be a match between generic trait param & function paramter types.
* replace function parameter type(s) with impl
* add new `impl_trait_type` function to generate the new trait bounds with `impl` keyword for use in the
function signature
fix: assists no longer break indentation
Fixes https://github.com/rust-lang/rust-analyzer/issues/14674
These are _ad hoc_ patches for a number of assists that can produce incorrectly indented code, namely:
- generate_derive
- add_missing_impl_members
- add_missing_default_members
Some general solution is required in future, as the same problem arises in many other assists, e.g.
- replace_derive_with...
- generate_default_from_enum...
- generate_default_from_new
- generate_delegate_methods
(the list is incomplete)
Fix: a TODO and some clippy fixes
- fix(todo): implement IntoIterator for ArenaMap<IDX, V>
- chore: remove unused method
- fix: remove useless `return`s
- fix: various clippy lints
- fix: simplify boolean test to a single negation
fix: introduce new type var when expectation for ref pat is not ref
Fixes#14840
When we infer the type of ref patterns, its expected type may not be reference type: 1) expected type is an unresolved inference variable, or 2) expected type is erroneously other kind of type. In either case, we should produce a reference type with a new type variable rather than an error type so that we can continue inferring the inner patterns without further errors because of the (possible) type mismatch of this pattern.
fix: consider all tokens in macro expr when analyzing locals
Fixes#14687
2 fixes for `extract_function` assist (related closely enough that I squashed into one commit):
- Locals in macro expressions have been analyzed only when they are in the top-level token tree the macro call wraps. We should consider all descendant tokens.
- `self` in macro expressions haven't been analyzed.
Fix `preorder_expr` skipping the `else` block of let-else statements
Fixes exit/yield points not getting highlighted in such blocks for `highlight_related` (#14813; and possibly other bugs in features that use `preorder_expr`).
Fixes exit/yield points not getting highlighted in such blocks for `highlight_related` (#14813; and possibly other bugs in features that use `preorder_expr`).
MIR episode 5
This PR inits drop support (it is very broken at this stage, some things are dropped multiple time, drop scopes are wrong, ...) and adds stdout support (`println!` doesn't work since its expansion is dummy, but `stdout().write(b"hello world\n")` works if you use `RA_SYSROOT_HACK`) for interpreting. There is no useful unit test that it can interpret yet, but it is a good sign that it didn't hit a major road block yet.
In MIR lowering, it adds support for slice pattern and anonymous const blocks, and some fixes so that we can evaluate `SmolStr::new_inline` in const eval. With these changes, 57 failed mir body remains.
fix: Diagnose non-value return and break type mismatches
Could definitely deserve more polished diagnostics, but this at least brings the message across for now.
fix(analysis-stats): divided by zero error
## What does this PR try to resolve?
2023-05-15 rust-analyzer suffers from
```
thread 'main' panicked at 'attempt to divide by zero', crates/rust-analyzer/src/cli/analysis_stats.rs:230:56
```
This commit <51e8b8ff14> might be the culprit.
This PR uses `percentage` function to avoid the classic “division by zero” bug.
## Reproducer
```console
cargo new ra-test
pushd ra-test
echo "pub type Foo = u32;" >> src/lib.rs
rust-analyzer analysis-stats .
```
Support `#[macro_use(name, ...)]`
This PR adds support for another form of the `macro_use` attribute: `#[macro_use(name, ...)]` ([reference]).
Note that this form of the attribute is only applicable to extern crate decls, not to mod decls.
[reference]: https://doc.rust-lang.org/reference/macros-by-example.html#the-macro_use-attribute
Parse associated return type bounds
This PR implements parser support for associated return type bounds: `T: Foo<bar(): Send>`. This PR does not implement associated return types (`T::bar(): Send`) because it's not implemented even in rustc, and also removes `(..)`-style return type notation because it has been removed in rust-lang/rust#110203 (effectively reverting #14465).
I don't plan to proactively follow this unstable feature unless an RFC is accepted and my main motivation here is to remove no-longer-valid syntax `(..)` from our parser, nevertheless adding minimal parser support so anyone interested (as can be seen in #14465) can experiment it without rust-analyzer's syntax errors.
Expand more single ident macro calls upon item collection
Addresses https://github.com/rust-lang/rust-analyzer/pull/14781#issuecomment-1546201022
I believe this (almost) brings the number of unresolved names back to pre-#14781:
|r-a version|`analysis-stats compiler/rustc` (rust-lang/rust@69fef92ab2) |
|---|---|
|pre-#14781 (b069eb720b) | exprs: 2747778, ??ty: 122236 (4%), ?ty: 107826 (3%), !ty: 728 |
| #14781 (a7944a93a1) | exprs: 2713080, ??ty: 139651 (5%), ?ty: 114444 (4%), !ty: 730 |
| with this fix | exprs: 2747871, ??ty: 122237 (4%), ?ty: 108171 (3%), !ty: 676 |
(I haven't investigated on the increase in some numbers but hopefully not too much of a problem)
This is only a temporary solution. The core problem is that we haven't fully implemented the textual scope of legacy macros. For example, we *have been* failing to resolve `foo` in the following snippet, even before #14781 or after this patch. As noted in a FIXME, we need a way to resolve names in textual scope without eager expansion during item collection.
```rust
//- /main.rs crate:main deps:lib
lib::mk_foo!();
const A: i32 = foo!();
//^^^^^^ unresolved-macro-call
//- /lib.rs crate:lib
#[macro_export]
macro_rules! mk_foo {
() => {
macro_rules! foo { () => { 42 } }
}
}
```
Introduce macro sub-namespaces and `macro_use` prelude
This PR implements two mechanisms needed for correct macro name resolution: macro sub-namespace and `macro_use` prelude.
- [macro sub-namespaces][subns-ref]
Macros have two sub-namespaces: one for function-like macro and the other for those in attributes (including custom derive macros). When we're resolving a macro name for function-like macro, we should ignore non-function-like macros, and vice versa.
This helps resolve single-segment macro names because we can (and should, as rustc does) fallback to names in preludes when the name in the current module scope is in different sub-namespace.
- [`macro_use` prelude][prelude-ref]
`#[macro_use]`'d extern crate declarations (including the standard library) bring their macros into scope, but they should not be prioritized over local macros (those defined in place and those explicitly imported).
We have been bringing them into legacy (textual) macro scope, which has the highest precedence in name resolution. This PR introduces the `macro_use` prelude in crate-level `DefMap`s, whose precedence is lower than local macros but higher than the standard library prelude.
The first 3 commits are drive-by fixes/refactors.
Fixes#8828 (prelude)
Fixes#12505 (prelude)
Fixes#12734 (prelude)
Fixes#13683 (prelude)
Fixes#13821 (prelude)
Fixes#13974 (prelude)
Fixes#14254 (namespace)
[subns-ref]: https://doc.rust-lang.org/reference/names/namespaces.html#sub-namespaces
[prelude-ref]: https://doc.rust-lang.org/reference/names/preludes.html#macro_use-prelude
We've already removed non-sysroot proc macro server, which effectively
removed support for Rust <1.64.0, so this removal of fallback path
shouldn't be problem at this point.
This function/lang_item was introduced in #104321 as a temporary workaround of future lowering.
The usage and need for it went away in #104833.
After a bootstrap update, the function itself can be removed from `std`.
Make line-index a lib, use nohash_hasher
These seem like they are not specific to rust-analyzer and could be pulled out to their own libraries. So I did.
https://github.com/azdavis/millet/issues/31
Provide links to locally built documentation for `experimental/externalDocs`
This pull request addresses issue #12867, which requested the ability to provide links to locally built documentation when using the "Open docs for symbol" feature. Previously, rust-analyzer always used docs.rs for this purpose. With these changes, the feature will provide both web (docs.rs) and local documentation links without verifying their existence.
Changes in this PR:
- Added support for local documentation links alongside web documentation links.
- Added `target_dir` path argument for external_docs and other related methods.
- Added `sysroot` argument for external_docs.
- Added `target_directory` path to `CargoWorkspace`.
API Changes:
- Added an experimental client capability `{ "localDocs": boolean }`. If this capability is set, the `Open External Documentation` request returned from the server will include both web and local documentation links in the `ExternalDocsResponse` object.
Here's the `ExternalDocsResponse` interface:
```typescript
interface ExternalDocsResponse {
web?: string;
local?: string;
}
```
By providing links to both web-based and locally built documentation, this update improves the developer experience for those using different versions of crates, git dependencies, or local crates not available on docs.rs. Rust-analyzer will now provide both web (docs.rs) and local documentation links, leaving it to the client to open the desired link. Please note that this update does not perform any checks to ensure the validity of the provided links.
