mirror of
https://github.com/rust-lang/rust-analyzer
synced 2025-01-26 11:55:04 +00:00
brush up docs
This commit is contained in:
Aleksey Kladov
parent
7e55aaeeed
commit
f77fdbc9d2
3 changed files with 146 additions and 133 deletions
127
ARCHITECTURE.md
Normal file
127
ARCHITECTURE.md
Normal file
|
|
@ -0,0 +1,127 @@
|
||||||
|
# Architecture
|
||||||
|
|
||||||
|
This document describes high-level architecture of rust-analyzer.
|
||||||
|
If you want to familiarize yourself with the code base, you are just
|
||||||
|
in the right place!
|
||||||
|
|
||||||
|
|
||||||
|
## Code generation
|
||||||
|
|
||||||
|
Some of the components of this repository are generated through automatic
|
||||||
|
processes. These are outlined below:
|
||||||
|
|
||||||
|
- `gen-kinds`: The kinds of tokens are reused in several places, so a generator
|
||||||
|
is used. This process uses [tera] to generate, using data in [grammar.ron],
|
||||||
|
the files:
|
||||||
|
- [ast/generated.rs][ast generated] in `ra_syntax` based on
|
||||||
|
[ast/generated.tera.rs][ast source]
|
||||||
|
- [syntax_kinds/generated.rs][syntax_kinds generated] in `ra_syntax` based on
|
||||||
|
[syntax_kinds/generated.tera.rs][syntax_kinds source]
|
||||||
|
|
||||||
|
[tera]: https://tera.netlify.com/
|
||||||
|
[grammar.ron]: ./crates/ra_syntax/src/grammar.ron
|
||||||
|
[ast generated]: ./crates/ra_syntax/src/ast/generated.rs
|
||||||
|
[ast source]: ./crates/ra_syntax/src/ast/generated.tera.rs
|
||||||
|
[syntax_kinds generated]: ./crates/ra_syntax/src/syntax_kinds/generated.rs
|
||||||
|
[syntax_kinds source]: ./crates/ra_syntax/src/syntax_kinds/generated.tera.rs
|
||||||
|
|
||||||
|
|
||||||
|
## Code Walk-Through
|
||||||
|
|
||||||
|
### `crates/ra_syntax`
|
||||||
|
|
||||||
|
Rust syntax tree structure and parser. See
|
||||||
|
[RFC](https://github.com/rust-lang/rfcs/pull/2256) for some design
|
||||||
|
notes.
|
||||||
|
|
||||||
|
- [rowan](https://github.com/rust-analyzer/rowan) library is used for constructing syntax trees.
|
||||||
|
- `grammar` module is the actual parser. It is a hand-written recursive descent parsers, which
|
||||||
|
produced a sequence of events like "start node X", "finish not Y". It works similarly to [kotlin parser](https://github.com/JetBrains/kotlin/blob/4d951de616b20feca92f3e9cc9679b2de9e65195/compiler/frontend/src/org/jetbrains/kotlin/parsing/KotlinParsing.java),
|
||||||
|
which is a good source for inspiration for dealing with syntax errors and incomplete input. Original [libsyntax parser](https://github.com/rust-lang/rust/blob/6b99adeb11313197f409b4f7c4083c2ceca8a4fe/src/libsyntax/parse/parser.rs)
|
||||||
|
is what we use for the definition of the Rust language.
|
||||||
|
- `parser_api/parser_impl` bridges the tree-agnostic parser from `grammar` with `rowan` trees.
|
||||||
|
This is the thing that turns a flat list of events into a tree (see `EventProcessor`)
|
||||||
|
- `ast` a type safe API on top of the raw `rowan` tree.
|
||||||
|
- `grammar.ron` RON description of the grammar, which is used to
|
||||||
|
generate `syntax_kinds` and `ast` modules, using `cargo gen-kinds` command.
|
||||||
|
- `algo`: generic tree algorithms, including `walk` for O(1) stack
|
||||||
|
space tree traversal (this is cool) and `visit` for type-driven
|
||||||
|
visiting the nodes (this is double plus cool, if you understand how
|
||||||
|
`Visitor` works, you understand rust-analyzer).
|
||||||
|
|
||||||
|
Test for ra_syntax are mostly data-driven: `tests/data/parser` contains a bunch of `.rs`
|
||||||
|
(test vectors) and `.txt` files with corresponding syntax trees. During testing, we check
|
||||||
|
`.rs` against `.txt`. If the `.txt` file is missing, it is created (this is how you update
|
||||||
|
tests). Additionally, running `cargo gen-tests` will walk the grammar module and collect
|
||||||
|
all `//test test_name` comments into files inside `tests/data` directory.
|
||||||
|
|
||||||
|
See [#93](https://github.com/rust-analyzer/rust-analyzer/pull/93) for an example PR which
|
||||||
|
fixes a bug in the grammar.
|
||||||
|
|
||||||
|
|
||||||
|
### `crates/ra_editor`
|
||||||
|
|
||||||
|
All IDE features which can be implemented if you only have access to a
|
||||||
|
single file. `ra_editor` could be used to enhance editing of Rust code
|
||||||
|
without the need to fiddle with build-systems, file
|
||||||
|
synchronization and such.
|
||||||
|
|
||||||
|
In a sense, `ra_editor` is just a bunch of pure functions which take a
|
||||||
|
syntax tree as an input.
|
||||||
|
|
||||||
|
The tests for `ra_editor` are `[cfg(test)] mod tests` unit-tests spread
|
||||||
|
throughout its modules.
|
||||||
|
|
||||||
|
### `crates/salsa`
|
||||||
|
|
||||||
|
An implementation of red-green incremental compilation algorithm from
|
||||||
|
rust compiler. It makes all rust-analyzer features on-demand. To be replaced
|
||||||
|
with `salsa-rs/salsa` soon.
|
||||||
|
|
||||||
|
|
||||||
|
### `crates/ra_analysis`
|
||||||
|
|
||||||
|
A stateful library for analyzing many Rust files as they change.
|
||||||
|
`AnalysisHost` is a mutable entity (clojure's atom) which holds
|
||||||
|
current state, incorporates changes and handles out `Analysis` --- an
|
||||||
|
immutable consistent snapshot of world state at a point in time, which
|
||||||
|
actually powers analysis.
|
||||||
|
|
||||||
|
|
||||||
|
### `crates/ra_lsp_server`
|
||||||
|
|
||||||
|
An LSP implementation which uses `ra_analysis` for managing state and
|
||||||
|
`ra_editor` for actually doing useful stuff.
|
||||||
|
|
||||||
|
See [#79](https://github.com/rust-analyzer/rust-analyzer/pull/79/) as an
|
||||||
|
example of PR which adds a new feature to `ra_editor` and exposes it
|
||||||
|
to `ra_lsp_server`.
|
||||||
|
|
||||||
|
|
||||||
|
### `crates/cli`
|
||||||
|
|
||||||
|
A CLI interface to rust-analyzer.
|
||||||
|
|
||||||
|
### `crate/tools`
|
||||||
|
|
||||||
|
Code-gen tasks, used to develop rust-analyzer:
|
||||||
|
|
||||||
|
- `cargo gen-kinds` -- generate `ast` and `syntax_kinds`
|
||||||
|
- `cargo gen-tests` -- collect inline tests from grammar
|
||||||
|
- `cargo install-code` -- build and install VS Code extension and server
|
||||||
|
|
||||||
|
### `editors/code`
|
||||||
|
|
||||||
|
VS Code plugin
|
||||||
|
|
||||||
|
|
||||||
|
## Common workflows
|
||||||
|
|
||||||
|
To try out VS Code extensions, run `cargo install-code`. To see logs from the language server,
|
||||||
|
set `RUST_LOG=info` env variable. To see all communication between the server and the client, use
|
||||||
|
`RUST_LOG=gen_lsp_server=debug` (will print quite a bit of stuff).
|
||||||
|
|
||||||
|
To run tests, just `cargo test`.
|
||||||
|
|
||||||
|
To work on VS Code extension, launch code inside `editors/code` and use `F5` to launch/debug.
|
||||||
|
|
||||||
|
|
@ -1,58 +1,18 @@
|
||||||
The project is in its early stages: contributions are welcome and would be
|
The project is in its early stages: contributions are welcome and would be
|
||||||
**very** helpful, but the project is not _yet_ optimized for contribution.
|
**very** helpful, but the project is not _yet_ optimized for contribution.
|
||||||
Moreover, it is doubly experimental, so there's no guarantee that any work here
|
Moreover, it is doubly experimental, so there's no guarantee that any work here
|
||||||
would reach production. That said, here are some areas where contributions would
|
would reach production.
|
||||||
be **especially** welcome:
|
|
||||||
|
|
||||||
- Designing internal data structures: RFC only outlines the constraints, it's an
|
To get an idea of how rust-analyzer works, take a look at the [ARCHITECTURE.md](./ARCHITECTURE.md)
|
||||||
open question how to satisfy them in the optimal way. See `ARCHITECTURE.md`
|
document.
|
||||||
for current design questions.
|
|
||||||
|
|
||||||
- Porting libsyntax parser to rust-analyzer: currently rust-analyzer parses only
|
Useful labels on the issue tracker:
|
||||||
a tiny subset of Rust. This should be fixed by porting parsing functions from
|
* [E-mentor](https://github.com/rust-analyzer/rust-analyzer/issues?q=is%3Aopen+is%3Aissue+label%3AE-mentor)
|
||||||
libsyntax one by one. Take a look at the [libsyntax parser] for "what to port"
|
issues have links to the code in question and tests,
|
||||||
and at the [Kotlin parser] for "how to port".
|
* [E-easy](https://github.com/rust-analyzer/rust-analyzer/issues?q=is%3Aopen+is%3Aissue+label%3AE-easy),
|
||||||
|
[E-medium](https://github.com/rust-analyzer/rust-analyzer/issues?q=is%3Aopen+is%3Aissue+label%3AE-medium),
|
||||||
|
[E-hard](https://github.com/rust-analyzer/rust-analyzer/issues?q=is%3Aopen+is%3Aissue+label%3AE-hard),
|
||||||
|
labels are *estimates* for how hard would be to write a fix.
|
||||||
|
|
||||||
- Writing validators: by design, rust-analyzer is very lax about the input. For
|
There's no formal PR check list: everything that passes CI (we use [bors](https://bors.tech/)) is valid,
|
||||||
example, the lexer happily accepts unclosed strings. The idea is that there
|
but it's a good idea to write nice commit messages, test code thoroughly, maintain consistent style, etc.
|
||||||
should be a higher level visitor, which walks the syntax tree after parsing
|
|
||||||
and produces all the warnings. Alas, there's no such visitor yet :( Would you
|
|
||||||
like to write one? :)
|
|
||||||
|
|
||||||
- Creating tests: it would be tremendously helpful to read each of libsyntax and
|
|
||||||
rust-analyzer parser functions and crate a small separate test cases to cover
|
|
||||||
each and every edge case.
|
|
||||||
|
|
||||||
- Building stuff with rust-analyzer: it would be really cool to compile
|
|
||||||
rust-analyzer to WASM and add _client side_ syntax validation to rust
|
|
||||||
playground!
|
|
||||||
|
|
||||||
Do take a look at the issue tracker.
|
|
||||||
|
|
||||||
If you don't know where to start, or have _any_ questions or suggestions, don't
|
|
||||||
hesitate to chat at [Gitter]!
|
|
||||||
|
|
||||||
# Code generation
|
|
||||||
|
|
||||||
Some of the components of this repository are generated through automatic
|
|
||||||
processes. These are outlined below:
|
|
||||||
|
|
||||||
- `gen-kinds`: The kinds of tokens are reused in several places, so a generator
|
|
||||||
is used. This process uses [tera] to generate, using data in [grammar.ron],
|
|
||||||
the files:
|
|
||||||
- [ast/generated.rs][ast generated] in `ra_syntax` based on
|
|
||||||
[ast/generated.tera.rs][ast source]
|
|
||||||
- [syntax_kinds/generated.rs][syntax_kinds generated] in `ra_syntax` based on
|
|
||||||
[syntax_kinds/generated.tera.rs][syntax_kinds source]
|
|
||||||
|
|
||||||
[libsyntax parser]:
|
|
||||||
https://github.com/rust-lang/rust/blob/6b99adeb11313197f409b4f7c4083c2ceca8a4fe/src/libsyntax/parse/parser.rs
|
|
||||||
[kotlin parser]:
|
|
||||||
https://github.com/JetBrains/kotlin/blob/4d951de616b20feca92f3e9cc9679b2de9e65195/compiler/frontend/src/org/jetbrains/kotlin/parsing/KotlinParsing.java
|
|
||||||
[gitter]: https://gitter.im/libsyntax2/Lobby
|
|
||||||
[tera]: https://tera.netlify.com/
|
|
||||||
[grammar.ron]: ./crates/ra_syntax/src/grammar.ron
|
|
||||||
[ast generated]: ./crates/ra_syntax/src/ast/generated.rs
|
|
||||||
[ast source]: ./crates/ra_syntax/src/ast/generated.tera.rs
|
|
||||||
[syntax_kinds generated]: ./crates/ra_syntax/src/syntax_kinds/generated.rs
|
|
||||||
[syntax_kinds source]: ./crates/ra_syntax/src/syntax_kinds/generated.tera.rs
|
|
||||||
|
|
|
||||||
84
README.md
84
README.md
|
|
@ -75,85 +75,6 @@ parsing, macro expansion and related bits of name resolution, but
|
||||||
leave the rest (including type inference and trait selection) to the
|
leave the rest (including type inference and trait selection) to the
|
||||||
existing rustc.
|
existing rustc.
|
||||||
|
|
||||||
## Code Walk-Through
|
|
||||||
|
|
||||||
### `crates/ra_syntax`
|
|
||||||
|
|
||||||
Rust syntax tree structure and parser. See
|
|
||||||
[RFC](https://github.com/rust-lang/rfcs/pull/2256) for some design
|
|
||||||
notes.
|
|
||||||
|
|
||||||
- `yellow`, red/green syntax tree, heavily inspired [by this](https://github.com/apple/swift/tree/ab68f0d4cbf99cdfa672f8ffe18e433fddc8b371/lib/Syntax)
|
|
||||||
- `grammar`, the actual parser
|
|
||||||
- `parser_api/parser_impl` bridges the tree-agnostic parser from `grammar` with `yellow` trees
|
|
||||||
- `grammar.ron` RON description of the grammar, which is used to
|
|
||||||
generate `syntax_kinds` and `ast` modules.
|
|
||||||
- `algo`: generic tree algorithms, including `walk` for O(1) stack
|
|
||||||
space tree traversal (this is cool) and `visit` for type-driven
|
|
||||||
visiting the nodes (this is double plus cool, if you understand how
|
|
||||||
`Visitor` works, you understand rust-analyzer).
|
|
||||||
|
|
||||||
|
|
||||||
### `crates/ra_editor`
|
|
||||||
|
|
||||||
All IDE features which can be implemented if you only have access to a
|
|
||||||
single file. `ra_editor` could be used to enhance editing of Rust code
|
|
||||||
without the need to fiddle with build-systems, file
|
|
||||||
synchronization and such.
|
|
||||||
|
|
||||||
In a sense, `ra_editor` is just a bunch of pure functions which take a
|
|
||||||
syntax tree as an input.
|
|
||||||
|
|
||||||
### `crates/salsa`
|
|
||||||
|
|
||||||
An implementation of red-green incremental compilation algorithm from
|
|
||||||
rust compiler. It makes all rust-analyzer features on-demand.
|
|
||||||
|
|
||||||
|
|
||||||
### `crates/ra_analysis`
|
|
||||||
|
|
||||||
A stateful library for analyzing many Rust files as they change.
|
|
||||||
`AnalysisHost` is a mutable entity (clojure's atom) which holds
|
|
||||||
current state, incorporates changes and handles out `Analysis` --- an
|
|
||||||
immutable consistent snapshot of world state at a point in time, which
|
|
||||||
actually powers analysis.
|
|
||||||
|
|
||||||
|
|
||||||
### `crates/ra_lsp_server`
|
|
||||||
|
|
||||||
An LSP implementation which uses `ra_analysis` for managing state and
|
|
||||||
`ra_editor` for actually doing useful stuff.
|
|
||||||
|
|
||||||
|
|
||||||
### `crates/cli`
|
|
||||||
|
|
||||||
A CLI interface to libsyntax
|
|
||||||
|
|
||||||
### `crate/tools`
|
|
||||||
|
|
||||||
Code-gen tasks, used to develop rust-analyzer:
|
|
||||||
|
|
||||||
- `cargo gen-kinds` -- generate `ast` and `syntax_kinds`
|
|
||||||
- `cargo gen-tests` -- collect inline tests from grammar
|
|
||||||
- `cargo install-code` -- build and install VS Code extension and server
|
|
||||||
|
|
||||||
### `editors/code`
|
|
||||||
|
|
||||||
VS Code plugin
|
|
||||||
|
|
||||||
|
|
||||||
## Performance
|
|
||||||
|
|
||||||
Non-incremental, but seems pretty fast:
|
|
||||||
|
|
||||||
```
|
|
||||||
$ cargo build --release --package ra_cli
|
|
||||||
$ wc -l ~/projects/rust/src/libsyntax/parse/parser.rs
|
|
||||||
7546 /home/matklad/projects/rust/src/libsyntax/parse/parser.rs
|
|
||||||
$ ./target/release/ra_cli parse < ~/projects/rust/src/libsyntax/parse/parser.rs --no-dump > /dev/null
|
|
||||||
parsing: 21.067065ms
|
|
||||||
```
|
|
||||||
|
|
||||||
## Getting in touch
|
## Getting in touch
|
||||||
|
|
||||||
@matklad can be found at Rust
|
@matklad can be found at Rust
|
||||||
|
|
@ -161,6 +82,11 @@ parsing: 21.067065ms
|
||||||
#ides-and-editors.
|
#ides-and-editors.
|
||||||
|
|
||||||
|
|
||||||
|
## Contributing
|
||||||
|
|
||||||
|
See [CONTRIBUTING.md](./CONTRIBUTING.md) and [ARCHITECTURE.md](./ARCHITECTURE.md)
|
||||||
|
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
Rust analyzer is primarily distributed under the terms of both the MIT
|
Rust analyzer is primarily distributed under the terms of both the MIT
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue