Commit graph

305 commits

Author SHA1 Message Date
darksv
bc94bf95ce correctly handle IDENTs when changed to contextual keywords 2018-09-14 19:26:48 +02:00
darksv
c300135322 create leaf directly without calling the parser 2018-09-14 19:23:10 +02:00
darksv
4356240fa4 Incremental reparsing for single tokens (WHITESPACE, COMMENT, DOC_COMMENT, IDENT, STRING, RAW_STRING) 2018-09-13 23:25:05 +02:00
Aleksey Kladov
b6f8037a6f don't get stuck in slice patterns 2018-09-12 11:26:52 +03:00
Aleksey Kladov
ccc75675b6 correctly setup path-map for fs-changes 2018-09-12 11:19:19 +03:00
bors[bot]
e240360ee2 Merge #68
68: Implement incremental reparsing for remaining braced blocks r=matklad a=darksv

Fixes #66

Co-authored-by: darksv <darek969-12@o2.pl>
2018-09-11 07:32:36 +00:00
darksv
d0cfeb4f16 Do not reparse token tree when it is not delimited by braces 2018-09-10 23:21:16 +02:00
darksv
64d07c1bd4 Implement reparsing for remaining blocks 2018-09-10 20:14:09 +02:00
Aleksey Kladov
505895a25f store file rsovler 2018-09-10 12:57:40 +03:00
bors[bot]
4f64709666 Merge #65
65: simplify r=matklad a=matklad



Co-authored-by: Aleksey Kladov <aleksey.kladov@gmail.com>
2018-09-08 16:24:24 +00:00
Aleksey Kladov
f19a82beac simplify 2018-09-08 19:16:11 +03:00
Aleksey Kladov
a5c333c3ed Fix yet another parser infinite loop
This commit is an example of fixing a common parser error: infinite
loop due to error recovery.

This error typically happens when we parse a list of items and fail to
parse a specific item at the current position.

One choices is to skip a token and try to parse a list item at the
next position. This is a good, but not universal, default. When
parsing a list of arguments in a function call, you, for example,
don't want to skip over `fn`, because it's most likely that it is a
function declaration, and not a mistyped arg:

```
fn foo() {
    quux(1, 2

fn bar() {
}
```

Another choice is to bail out of the loop immediately, but it isn't
perfect either: sometimes skipping over garbage helps:

```
quux(1, foo:, 92) // should skip over `:`, b/c that's part of `foo::bar`
```

In general, parser tries to balance these two cases, though we don't
have a definitive strategy yet.

However, if the parser accidentally neither skips over a token, nor
breaks out of the loop, then it becomes stuck in the loop infinitely
(there's an internal counter to self-check this situation and panic
though), and that's exactly what is demonstrated by the test.

To fix such situation, first of all, add the test case to tests/data/parser/{err,fuzz-failures}.

Then, run

```
RUST_BACKTRACE=short cargo test --package libsyntax2
````

to verify that parser indeed panics, and to get an idea what grammar
production is the culprit (look for `_list` functions!).

In this case, I see

```
  10: libsyntax2::grammar::expressions::atom::match_arm_list
             at crates/libsyntax2/src/grammar/expressions/atom.rs:309
```

and that's look like it might be a culprit. I verify it by adding
`eprintln!("loopy {:?}", p.current());` and indeed I see that this is
printed repeatedly.

Diagnosing this a bit shows that the problem is that
`pattern::pattern` function does not consume anything if the next
token is `let`. That is a good default to make cases like

```
let
let foo = 92;
```

where the user hasn't typed the pattern yet, to parse in a reasonable
they correctly.

For match arms, pretty much the single thing we expect is a pattern,
so, for a fix, I introduce a special variant of pattern that does not
do recovery.
2018-09-08 19:10:40 +03:00
Aleksey Kladov
3ab9f4ad7f Add fuzz failures dir 2018-09-08 18:42:59 +03:00
Aleksey Kladov
ba4a697d8c move fuzz-invariants to the library 2018-09-08 18:34:41 +03:00
Pascal Hertleif
a37cd5ad43 Add trivial fuzzer for parser
As described in #61, fuzz testing some parts of this would be ~~fun~~
helpful. So, I started with the most trivial fuzzer I could think of:
Put random stuff into File::parse and see what happens.

To speed things up, I also did

    cp src/**/*.rs fuzz/corpus/parser/

in the `crates/libsyntax2/` directory (running the fuzzer once will
generate the necessary directories).
2018-09-08 16:55:53 +02:00
Aleksey Kladov
df05c5c3e2 Don't overflow when limiting symbol search 2018-09-08 15:39:28 +03:00
Aleksey Kladov
7daaddb2ac Some abstraction around workers 2018-09-08 13:15:01 +03:00
Aleksey Kladov
326ffcefe0 Deal with deadlocks in a more principaled way 2018-09-08 12:36:02 +03:00
Aleksey Kladov
d9ccebd913 fix deadlock 2018-09-08 12:08:46 +03:00
Aleksey Kladov
f48b9d9be7 Fix block structure in enums 2018-09-08 10:55:09 +03:00
Aleksey Kladov
749907d330 simplify 2018-09-08 10:38:53 +03:00
Aleksey Kladov
febbc9acdd Don't get stuck in tuple exprs 2018-09-08 10:35:05 +03:00
Aleksey Kladov
a0a347eac9 Don't get stuck in macros 2018-09-08 10:28:53 +03:00
Aleksey Kladov
bd3a26493f fix stuck parser 2018-09-08 10:13:32 +03:00
Aleksey Kladov
44334f6f56 fix labled expressions 2018-09-08 09:18:42 +03:00
Aleksey Kladov
127814d9a7 nested mod completion 2018-09-08 01:35:20 +03:00
Aleksey Kladov
ff1c82216c Remove dyn dispatch 2018-09-08 01:16:07 +03:00
Aleksey Kladov
fcfda94664 Separete API from IMPL
Looks like there's a rule of thumb: don't call API functions from an
implementation! In this case, following this rule of thumb saves us an
Arc-bump!
2018-09-07 22:05:05 +03:00
Zac Winter
518cc87496 Moved TokenSet into it's own file. 2018-09-06 21:57:04 +08:00
Aleksey Kladov
751562d2f7 better introduce 2018-09-06 01:19:24 +03:00
Aleksey Kladov
bb64edf8ba introduce variable 2018-09-06 00:59:07 +03:00
Aleksey Kladov
47e8b80e9b use correct workdir for the server 2018-09-05 21:38:43 +03:00
Aleksey Kladov
669eabe892 even less hacks 2018-09-05 20:22:52 +03:00
Aleksey Kladov
d0e22d7578 less hacky paths 2018-09-05 18:27:44 +03:00
bors[bot]
ad451686a8 Merge #56
56: Unify lookahead naming between parser and lexer. r=matklad a=zachlute

Resolves Issue #26.

I wanted to play around with libsyntax2, and fixing a random issue seemed like a good way to mess around in the code.

This PR mostly does what's suggested in that issue. I elected to go with `at` and `at_str` instead of trying to do any fancy overloading shenanigans, because...uh, well, frankly I don't really know how to do any fancy overloading shenanigans. The only really questionable bit is `nth_is_p`, which could also have potentially been named `nth_at_p`, but `is` seemed more apropos.

I also added simple tests for `Ptr` so I could be less terrified I broke something. 

Comments and criticisms very welcome. I'm still pretty new to Rust.

Co-authored-by: Zach Lute <zach.lute@gmail.com>
2018-09-05 15:07:17 +00:00
Aleksey Kladov
649f7faf7d fix tests on windows 2018-09-05 15:03:27 +01:00
Zach Lute
d21fead150 Added tests for Ptr. 2018-09-04 23:26:11 -07:00
Zach Lute
af0ae9ee04 Updated Ptr methods to better match Parser method names. 2018-09-04 22:56:16 -07:00
Aleksey Kladov
f87771092c switch to rayon threadpool 2018-09-04 20:43:37 +03:00
Aleksey Kladov
8b0210d233 simplify 2018-09-04 19:00:01 +03:00
Aleksey Kladov
8ed06d766f better extend selection 2018-09-04 12:48:39 +03:00
Aleksey Kladov
e44a6bcc82 for types in bounds 2018-09-04 12:25:23 +03:00
Aleksey Kladov
3a017aaa52 dont change readonly files 2018-09-04 11:40:45 +03:00
Aleksey Kladov
a668f703fa micro-optimize 2018-09-04 05:09:39 +03:00
Aleksey Kladov
294534abc7 accidentally quadratic 2018-09-04 05:04:55 +03:00
Aleksey Kladov
4df965a002 work 2018-09-04 04:13:22 +03:00
Aleksey Kladov
c3e28f0646 extern blocks 2018-09-04 00:49:21 +03:00
Aleksey Kladov
952da31f44 switch to internal feedback 2018-09-03 23:32:42 +03:00
Aleksey Kladov
971054e4d0 fix the test 2018-09-03 21:51:17 +03:00
Aleksey Kladov
f590635f57 faster text len 2018-09-03 21:48:26 +03:00
Aleksey Kladov
5ba645c009 index libraies off the main thread 2018-09-03 21:26:59 +03:00
Aleksey Kladov
47cbaeba6f Index deps 2018-09-03 21:03:37 +03:00
Aleksey Kladov
b04c14d4ad dispatch acros roots 2018-09-03 19:46:30 +03:00
Aleksey Kladov
2f2feef9af completion for trait params 2018-09-03 15:46:14 +03:00
Aleksey Kladov
4798a89a12 Complete params 2018-09-03 15:10:06 +03:00
Aleksey Kladov
58480b9190 method call scope 2018-09-03 02:01:43 +03:00
Aleksey Kladov
fdd282ee0c improve test 2018-09-03 01:52:59 +03:00
Aleksey Kladov
23303cd0f8 match scope 2018-09-03 01:51:46 +03:00
Aleksey Kladov
83e2ab434c store file id inside symbol 2018-09-02 23:36:23 +03:00
Aleksey Kladov
a5e319ec7e Store symbols separately from file data 2018-09-02 23:09:47 +03:00
Aleksey Kladov
440dc41dd8 Add source root 2018-09-02 22:51:59 +03:00
Aleksey Kladov
e98d8cd255 nail down runnables 2018-09-02 16:36:03 +03:00
Aleksey Kladov
1329dd4e28 Avoid clones 2018-09-02 15:18:43 +03:00
Aleksey Kladov
80be61ed78 project model 2018-09-02 14:46:15 +03:00
Aleksey Kladov
7fad13de73 store messages in tests 2018-09-02 12:34:06 +03:00
Aleksey Kladov
d752455637 introduce project model 2018-09-02 11:37:57 +03:00
Aleksey Kladov
541170420b Add an integration test 2018-09-01 20:21:11 +03:00
Aleksey Kladov
e8515fecd7 split lib 2018-09-01 18:16:08 +03:00
Aleksey Kladov
4268fbeaa1 simplify 2018-09-01 18:03:57 +03:00
Aleksey Kladov
8f1ce82753 move to gen-server impl 2018-09-01 17:40:45 +03:00
Aleksey Kladov
3588d6b2da add gen_lsp_server 2018-09-01 16:18:02 +03:00
Aleksey Kladov
f5669dfc56 No self-imports in completion 2018-09-01 12:46:43 +03:00
Aleksey Kladov
2161a1689d Type aliases to scope 2018-09-01 12:30:53 +03:00
Aleksey Kladov
f2772e29ae add crate graph 2018-08-31 19:14:08 +03:00
Aleksey Kladov
7a5bc94774 complete self 2018-08-31 16:30:42 +03:00
Aleksey Kladov
cdb9b4cbf4 handle shadowing 2018-08-31 15:53:52 +03:00
Aleksey Kladov
78d60a549d default method name to type name 2018-08-31 15:10:37 +03:00
Aleksey Kladov
05a9d42f54 tweak extend selection 2018-08-31 14:52:29 +03:00
Aleksey Kladov
8fc7f438c4 start item recovery 2018-08-31 13:35:48 +03:00
Aleksey Kladov
faebae74e4 fix tests 2018-08-31 12:13:02 +03:00
Aleksey Kladov
d999f4b568 cancelation 2018-08-31 12:04:33 +03:00
Aleksey Kladov
efa6a952b4 cancelation tokens 2018-08-31 10:27:38 +03:00
Aleksey Kladov
902df0fc05 add test 2018-08-30 21:42:23 +03:00
Aleksey Kladov
8f552ab352 break/continue completion 2018-08-30 21:32:12 +03:00
Aleksey Kladov
80ab3433d3 complete imports 2018-08-30 20:37:33 +03:00
Aleksey Kladov
49e14a99ed Complete types 2018-08-30 20:03:18 +03:00
Aleksey Kladov
9fcebbc512 subscriptions 2018-08-30 16:27:09 +03:00
Aleksey Kladov
7570d85869 loop scope 2018-08-30 16:05:49 +03:00
Aleksey Kladov
c2c64145cb move 2018-08-30 13:12:49 +03:00
Aleksey Kladov
1f2fb4e27f move 2018-08-30 12:52:21 +03:00
Aleksey Kladov
0d6d74e78e minor 2018-08-30 12:34:31 +03:00
Aleksey Kladov
7d95d38ecb fix join lines selection 2018-08-29 18:35:28 +03:00
Aleksey Kladov
09ea0ca7e5 rename world -> analysis impl 2018-08-29 18:23:57 +03:00
Aleksey Kladov
0f968ee430 minor 2018-08-29 18:12:28 +03:00
Aleksey Kladov
4dd4571bfe minor 2018-08-29 18:09:08 +03:00
Aleksey Kladov
1baba9a2e2 Minor 2018-08-29 18:07:56 +03:00
Aleksey Kladov
2e2c2e62eb Remove dead code 2018-08-29 18:06:46 +03:00
Aleksey Kladov
fce6bc0acc Make world private 2018-08-29 18:05:54 +03:00
Aleksey Kladov
8abf536343 Grand refactoring 2018-08-29 18:03:14 +03:00
Aleksey Kladov
754c034a81 fix tests 2018-08-29 11:15:51 +03:00