Commit graph

380 commits

Author SHA1 Message Date
Rust大闸蟹
088e662285
Replace git current_branch to shadow-rs branch (#2935)
* update to shadow-rs 0.4. use easy

* update shadow-rs to 0.5

* fix version not used

* update

* update Cargo.lock

* update Cargo.lock

* fix wasm build error when use dependence git2
fix error link:https://dev.azure.com/nushell/nushell/_build/results?buildId=4858&view=logs&j=1a745d4c-b027-5f34-06d8-d6f256bfe9f9&t=a0a335cb-fa1f-5bbf-be01-1a90d6899e54

* remove code not used; fix warning by RUSTFLAGS="-D warnings" build error

* upgrade shadow-rs 0.5.2

* upgrade shadow-rs 0.5.7

make nushell reduce dependence crates smaller and  build fast.

* upgrade shadow-rs 0.5.8

fix when use api 'strip_prefix()' method in less than rust1.45.0 build failed

* use shadow-rs branch replace with current_branch method;
remove and reduce git dependencies.

* upgrade shadow-rs 0.5.12-pre,test build error with wasm

* upgrade Cargo.lock

* upgarde shadow-rs depencdence

* fix build error in wasm

* add clippy warning
2021-01-16 07:06:29 +13:00
Jonathan Turner
99117ff2ef
Fix reading/writing bigint and bigdecimal (#2893) 2021-01-09 12:53:59 +13:00
Jonathan Turner
17a433996e
rename set/set-env to let/let-env (#2859) 2021-01-05 12:30:55 +13:00
Jonathan Turner
27fe356214
Add proper shadowing (#2851) 2021-01-03 20:48:02 +13:00
Jonathan Turner
fc44df1e45
Don't leak set/set-env/source scopes via actions (#2849) 2021-01-03 19:44:21 +13:00
Jonathan Turner
43c10b0625
Properly handle commands defined inside of other commands (#2837)
* Fix function inner scopes

* tweak error
2021-01-01 19:23:54 +13:00
Jonathan Turner
f22938fc4a
Add support for custom subcommands (#2814)
* Add support for custom subcommands

* clippy
2020-12-23 20:43:56 +13:00
Jonathan Turner
c012d648fb
Add experimental support for flags in custom commands (#2808)
* Add experimental support for flags in custom commands

* clippy
2020-12-21 20:36:59 +13:00
Jonathan Turner
e3da546e23
Simplifies 'if' to work on the available scope rather than a stream (#2805) 2020-12-21 16:02:39 +13:00
Jonathan Turner
058ef69da3
Add set-env for setting environment variables (#2802) 2020-12-19 19:25:03 +13:00
Jonathan Turner
ac578b8491
Multiline scripts part 2 (#2795)
* Begin allowing comments and multiline scripts.

* clippy

* Finish moving to groups. Test pass

* Keep going

* WIP

* WIP

* BROKEN WIP

* WIP

* WIP

* Fix more tests

* WIP: alias starts working

* Broken WIP

* Broken WIP

* Variables begin to work

* captures start working

* A little better but needs fixed scope

* Shorthand env setting

* Update main merge

* Broken WIP

* WIP

* custom command parsing

* Custom commands start working

* Fix coloring and parsing of block

* Almost there

* Add some tests

* Add more param types

* Bump version

* Fix benchmark

* Fix stuff
2020-12-18 20:53:49 +13:00
Jonathan Turner
6951fb440c
Remove it expansion (#2701)
* Remove it-expansion, take 2

* Cleanup

* silly update to test CI
2020-10-26 19:55:52 +13:00
Andrés N. Robalino
e02b4f1443
Update wix plugin check with base test. (#2666) 2020-10-13 16:22:49 -05:00
Andrés N. Robalino
4e931fa73f
Extract out xpath to a plugin. (#2661) 2020-10-12 18:18:39 -05:00
RedlineTriad
5770b15270
Use iterator chain instead of string concat. (#2655)
* Use iterator chain instead of string concat

* Add regression test for multi-value lines
2020-10-10 18:30:48 +13:00
Samuel Vanderwaal
2b076369e0
handle duration overflow error (#2616)
* handle duration overflow error

* handle checked_add_signed result
2020-10-09 14:51:47 +13:00
Andrés N. Robalino
2ea5235aea
Ensure Wix lists Nu plugin binaries. (#2637) 2020-10-05 14:29:04 -05:00
Andrés N. Robalino
0178b53289
Core nu plugin load capability. (#2544)
We introduce the `plugin` nu sub command (`nu plugin`) with basic plugin
loading support. We can choose to load plugins from a directory. Originally
introduced to make integration tests faster (by not loading any plugins on startup at all)
but `nu plugin --load some_path ; test_pipeline_that_uses_plugins_just_loaded` does not see it.

Therefore, a `nu_with_plugins!` macro for tests was introduced on top of nu`s `--skip-plugins`
switch executable which is set to true when running the integration tests that use the `nu!` macro now..
2020-09-14 09:07:02 -05:00
Radek Vít
599bb9797d
Implement exclusive and inclusive ranges with ..< and .. (#2541)
* Implement exclusive and inclusive ranges with .. and ..=

This commit adds right-exclusive ranges.

The original a..b inclusive syntax was changed to reflect the Rust notation.
New a..=b syntax was introduced to have the old behavior.

Currently, both a.. and b..= is valid, and it is unclear whether it's valid
to impose restrictions.

The original issue suggests .. for inclusive and ..< for exclusive ranges,
this can be implemented by making simple changes to this commit.

* Fix collect tests by changing ranges to ..=

* Fix clippy lints in exclusive range matching

* Implement exclusive ranges using `..<`
2020-09-14 09:53:08 +12:00
Jonathan Turner
d9ae66791a
allow decimals as a range boundary (#2509) 2020-09-08 05:30:11 +12:00
Jonathan Turner
c9ffd6afc0
Improve range parsing and handling (#2506)
* Improve range parsing and handling

* linting
2020-09-07 14:43:58 +12:00
Jonathan Turner
df691c6c91
Light fixes (#2455)
* Add optional commas for items in lists and tables

* A couple last fixes
2020-08-30 19:03:18 +12:00
Jonathan Turner
abc05ece21
Add optional commas for items in lists and tables (#2454) 2020-08-30 18:19:54 +12:00
Jonathan Turner
6f69ae8707
Add table literals (#2453)
* Add table literals

* clippy
2020-08-30 16:55:33 +12:00
Chris Gillespie
0390ec97f4
Create benign email test fixture (#2445)
1. The previous one was an Ebay email and flagged by AVs, create something simple.
2. Add test case for another header
2020-08-29 12:57:50 -04:00
Sam Hedin
57101d5022
Run exitscripts in original dir (#2352)
* Modify testcase

* Run exitscript in the folder it was specified

* Update documentation

* Add comment

* Borrow instead of clone

* Does this just... work on windows?

* fmt

* as_str

* Collapse if by order of clippy

* Support windows

* fmt

* refactor tests

* fmt

* This time it will work on windows FOR SURE

* Remove debug prints

* Comment

* Refactor tests

* fmt

* fix spelling

* update comment
2020-08-18 17:36:09 +12:00
Joseph T Lyons
8fe269a3d8 Add random_numbers.csv to repo, so it is easier to update histogram examples 2020-08-15 07:51:12 -05:00
Sam Hedin
3c18169f63
Autoenv fix: Exitscripts incorrectly running when visiting a subdirectory (#2326)
* Add test case for issue

* Preliminary fix

* fmt

* Reorder asserts

* move insertion

* Touch nu-env.toml

* Cleanup

* touch nu-env toml

* Remove touch

* Change feature flags
2020-08-12 07:54:49 +12:00
k-brk
6b4634b293
Convert table of primitives to positional arguments for external cmd (#2232)
* Convert table of primitives to positional arguments for external cmd

* Multiple file test, fix for cococo
2020-07-23 09:41:34 +12:00
Joseph T. Lyons
6eb2c94209
Add flag for case-insensitive sort-by (#2225)
* Add flag for case-insensitive sort-by

* Fix test names

* Fix documentation comments
2020-07-21 05:31:58 +12:00
Jonathan Turner
d8594a62c2
Add wasm support (#2199)
* Working towards a PoC for wasm

* Move bson and sqlite to plugins

* proof of concept now working

* tests are green

* Add CI test for --no-default-features

* Fix some tests

* Fix clippy and windows build

* More fixes

* Fix the windows build

* Fix the windows test
2020-07-18 13:59:23 +12:00
Sam Hedin
80d2a7ee7a
Fix autoenv executing scripts multiple times (#2171)
* Fix autoenv executing scripts multiple times

Previously, if the user had only specified entry or exitscripts the scripts
would execute many times. This should be fixed now

* Add tests

* Run exitscripts

* More tests and fixes to existing tests

* Test solution with visited dirs

* Track visited directories

* Comments and fmt
2020-07-15 07:16:50 +12:00
Sam Hedin
f3f40df4dd
Tests for autoenv (and fixes for bugs the tests found) (#2148)
* add test basic_autoenv_vars_are_added

* Tests

* Entry and exit scripts

* Recursive set and overwrite

* Make sure that overwritten vals are restored

* Move tests to autoenv

* Move tests out of cli crate

* Tests help, apparently. Windows has issues

On windows, .nu-env is not applied immediately after running autoenv trust.
You have to cd out of the directory for it to work.

* Sort paths non-lexicographically

* Sibling dir test

* Revert "Sort paths non-lexicographically"

This reverts commit 72e4b856af.

* Rename test

* Change conditions

* Revert "Revert "Sort paths non-lexicographically""

This reverts commit 71606bc62f.

* Set vars as they are discovered

This means that if a parent directory is untrusted,
the variables in its child directories are still set properly.

* format

* Fix cleanup issues too

* Run commands in their separate functions

* Make everything into one large function like all the cool kids

* Refactoring

* fmt

* Debugging windows path issue

* Canonicalize

* Trim whitespace

* On windows, use echo nul instead of touch to create file in test

* Avoid cloning by using drain()
2020-07-12 16:14:09 +12:00
Jonathan Turner
bbc5a28fe9
Fix buffering in lines command (#2111) 2020-07-05 12:20:58 +12:00
Jason Gedge
180290f3a8
Remove custom escaping for external args. (#2095)
Our own custom escaping unfortunately is far too simple to cover all cases.
Instead, the parser will now do no transforms on the args passed to an external
command, letting the process spawning library deal with doing the appropriate
escaping.
2020-07-03 11:29:28 +12:00
Jonathan Turner
ed10aafa6f
Bubble errors even if pipeline isn't used (#2080) 2020-06-30 05:39:11 +12:00
Jonathan Turner
dffc9c9b1c
Properly redirect invocations (#2070)
* Properly redirect invocations

* Don't convert with-env yet, as there's a random test failure
2020-06-28 09:04:57 +12:00
Jonathan Turner
4e2a4236f8
Fix it expansion and add collect (#2065) 2020-06-27 17:38:19 +12:00
Joseph T. Lyons
53a6e9f0bd
Convert sum command into subcommand of the math command (#2004)
* Convert sum command into subcommand of the math command

* Add bullet points to math.md documentation
2020-06-18 21:02:01 -05:00
Andrés N. Robalino
ad43ef08e5 Support average for tables. 2020-05-30 10:33:09 -05:00
Andrés N. Robalino
48ee20782f Ensure end_filter plugin lifecycle stage gets called. 2020-05-29 04:03:25 -05:00
Andrés N. Robalino
fe01a223a4 Str plugin promoted to built-in Nu command. 2020-05-28 11:18:58 -05:00
Jonathan Turner
98a3d9fff6
Allow echo to iterate ranges (#1905) 2020-05-28 06:07:53 +12:00
Jonathan Turner
e2dabecc0b Make it-expansion work when in a list (#1903) 2020-05-27 20:29:05 +12:00
Ryan Blecher
bf212a5a3a
change the test to use the origin column (#1878) 2020-05-25 18:50:54 -04:00
k-brk
8a99d112fc
Add --to-float to str plugin (#1872) 2020-05-24 18:11:49 -04:00
Alexander James
aadbcf5ce8
Issue 1787 (#1827) 2020-05-23 20:08:39 -04:00
Jonathan Turner
ed80933806
String interpolation (#1849)
* Add string interpolation

* fix coloring

* A few more fixups + tests

* merge master again
2020-05-20 07:27:26 +12:00
Chris Pearce
ae87582cb6
Fix missing invocation errors (#1846) 2020-05-19 08:57:25 -04:00
Jason Gedge
6efabef8d3
Remove interpretation of Primitive::Nothing as the number 0. (#1836) 2020-05-18 15:18:46 -04:00
Andrés N. Robalino
6ec6eb5199 Call external correctly. 2020-05-17 23:32:55 -05:00
Andrés N. Robalino
f18424a6f6 Remove test-bins feature. 2020-05-17 23:32:55 -05:00
Jonathan Turner
f43ed23ed7
Fix parsing of invocations with a dot (#1804) 2020-05-16 19:25:18 +12:00
Jonathan Turner
076fde16dd
Evaluation of command arguments (#1801)
* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* Finish adding the baseline refactors for argument invocation

* Finish cleanup and add test

* Add missing plugin references
2020-05-16 15:18:24 +12:00
Xavier L'Heureux
5fbe5cf785
Use the directories crate instead of app_dirs (#1782)
The app_dirs crate is abandonned since quite a bit of time. Use the directories
crate instead, which is maintained and have more OS support.
2020-05-14 20:17:23 +12:00
Andrés N. Robalino
96e5fc05a3
Pick->Select rename. Integration tests changes. (#1725)
Pick->Select rename. Integration tests changes.
2020-05-07 06:03:43 -05:00
Jonathan Turner
a9968046ed
Add subcommands. Switch from-* and to-* to them (#1708) 2020-05-04 20:44:33 +12:00
Jonathan Turner
7d403a6cc7
Escape some symbols in external args (#1687)
* Escape some symbols in external args

* Don't escape on Windows, which does its own

* fix warning
2020-04-30 16:54:07 +12:00
Jonathan Turner
db8219e798
extend it-expansion to externals (#1682)
* extend it-expansion to externals

* trim the carriage return for external strings
2020-04-30 07:09:14 +12:00
Jonathan Turner
73d5310c9c
make it-expansion work through blocks when necessary (#1681) 2020-04-29 19:51:46 +12:00
Jason Gedge
6f2ef05195
Resolves https://github.com/nushell/nushell/issues/1658 (#1660)
For example, when running the following:

    crates/nu-cli/src

nushell currently parses this as an external command. Before running the command, we check to see if
it's a directory. If it is, we "auto cd" into that directory, otherwise we go through normal
external processing.

If we put a trailing slash on it though, shells typically interpret that as "user is explicitly
referencing directory". So

    crates/nu-cli/src/

should not be interpreted as "run an external command". We intercept a trailing slash in the head
position of a command in a pipeline as such, and inject a `cd` internal command.
2020-04-27 13:22:01 +12:00
Adam Shirey
ad8ab5b04d
Add from-eml command (#1656)
* from-eml initial ver

* Adding tests for `from-eml`

* Add eml to prepares_and_decorates_filesystem_source_files

* Sort the file order

Co-authored-by: Jonathan Turner <jonathandturner@users.noreply.github.com>
2020-04-26 16:26:35 +12:00
Andrés N. Robalino
10768b6ecf
str plugin can capitalize and trim strings. (#1652)
* Str plugin can capitalize.

* Str plugin can trim.
2020-04-24 16:37:58 -05:00
Jonathan Turner
c4daa2e40f
Add experimental new parser (#1554)
Move to an experimental new parser
2020-04-06 19:16:14 +12:00
Sam Hedin
ae5f3c8210
WIP: 1486/first row as headers (#1530)
* headers plugin

* Remove plugin

* Add non-functioning headers command

* Add ability to extract headers from first row

* Refactor header extraction

* Rebuild indexmap with proper headers

* Rebuild result properly

* Compiling, probably wrapped too much?

* Refactoring

* Deal with case of empty header cell

* Deal with case of empty header cell

* Fix formatting

* Fix linting, attempt 2.

* Move whole_stream_command(Headers) to more appropriate section

* ... more linting

* Return Err(ShellError...) instead of panic, yield each row instead of entire table

* Insert Column[index] if no header info is found.

* Update error description

* Add initial test

* Add tests for headers command

* Lint test cases in headers

* Change ShellError for headers, Add sample_headers file to utils.rs

* Add empty sheet to test file

* Revert "Add empty sheet to test file"

This reverts commit a4bf38a31d.

* Show error message when given empty table
2020-03-29 15:05:57 +13:00
Jonathan Turner
1ec2ec72b5
Add automatic change directory (#1496)
* Allow automatic cd in cli mode

* Set correct priority for auto-cd and add test
2020-03-18 07:13:38 +13:00
Andrés N. Robalino
b36d21e76f
Infer types from regular delimited plain text unstructured files. (#1494)
* Infer types from regular delimited plain text unstructured files.

* Nothing resolves to an empty string.
2020-03-16 15:50:45 -05:00
Andrés N. Robalino
db16b56fe1
Columnpath support when passing fields for formatting. (#1472) 2020-03-10 01:55:03 -05:00
Jonathan Turner
8925ca5da3
Move to bytes/string hybrid codec (#1457)
* WIP: move to bytes codec

* Progress on adding collect helpers

* Progress on adding collect helpers

* Add in line splitting back to lines

* Lines outputting line primitives

* Close to ready?

* Finish fixing lines

* clippy fixes

* fmt fixes

* removed unused code

* Cleanup a few bits

* Cleanup a few bits

* Cleanup a few more bits

* Fix failing test with corrected test case
2020-03-07 05:06:39 +13:00
Jason Gedge
b2c5af457e
Move most of the root package into a subcrate. (#1445)
This improves incremental build time when working on what was previously
the root package. For example, previously all plugins would be rebuilt
with a change to `src/commands/classified/external.rs`, but now only
`nu-cli` will have to be rebuilt (and anything that depends on it).
2020-03-04 13:58:20 -05:00
Andrés N. Robalino
c731a5b628
Columns can be renamed. (#1447) 2020-03-03 16:01:24 -05:00
Jason Gedge
7304d06c0b
Use threads to avoid blocking reads/writes in externals. (#1440)
In particular, one thing that we can't (properly) do before this commit
is consuming an infinite input stream. For example:

```
yes | grep y | head -n10
```

will give 10 "y"s in most shells, but blocks indefinitely in nu. This PR
resolves that by doing blocking I/O in threads, and reducing the `await`
calls we currently have in our pipeline code.
2020-03-02 06:19:09 +13:00
Sean Hellum
b98f893217
add a touch command (#1399) 2020-02-19 09:54:32 +13:00
Andrés N. Robalino
18d988d4c8
Restrict short-hand flag detection to exact match. (#1406) 2020-02-18 01:58:30 -05:00
Jonathan Turner
afce2fd0f9
Revert "Display rows in the same table regardless of their column order given they are equal. (#1392)" (#1401)
This reverts commit 4fd9974204.
2020-02-17 17:34:37 -08:00
Andrés N. Robalino
4fd9974204
Display rows in the same table regardless of their column order given they are equal. (#1392) 2020-02-16 20:35:01 -05:00
Jonathan Turner
8ae8ebd107
Add support for multiline script files (#1386)
* Add support for multiline script files

* clippy
2020-02-13 21:24:18 -08:00
Andrés N. Robalino
84927d52b5
Refuse internal command execution given unexpected arguments. (#1383) 2020-02-13 02:34:43 -05:00
Corvus Corax
c0be02a434
Short-hand flags (#1378)
* typo fixes

* Change signature to take in short-hand flags

* update help information

* Parse short-hand flags as their long counterparts

* lints

* Modified a couple tests to use shorthand flags
2020-02-11 18:24:31 -08:00
Andrés N. Robalino
2ab8d035e6
External it and nu variable column path fetch support. (#1379) 2020-02-11 18:25:56 -05:00
Andrés N. Robalino
24094acee9
Allow switch flags anywhere in the pipeline. (#1375) 2020-02-11 03:49:00 -05:00
Andrés N. Robalino
29ccb9f5cd
Ensure stable plugins get installed. (#1373) 2020-02-10 15:32:10 -05:00
Jonathan Turner
fb532f3f4e
Prototype shebang support (#1368)
* Add shebang support to nu.

* Move test file

* Add test for scripts

Co-authored-by: Jason Gedge <jason.gedge@shopify.com>
2020-02-10 08:49:45 -08:00
Jason Gedge
a29d52158e
Do not panic when failing to decode lines from external stdout (#1364) 2020-02-10 07:37:48 -08:00
Jonathan Turner
dc50e61f26 Switch stdin redirect to manual. Add test (#1367) 2020-02-09 22:55:07 -08:00
Jason Gedge
3687603799
Only spawn external once when no $it argument (#1358) 2020-02-08 17:57:05 -08:00
George Tsomlektsis
643b532537
Fixed mv not throwing error when the source path was invalid (#1351)
* Fixed mv not throwing error when the source path was invalid

* Fixed failing test

* Fixed another lint error

* Fix $PATH conflicts in .gitpod.Dockerfile (#1349)

- Use the correct user for gitpod Dockerfile.
- Remove unneeded packages (curl, rustc) from gitpod Dockerfile.

* Added test to check for the error

* Fixed linting error

* Fixed mv not moving files on Windows. (#1342)

Move files correctly in windows.

* Fixed mv not throwing error when the source path was invalid

* Fixed failing test

* Fixed another lint error

* Added test to check for the error

* Fixed linting error

* Changed error message

* Typo and fixed test

Co-authored-by: Sean Hellum <seanhellum45@gmail.com>
2020-02-07 12:40:48 -05:00
George Tsomlektsis
ed86b1fbe8
Fixed mv not moving files on Windows. (#1342)
Move files correctly in windows.
2020-02-07 11:24:01 -05:00
Ishan Bhanuka
427bde83f7
Allow cp to overwrite existing files (#1339) 2020-02-05 01:54:05 -05:00
Jason Gedge
dcdfa2a866
Improve tests and labeling in FilesystemShell (#1305)
Additional `ls` command tests and better FilesystemShell error and label messages.
2020-02-01 03:34:34 -05:00
Shaurya Shubham
49a1385543
Make tests work from directory names with spaces (#1325) 2020-01-31 22:12:56 -08:00
Shaurya Shubham
dc3370b103
Make a calc command (#1280) 2020-01-29 08:34:36 -05:00
Jason Gedge
32dfb32741 Switch from subprocess crate to the builtin std::process (#1284)
* Switch from subprocess crate to the builtin std::process

* Update external.rs

* Update external.rs

* Update external.rs

Co-authored-by: Jonathan Turner <jonathandturner@users.noreply.github.com>
2020-01-26 16:03:21 +13:00
Borimino
583f27dc41 Added attributes to from-xml command (#1272)
* Added attributes to from-xml command

* Added attributes as their own rows

* Removed unneccesary lifetime declarations

* from-xml now has children and attributes side by side

* Fixed tests and linting

* Fixed lint-problem
2020-01-26 05:16:40 +13:00
Corvus Corax
a5e1372bc2 RM error on bad filename (#1244)
* rm error on bad filename

* De-lint

* Fix error message in test
2020-01-25 08:16:41 +13:00
Jonathan Turner
798a24eda5
Soften restrictions for external parameters (#1277)
* Soften restrictions for external parameters

* Add test
2020-01-25 08:14:49 +13:00
Jonathan Turner
d38a63473b
Improve shelling out (#1273)
Improvements to shelling out
2020-01-24 08:24:31 +13:00
Jonathan Turner
2b37ae3e81
Switch to using subprocess::shell (#1264)
* Switch to using `shell`

Switch to using the shell for subprocess to enable more natural shelling out.

* Update external.rs

* This is a test with .shell() for external

* El pollo loco's PR

* co co co

* Attempt to fix windows

* Fmt

* Less is more?

Co-authored-by: Andrés N. Robalino <andres@androbtech.com>
2020-01-24 05:21:05 +13:00
Andrés N. Robalino
762c798670
It ls test setup rewrite. (#1260) 2020-01-21 22:56:12 -05:00
Yehuda Katz
7efb31a4e4 Restructure and streamline token expansion (#1123)
Restructure and streamline token expansion

The purpose of this commit is to streamline the token expansion code, by
removing aspects of the code that are no longer relevant, removing
pointless duplication, and eliminating the need to pass the same
arguments to `expand_syntax`.

The first big-picture change in this commit is that instead of a handful
of `expand_` functions, which take a TokensIterator and ExpandContext, a
smaller number of methods on the `TokensIterator` do the same job.

The second big-picture change in this commit is fully eliminating the
coloring traits, making coloring a responsibility of the base expansion
implementations. This also means that the coloring tracer is merged into
the expansion tracer, so you can follow a single expansion and see how
the expansion process produced colored tokens.

One side effect of this change is that the expander itself is marginally
more error-correcting. The error correction works by switching from
structured expansion to `BackoffColoringMode` when an unexpected token
is found, which guarantees that all spans of the source are colored, but
may not be the most optimal error recovery strategy.

That said, because `BackoffColoringMode` only extends as far as a
closing delimiter (`)`, `]`, `}`) or pipe (`|`), it does result in
fairly granular correction strategy.

The current code still produces an `Err` (plus a complete list of
colored shapes) from the parsing process if any errors are encountered,
but this could easily be addressed now that the underlying expansion is
error-correcting.

This commit also colors any spans that are syntax errors in red, and
causes the parser to include some additional information about what
tokens were expected at any given point where an error was encountered,
so that completions and hinting could be more robust in the future.

Co-authored-by: Jonathan Turner <jonathandturner@users.noreply.github.com>
Co-authored-by: Andrés N. Robalino <andres@androbtech.com>
2020-01-21 17:45:03 -05:00