Commit graph

626 commits

Author SHA1 Message Date
Devyn Cairns
73f3c0b60b
Support for all custom value operations on plugin custom values (#12088)
# Description

Adds support for the following operations on plugin custom values, in
addition to `to_base_value` which was already present:

- `follow_path_int()`
- `follow_path_string()`
- `partial_cmp()`
- `operation()`
- `Drop` (notification, if opted into with
`CustomValue::notify_plugin_on_drop`)

There are additionally customizable methods within the `Plugin` and
`StreamingPlugin` traits for implementing these functions in a way that
requires access to the plugin state, as a registered handle model such
as might be used in a dataframes plugin would.

`Value::append` was also changed to handle custom values correctly.

# User-Facing Changes

- Signature of `CustomValue::follow_path_string` and
`CustomValue::follow_path_int` changed to give access to the span of the
custom value itself, useful for some errors.
- Plugins using custom values have to be recompiled because the engine
will try to do custom value operations that aren't supported
- Plugins can do more things 🎉 

# Tests + Formatting
Tests were added for all of the new custom values functionality.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
- [ ] Document protocol reference `CustomValueOp` variants:
  - [ ] `FollowPathInt`
  - [ ] `FollowPathString`
  - [ ] `PartialCmp`
  - [ ] `Operation`
  - [ ] `Dropped`
- [ ] Document `notify_on_drop` optional field in `PluginCustomValue`
2024-03-12 10:37:08 +01:00
Devyn Cairns
afce380530
Make the plugin persistence GC delay test more reliable (#12153)
# Description

This makes the test a bit more complicated, but implements a timeout
loop in the script. As long as the test completes in 5 seconds it's
considered to be ok. The default is 10 seconds, so that would still be
half that.

This should help with running on the busy CI where things sometimes take
a while. Unfortunately this is a timing sensitive test. The alternative
is basically to just not test this at all because it's too difficult to
guarantee that it will complete in time. If we continue to have issues,
I might just have to take that route instead.
2024-03-11 12:01:48 +01:00
Wind
5596190377
do command: Make closure support default parameters and type checking (#12056)
# Description
Fixes: #11287
Fixes: #11318

It's implemented by porting the similar logic in `eval_call`, I've tried
to reduce duplicate code, but it seems that it's hard without using
macros.

3ee2fc60f9/crates/nu-engine/src/eval.rs (L60-L130)

It only works for `do` command.

# User-Facing Changes
## Closure supports optional parameter
```nushell
let code = {|x?| print ($x | default "i'm the default")}
do $code
```
Previously it raises an error, after this change, it prints `i'm the
default`.

## Closure supports type checking
```nushell
let code = {|x: int| echo $x}
do $code "aa"
```
After this change, it will raise an error with a message: `can't convert
string to int`

# Tests + Formatting
Done

# After Submitting
NaN
2024-03-11 18:11:08 +08:00
Stefan Holderbach
067ceedf79
Remove feat extra and include in default (#12140)
# Description
The intended effect of the `extra` feature has been undermined by
introducing the full builds on our release pages and having more
activity on some of the extra commands.

To simplify the feature matrix let's get rid of it and focus our effort
on truly either refining a command to well-specified behavior or
discarding it entirely from the `nu` binary and moving it into plugins.

## Details
- Remove `--features extra` from CI
- Don't explicitly name `extra` in full build wf
- Remove feature extra from build-help scripts
- Update README in `nu-cmd-extra`
- Remove feature `extra`
- Fix previously dead `format pattern` tests
- Relax signature of `to html`
- Fix/ignore `html::test_no_color_flag`
- Remove dead features from `version`
- Refine `to html` type signature

# User-Facing Changes
The commands that were previously only available when building with
`--features extra` will now be available to everyone. This increases the
number of dependencies slightly but has a limited impact on the overall
binary size.

# Tests + Formatting
Some tests that were left in `nu-command` during cratification were dead
because the feature was not passed to `nu-command` and only to
`nu-cmd-lang` for feature-flag mention in `version`.
Those tests have now been either fixed or ignored in one case.

# After Submitting
There may be places in the documentation where we point to `--features
extra` that will now be moot (apart from the generated command help)
2024-03-10 17:29:02 +01:00
Devyn Cairns
bc19be25b1
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.

Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.

This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.

Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:

```nushell
  $env.config.plugin_gc = {
      # Configuration for plugin garbage collection
      default: {
          enabled: true # true to enable stopping of inactive plugins
          stop_after: 10sec # how long to wait after a plugin is inactive to stop it
      }
      plugins: {
          # alternate configuration for specific plugins, by name, for example:
          #
          # gstat: {
          #     enabled: false
          # }
      }
  }
```

If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.

The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.

Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.

# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.

# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 17:10:22 -06:00
Devyn Cairns
430fb1fcb6
Add support for engine calls from plugins (#12029)
# Description

This allows plugins to make calls back to the engine to get config,
evaluate closures, and do other things that must be done within the
engine process.

Engine calls can both produce and consume streams as necessary. Closures
passed to plugins can both accept stream input and produce stream output
sent back to the plugin.

Engine calls referring to a plugin call's context can be processed as
long either the response hasn't been received, or the response created
streams that haven't ended yet.

This is a breaking API change for plugins. There are some pretty major
changes to the interface that plugins must implement, including:

1. Plugins now run with `&self` and must be `Sync`. Executing multiple
plugin calls in parallel is supported, and there's a chance that a
closure passed to a plugin could invoke the same plugin. Supporting
state across plugin invocations is left up to the plugin author to do in
whichever way they feel best, but the plugin object itself is still
shared. Even though the engine doesn't run multiple plugin calls through
the same process yet, I still considered it important to break the API
in this way at this stage. We might want to consider an optional
threadpool feature for performance.

2. Plugins take a reference to `EngineInterface`, which can be cloned.
This interface allows plugins to make calls back to the engine,
including for getting config and running closures.

3. Plugins no longer take the `config` parameter. This can be accessed
from the interface via the `.get_plugin_config()` engine call.


# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Not only does this have plugin protocol changes, it will require plugins
to make some code changes before they will work again. But on the plus
side, the engine call feature is extensible, and we can add more things
to it as needed.

Plugin maintainers will have to change the trait signature at the very
least. If they were using `config`, they will have to call
`engine.get_plugin_config()` instead.

If they were using the mutable reference to the plugin, they will have
to come up with some strategy to work around it (for example, for `Inc`
I just cloned it). This shouldn't be such a big deal at the moment as
it's not like plugins have ever run as daemons with persistent state in
the past, and they don't in this PR either. But I thought it was
important to make the change before we support plugins as daemons, as an
exclusive mutable reference is not compatible with parallel plugin
calls.

I suggest this gets merged sometime *after* the current pending release,
so that we have some time to adjust to the previous plugin protocol
changes that don't require code changes before making ones that do.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`


# After Submitting
I will document the additional protocol features (`EngineCall`,
`EngineCallResponse`), and constraints on plugin call processing if
engine calls are used - basically, to be aware that an engine call could
result in a nested plugin call, so the plugin should be able to handle
that.
2024-03-09 11:26:30 -06:00
Yash Thakur
c90640411d
Update tests Playground (#12134)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

It looks like `Playground` and `Director` in nu-tests-support haven't
gotten much love recently, so this PR is for updating them to work with
newer Nushell versions.

- `Director` adds a `--skip-plugins` argument before running `nu`, but
that doesn't exist anymore, so I removed it.
- `Director` also adds a `--perf` argument, which also doesn't exist
anymore. I added `--log-level info` instead to get the performance
output.
- It doesn't seem like anyone was using `playground::matchers`, and it
used the [hamcrest2](https://github.com/Valloric/hamcrest2-rust) crate,
which appears to be unmaintained, so I got rid of that (and the
`hamcrest2` dependency).
- Inside `tests/fixtures/playground/config` were two files in the old
config format: `default.toml` and `startup.toml`. I removed those too.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

None, these changes only mess with tests.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-03-08 20:31:21 -08:00
Yash Thakur
98525043ed
Fix const string interpolation test (#12038)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Currently, in the test for interpolating strings at parse-time, the
formatted string includes `(X years ago)` (from formatting a date) (test
came from https://github.com/nushell/nushell/pull/11562). I didn't
realize when I was writing it that it would have to be updated every
year. This PR uses regex to check the output instead.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-03-01 16:35:07 -08:00
Wind
387328fe73
Glob: don't allow implicit casting between glob and string (#11992)
# Description
As title, currently on latest main, nushell confused user if it allows
implicit casting between glob and string:
```nushell
let x = "*.txt"
def glob-test [g: glob] { open $g } 
glob-test $x
```
It always expand the glob although `$x` is defined as a string.
This pr implements a solution from @kubouch :
> We could make it really strict and disallow all autocasting between
globs and strings because that's what's causing the "magic" confusion.
Then, modify all builtins that accept globs to accept oneof(glob,
string) and the rules would be that globs always expand and strings
never expand

# User-Facing Changes
After this pr, user needs to use `into glob` to invoke `glob-test`, if
user pass a string variable:
```nushell
let x = "*.txt"
def glob-test [g: glob] { open $g } 
glob-test ($x | into glob)
```
Or else nushell will return an error.
```
 3 │ glob-test $x
   ·           ─┬
   ·            ╰── can't convert string to glob
```

# Tests + Formatting
Done

# After Submitting
Nan
2024-02-28 23:05:35 +08:00
Wind
eaedb30a8c
Don't expanding globs if user pass variables. (#11946)
# Description
Fixes: #11912

# User-Facing Changes
After this change: 
```
let x = '*.nu'; ^echo $x
```
will no longer expand glob.
If users still want to expand glob, there are also 3 ways to do this:
```
# 1. use spread operation with `glob` command
let x = '*.nu'; ^echo ...(glob $x)
```
# Tests + Formatting
Done

# After Submitting
NaN
2024-02-28 23:05:09 +08:00
Filip Andersson
0aae485395
Fixes test 1 year more. (#11994)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
The test checks the time that has passed, bumped year by 1.


# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-27 08:55:37 -06:00
132ikl
f4d9ddd3ad
Fix completions for directories with hidden files (#11921)
# Description
Attempting to complete a directory with hidden files could cause a
variety of issues. When Rust parses the partial path to be completed
into components, it removes the trailing `.` since it interprets this to
mean "the current directory", but in the case of the completer we
actually want to treat the trailling `.` as a literal `.`. This PR fixes
this by adding a `.` back into the Path components if the last character
of the path is a `.` AND the path is longer than 1 character (eg., not
just a ".", since that correctly gets interpreted as Component::CurDir).

Here are some things this fixes:
- Panic when tab completing for hidden files in a directory with hidden
files (ex. `ls test/.`)
- Panic when tab completing a directory with only hidden files (since
the common prefix ends with a `.`, causing the previous issue)
- Mishandling of tab completing hidden files in directory (ex. `ls
~/.<TAB>` lists all files instead of just hidden files)
- Trailing `.` being inexplicably removed when tab completing a
directory without hidden files

While testing for this PR I also noticed there is a similar issue when
completing with `..` (ex. `ls ~/test/..<TAB>`) which is not fixed by
this PR (edit: see #11922).

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
N/A

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Added a hidden-files-within-directories test to the `file_completions`
test.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-26 12:14:19 -06:00
Devyn Cairns
88f1f386bb
Bidirectional communication and streams for plugins (#11911) 2024-02-25 16:32:50 -06:00
Wind
1058707a29
make stderr works for failed external command (#11914)
# Description
Fixes: #11913

When running external command, nushell shouldn't consumes stderr
messages, if user want to redirect stderr.

# User-Facing Changes
NaN

# Tests + Formatting
Done

# After Submitting
NaN
2024-02-21 21:15:05 +08:00
Wind
58c6fea60b
Support redirect stderr and stdout+stderr with a pipe (#11708)
# Description
Close: #9673
Close: #8277
Close: #10944

This pr introduces the following syntax:
1. `e>|`, pipe stderr to next command. Example: `$env.FOO=bar nu
--testbin echo_env_stderr FOO e>| str length`
2. `o+e>|` and `e+o>|`, pipe both stdout and stderr to next command,
example: `$env.FOO=bar nu --testbin echo_env_mixed out-err FOO FOO e+o>|
str length`

Note: it only works for external commands. ~There is no different for
internal commands, that is, the following three commands do the same
things:~ Edit: it raises errors if we want to pipes for internal
commands
``` 
❯ ls e>| str length
Error:   × `e>|` only works with external streams
   ╭─[entry #1:1:1]
 1 │ ls e>| str length
   ·    ─┬─
   ·     ╰── `e>|` only works on external streams
   ╰────

❯ ls e+o>| str length
Error:   × `o+e>|` only works with external streams
   ╭─[entry #2:1:1]
 1 │ ls e+o>| str length
   ·    ──┬──
   ·      ╰── `o+e>|` only works on external streams
   ╰────
```

This can help us to avoid some strange issues like the following:

`$env.FOO=bar (nu --testbin echo_env_stderr FOO) e>| str length`

Which is hard to understand and hard to explain to users.

# User-Facing Changes
Nan

# Tests + Formatting
To be done

# After Submitting
Maybe update documentation about these syntax.
2024-02-09 01:30:46 +08:00
Andrej Kolchin
fb7f6fc08b
Fix a panic when parsing empty file (#11314)
The previous implementation presumed that if files were given, they had
contents. The change makes the fallback to permanent files uniform.

Fix #11256
2024-02-07 18:47:44 -06:00
Wind
20aa59085b
Fix file completions which contains glob pattern (#11766)
# Description
Fixes: https://github.com/nushell/nushell/issues/11762

The auto-completion is somehow annoying if a path contains a glob
pattern, let's say if user type `ls` and it auto-completes to <code>ls
`[a] bc.txt`</code>, and user can't list the file because it's backtick
quoted.

This pr is going to fix it.

# User-Facing Changes
### Before
```
❯ | ls
`[a] bc.txt`        `a bc`
```
### After
```
❯ | ls
"[a] bc.txt"        `a bc`
```
# Tests + Formatting
Done

# After Submitting
NaN
2024-02-08 06:42:50 +08:00
Antoine Stevan
859f7b3dc7
add $.extra_usage to modules (#11649)
- should fix https://github.com/nushell/nushell/issues/11648

# Description
this PR
- adds a test that should pass but fails
- adds `$.extra_usage` to the output of `scope modules`, fixing both the
new test and the linked issue

# User-Facing Changes
`$.extra_usage` is now a column in the output of `scope modules`

# Tests + Formatting
a new test case has been added to `correct_scope_modules_fields`

# After Submitting
2024-01-27 17:49:21 +02:00
Artemiy
092d496ff5
Plugin explicit flags (#11581)
# Description
#11492 fixed flags for builtin commands but I missed that plugins don't
use the same `has_flag` that builtins do. This PR addresses this.

Unfortunately this means that return value of `has_flag` needs to change
from `bool` to `Result<bool, ShellError>` to produce an error when
explicit value is not a boolean (just like in case of `has_flag` for
builtin commands. It is not possible to check this in
`EvaluatedCall::try_from_call` because

# User-Facing Changes
Passing explicit values to flags of plugin commands (like `--flag=true`
`--flag=false`) should work now.
BREAKING: changed return value of `EvaluatedCall::has_flag` method from
`bool` to `Result<bool, ShellError>`

# Tests + Formatting
Added tests and updated documentation and examples
2024-01-22 15:00:43 -06:00
Yash Thakur
90d65bb987
Evaluate string interpolation at parse time (#11562)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

Closes #11561

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

This PR will allow string interpolation at parse time.

Since the actual config hasn't been loaded at parse time, this uses the
`get_config()` method on `StateWorkingSet`. So file sizes and datetimes
(I think those are the only things whose string representations depend
on the config) may be formatted differently from how users have
configured things, which may come as a surprise to some. It does seem
unlikely that anyone would be formatting file sizes or date times at
parse time. Still, something to think about if/before this PR merged.

Also, I changed the `ModuleNotFound` error to include the name of the
module.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

Users will be able to do stuff like:

```nu
const x = [1 2 3]
const y = $"foo($x)" // foo[1, 2, 3]
```

The main use case is `use`-ing and `source`-ing files at parse time:

```nu
const file = "foo.nu"
use $"($file)"
```

If the module isn't found, you'll see an error like this:
```
Error: nu::parser::module_not_found

  × Module not found.
   ╭─[entry #3:1:1]
 1 │  use $"($file)"
   ·      ─────┬────
   ·           ╰── module foo.nu not found
   ╰────
  help: module files and their paths must be available before your script is run as parsing occurs before anything is evaluated
```

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

Although there's user-facing changes, there's probably no need to change
the docs since people probably already expect string interpolation to
work at parse time.

Edit: @kubouch pointed out that we'd need to document the fact that
stuff like file sizes and datetimes won't get formatted according to
user's runtime configs, so I'll make a PR to nushell.github.io after
this one
2024-01-22 09:13:48 +02:00
Eric Hodel
7071617f18
Allow plugins to receive configuration from the nushell configuration (#10955)
# Description

When nushell calls a plugin it now sends a configuration `Value` from
the nushell config under `$env.config.plugins.PLUGIN_SHORT_NAME`. This
allows plugin authors to read configuration provided by plugin users.

The `PLUGIN_SHORT_NAME` must match the registered filename after
`nu_plugin_`. If you register `target/debug/nu_plugin_config` the
`PLUGIN_NAME` will be `config` and the nushell config will loook like:

        $env.config = {
          # ...
          plugins: {
            config: [
              some
              values
            ]
          }
        }

Configuration may also use a closure which allows passing values from
`$env` to a plugin:

        $env.config = {
          # ...
          plugins: {
            config: {||
              $env.some_value
            }
          }
        }

This is a breaking change for the plugin API as the `Plugin::run()`
function now accepts a new configuration argument which is an
`&Option<Value>`. If no configuration was supplied the value is `None`.

Plugins compiled after this change should work with older nushell, and
will behave as if the configuration was not set.

Initially discussed in #10867

# User-Facing Changes

* Plugins can read configuration data stored in `$env.config.plugins`
* The plugin `CallInfo` now includes a `config` entry, existing plugins
will require updates

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- [ ] Update [Creating a plugin (in
Rust)](https://www.nushell.sh/contributor-book/plugins.html#creating-a-plugin-in-rust)
[source](https://github.com/nushell/nushell.github.io/blob/main/contributor-book/plugins.md)
- [ ] Add "Configuration" section to [Plugins
documentation](https://www.nushell.sh/contributor-book/plugins.html)
2024-01-15 16:59:47 +08:00
Marc Schreiber
41119d3f88
Fix "Char index out of bounds" Error (#11526)
# Description

The code that converts Nushell's span into LSP line and character
indices accidentally treated the span as character indices while they
are byte indices. Fixes #11522.

# User-Facing Changes

None, just a bugfix.
2024-01-11 15:24:49 -06:00
Antoine Büsch
1920ece759
fix: closure captures can also be constants (#11493)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
When evaluating a closure (in
`EvalRuntime::eval_row_condition_or_closure()`), we try to resolve the
closure's block's captures, but we only check if they're variables on
the stack. We need to also check if they are constants (see the logic in
`Stack::gather_captures()`).

fixes #10701
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-01-07 12:51:39 +02:00
tomoda
ad95e4cc27
Refactor tests (using cococo instead of ^echo) (#11479)
- related PR: #11478 

# Description

Now we can use `nu --testbin cococo` instead of `^echo` to echo messages
to stdout in tests.

But `nu` treats parameters as its own flags when parameter starts with
`-`. So `^echo --foo='bar'` still use `^echo`.

# User-Facing Changes

(none)

# Tests + Formatting

- [x] `cargo fmt --all -- --check` to check standard code formatting
(`cargo fmt --all` applies these changes)
- [x] `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used`
to check that you're using the standard code style
- [x] `cargo test --workspace` to check that all tests pass (on Windows
make sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- [x] `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library


# After Submitting

(none)
2024-01-05 11:40:56 +08:00
Yash Thakur
21b3eeed99
Allow spreading arguments to commands (#11289)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

Finishes implementing https://github.com/nushell/nushell/issues/10598,
which asks for a spread operator in lists, in records, and when calling
commands.

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

This PR will allow spreading arguments to commands (both internal and
external). It will also deprecate spreading arguments automatically when
passing to external commands.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

- Users will be able to use `...` to spread arguments to custom/builtin
commands that have rest parameters or allow unknown arguments, or to any
external command
- If a custom command doesn't have a rest parameter and it doesn't allow
unknown arguments either, the spread operator will not be allowed
- Passing lists to external commands without `...` will work for now but
will cause a deprecation warning saying that it'll stop working in 0.91
(is 2 versions enough time?)

Here's a function to help with demonstrating some behavior:
```nushell
> def foo [ a, b, c?, d?, ...rest ] { [$a $b $c $d $rest] | to nuon }
```

You can pass a list of arguments to fill in the `rest` parameter using
`...`:
```nushell
> foo 1 2 3 4 ...[5 6]
[1, 2, 3, 4, [5, 6]]
```

If you don't use `...`, the list `[5 6]` will be treated as a single
argument:

```nushell
> foo 1 2 3 4 [5 6] # Note the double [[]]
[1, 2, 3, 4, [[5, 6]]]
```

You can omit optional parameters before the spread arguments:
```nushell
> foo 1 2 3 ...[4 5] # d is omitted here
[1, 2, 3, null, [4, 5]]
```

If you have multiple lists, you can spread them all:
```nushell
> foo 1 2 3 ...[4 5] 6 7 ...[8] ...[]
[1, 2, 3, null, [4, 5, 6, 7, 8]]
```

Here's the kind of error you get when you try to spread arguments to a
command with no rest parameter:

![image](https://github.com/nushell/nushell/assets/45539777/93faceae-00eb-4e59-ac3f-17f98436e6e4)

And this is the warning you get when you pass a list to an external now
(without `...`):


![image](https://github.com/nushell/nushell/assets/45539777/d368f590-201e-49fb-8b20-68476ced415e)


# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

Added tests to cover the following cases:
- Spreading arguments to a command that doesn't have a rest parameter
(unexpected spread argument error)
- Spreading arguments to a command that doesn't have a rest parameter
*but* there's also a missing positional argument (missing positional
error)
- Spreading arguments to a command that doesn't have a rest parameter
but does allow unknown arguments, such as `exec` (allowed)
- Spreading a list literal containing arguments of the wrong type (parse
error)
- Spreading a non-list value, both to internal and external commands
- Having named arguments in the middle of rest arguments
- `explain`ing a command call that spreads its arguments

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

# Examples

Suppose you have multiple tables:
```nushell
let people = [[id name age]; [0 alice 100] [1 bob 200] [2 eve 300]]
let evil_twins = [[id name age]; [0 ecila 100] [-1 bob 200] [-2 eve 300]]
```

Maybe you often find yourself needing to merge multiple tables and want
a utility to do that. You could write a function like this:
```nushell
def merge_all [ ...tables ] { $tables | reduce { |it, acc| $acc | merge $it } }
```

Then you can use it like this:
```nushell
> merge_all ...([$people $evil_twins] | each { |$it| $it | select name age })
╭───┬───────┬─────╮
│ # │ name  │ age │
├───┼───────┼─────┤
│ 0 │ ecila │ 100 │
│ 1 │ bob   │ 200 │
│ 2 │ eve   │ 300 │
╰───┴───────┴─────╯
```

Except they had duplicate columns, so now you first want to suffix every
column with a number to tell you which table the column came from. You
can make a command for that:
```nushell
def select_and_merge [ --cols: list<string>, ...tables ] {
  let renamed_tables = $tables
    | enumerate
    | each { |it|
      $it.item | select $cols | rename ...($cols | each { |col| $col + ($it.index | into string) })
    };
  merge_all ...$renamed_tables
}
```
And call it like this:
```nushell
> select_and_merge --cols [name age] $people $evil_twins
╭───┬───────┬──────┬───────┬──────╮
│ # │ name0 │ age0 │ name1 │ age1 │
├───┼───────┼──────┼───────┼──────┤
│ 0 │ alice │  100 │ ecila │  100 │
│ 1 │ bob   │  200 │ bob   │  200 │
│ 2 │ eve   │  300 │ eve   │  300 │
╰───┴───────┴──────┴───────┴──────╯
```

---

Suppose someone's made a command to search for APT packages:

```nushell
# The main command
def search-pkgs [
    --install                   # Whether to install any packages it finds
    log_level: int              # Pretend it's a good idea to make this a required positional parameter
    exclude?: list<string>      # Packages to exclude
    repositories?: list<string> # Which repositories to look in (searches in all if not given)
    ...pkgs                     # Package names to search for
] {
  { install: $install, log_level: $log_level, exclude: ($exclude | to nuon), repositories: ($repositories | to nuon), pkgs: ($pkgs | to nuon) }
}
```

It has a lot of parameters to configure it, so you might make your own
helper commands to wrap around it for specific cases. Here's one
example:
```nushell
# Only look for packages locally
def search-pkgs-local [
    --install              # Whether to install any packages it finds
    log_level: int
    exclude?: list<string> # Packages to exclude
    ...pkgs                # Package names to search for
] {
  # All required and optional positional parameters are given
  search-pkgs --install=$install $log_level [] ["<local URI or something>"] ...$pkgs
}
```
And you can run it like this:
```nushell
> search-pkgs-local --install=false 5 ...["python2.7" "vim"]
╭──────────────┬──────────────────────────────╮
│ install      │ false                        │
│ log_level    │ 5                            │
│ exclude      │ []                           │
│ repositories │ ["<local URI or something>"] │
│ pkgs         │ ["python2.7", vim]           │
╰──────────────┴──────────────────────────────╯
```

One thing I realized when writing this was that if we decide to not
allow passing optional arguments using the spread operator, then you can
(mis?)use the spread operator to skip optional parameters. Here, I
didn't want to give `exclude` explicitly, so I used a spread operator to
pass the packages to install. Without it, I would've needed to do
`search-pkgs-local --install=false 5 [] "python2.7" "vim"` (explicitly
pass `[]` (or `null`, in the general case) to `exclude`). There are
probably more idiomatic ways to do this, but I just thought it was
something interesting.

If you're a virologist of the [xkcd](https://xkcd.com/350/) kind,
another helper command you might make is this:
```nushell
# Install any packages it finds
def live-dangerously [ ...pkgs ] {
  # One optional argument was given (exclude), while another was not (repositories)
  search-pkgs 0 [] ...$pkgs --install # Flags can go after spread arguments
}
```

Running it:
```nushell
> live-dangerously "git" "*vi*" # *vi* because I don't feel like typing out vim and neovim
╭──────────────┬─────────────╮
│ install      │ true        │
│ log_level    │ 0           │
│ exclude      │ []          │
│ repositories │ null        │
│ pkgs         │ [git, *vi*] │
╰──────────────┴─────────────╯
```

Here's an example that uses the spread operator more than once within
the same command call:
```nushell
let extras = [ chrome firefox python java git ]

def search-pkgs-curated [ ...pkgs ] {
  (search-pkgs
      1
      [emacs]
      ["example.com", "foo.com"]
      vim # A must for everyone!
      ...($pkgs | filter { |p| not ($p | str contains "*") }) # Remove packages with globs
      python # Good tool to have
      ...$extras
      --install=false
      python3) # I forget, did I already put Python in extras?
}
```

Running it:
```nushell
> search-pkgs-curated "git" "*vi*"
╭──────────────┬───────────────────────────────────────────────────────────────────╮
│ install      │ false                                                             │
│ log_level    │ 1                                                                 │
│ exclude      │ [emacs]                                                           │
│ repositories │ [example.com, foo.com]                                            │
│ pkgs         │ [vim, git, python, chrome, firefox, python, java, git, "python3"] │
╰──────────────┴───────────────────────────────────────────────────────────────────╯
```
2023-12-28 15:43:20 +08:00
Antoine Stevan
c6043eb500
improve completions of use and overlay use (#11330)
# Description
this PR is two-fold
- make `use` and `overlay use` use the same completion algorithm in
48f29b633
- list directory modules in completions of both with 402acde5c

# User-Facing Changes
i currently have the following in my `NU_LIB_DIRS`
<details>
<summary>click to see the script</summary>

```nushell
for dir in $env.NU_LIB_DIRS {
    print $dir
    print (ls $dir --short-names | select name type)
}
```
</details>

```
/home/amtoine/.local/share/nupm/modules
#┬────────name────────┬type
0│nu-git-manager      │dir
1│nu-git-manager-sugar│dir
2│nu-hooks            │dir
3│nu-scripts          │dir
4│nu-themes           │dir
5│nupm                │dir
─┴────────────────────┴────

/home/amtoine/.config/nushell/overlays
#┬──name──┬type
0│ocaml.nu│file
─┴────────┴────
```

> **Note**
> all the samples below are run from the Nushell repo, i.e. a directory
with a `toolkit.nu` module

## before the changes
- `use` would give me `["ocaml.nu", "toolkit.nu"]` 
- `overlay use` would give me `[]` 

## after the changes
both commands give me
```nushell
[
    "nupm/",
    "ocaml.nu",
    "toolkit.nu",
    "nu-scripts/",
    "nu-git-manager/",
    "nu-git-manager-sugar/",
]
```

# Tests + Formatting
- adds a new `directory_completion/mod.nu` to the completion fixtures
- make sure `source-env`, `use` and `overlay-use` are all tested in the
_dotnu_ test
- fix all the other tests that use completions in the fixtures directory
for completions

# After Submitting
2023-12-19 17:14:34 +08:00
Antoine Stevan
156232fe08
disable directory submodule auto export (#11157)
should
- close https://github.com/nushell/nushell/issues/11133

# Description
to allow more freedom when writing complex modules, we are disabling the
auto-export of director modules.

the change was as simple as removing the crawling of files and modules
next to any `mod.nu` and update the standard library.

# User-Facing Changes
users will have to explicitely use `export module <mod>` to define
submodules and `export use <mod> <cmd>` to re-export definitions, e.g.
```nushell
# my-module/mod.nu
export module foo.nu     # export a submodule
export use bar.nu bar-1  # re-export an internal command

export def top [] {
    print "`top` from `mod.nu`"
}
```
```nushell
# my-module/foo.nu
export def "foo-1" [] {
    print "`foo-1` from `lib/foo.nu`"
}

export def "foo-2" [] {
    print "`foo-2` from `lib/foo.nu`"
}
```
```nushell
# my-module/bar.nu
export def "bar-1" [] {
    print "`bar-1` from `lib/bar.nu`"
}
```

# Tests + Formatting
i had to add `export module` calls in the `tests/modules/samples/spam`
directory module and allow the `not_allowed` module to not give an
error, it is just empty, which is fine.

# After Submitting
- mention in the release note
- update the following repos
```
#┬─────name─────┬version┬─type─┬─────────repo─────────
0│nu-git-manager│0.4.0  │module│amtoine/nu-git-manager
1│nu-scripts    │0.1.0  │module│amtoine/scripts       
2│nu-zellij     │0.1.0  │module│amtoine/zellij-layouts
3│nu-scripts    │0.1.0  │module│nushell/nu_scripts    
4│nupm          │0.1.0  │module│nushell/nupm          
─┴──────────────┴───────┴──────┴──────────────────────
```
2023-12-15 12:37:55 +01:00
nibon7
d1390ac95b
Fix overlay_use_main_not_exported hanging when an external spam command exists (#11261)
# Description
The `spam` command is provided by
[opensp](https://openjade.sourceforge.net/) which causes
`overlay_use_main_not_exported` to hang. `opensp` is pulled by
`gnome-control-center` on my system.

[opensp package
list](https://archlinux.org/packages/extra/x86_64/opensp/files/)

```
opensp 1.5.2-10 File List

Package has 224 files and 14 directories.

[Back to Package](https://archlinux.org/packages/extra/x86_64/opensp/)

    usr/
    usr/bin/
    usr/bin/nsgmls
    usr/bin/onsgmls
    usr/bin/osgmlnorm
    usr/bin/ospam
    usr/bin/ospcat
    usr/bin/ospent
    usr/bin/osx
    usr/bin/sgml2xml
    usr/bin/sgmlnorm
    usr/bin/spam
    ...snip...
```

`cargo test` output
```
...snip...
test shell::pipeline::commands::internal::unlet_variable_in_parent_scope ... ok
test shell::pipeline::commands::internal::unlet_env_variable ... ok
test shell::pipeline::doesnt_break_on_utf8 ... ok
test shell::run_export_extern ... ok
test shell::run_script_that_looks_like_module ... ok
test shell::pipeline::commands::internal::can_process_one_row_from_internal_and_pipes_it_to_stdin_of_external ... ok
test shell::pipeline::commands::internal::variable_scoping::access_variables_in_scopes ... ok
test shell::run_in_login_mode ... ok
test shell::run_in_interactive_mode ... ok
test shell::run_in_noninteractive_mode ... ok
test shell::run_in_not_login_mode ... ok
test shell::pipeline::commands::internal::subexpression_properly_redirects ... ok
test shell::pipeline::commands::internal::subexpression_handles_dot ... ok
test shell::pipeline::commands::internal::takes_rows_of_nu_value_strings_and_pipes_it_to_stdin_of_external ... ok
test overlays::overlay_use_main_not_exported has been running for over 60 seconds
```
# User-Facing Changes
N/A

# Tests + Formatting
Make sure you've run and fixed any issues with these commands:

- [x] `cargo fmt --all -- --check` to check standard code formatting
(`cargo fmt --all` applies these changes)
- [x] `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used`
to check that you're using the standard code style
- [x] `cargo test --workspace` to check that all tests pass (on Windows
make sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- [x] `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library



# After Submitting
N/A
2023-12-08 06:08:38 -06:00
Renan Ribeiro
54d73748e4
Remove file I/O from tests that don't need it (#11182)
# Description

This PR implements modifications to command tests that write unnecessary
json and csv to disk then load it with open, by using nuon literals
instead.

- Fixes #7189



# User-Facing Changes
None

# Tests + Formatting
This only affects existing tests, which still pass.
2023-11-29 23:21:34 +01:00
Andrej Kolchin
0e1322e6d6
Forbid reserved variable names for function arguments (#11169)
Works for all arguments and flags. Because the signature parsing doesn't
give the spans, it is flags the entire signature.

Also added a constant with reserved variable names.

Fix #11158.
2023-11-29 18:29:07 +01:00
Marc Schreiber
c110ddff66
Implement LSP Text Document Synchronization (#10941) 2023-11-15 17:35:48 -06:00
Eric Hodel
81d00f71a9
Show plugin extra usage and search terms (#10952)
# Description

The `PluginSignature` type supports extra usage but this was not
available in `plugin_name --help`. It also supports search terms but
these did not appear in `help commands`

New behavior show below is the "Extra usage for nu-example-1" line and
the "Search terms:" line

```
❯ nu-example-1 --help
PluginSignature test 1 for plugin. Returns Value::Nothing

Extra usage for nu-example-1

Search terms: example

Usage:
  > nu-example-1 {flags} <a> <b> (opt) ...(rest)

Flags:
  -h, --help - Display the help message for this command
  -f, --flag - a flag for the signature
  -n, --named <String> - named string

Parameters:
  a <int>: required integer value
  b <string>: required string value
  opt <int>: Optional number (optional)
  ...rest <string>: rest value string

Examples:
  running example with an int value and string value
  > nu-example-1 3 bb
```

Search terms are also available in `help commands`:

```
❯ help commands | where name == "nu-example-1" | select name search_terms
╭──────────────┬──────────────╮
│     name     │ search_terms │
├──────────────┼──────────────┤
│ nu-example-1 │ example      │
╰──────────────┴──────────────╯
```

# User-Facing Changes

Users can now see plugin extra usage and search terms 

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

N/A
2023-11-04 15:12:58 -05:00
Marc Schreiber
0ca8fcf58c
Integrated Language Server (#10723)
# Description

This commit integrates a language server into nushell so that IDEs don't
have to convert CLI option back and forth.

- fixes https://github.com/nushell/vscode-nushell-lang/issues/117
- fixes https://github.com/jokeyrhyme/nuls/issues/8

Tracking tasks


- [x]
[textDocument/hover](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_hover)
-> `nu --ide-hover`
- [x]
[textDocument/completion](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_completion)
-> `nu --ide-complete`
- [x]
[textDocument/definition](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_definition)
-> `nu --ide-goto-def`
- ~~[ ]
[textDocument/didChange](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_didChange),
[textDocument/didClose](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_didClose),
and
[textDocument/didOpen](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_didOpen)~~
(will be done in a follow-up PR)
- ~~[ ]
[textDocument/inlayHint](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_inlayHint)
-> `nu --ide-check`~~ (will be done in a follow-up PR)
- ~~[ ]
[textDocument/publishDiagnostics](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_publishDiagnostics)
-> `nu --ide-check`~~ (will be done in a follow-up PR)
- ~~[ ]
[workspace/configuration](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#workspace_configuration)~~
(will be done in a follow-up PR)
- ~~[ ]
[workspace/didChangeConfiguration](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#workspace_didChangeConfiguration)~~
(will be done in a follow-up PR)


# User-Facing Changes

The command line options `--lsp` will start a LSP server.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2023-11-02 10:18:57 -05:00
Stefan Holderbach
0569a9c92e
Disallow duplicated columns in table literals (#10875)
# Description
Pretty much all operations/commands in Nushell assume that the column
names/keys in a record and thus also in a table (which consists of a
list of records) are unique.
Access through a string-like cell path should refer to a single column
or key/value pair and our output through `table` will only show the last
mention of a repeated column name.

```nu
[[a a]; [1 2]]
╭─#─┬─a─╮
│ 0 │ 2 │
╰───┴───╯
```

While the record parsing already either errors with the
`ShellError::ColumnDefinedTwice` or silently overwrites the first
occurence with the second occurence, the table literal syntax `[[header
columns]; [val1 val2]]` currently still allowed the creation of tables
(and internally records with more than one entry with the same name.

This is not only confusing, but also breaks some assumptions around how
we can efficiently perform operations or in the past lead to outright
bugs (e.g. #8431 fixed by #8446).

This PR proposes to make this an error.
After this change another hole which allowed the construction of records
with non-unique column names will be plugged.

## Parts
- Fix `SE::ColumnDefinedTwice` error code
- Remove previous tests permitting duplicate columns
- Deny duplicate column in table literal eval
- Deny duplicate column in const eval
- Deny duplicate column in `from nuon`

# User-Facing Changes
`[[a a]; [1 2]]` will now return an error:

```
Error: nu:🐚:column_defined_twice

  × Record field or table column used twice
   ╭─[entry #2:1:1]
 1 │ [[a a]; [1 2]]
   ·   ┬ ┬
   ·   │ ╰── field redefined here
   ·   ╰── field first defined here
   ╰────
```

this may under rare circumstances block code from evaluating.

Furthermore this makes some NUON files invalid if they previously
contained tables with repeated column names.

# Tests + Formatting
Added tests for each of the different evaluation paths that materialize
tables.
2023-11-01 21:25:35 +01:00
Himadri Bhattacharjee
7162d4d9aa
Escape path that could be a flag (#10721)
# Description
Files that begin with dashes can be ambiguous when passed to commands
like `ls`. For example if there exists a file `--help`, it might be
considered a flag if not properly escaped. This PR escapes any file that
begins with a dash.

# User-Facing Changes

Files beginning with dashes will be escaped.

# Tests + Formatting

Tests are added.
2023-10-18 23:02:11 +02:00
Darren Schroeder
814a5caf9a
change canonicalize test use a more deeply rooted folder (#10685)
# Description

This PR changes the `canonicalize_ndots` tests (renames to
canonicalize_ndots2) so that when it's checking for
`canonicalize_with("...", cwd)` it guarantees it begins in a more deeply
nested folder. I was having problems because my new DevDrive is on
D:\nushell and it can't do `cd ...` from that folder.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2023-10-11 11:27:25 -05:00
Hofer-Julian
8a77d1ed92
Let run_in_login_mode succeed even with broken local config (#10622)
I wondered why this test failed for me.
Turns out my config file is not compatible with current main, but the
error message was useless. I've added `--no-config-file`
2023-10-10 14:12:19 +02:00
Jakub Žádník
4efccb2b1c
Fix parsing of signature inp/out types in predecls (#10642)
# Description
Fixes https://github.com/nushell/nushell/issues/10605 (again).

The loop looking for `[` to determine signature position didn't stop
early enough, so it thought the second `[` denoting the inp/out types
marks the beginning of the signature.

# User-Facing Changes

# Tests + Formatting
adds a new `predecl_signature_multiple_inp_out_types` test

# After Submitting
2023-10-08 12:58:26 +02:00
Jakub Žádník
67b5e1bde9
Fix wrong parsing of signatures in predecl scan (#10637) 2023-10-07 16:42:09 +03:00
Himadri Bhattacharjee
d064d187ab
fix: complete paths surrounded by quotes or backticks (#10600)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->
Fixes #10586

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Any partial path that begins with or is surrounded by a quote or
backtick will be tab completed. The completed result would be surrounded
by backticks unconditionally.


![output](https://github.com/nushell/nushell/assets/107522312/13e01104-18a1-4483-b010-79985294748b)

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

See above.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

Formatted and added test cases.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2023-10-07 05:45:30 +13:00
joergsch
e8da57b05e
Correct line folding in from ics/from vcf (#10577)
# Description
Potential fix for #10398

datatracker.ietf.org/doc/html/rfc5545#section-3.1
datatracker.ietf.org/doc/html/rfc6350#section-3.2

---------

Co-authored-by: Joerg <joerg@schuetter.org>
2023-10-03 19:11:16 +02:00
Jakub Žádník
eb6870cab5
Add --env and --wrapped flags to def (#10566) 2023-10-02 21:13:31 +03:00
Himadri Bhattacharjee
5c15a4dd6e
Fish-like completions for nested directories (#10543)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #5683

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

This PR allows tab completion for nested directories while only
specifying a part of the directory names. To illustrate this, if I type
`tar/de/inc` and hit tab, it autocompletes to
`./target/debug/incremental`.



# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Nested paths can be tab completed by typing lesser characters.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Tests cases are added.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2023-10-03 06:44:51 +13:00
WindSoilder
9c52b93975
allow early return outside of main (#10514)
# Description
Fixes: #9792

When evaluating file, we need to allow early return if we evaluate
script file first.
2023-09-28 18:49:42 +02:00
Antoine Stevan
6c026242d4
remove the $nothing variable (#10478)
related to 
- https://github.com/nushell/nushell/pull/9973
- https://github.com/nushell/nushell/pull/9918

thanks to @jntrnr and their super useful tips on this PR, i learned
about the parser + evaluation, so 🙏

# Description
because we already have `null` as the value of the type `nothing` and as
a followup to the two other attempts of mine, i propose to remove the
redundant `$nothing` built-in variable 😋

this PR is the first step, deprecating `$nothing`.
a followup PR will remove it altogether and wait for 0.87 👍 

⚙️ **details**: a new `NOTHING_VARIABLE_ID = 3` has been added,
parsing `$nothing` will create it, finally a `Value::Nothing` will be
produced and a warning will be reported.

this PR already fixes the `toolkit.nu` module so that it does not throw
a bunch of warnings each time 👌

# User-Facing Changes
`$nothing` is now deprecated and will be removed in 0.87
```nushell
> $nothing
Error:   × Deprecated variable
   ╭─[entry #1:1:1]
 1 │ $nothing
   · ────┬───
   ·     ╰── `$nothing` is deprecated and will be removed in 0.87.
   ╰────
  help: Use `null` instead
```

# Tests + Formatting
tests have been updated, especially
- `nothing_fails_string`
- `nothing_fails_int`
which use a variable called `nil` now to make sure `nothing` does not
support cell paths 👍

# After Submitting
classic deprecation mention 👍
2023-09-26 18:49:28 +02:00
Artemiy
e96039fb1b
Fix default argument value type checking (#10460)
# Description
Fix type checking in arguments default values not adhering to subtyping
rules
Currently following examples produce a parse error:
```nu
def test [ --qwe: record<a: int> = {a: 1 b: 1} ] { }
def test [ --qwe: list<any> = [ 1 2 3 ] ] { }
```
despite types matching. Type equality check is replaced with subtyping
check and everything parses fine:
# User-Facing Changes
Default values of flag arguments type checking behavior is in line with
`let` statements
2023-09-24 11:30:58 +02:00
Artemiy
65e2733571
Allow complex types in input/output and let (#10405)
# Description
This PR fixes #9702 on the side of parse. I.e. input/output types in
signature and type annotations in `let` now should correctly parse with
type annotations that contain commas and spaces:

![image](https://github.com/nushell/nushell/assets/17511668/babc0a69-5cb3-46c2-98ef-6da69ee3d3be)

# User-Facing Changes
Return values and let type annotations now can contain stuff like
`table<a: int b: record<c: string d: datetime>>` e.t.c
2023-09-24 11:01:21 +02:00
Stefan Holderbach
0c7a8e3634
Remove dead BSON related tests (#10458)
Those tests where behind a feature flag `bson` that is never defined in
`Cargo.toml`

Remove them.
2023-09-21 20:01:49 +02:00
Stefan Holderbach
bbf0b45c59
Update internal use of decimal to float (#10333)
# Description
We made the decision that our floating point type should be referred to
as `float` over `decimal`.
Commands were updated by #9979 and #10320

Now make the internal codebase consistent in referring to this data type
as `float`.

Work for #10332

# User-Facing Changes

`decimal` has been removed as a type name/symbol. 

Instead of 
```nushell
def foo [bar: decimal] decimal -> decimal {}
```
use 
```nushell
def foo [bar: float] float -> float {}
```

Potential effect of `SyntaxShape`'s `Display` implementation now also
referring to `float` instead of `decimal`

# Details
- Rename `SyntaxShape::Decimal` to `Float`
- Update `Display for SyntaxShape` to `float`
- Update error message + fn name in dataframe code
- Fix docs in command examples
- Rename tests that are float specific
- Update doccomment on `SyntaxShape`
- Update comment in script

# Tests + Formatting
Updates the names of some tests
2023-09-13 23:53:55 +02:00
Stefan Holderbach
7f39609d9a
Remove python-like string multiplication (#10293)
# Description
Currently we support "multiplication" of strings, resulting in a terse
way to repeat a particular string.
This can have unintended side effects when dealing with mixed data (e.g.
after parsing data that is not all numbers).
Furthermore as we frequently fall-back to strings while parsing source
code, this introduced a runaway edge case in const evaluation (#10212)

Work for #10233

## Details
- Remove python-like string multiplication.
- Workaround for indentation
  - This should probably be addressed with a purpose built command
- Remove special const-eval error test

# User-Facing Changes
**Major breaking change!**
`"string" * 42` will stop working. (This was used for example in the
stdlib)

We should bless a good alternative before landing this

---------

Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
2023-09-13 19:52:04 +12:00