Commit graph

5629 commits

Author SHA1 Message Date
Devyn Cairns
d126793290
Add plugin error propagation on write/flush (#12670)
# Description
Yet another attempt to fix the `stress_internals::test_wrong_version()`
test...

This time I think it's probably because we are getting a broken pipe
write error when we try to send `Hello` or perhaps something after it,
because the pipe has already been closed by the reader when it saw the
invalid version. In that case, an error should be available in state. It
probably makes more sense to send that back to the user rather than an
unhelpful I/O error.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-26 06:23:58 -05:00
Devyn Cairns
adf38c7c76
Msgpack commands (#12664)
# Description

I thought about bringing `nu_plugin_msgpack` in, but that is MPL with a
clause that prevents other licenses, so rather than adapt that code I
decided to take a crack at just doing it straight from `rmp` to `Value`
without any `rmpv` in the middle. It seems like it's probably faster,
though I can't say for sure how much with the plugin overhead.

@IanManske I started on a `Read` implementation for `RawStream` but just
specialized to `from msgpack` here, but I'm thinking after release maybe
we can polish it up and make it a real one. It works!

# User-Facing Changes
New commands:

- `from msgpack`
- `from msgpackz`
- `to msgpack`
- `to msgpackz`

# Tests + Formatting
Pretty thorough tests added for the format deserialization, with a
roundtrip for serialization. Some example tests too for both `from
msgpack` and `to msgpack`.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`


# After Submitting
- [ ] update release notes
2024-04-26 06:23:16 -05:00
Ian Manske
79ebf0c0a9
Fix an into bits example (#12668)
# Description
One example for `into bits` says it uses binary value when it actually
uses a filesize. This lead to issue #11412, but I never got around to
fixing the example until this PR.
2024-04-25 19:38:28 -05:00
Ian Manske
f234a0ea33
each signature fix (#12666)
# Description
Removes the second `Int` parameter from the closure in the signature of
`each`. This parameter doesn't exist / isn't supported.
2024-04-25 22:47:30 +02:00
Andrej Kolchin
2466a39574
Fix example wording in seq date (#12665)
# Description

The previous messages said that the command printed dates separated by
newlines. But the current iteration of `seq date` returns a list.

# User-Facing Changes

Minor wording edit.

---------

Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2024-04-25 22:12:42 +02:00
Ian Manske
1239e86320
Remove deprecated flags on run-external (#12659)
# Description
Removes the `run-external` flags that were deprecated in 0.92.0 with
#11934.
2024-04-25 14:17:21 +00:00
Ian Manske
5b0546cfce
Remove deprecated flags on commandline (#12658)
# Description
Removes the `commandline` flags and API that was deprecated in 0.91.0
with #11877.

# User-Facing Changes
Users need to migrate to the new `commandline` subcommands introduced in
0.91.0.
2024-04-25 14:16:42 +00:00
Ian Manske
b6d765a2d8
nu-cmd-lang cleanup (#12609)
# Description
This PR does miscellaneous cleanup in some of the commands from
`nu-cmd-lang`.

# User-Facing Changes
None.

# After Submitting
Cleanup the other commands in `nu-cmd-lang`.
2024-04-25 14:16:12 +00:00
Ian Manske
530162b4c4
Lazy record deprecation (#12656)
# Description
In this week's nushell meeting, we decided to go ahead with #12622 and
remove lazy records in 0.94.0. For 0.93.0, we will only deprecate `lazy
make`, and so this PR makes `lazy make` print a deprecation warning.

# User-Facing Changes
None, besides the deprecation warning.

# After Submitting
Remove lazy records.
2024-04-25 07:05:24 +08:00
Devyn Cairns
25cbcb511d
Rename plugin cache file ⇒ plugin registry file (#12634)
# Description
So far this seems like the winner of my poll on what the name should be.
I'll take this off draft once the poll expires, if this is indeed the
winner.
2024-04-24 17:40:39 -05:00
Ian Manske
9996e4a1f8
Shrink the size of Expr (#12610)
# Description
Continuing from #12568, this PR further reduces the size of `Expr` from
64 to 40 bytes. It also reduces `Expression` from 128 to 96 bytes and
`Type` from 32 to 24 bytes.

This was accomplished by:
- for `Expr` with multiple fields (e.g., `Expr::Thing(A, B, C)`),
merging the fields into new AST struct types and then boxing this struct
(e.g. `Expr::Thing(Box<ABC>)`).
- replacing `Vec<T>` with `Box<[T]>` in multiple places. `Expr`s and
`Expression`s should rarely be mutated, if at all, so this optimization
makes sense.

By reducing the size of these types, I didn't notice a large performance
improvement (at least compared to #12568). But this PR does reduce the
memory usage of nushell. My config is somewhat light so I only noticed a
difference of 1.4MiB (38.9MiB vs 37.5MiB).

---------

Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
2024-04-24 15:46:35 +00:00
Devyn Cairns
c52884b3c8
Fix (and test) for a deadlock that can happen while waiting for protocol info (#12633)
# Description

The local socket PR introduced a `Waitable` type, which could either
hold a value or be waited on until a value is available. Unlike a
channel, it would always return that value once set.

However, one issue with this design was that there was no way to detect
whether a value would ever be written. This splits the writer into a
different type `WaitableMut`, so that when it is dropped, waiting
threads can fail (because they'll never get a value).

# Tests + Formatting

A test has been added to `stress_internals` to make sure this fails in
the right way.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-24 08:44:04 -05:00
Devyn Cairns
0f645b3bb6
Futher improve messages related to error propagation on plugin calls (#12646)
# Description
Trying to give as much context as possible. Now there should be a
spanned error with the call span if possible, and the propagated error
as an inner error if there was one in every case.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-24 08:39:04 -05:00
dependabot[bot]
9d65c47313
Bump rust-ini from 0.20.0 to 0.21.0 (#12637) 2024-04-24 11:55:20 +00:00
Devyn Cairns
b576123b0a
Accept filenames in other plugin management commands (#12639)
# Description

This allows the following commands to all accept a filename instead of a
plugin name:

- `plugin use`
- `plugin rm`
- `plugin stop`

Slightly complicated because of the need to also check against
`NU_PLUGIN_DIRS`, but I also fixed some issues with that at the same
time

Requested by @fdncred

# User-Facing Changes

The new commands are updated as described.

# Tests + Formatting

Tests for `NU_PLUGIN_DIRS` handling also made more robust.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- [ ] Double check new docs to make sure they describe this capability
2024-04-24 06:28:45 -05:00
Devyn Cairns
c9bc0c7d3e
Fix error message propagation on plugin call failure (#12632)
# Description

This should fix the sometimes failing wrong version test for
stress_internals.

The plugin interface state stores an error if some kind of critical
error happened, and this error should be propagated to any future
operations on the interface, but this wasn't being propagated to plugin
calls that were already waiting.

During plugin registration, the wrong version error needs to be received
as a response to the `get_signature()` to show up properly, but this
would only happen if `get_signature()` started after the `Hello` was
already received and processed. That would be a race condition, which
this commit solves.

cc @sholderbach - this should fix the CI issue

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-23 17:30:51 -05:00
Devyn Cairns
1f4131532d
Deprecate register and add plugin use (#12607)
# Description

Adds a new keyword, `plugin use`. Unlike `register`, this merely loads
the signatures from the plugin cache file. The file is configurable with
the `--plugin-config` option either to `nu` or to `plugin use` itself,
just like the other `plugin` family of commands. At the REPL, one might
do this to replace `register`:

```nushell
> plugin add ~/.cargo/bin/nu_plugin_foo
> plugin use foo
```

This will not work in a script, because `plugin use` is a keyword and
`plugin add` does not evaluate at parse time (intentionally). This means
we no longer run random binaries during parse.

The `--plugins` option has been added to allow running `nu` with certain
plugins in one step. This is used especially for the `nu_with_plugins!`
test macro, but I'd imagine is generally useful. The only weird quirk is
that it has to be a list, and we don't really do this for any of our
other CLI args at the moment.

`register` now prints a deprecation parse warning.

This should fix #11923, as we now have a complete alternative to
`register`.

# User-Facing Changes

- Add `plugin use` command
- Deprecate `register`
- Add `--plugins` option to `nu` to replace a common use of `register`

# Tests + Formatting

I think I've tested it thoroughly enough and every existing test passes.
Testing nu CLI options and alternate config files is a little hairy and
I wish there were some more generic helpers for this, so this will go on
my TODO list for refactoring.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- [ ] Update plugins sections of book
- [ ] Release notes
2024-04-23 06:37:50 -05:00
Devyn Cairns
5c7f7883c8
Add ErrSpan extension trait for Result (#12626)
# Description
This adds an extension trait to `Result` that wraps errors in `Spanned`,
saving the effort of calling `.map_err(|err| err.into_spanned(span))`
every time. This will hopefully make it even more likely that someone
will want to use a spanned `io::Error` and make it easier to remove the
impl for `From<io::Error> for ShellError` because that doesn't have span
information.
2024-04-23 10:39:55 +02:00
Wind
b0acc1d890
Avoid panic when pipe a variable to a custom command which have recursive call (#12491)
# Description
Fixes: #11351

And comment here is also fixed:
https://github.com/nushell/nushell/issues/11351#issuecomment-1996191537

The panic can happened if we pipe a variable to a custom command which
recursively called itself inside another block.

TBH, I think I figure out how it works to panic, but I'm not sure if
there is a potention issue if nushell don't mutate a block in such case.

# User-Facing Changes
Nan

# Tests + Formatting
Done

# After Submitting
Done

---------

Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
2024-04-23 06:10:35 +08:00
Ian Manske
797a90520c
Improve the "input and output are the same file" error text (#12619)
# Description
Continuing from #12601, this PR improves the error message help text and
adds an example to `collect`.
2024-04-22 09:00:38 -05:00
Ian Manske
bae6d694ca
Refactor using ClosureEval types (#12541)
# Description
Adds two new types in `nu-engine` for evaluating closures: `ClosureEval`
and `ClosureEvalOnce`. This removed some duplicate code and centralizes
our logic for setting up, running, and cleaning up closures. For
example, in the future if we are able to reduce the cloning necessary to
run a closure, then we only have to change the code related to these
types.

`ClosureEval` and `ClosureEvalOnce` are designed with a builder API.
`ClosureEval` is used to run a closure multiple times whereas
`ClosureEvalOnce` is used for a one-shot closure.

# User-Facing Changes
Should be none, unless I messed up one of the command migrations.
Actually, this will fix any unreported environment bugs for commands
that didn't reset the env after running a closure.
2024-04-22 14:15:09 +08:00
Ian Manske
83720a9f30
Make the same file error more likely to appear (#12601)
# Description
When saving to a file we currently try to check if the data source in
the pipeline metadata is the same as the file we are saving to. If so,
we create an error, since reading and writing to a file at the same time
is currently not supported/handled gracefully. However, there are still
a few instances where this error is not properly triggered, and so this
PR attempts to reduce these cases. Inspired by #12599.

# Tests + Formatting
Added a few tests.

# After Submitting
Some commands still do not properly preserve metadata (e.g., `str trim`)
and so prevent us from detecting this error.
2024-04-22 01:12:13 +00:00
Jack Wright
a60381a932
Added commands for working with the plugin cache. (#12576)
# Description
This pull request provides three new commands:
`polars store-ls` - moved from `polars ls`. It provides the list of all
object stored in the plugin cache
`polars store-rm` - deletes a cached object
`polars store-get` - gets an object from the cache. 

The addition of `polars store-get` required adding a reference_count to
cached entries. `polars get` is the only command that will increment
this value. `polars rm` will remove the value despite it's count. Calls
to PolarsPlugin::custom_value_dropped will decrement the value.

The prefix store- was chosen due to there already being a `polars cache`
command. These commands were not made sub-commands as there isn't a way
to display help for sub commands in plugins (e.g. `polars store`
displaying help) and I felt the store- seemed fine anyways.

The output of `polars store-ls` now shows the reference count for each
object.

# User-Facing Changes
polars ls has now moved to polars store-ls

---------

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-04-21 19:43:43 -05:00
Devyn Cairns
8b7696f4c1
stress_internals: exit(1) on io error (#12612)
# Description

The `stress_internals` tests can fail sometimes, but usually not on the
CI, because Nushell exits while the plugin is still trying to read or
maybe write something, leading to a broken pipe.

`nu-plugin` already exits with 1 without printing a message on a
protocol-level I/O error, so this just doing the same thing.

I think there's probably a way to correct the plugin handling so that we
wait for plugins to shut down before exiting and this doesn't happen,
but this is the quick fix in the meantime.
2024-04-21 23:48:09 +02:00
Antoine Stevan
a900166e27
fix typo in the documentation of nuon::ToStyle (#12608)
follow-up to
- https://github.com/nushell/nushell/pull/12591

cc/ @fdncred 

# Description
there was a typo in the doc of `nuon::ToStyle`.

# User-Facing Changes

# Tests + Formatting

# After Submitting
2024-04-21 10:53:53 -05:00
Devyn Cairns
2595f31541
Overhaul the plugin cache file with a new msgpack+brotli format (#12579)
# Description

- Plugin signatures are now saved to `plugin.msgpackz`, which is
brotli-compressed MessagePack.
- The file is updated incrementally, rather than writing all plugin
commands in the engine every time.
- The file always contains the result of the `Signature` call to the
plugin, even if commands were removed.
- Invalid data for a particular plugin just causes an error to be
reported, but the rest of the plugins can still be parsed

# User-Facing Changes

- The plugin file has a different filename, and it's not a nushell
script.
- The default `plugin.nu` file will be automatically migrated the first
time, but not other plugin config files.
- We don't currently provide any utilities that could help edit this
file, beyond `plugin add` and `plugin rm`
  - `from msgpackz`, `to msgpackz` could also help
- New commands: `plugin add`, `plugin rm`

# Tests + Formatting

Tests added for the format and for the invalid handling.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- [ ] Check for documentation changes
- [ ] Definitely needs release notes
2024-04-21 07:36:26 -05:00
Stefan Holderbach
6cba7c6b40
Small refactor in cal (#12604)
Removed some unnecessary code found when looking at #12597

- **Simplify day of the week selection**
- **Use `Record` directly instead of `IndexMap`**
2024-04-21 10:41:26 +02:00
Ian Manske
3b1d405b96
Remove the Value::Block case (#12582)
# Description
`Value` describes the types of first-class values that users and scripts
can create, manipulate, pass around, and store. However, `Block`s are
not first-class values in the language, so this PR removes it from
`Value`. This removes some unnecessary code, and this change should be
invisible to the user except for the change to `scope modules` described
below.

# User-Facing Changes
Breaking change: the output of `scope modules` was changed so that
`env_block` is now `has_env_block` which is a boolean value instead of a
`Block`.

# After Submitting
Update the language guide possibly.
2024-04-21 07:03:33 +02:00
Maxim Uvarov
5fd34320e9
add search_term "str extract" to parse command (#12600)
# Description

For a long time, I was searching for the `str extract` command to
extract regexes from strings. I often painfully used `str replace -r
'(.*)(pattern_to_find)(.*)' '$2'` for such purposes.
Only this morning did I realize that `parse` is what I needed for so
many times, which I had only used for parsing data in tables.
2024-04-21 07:01:42 +02:00
woosaaahh
5e52bd77e0
Change cal --week-start examples + error message (#12597)
This PR fixes the example for `cal --week-start` and adds the list of
expected values to the error message.
2024-04-20 22:39:46 +02:00
Poliorcetics
47867a58df
Ab/version details (#12593)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->
Closes #12561

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Add version details in `version`'s output. The intended use is for
third-party tools to be able to quickly check version numbers without
having to the parsing from `(version).version` or `$env.NU_VERSION`.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

This adds 4 new values to the record from `version`:

```
...
│ major              │ 0                                               │
│ minor              │ 92                                              │
│ patch              │ 3                                               │
│ pre                │ a-value                                         │
...
```

`pre` is optional and won't be present most of the time I think.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

I ran the new command using `cargo run -- -c version`:

```
╭────────────────────┬─────────────────────────────────────────────────╮
│ version            │ 0.92.3                                          │
│ major              │ 0                                               │
│ minor              │ 92                                              │
│ patch              │ 3                                               │
│ branch             │                                                 │
│ commit_hash        │                                                 │
│ build_os           │ macos-aarch64                                   │
│ build_target       │ aarch64-apple-darwin                            │
│ rust_version       │ rustc 1.77.2 (25ef9e3d8 2024-04-09)             │
│ rust_channel       │ 1.77.2-aarch64-apple-darwin                     │
│ cargo_version      │ cargo 1.77.2 (e52e36006 2024-03-26)             │
│ build_time         │ 2024-04-20 15:09:36 +02:00                      │
│ build_rust_channel │ release                                         │
│ allocator          │ mimalloc                                        │
│ features           │ default, sqlite, system-clipboard, trash, which │
│ installed_plugins  │                                                 │
╰────────────────────┴─────────────────────────────────────────────────╯
```

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

After this is merged I would like to write docs somewhere for scripts
writer to advise using these members instead of the string values. Where
should I put said docs ?
2024-04-20 10:01:51 -05:00
Darren Schroeder
b274ec19fd
improve nu --lsp command tooltips (#12589)
# Description

This PR improves the `nu --lsp` tooltips by using nu code blocks around
the examples and a few other places.

This is what it looks like in Zed.
![Screenshot 2024-04-19 at 8 20
53 PM](https://github.com/nushell/nushell/assets/343840/20d51dcc-f3b2-4f2b-9d43-5817dd3913df)

Here it is in Helix.

![image](https://github.com/nushell/nushell/assets/343840/a9e7d6b9-cd21-4a5a-9c88-9af17a2b2363)

This coloring is far from perfect, but it's what the tree-sitter-nu
queries generate.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-04-20 07:16:38 -05:00
Antoine Stevan
be5ed3290c
add "to nuon" enumeration of possible styles (#12591)
# Description
in order to change the style of the _serialized_ NUON data,
`nuon::to_nuon` takes three mutually exclusive arguments, `raw: bool`,
`tabs: Option<usize>` and `indent: Option<usize>` 🤔
this begs to use an enumeration with all possible alternatives, right?

this PR changes the signature of `nuon::to_nuon` to use `nuon::ToStyle`
which has three variants
- `Raw`: no newlines
- `Tabs(n: usize)`: newlines and `n` tabulations as indent
- `Spaces(n: usize)`: newlines and `n` spaces as indent

# User-Facing Changes
the signature of `nuon::to_nuon` changes from
```rust
to_nuon(
    input: &Value,
    raw: bool,
    tabs: Option<usize>,
    indent: Option<usize>,
    span: Option<Span>,
) -> Result<String, ShellError>
```
to
```rust
to_nuon(
    input: &Value,
    style: ToStyle,
    span: Option<Span>
) -> Result<String, ShellError>
```

# Tests + Formatting

# After Submitting
2024-04-20 11:40:52 +02:00
Wind
187b87c61c
Don't allow skip on external stream (#12559)
# Description
Close: #12514

# User-Facing Changes
`^ls | skip 1` will raise an error
```nushell
❯ ^ls | skip 1
Error: nu:🐚:only_supports_this_input_type

  × Input type not supported.
   ╭─[entry #1:1:2]
 1 │ ^ls | skip 1
   ·  ─┬   ──┬─
   ·   │     ╰── only list, binary or range input data is supported
   ·   ╰── input type: raw data
   ╰────
```

# Tests + Formatting
Sorry I can't add it because of the issue:
https://github.com/nushell/nushell/issues/12558

# After Submitting
Nan
2024-04-19 14:54:59 +00:00
Devyn Cairns
f2169c8022
Switch plugin msgpack protocol to named format (#12580)
# Description
In conflict with the documentation, the msgpack serializer for plugins
is actually using the compact format, which doesn't name struct fields,
and is instead dependent on their ordering, rendering them as tuples.
This is not a good idea for a robust protocol even if it makes the
serialization and deserialization faster.

I expect this to have some impact on performance, but I think the
robustness is probably worth it.

Deserialization always accepts either format, so this shouldn't cause
too many incompatibilities.

# User-Facing Changes
This does technically change the protocol, but it makes it reflect the
documentation. It shouldn't break deserialization, so plugins shouldn't
necessarily need a recompile.

Performance is likely worse and I should benchmark the difference.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-19 07:50:51 -05:00
Jack Wright
9fb59a6f43
Removed the polars dtypes command (#12577)
# Description
The polars dtype command is largerly redundant since the introduction of
the schema command. The schema command also has the added benefit that
it's output can be used as a parameter to other schema commands:

```nushell
[[a b]; [5 6] [5 7]] | polars into-df -s ($df | polars schema
```

# User-Facing Changes
`polars dtypes` has been removed. Users should use `polars schema`
instead.

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-04-19 07:01:47 -05:00
Antoine Stevan
55edef5dda
create nuon crate from from nuon and to nuon (#12553)
# Description
playing with the NUON format in Rust code in some plugins, we agreed
with the team it was a great time to create a standalone NUON format to
allow Rust devs to use this Nushell file format.

> **Note**
> this PR almost copy-pastes the code from
`nu_commands/src/formats/from/nuon.rs` and
`nu_commands/src/formats/to/nuon.rs` to `nuon/src/from.rs` and
`nuon/src/to.rs`, with minor tweaks to make then standalone functions,
e.g. remove the rest of the command implementations

### TODO
- [x] add tests
- [x] add documentation

# User-Facing Changes
devs will have access to a new crate, `nuon`, and two functions,
`from_nuon` and `to_nuon`
```rust
from_nuon(
    input: &str,
    span: Option<Span>,
) -> Result<Value, ShellError>
```
```rust
to_nuon(
    input: &Value,
    raw: bool,
    tabs: Option<usize>,
    indent: Option<usize>,
    span: Option<Span>,
) -> Result<String, ShellError>
```

# Tests + Formatting
i've basically taken all the tests from
`crates/nu-command/tests/format_conversions/nuon.rs` and converted them
to use `from_nuon` and `to_nuon` instead of Nushell commands
- i've created a `nuon_end_to_end` to run both conversions with an
optional middle value to check that all is fine

> **Note** 
> the `nuon::tests::read_code_should_fail_rather_than_panic` test does
give different results locally and in the CI...
> i've left it ignored with comments to help future us :)

# After Submitting
mention that in the release notes for sure!!
2024-04-19 13:54:16 +02:00
Devyn Cairns
fac2f43aa4
Add an example Nushell plugin written in Nushell itself (#12574)
# Description

As suggested by @fdncred.

It's neat that this is possible, but the particularly useful part of
this is that we can actually
test it because it doesn't have any external dependencies, unlike the
python plugin.

Right now this just implements exactly the same behavior as the python
plugin, but we could have it
exercise a few more things.

Also fixes a couple of bugs:

- `.nu` plugins were not run with `nu --stdin`, so they couldn't take
input.
- `register` couldn't be called if `--no-config-file` was set, because
it would error on trying to
  update the plugin file.

# User-Facing Changes

- `nu_plugin_nu_example` plugin added.
- `register` now works in `--no-config-file` mode.

# Tests + Formatting
Tests added for `nu_plugin_nu_example`.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- [ ] Add the version bump to the release script just like for python
2024-04-19 09:53:30 +03:00
YizhePKU
6d2cb4382a
Fix circular source causing Nushell to crash (#12262)
# Description

EngineState now tracks the script currently running, instead of the
parent directory of the script. This also provides an easy way to expose
the current running script to the user (Issue #12195).

Similarly, StateWorkingSet now tracks scripts instead of directories.
`parsed_module_files` and `currently_parsed_pwd` are merged into one
variable, `scripts`, which acts like a stack for tracking the current
running script (which is on the top of the stack).

Circular import check is added for `source` operations, in addition to
module import. A simple testcase is added for circular source.

<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->


<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

It shouldn't have any user facing changes.
2024-04-19 09:38:08 +03:00
Brooks Rady
351bff8233
fix(shell_integration): set window title on startup (#12569)
Should close #10833 — though I'd imagine that should have already been
closed.

# Description

Very minor tweak, but it was quite noticeable when using Zellij which
relies on OSC 2 to set pane titles. Before the change:

![image](https://github.com/nushell/nushell/assets/6251883/b944bbce-2040-4886-9955-3c5b57d368e9)

Note that the default `Pane #1` is still showing for the untouched
shell, but running a command like `htop` or `ls` correctly sets the
title during / afterwards.

After this PR:

![image](https://github.com/nushell/nushell/assets/6251883/dd513cfe-923c-450f-b0f2-c66938b0d6f0)

There are now no-longer any unset titles — even if the shell hasn't been
touched.

**As an aside:** I feel quite strongly that (at least OSC 2) shell
integration should be enabled by default, as it is for every other Linux
shell I've used, but I'm not sure which issues that caused that the
default config refers to? Which terminals are broken by shell
integration, and could some of the shell integrations be turned on by
default after splitting things into sub-options as suggested in #11301 ?

# User-Facing Changes

You'll just have shell integrations working from right after the shell
has been launched, instead of needing to run something first.

# Tests + Formatting

Not quite sure how to test this one? Are there any other tests that
currently exist for shell integration? I couldn't quite track them
down...

# After Submitting

Let me know if you think this needs any user-facing docs changes!
2024-04-18 20:30:37 -05:00
Darren Schroeder
7fe2e60af7
add ability to set metadata (#12564)
# Description

This PR adds the ability to set metadata. This is especially useful for
activating LS_COLORS when using table literals.


![image](https://github.com/nushell/nushell/assets/343840/feef6433-f592-43ea-890a-38cb2df35686)

You can also set the filepath metadata, although I'm not really user how
useful this is. We may end up removing this option entirely.
```nushell
❯ "crates" | metadata set --datasource-filepath $'(pwd)/crates' | metadata
╭────────┬───────────────────────────────────╮
│ source │ /Users/fdncred/src/nushell/crates │
╰────────┴───────────────────────────────────╯
```

No file paths are checked. You could also do this.
```nushell
❯ "crates" | metadata set --datasource-filepath $'a/b/c/d/crates' | metadata
╭────────┬────────────────╮
│ source │ a/b/c/d/crates │
╰────────┴────────────────╯
```

The command name and parameter names are still WIP. We could change
them.

There are currently 3 kinds of metadata in nushell.
```rust
pub enum DataSource {
    Ls,
    HtmlThemes,
    FilePath(PathBuf),
}
```

I've skipped adding `HtmlThemes` because it seems to be specific to our
`to html` command only.
2024-04-19 09:03:59 +08:00
Devyn Cairns
999dfdf936
Fix the error output in the python plugin to match LabeledError (#12575)
# Description

Forgot to update this after updating the format of `LabeledError`.

# Tests + Formatting

Not applicable to the python plugin.
2024-04-18 20:01:35 -05:00
Jack Wright
cc7b5c5a26
Only mark collected dataframes as from_lazy=false when collect is called from the collect command. (#12571)
I had previously changed NuLazyFrame::collect to set the NuDataFrame's
from_lazy field to false to prevent conversion back to a lazy frame. It
appears there are cases where this should happen. Instead, I am only
setting from_lazy=false inside the `polars collect` command.

[Related discord
message](https://discord.com/channels/601130461678272522/1227612017171501136/1230600465159421993)

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-04-18 17:10:38 -05:00
Ian Manske
9a265847e2
Box ImportPattern in Expr (#12568)
# Description
Adds a `Box` around the `ImportPattern` in `Expr` which decreases the
size of `Expr` from 152 to 64 bytes (and `Expression` from 216 to 128
bytes). This seems to speed up parsing a little bit according to the
benchmarks (main is top, PR is bottom):
```
benchmarks                       fastest       │ slowest       │ median        │ mean          │ samples │ iters
benchmarks                       fastest       │ slowest       │ median        │ mean          │ samples │ iters
├─ parser_benchmarks                           │               │               │               │         │
├─ parser_benchmarks                           │               │               │               │         │
│  ├─ parse_default_config_file  2.287 ms      │ 4.532 ms      │ 2.311 ms      │ 2.437 ms      │ 100     │ 100
│  ├─ parse_default_config_file  2.255 ms      │ 2.781 ms      │ 2.281 ms      │ 2.312 ms      │ 100     │ 100
│  ╰─ parse_default_env_file     421.8 µs      │ 824.6 µs      │ 494.3 µs      │ 527.5 µs      │ 100     │ 100
│  ╰─ parse_default_env_file     402 µs        │ 486.6 µs      │ 414.8 µs      │ 416.2 µs      │ 100     │ 100

```
2024-04-18 17:57:01 +02:00
Stefan Holderbach
b088f395dc
Update crate feature flags (#12566)
# Description
Remove unused/effect-less features, make sure we show all relevant
features in `version`

# User-Facing Changes
- **Remove unused feature `wasi`**
- will cause failure to build should you enable it. Otherwise no effect
- **Include feat `system-clipboard` in `version`**
2024-04-18 16:33:41 +02:00
Ian Manske
6ccd547d81
Add ListItem type for Expr::List (#12529)
# Description
This PR adds a `ListItem` enum to our set of AST types. It encodes the
two possible expressions inside of list expression: a singular item or a
spread. This is similar to the existing `RecordItem` enum. Adding
`ListItem` allows us to remove the existing `Expr::Spread` case which
was previously used for list spreads. As a consequence, this guarantees
(via the type system) that spreads can only ever occur inside lists,
records, or as command args.

This PR also does a little bit of cleanup in relevant parser code.
2024-04-18 13:21:05 +02:00
Jack Wright
57b0c722c6
Upgrading nu-cmd-dataframe to polars 0.39 (#12554)
#Description
Upgrading nu-cmd-dataframe to polars 0.39

---------

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-04-17 12:50:17 -05:00
Devyn Cairns
13160b3ec3
Replace subtraction of Instants and Durations with saturating subtractions (#12549)
# Description
Duration can not be negative, and an underflow causes a panic.

This should fix #12539 as from what I can tell that bug was caused in
`nu-explore:📟:events` from subtracting durations, but I figured
this might be more widespread, and saturating to zero generally makes
sense.

I also added the relevant clippy lint to try to prevent this from
happening in the future. I can't think of a reason we would ever want to
subtract durations without checking first.

cc @fdncred

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-17 07:25:16 -05:00
Jack Wright
410f3c5c8a
Upgrading nu_plugin_polars to polars 0.39.1 (#12551)
# Description
Upgrading nu_plugin_polars to polars 0.39.1

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-04-17 06:35:09 -05:00
Devyn Cairns
b296d6ee3c
Improve safety of get_unchecked_str in nu_system::macos (#12550)
# Description
The implementation of this function had a few issues before:

- It didn't check that the `cp` pointer is actually ahead of the `start`
pointer, so `len` could potentially underflow and wrap around, which
would be a violation of memory safety
- It used `Vec::from_raw_parts` even though the buffer is borrowed, not
owned. Although `std::mem::forget` is used later to ensure the
destructor doesn't run, there is a risk that the destructor would run if
a panic happened during `String::from_utf8_unchecked`, which would lead
to a `free()` of a pointer we don't own
2024-04-17 17:10:05 +08:00