# Description
Adds subcommands to `sys` corresponding to each column of the record
returned by `sys`. This is to alleviate the fact that `sys` now returns
a regular record, meaning that it must compute every column which might
take a noticeable amount of time. The subcommands, on the other hand,
only need to compute and return a subset of the data which should be
much faster. In fact, it should be as fast as before, since this is how
the lazy record worked (it would compute only each column as necessary).
I choose to add subcommands instead of having an optional cell-path
parameter on `sys`, since the cell-path parameter would:
- increase the code complexity (can access any value at any row or
nested column)
- prevents discovery with tab-completion
- hinders type checking and allows users to pass potentially invalid
columns
# User-Facing Changes
Deprecates `sys` in favor of the new `sys` subcommands.
This moves to predominantly supporting only lazy dataframes for most
operations. It removes a lot of the type conversion between lazy and
eager dataframes based on what was inputted into the command.
For the most part the changes will mean:
* You will need to run `polars collect` after performing operations
* The into-lazy command has been removed as it is redundant.
* When opening files a lazy frame will be outputted by default if the
reader supports lazy frames
A list of individual command changes can be found
[here](https://hackmd.io/@nucore/Bk-3V-hW0)
---------
Co-authored-by: Ian Manske <ian.manske@pm.me>
# Description
Fixes: #12744
This pr is moving raw string lex logic into `lex_item` function, so we
can use raw string inside subexpression, list, closure.
```nushell
> [r#'abc'#]
╭───┬─────╮
│ 0 │ abc │
╰───┴─────╯
> (r#'abc'#)
abc
> do {r#'aa'#}
aa
```
# Tests + Formatting
Done
# After Submitting
NaN
# Description
Fixes: #12429
To fix the issue, we need to pass the `input pattern` itself to
`glob_from` function, but currently on latest main, nushell pass
`expanded path of input pattern` to `glob_from` function.
It causes globbing failed if expanded path includes `[]` brackets.
It's a pity that I have to duplicate `nu_engine::glob_from` function
into `ls`, because `ls` might convert from `NuGlob::NotExpand` to
`NuGlob::Expand`, in that case, `nu_engine::glob_from` won't work if
user want to ls for a directory which includes tilde:
```
mkdir "~abc"
ls "~abc"
```
So I need to duplicate `glob_from` function and pass original
`expand_tilde` information.
# User-Facing Changes
Nan
# Tests + Formatting
Done
# After Submitting
Nan
# Description
Does some misc changes to `ListStream`:
- Moves it into its own module/file separate from `RawStream`.
- `ListStream`s now have an associated `Span`.
- This required changes to `ListStreamInfo` in `nu-plugin`. Note sure if
this is a breaking change for the plugin protocol.
- Hides the internals of `ListStream` but also adds a few more methods.
- This includes two functions to more easily alter a stream (these take
a `ListStream` and return a `ListStream` instead of having to go through
the whole `into_pipeline_data(..)` route).
- `map`: takes a `FnMut(Value) -> Value`
- `modify`: takes a function to modify the inner stream.
# Description
this PR
- moves the documentation from `lib.rs` to `README.md` while still
including it in the lib file, so that both the [crates.io
page](https://crates.io/crates/nuon) and the
[documentation](https://docs.rs/nuon/latest/nuon/) show the top-level
doc
- mention that comments are allowed in NUON
- add a JSON-NUON example
- put back the formatting of NOTE blocks in the doc
# User-Facing Changes
# Tests + Formatting
# After Submitting
# Description
Our current flattening code creates a bunch of intermediate `Vec`s for
each function call. These intermediate `Vec`s are then usually appended
to the current `output` `Vec`. By instead passing a mutable reference of
the `output` `Vec` to each flattening function, this `Vec` can be
reused/appended to directly thereby eliminating the need for
intermediate `Vec`s in most cases.
# Description
Make typos config more strict: ignore false positives where they occur.
1. Ignore only files with typos
2. Add regexp-s with context
3. Ignore variable names only in Rust code
4. Ignore only 1 "identifier"
5. Check dot files
🎁 Extra bonus: fix typos!!
# Description
Spaces were causing an issue with into_sqlite when they appeared in
column names.
This is because the column names were not properly wrapped with
backticks that allow sqlite to properly interpret the column.
The issue has been addressed by adding backticks to the column names of
into sqlite. The output of the column names when using open is
unchanged, and the column names appear without backticks as expected.
fixes#12700
# User-Facing Changes
N/A
# Tests + Formatting
Formatting has been respected.
Repro steps from the issue have been done, and ran multiple times. New
values get added to the correct columns as expected.
# Description
Bumps `base64` to 0.22.1 which fixes the alphabet used for binhex
encoding and decoding. This required updating some test expected output.
Related to PR #12469 where `base64` was also bumped and ran into the
failing tests.
# User-Facing Changes
Bug fix, but still changes binhex encoding and decoding output.
# Tests + Formatting
Updated test expected output.
PR https://github.com/nushell/nushell/pull/12603 made it so that PWD can
never contain a trailing slash. However, a root path (such as `/` or
`C:\`) technically counts as "having a trailing slash", so now `cd /`
doesn't work.
I feel dumb for missing such an obvious edge case. Let's just merge this
quickly before anyone else finds out...
EDIT: It appears I'm too late.
# Description
Fixes#12758.
#12662 introduced a bug where calling `cd` with a path with a trailing
slash would cause `PWD` to be set to a path including a trailing slash,
which is not allowed. This adds a helper to `nu_path` to remove this,
and uses it in the `cd` command to clean it up before setting `PWD`.
# Tests + Formatting
I added some tests to make sure we don't regress on this in the future.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
- fixes#11922
- fixes#12203
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
This is a rewrite for some parts of the recursive completion system. The
Rust `std::path` structures often ignores things like a trailing `.`
because for a complete path, it implies the current directory. We are
replacing the use of some of these structs for Strings.
A side effect is the slashes being normalized in Windows. For example if
we were to type `foo/bar/b`, it would complete it to `foo\bar\baz`
because a backward slash is the main separator in windows.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Relative paths are preserved. `..`s in the paths won't eagerly show
completions from the parent path. For example, `asd/foo/../b` will now
complete to `asd/foo/../bar` instead of `asd/bar`.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
I added some more tests to our mighty `polars` ~~, yet I don't know how
to add expected results in some of them. I would like to ask for help.~~
~~My experiments are in the last commit: [polars:
experiments](f7e5e72019).
Without those experiments `cargo test` goes well.~~
UPD. I moved out my unsuccessful test experiments into a separate
[branch](https://github.com/maxim-uvarov/nushell/blob/polars-tests-broken2/).
So, this branch seems ready for a merge.
@ayax79, maybe you'll find time for me please? It's not urgent for sure.
P.S. I'm very new to git. Please feel free to give me any suggestions on
how I should use it better
# Description
Judiciously try to avoid allocations/clone by changing the signature of
functions
- **Don't pass str by value unnecessarily if only read**
- **Don't require a vec in `Sandbox::with_files`**
- **Remove unnecessary string clone**
- **Fixup unnecessary borrow**
- **Use `&str` in shape color instead**
- **Vec -> Slice**
- **Elide string clone**
- **Elide `Path` clone**
- **Take &str to elide clone in tests**
# User-Facing Changes
None
# Tests + Formatting
This touches many tests purely in changing from owned to borrowed/static
data
Currently errors just create empty entries inside of resulting
dataframes.
This changeset is meant to help debug #12004, though generally speaking
I do think it's worth having ways to make errors be visible in this kind
of pipeline be visible
An example of what this looks like
<img width="954" alt="image"
src="https://github.com/nushell/nushell/assets/1408472/2c3c9167-2aaf-4f87-bab5-e8302d7a1170">
This is the first PR towards migrating to a new `$env.PWD` API that
returns potentially un-canonicalized paths. Refer to PR #12515 for
motivations.
## New API: `EngineState::cwd()`
The goal of the new API is to cover both parse-time and runtime use
case, and avoid unintentional misuse. It takes an `Option<Stack>` as
argument, which if supplied, will search for `$env.PWD` on the stack in
additional to the engine state. I think with this design, there's less
confusion over parse-time and runtime environments. If you have access
to a stack, just supply it; otherwise supply `None`.
## Deprecation of other PWD-related APIs
Other APIs are re-implemented using `EngineState::cwd()` and properly
documented. They're marked deprecated, but their behavior is unchanged.
Unused APIs are deleted, and code that accesses `$env.PWD` directly
without using an API is rewritten.
Deprecated APIs:
* `EngineState::current_work_dir()`
* `StateWorkingSet::get_cwd()`
* `env::current_dir()`
* `env::current_dir_str()`
* `env::current_dir_const()`
* `env::current_dir_str_const()`
Other changes:
* `EngineState::get_cwd()` (deleted)
* `StateWorkingSet::list_env()` (deleted)
* `repl::do_run_cmd()` (rewritten with `env::current_dir_str()`)
## `cd` and `pwd` now use logical paths by default
This pulls the changes from PR #12515. It's currently somewhat broken
because using non-canonicalized paths exposed a bug in our path
normalization logic (Issue #12602). Once that is fixed, this should
work.
## Future plans
This PR needs some tests. Which test helpers should I use, and where
should I put those tests?
I noticed that unquoted paths are expanded within `eval_filepath()` and
`eval_directory()` before they even reach the `cd` command. This means
every paths is expanded twice. Is this intended?
Once this PR lands, the plan is to review all usages of the deprecated
APIs and migrate them to `EngineState::cwd()`. In the meantime, these
usages are annotated with `#[allow(deprecated)]` to avoid breaking CI.
---------
Co-authored-by: Jakub Žádník <kubouch@gmail.com>
# Description
This fixes#12724. NetBSD confirmed to work with this change.
The update also behaves a bit better in some ways - it automatically
unlinks and reclaims sockets on Unix, and doesn't try to flush/sync the
socket on Windows, so I was able to remove that platform-specific logic.
They also have a way to split the socket so I could just use one socket
now, but I haven't tried to do that yet. That would be more of a
breaking change but I think it's more straightforward.
# User-Facing Changes
- Hopefully more platforms work
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
_cell paths_ can be easily serialized back and forth to NUON with the
leading `$.` syntax.
# User-Facing Changes
```nushell
$.foo.bar.0 | to nuon
```
and
```nushell
"$.foo.bar.0" | from nuon
```
are now possible
# Tests + Formatting
a new `cell_path` test has been added to `nuon`
# After Submitting
# Description
Nightly clippy found some unused fields leading me down a rabbit hole of
dead code hidden behind `pub`
Generally removing any already dead code or premature configurability
that is not exposed to the user.
# User-Facing Changes
None in effect.
Removed some options from the `$env.config.explore.hex-dump` record that
were only read into a struct but never used and also not validated.
# Description
Removes lazy records from the language, following from the reasons
outlined in #12622. Namely, this should make semantics more clear and
will eliminate concerns regarding maintainability.
# User-Facing Changes
- Breaking change: `lazy make` is removed.
- Breaking change: `describe --collect-lazyrecords` flag is removed.
- `sys` and `debug info` now return regular records.
# After Submitting
- Update nushell book if necessary.
- Explore new `sys` and `debug info` APIs to prevent them from taking
too long (e.g., subcommands or taking an optional column/cell-path
argument).
# Description
This helps to ensure data produced on a stream is immediately available
to the consumer of the stream. The BufWriter introduced for performance
reasons in 0.93 exposed the behavior that data messages wouldn't make it
to the other side until they filled the buffer in @cablehead's
[`nu_plugin_from_sse`](https://github.com/cablehead/nu_plugin_from_sse).
I had originally not flushed on every `Data` message because I figured
that it isn't really critical that the other side sees those messages
immediately, since they're not used for control and they are flushed
when waiting for acknowledgement or when the buffer is too full anyway.
Increasing the amount of data that can be sent with a single underlying
write increases performance, but this interferes with some plugins that
want to use streams in a more real-time way. In the future I would like
to make this configurable, maybe even per-command, so that a command can
decide what the priority is. But for now I think this is reasonable.
In the worst case, this decreases performance by about 40%, when sending
very small values (just numbers). But for larger values, this PR
actually increases performance by about 20%, because I've increased the
buffer size about 2x to 16,384 bytes. The previous value of 8,192 bytes
was too small to fit a full buffer coming from an external command, so
doubling it makes sense, and now a write of a buffer from an external
command can be done in exactly one write call, which I think makes
sense. I'm doing this at the same time because flushing each data
message would make it very likely that each individual data message from
an external stream would require exactly two writes rather than
approximately one (amortized).
Again, hopefully the tradeoff isn't too bad, and if it is I'll just make
it configurable.
# User-Facing Changes
- Performance of plugin streams will be a bit different
- Plugins that expect to send streams in real-time will work again
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
This PR overhauls the shell_integration system by allowing individual
control over which ansi escape sequences are used. As we continue to
broaden our support for more ansi escape sequences, we can't really have
an all-or-nothing strategy. Some ansi escapes cause problems in certain
operating systems or terminals. We should allow the user to choose which
escapes they want.
TODO:
* Gather feedback
* Should osc7, osc9_9 and osc633p be mutually exclusive?
* Is the naming convention for these settings too nerdy osc2, osc7, etc?
closes#11301
# User-Facing Changes
shell_integration is no longer a boolean value. This is what is
supported in the default_config.nu
```nushell
shell_integration: {
# osc2 abbreviates the path if in the home_dir, sets the tab/window title, shows the running command in the tab/window title
osc2: true
# osc7 is a way to communicate the path to the terminal, this is helpful for spawning new tabs in the same directory
osc7: true
# osc8 is also implemented as the deprecated setting ls.show_clickable_links, it shows clickable links in ls output if your terminal supports it
osc8: true
# osc9_9 is from ConEmu and is starting to get wider support. It's similar to osc7 in that it communicates the path to the terminal
osc9_9: false
# osc133 is several escapes invented by Final Term which include the supported ones below.
# 133;A - Mark prompt start
# 133;B - Mark prompt end
# 133;C - Mark pre-execution
# 133;D;exit - Mark execution finished with exit code
# This is used to enable terminals to know where the prompt is, the command is, where the command finishes, and where the output of the command is
osc133: true
# osc633 is closely related to osc133 but only exists in visual studio code (vscode) and supports their shell integration features
# 633;A - Mark prompt start
# 633;B - Mark prompt end
# 633;C - Mark pre-execution
# 633;D;exit - Mark execution finished with exit code
# 633;E - NOT IMPLEMENTED - Explicitly set the command line with an optional nonce
# 633;P;Cwd=<path> - Mark the current working directory and communicate it to the terminal
# and also helps with the run recent menu in vscode
osc633: true
# reset_application_mode is escape \x1b[?1l and was added to help ssh work better
reset_application_mode: true
}
```
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
This PR adds raw string support by using `r#` at the beginning of single
quoted strings and `#` at the end.
Notice that escapes do not process, even within single quotes,
parentheses don't mean anything, $variables don't mean anything. It's
just a string.
```nushell
❯ echo r#'one\ntwo (blah) ($var)'#
one\ntwo (blah) ($var)
```
Notice how they work without `echo` or `print` and how they work without
carriage returns.
```nushell
❯ r#'adsfa'#
adsfa
❯ r##"asdfa'@qpejq'##
asdfa'@qpejq
❯ r#'asdfasdfasf
∙ foqwejfqo@'23rfjqf'#
```
They also have a special configurable color in the repl. (use single
quotes though)
They should work like rust raw literals and allow `r##`, `r###`,
`r####`, etc, to help with having one or many `#`'s in the middle of
your raw-string.
They should work with `let` as well.
```nushell
r#'some\nraw\nstring'# | str upcase
```
closes https://github.com/nushell/nushell/issues/5091
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect -A clippy::result_large_err` to check that
you're using the standard code style
- `cargo test --workspace` to check that all tests pass
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
---------
Co-authored-by: WindSoilder <WindSoilder@outlook.com>
Co-authored-by: Ian Manske <ian.manske@pm.me>
This PR:
1. Adds basic support for `CustomValue` to `explore`. Previously `open
foo.db | explore` didn't really work, now we "materialize" the whole
database to a `Value` before loading it
2. Adopts `anyhow` for error handling in `explore`. Previously we were
kind of rolling our own version of `anyhow` by shoving all errors into a
`std::io::Error`; I think this is much nicer. This was necessary because
as part of 1), collecting input is now fallible...
3. Removes a lot of `explore`'s fancy command help system.
- Previously each command (`:help`, `:try`, etc.) had a sophisticated
help system with examples etc... but this was not very visible to users.
You had to know to run `:help :try` or view a list of commands with
`:help :`
- As discussed previously, we eventually want to move to a less modal
approach for `explore`, without the Vim-like commands. And so I don't
think it's worth keeping this command help system around (it's
intertwined with other stuff, and making these changes would have been
harder if keeping it).
4. Rename the `--reverse` flag to `--tail`. The flag scrolls to the end
of the data, which IMO is described better by "tail"
5. Does some renaming+commenting to clear up things I found difficult to
understand when navigating the `explore` code
I initially thought 1) would be just a few lines, and then this PR blew
up into much more extensive changes 😅
## Before
The whole database was being displayed as a single Nuon/JSON line 🤔
## After
The database gets displayed like a record
## Future work
It is sort of annoying that we have to load a whole SQLite database into
memory to make this work; it will be impractical for large databases.
I'd like to explore improvements to `CustomValue` that can make this
work more efficiently.
# Description
This creates an option for building binary data from byte integers.
Previously I think you could only do this by formatting the integers to
hex and using `decode hex`.
One potentially confusing thing is that this is different from the `into
binary` behavior. But since this doesn't support any of the other `into
binary` behaviors, it might be okay.
# User-Facing Changes
- `bytes build` accepts single byte arguments as integers
# Tests + Formatting
Example added.
# After Submitting
- [ ] release notes
This PR changes `nu_path::expand_path_with()` to no longer remove
trailing slashes. It also fixes bugs in the current implementation due
to ineffective tests (Fixes#12602).
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Prior, it seemed that nested errors would not get detected and shown.
This PR fixes that.
Resolves#10176:
```
~/CodingProjects/nushell> [[1,2]] | each {|x| $x | each {|y| error make {msg: "oh noes"} } } 05/04/2024 21:34:08
Error: nu:🐚:eval_block_with_input
× Eval block failed with pipeline input
╭─[entry #1:1:3]
1 │ [[1,2]] | each {|x| $x | each {|y| error make {msg: "oh noes"} } }
· ┬
· ╰── source value
╰────
Error: × oh noes
╭─[entry #1:1:36]
1 │ [[1,2]] | each {|x| $x | each {|y| error make {msg: "oh noes"} } }
· ─────┬────
· ╰── originates from here
╰────
```
Resolves#11224:
```
~/CodingProjects/nushell> [0] | each { |_| 05/04/2024 21:35:40
::: [0] | each { |_|
::: non-existent-command
::: }
::: }
Error: nu:🐚:eval_block_with_input
× Eval block failed with pipeline input
╭─[entry #1:2:6]
1 │ [0] | each { |_|
2 │ [0] | each { |_|
· ┬
· ╰── source value
3 │ non-existent-command
╰────
Error: nu:🐚:external_command
× External command failed
╭─[entry #1:3:9]
2 │ [0] | each { |_|
3 │ non-existent-command
· ──────────┬─────────
· ╰── executable was not found
4 │ }
╰────
help: No such file or directory (os error 2)
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
This PR changes `$env` to be **case-preserving** instead of
case-sensitive. That is, it preserves the case of the environment
variable when it is first assigned, but subsequent retrieval and update
ignores the case.
Notably, both `$env.PATH` and `$env.Path` can now be used to read or set
the environment variable, but child processes will always see the
correct case based on the platform.
Fixes#11268.
---
This feature was surprising simple to implement, because most of the
infrastructure to support case-insensitive cell path access already
exists. The `get` command extracts data using a cell path in a
case-insensitive way (!), but accepts a `--sensitive` flag. (I think
this should be flipped around?)
# Description
So sorry to do this during the pre-release freeze, but my plugin crate
split PR broke local socket mode, because `nu-plugin-protocol` didn't
have the compile feature to advertise the `LocalSocket` protocol
feature.
This is a very simple, configuration-only bugfix that I think really
needs to be merged before the release, or else local socket mode won't
work at all.
# Tests + Formatting
There's an oversight in my testing that caused this to not be caught:
the engine really did have the feature, but it just wasn't advertising
it, so for `stress_internals` it was still able to use it successfully.
Post-release I'll try to make sure this is properly handled somehow.
# Description
Minor change but fixes a few rust-analyzer warnings.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
This PR fixes a problem introduced with PR
https://github.com/nushell/nushell/pull/12236. That PR accidentally
stopped `--as-table` from working.
Closes https://github.com/nushell/nushell/issues/12689
It works again.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
This breaks `nu-plugin` up into four crates:
- `nu-plugin-protocol`: just the type definitions for the protocol, no
I/O. If someone wanted to wire up something more bare metal, maybe for
async I/O, they could use this.
- `nu-plugin-core`: the shared stuff between engine/plugin. Less stable
interface.
- `nu-plugin-engine`: everything required for the engine to talk to
plugins. Less stable interface.
- `nu-plugin`: everything required for the plugin to talk to the engine,
what plugin developers use. Should be the most stable interface.
No changes are made to the interface exposed by `nu-plugin` - it should
all still be there. Re-exports from `nu-plugin-protocol` or
`nu-plugin-core` are used as required. Plugins shouldn't ever have to
use those crates directly.
This should be somewhat faster to compile as `nu-plugin-engine` and
`nu-plugin` can compile in parallel, and the engine doesn't need
`nu-plugin` and plugins don't need `nu-plugin-engine` (except for test
support), so that should reduce what needs to be compiled too.
The only significant change here other than splitting stuff up was to
break the `source` out of `PluginCustomValue` and create a new
`PluginCustomValueWithSource` type that contains that instead. One bonus
of that is we get rid of the option and it's now more type-safe, but it
also means that the logic for that stuff (actually running the plugin
for custom value ops) can live entirely within the `nu-plugin-engine`
crate.
# User-Facing Changes
- New crates.
- Added `local-socket` feature for `nu` to try to make it possible to
compile without that support if needed.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
I would like to help with `polars` plugin development and add tests to
all the `polars` command's existing params.
Since I have never written any lines of Rust, even though the task of
creating tests is relatively simple, I would like to ask for feedback to
ensure I did everything correctly here.
# Description
I found a bunch of issues relating to the specialized reimplementation
of `print()` that's done in `nu-cli` and it just didn't seem necessary.
So I tried to unify the behavior reasonably. `PipelineData::print()`
already handles the call to `table` and it even has a `no_newline`
option.
One of the most major issues before was that we were using the value
iterator, and then converting to string, and then printing each with
newlines. This doesn't work well for an external stream, because its
iterator ends up creating `Value::binary()` with each buffer... so we
were doing lossy UTF-8 conversion on those and then printing them with
newlines, which was very weird:
You can see the random newline inserted in a break between buffers, but
this would be even worse if it were on a multibyte UTF-8 character. You
can produce this by writing a large amount of text to a text file, and
then doing `nu -c 'open file.txt'` - in my case I just wrote `^find .`;
it just has to be large enough to trigger a buffer break.
Using `print()` instead led to a new issue though, because it doesn't
abort on errors. This is so that certain commands can produce a stream
of errors and have those all printed. There are tests for e.g. `rm` that
depend on this behavior. I assume we want to keep that, so instead I
made my target `BufferedReader`, and had that fuse closed if an error
was encountered. I can't imagine we want to keep reading from a wrapped
I/O stream if an error occurs; more often than not the error isn't going
to magically resolve itself, it's not going to be a different error each
time, and it's just going to lead to an infinite stream of the same
error.
The test that broke without that was `open . | lines`, because `lines`
doesn't fuse closed on error. But I don't know if it's expected or not
for it to do that, so I didn't target that.
I think this PR makes things better but I'll keep looking for ways to
improve on how errors and streams interact, especially trying to
eliminate cases where infinite error loops can happen.
# User-Facing Changes
- **Breaking**: `BufferedReader` changes + no more public fields
- A raw I/O stream from e.g. `open` won't produce infinite errors
anymore, but I consider that to be a plus
- the implicit `print` on script output is the same as the normal one
now
# Tests + Formatting
Everything passes but I didn't add anything specific.
# Description
Bandaid fix for #12643, where it is not possible to get the exit code of
a failed external command while also having the external command inherit
nushell's stdout and stderr. This changes `try` so that the exit code of
external command is available in the `catch` block via the usual
`$env.LAST_EXIT_CODE`.
# Tests + Formatting
Added one test.
# After Submitting
Rework I/O redirection and possibly exit codes.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Resolves#12654.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
`grid` can now throw an error.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Added relevant test.
# Description
Yet another attempt to fix the `stress_internals::test_wrong_version()`
test...
This time I think it's probably because we are getting a broken pipe
write error when we try to send `Hello` or perhaps something after it,
because the pipe has already been closed by the reader when it saw the
invalid version. In that case, an error should be available in state. It
probably makes more sense to send that back to the user rather than an
unhelpful I/O error.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
I thought about bringing `nu_plugin_msgpack` in, but that is MPL with a
clause that prevents other licenses, so rather than adapt that code I
decided to take a crack at just doing it straight from `rmp` to `Value`
without any `rmpv` in the middle. It seems like it's probably faster,
though I can't say for sure how much with the plugin overhead.
@IanManske I started on a `Read` implementation for `RawStream` but just
specialized to `from msgpack` here, but I'm thinking after release maybe
we can polish it up and make it a real one. It works!
# User-Facing Changes
New commands:
- `from msgpack`
- `from msgpackz`
- `to msgpack`
- `to msgpackz`
# Tests + Formatting
Pretty thorough tests added for the format deserialization, with a
roundtrip for serialization. Some example tests too for both `from
msgpack` and `to msgpack`.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] update release notes
# Description
One example for `into bits` says it uses binary value when it actually
uses a filesize. This lead to issue #11412, but I never got around to
fixing the example until this PR.
# Description
The previous messages said that the command printed dates separated by
newlines. But the current iteration of `seq date` returns a list.
# User-Facing Changes
Minor wording edit.
---------
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
# Description
Removes the `commandline` flags and API that was deprecated in 0.91.0
with #11877.
# User-Facing Changes
Users need to migrate to the new `commandline` subcommands introduced in
0.91.0.
# Description
This PR does miscellaneous cleanup in some of the commands from
`nu-cmd-lang`.
# User-Facing Changes
None.
# After Submitting
Cleanup the other commands in `nu-cmd-lang`.
# Description
In this week's nushell meeting, we decided to go ahead with #12622 and
remove lazy records in 0.94.0. For 0.93.0, we will only deprecate `lazy
make`, and so this PR makes `lazy make` print a deprecation warning.
# User-Facing Changes
None, besides the deprecation warning.
# After Submitting
Remove lazy records.
# Description
So far this seems like the winner of my poll on what the name should be.
I'll take this off draft once the poll expires, if this is indeed the
winner.
# Description
Continuing from #12568, this PR further reduces the size of `Expr` from
64 to 40 bytes. It also reduces `Expression` from 128 to 96 bytes and
`Type` from 32 to 24 bytes.
This was accomplished by:
- for `Expr` with multiple fields (e.g., `Expr::Thing(A, B, C)`),
merging the fields into new AST struct types and then boxing this struct
(e.g. `Expr::Thing(Box<ABC>)`).
- replacing `Vec<T>` with `Box<[T]>` in multiple places. `Expr`s and
`Expression`s should rarely be mutated, if at all, so this optimization
makes sense.
By reducing the size of these types, I didn't notice a large performance
improvement (at least compared to #12568). But this PR does reduce the
memory usage of nushell. My config is somewhat light so I only noticed a
difference of 1.4MiB (38.9MiB vs 37.5MiB).
---------
Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
# Description
The local socket PR introduced a `Waitable` type, which could either
hold a value or be waited on until a value is available. Unlike a
channel, it would always return that value once set.
However, one issue with this design was that there was no way to detect
whether a value would ever be written. This splits the writer into a
different type `WaitableMut`, so that when it is dropped, waiting
threads can fail (because they'll never get a value).
# Tests + Formatting
A test has been added to `stress_internals` to make sure this fails in
the right way.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
Trying to give as much context as possible. Now there should be a
spanned error with the call span if possible, and the propagated error
as an inner error if there was one in every case.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
This allows the following commands to all accept a filename instead of a
plugin name:
- `plugin use`
- `plugin rm`
- `plugin stop`
Slightly complicated because of the need to also check against
`NU_PLUGIN_DIRS`, but I also fixed some issues with that at the same
time
Requested by @fdncred
# User-Facing Changes
The new commands are updated as described.
# Tests + Formatting
Tests for `NU_PLUGIN_DIRS` handling also made more robust.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] Double check new docs to make sure they describe this capability
# Description
This should fix the sometimes failing wrong version test for
stress_internals.
The plugin interface state stores an error if some kind of critical
error happened, and this error should be propagated to any future
operations on the interface, but this wasn't being propagated to plugin
calls that were already waiting.
During plugin registration, the wrong version error needs to be received
as a response to the `get_signature()` to show up properly, but this
would only happen if `get_signature()` started after the `Hello` was
already received and processed. That would be a race condition, which
this commit solves.
cc @sholderbach - this should fix the CI issue
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
Adds a new keyword, `plugin use`. Unlike `register`, this merely loads
the signatures from the plugin cache file. The file is configurable with
the `--plugin-config` option either to `nu` or to `plugin use` itself,
just like the other `plugin` family of commands. At the REPL, one might
do this to replace `register`:
```nushell
> plugin add ~/.cargo/bin/nu_plugin_foo
> plugin use foo
```
This will not work in a script, because `plugin use` is a keyword and
`plugin add` does not evaluate at parse time (intentionally). This means
we no longer run random binaries during parse.
The `--plugins` option has been added to allow running `nu` with certain
plugins in one step. This is used especially for the `nu_with_plugins!`
test macro, but I'd imagine is generally useful. The only weird quirk is
that it has to be a list, and we don't really do this for any of our
other CLI args at the moment.
`register` now prints a deprecation parse warning.
This should fix#11923, as we now have a complete alternative to
`register`.
# User-Facing Changes
- Add `plugin use` command
- Deprecate `register`
- Add `--plugins` option to `nu` to replace a common use of `register`
# Tests + Formatting
I think I've tested it thoroughly enough and every existing test passes.
Testing nu CLI options and alternate config files is a little hairy and
I wish there were some more generic helpers for this, so this will go on
my TODO list for refactoring.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] Update plugins sections of book
- [ ] Release notes
# Description
This adds an extension trait to `Result` that wraps errors in `Spanned`,
saving the effort of calling `.map_err(|err| err.into_spanned(span))`
every time. This will hopefully make it even more likely that someone
will want to use a spanned `io::Error` and make it easier to remove the
impl for `From<io::Error> for ShellError` because that doesn't have span
information.
# Description
Fixes: #11351
And comment here is also fixed:
https://github.com/nushell/nushell/issues/11351#issuecomment-1996191537
The panic can happened if we pipe a variable to a custom command which
recursively called itself inside another block.
TBH, I think I figure out how it works to panic, but I'm not sure if
there is a potention issue if nushell don't mutate a block in such case.
# User-Facing Changes
Nan
# Tests + Formatting
Done
# After Submitting
Done
---------
Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
# Description
Adds two new types in `nu-engine` for evaluating closures: `ClosureEval`
and `ClosureEvalOnce`. This removed some duplicate code and centralizes
our logic for setting up, running, and cleaning up closures. For
example, in the future if we are able to reduce the cloning necessary to
run a closure, then we only have to change the code related to these
types.
`ClosureEval` and `ClosureEvalOnce` are designed with a builder API.
`ClosureEval` is used to run a closure multiple times whereas
`ClosureEvalOnce` is used for a one-shot closure.
# User-Facing Changes
Should be none, unless I messed up one of the command migrations.
Actually, this will fix any unreported environment bugs for commands
that didn't reset the env after running a closure.
# Description
When saving to a file we currently try to check if the data source in
the pipeline metadata is the same as the file we are saving to. If so,
we create an error, since reading and writing to a file at the same time
is currently not supported/handled gracefully. However, there are still
a few instances where this error is not properly triggered, and so this
PR attempts to reduce these cases. Inspired by #12599.
# Tests + Formatting
Added a few tests.
# After Submitting
Some commands still do not properly preserve metadata (e.g., `str trim`)
and so prevent us from detecting this error.
# Description
This pull request provides three new commands:
`polars store-ls` - moved from `polars ls`. It provides the list of all
object stored in the plugin cache
`polars store-rm` - deletes a cached object
`polars store-get` - gets an object from the cache.
The addition of `polars store-get` required adding a reference_count to
cached entries. `polars get` is the only command that will increment
this value. `polars rm` will remove the value despite it's count. Calls
to PolarsPlugin::custom_value_dropped will decrement the value.
The prefix store- was chosen due to there already being a `polars cache`
command. These commands were not made sub-commands as there isn't a way
to display help for sub commands in plugins (e.g. `polars store`
displaying help) and I felt the store- seemed fine anyways.
The output of `polars store-ls` now shows the reference count for each
object.
# User-Facing Changes
polars ls has now moved to polars store-ls
---------
Co-authored-by: Jack Wright <jack.wright@disqo.com>
# Description
The `stress_internals` tests can fail sometimes, but usually not on the
CI, because Nushell exits while the plugin is still trying to read or
maybe write something, leading to a broken pipe.
`nu-plugin` already exits with 1 without printing a message on a
protocol-level I/O error, so this just doing the same thing.
I think there's probably a way to correct the plugin handling so that we
wait for plugins to shut down before exiting and this doesn't happen,
but this is the quick fix in the meantime.
follow-up to
- https://github.com/nushell/nushell/pull/12591
cc/ @fdncred
# Description
there was a typo in the doc of `nuon::ToStyle`.
# User-Facing Changes
# Tests + Formatting
# After Submitting
# Description
- Plugin signatures are now saved to `plugin.msgpackz`, which is
brotli-compressed MessagePack.
- The file is updated incrementally, rather than writing all plugin
commands in the engine every time.
- The file always contains the result of the `Signature` call to the
plugin, even if commands were removed.
- Invalid data for a particular plugin just causes an error to be
reported, but the rest of the plugins can still be parsed
# User-Facing Changes
- The plugin file has a different filename, and it's not a nushell
script.
- The default `plugin.nu` file will be automatically migrated the first
time, but not other plugin config files.
- We don't currently provide any utilities that could help edit this
file, beyond `plugin add` and `plugin rm`
- `from msgpackz`, `to msgpackz` could also help
- New commands: `plugin add`, `plugin rm`
# Tests + Formatting
Tests added for the format and for the invalid handling.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] Check for documentation changes
- [ ] Definitely needs release notes
# Description
`Value` describes the types of first-class values that users and scripts
can create, manipulate, pass around, and store. However, `Block`s are
not first-class values in the language, so this PR removes it from
`Value`. This removes some unnecessary code, and this change should be
invisible to the user except for the change to `scope modules` described
below.
# User-Facing Changes
Breaking change: the output of `scope modules` was changed so that
`env_block` is now `has_env_block` which is a boolean value instead of a
`Block`.
# After Submitting
Update the language guide possibly.
# Description
For a long time, I was searching for the `str extract` command to
extract regexes from strings. I often painfully used `str replace -r
'(.*)(pattern_to_find)(.*)' '$2'` for such purposes.
Only this morning did I realize that `parse` is what I needed for so
many times, which I had only used for parsing data in tables.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
Closes#12561
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Add version details in `version`'s output. The intended use is for
third-party tools to be able to quickly check version numbers without
having to the parsing from `(version).version` or `$env.NU_VERSION`.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
This adds 4 new values to the record from `version`:
```
...
│ major │ 0 │
│ minor │ 92 │
│ patch │ 3 │
│ pre │ a-value │
...
```
`pre` is optional and won't be present most of the time I think.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
I ran the new command using `cargo run -- -c version`:
```
╭────────────────────┬─────────────────────────────────────────────────╮
│ version │ 0.92.3 │
│ major │ 0 │
│ minor │ 92 │
│ patch │ 3 │
│ branch │ │
│ commit_hash │ │
│ build_os │ macos-aarch64 │
│ build_target │ aarch64-apple-darwin │
│ rust_version │ rustc 1.77.2 (25ef9e3d8 2024-04-09) │
│ rust_channel │ 1.77.2-aarch64-apple-darwin │
│ cargo_version │ cargo 1.77.2 (e52e36006 2024-03-26) │
│ build_time │ 2024-04-20 15:09:36 +02:00 │
│ build_rust_channel │ release │
│ allocator │ mimalloc │
│ features │ default, sqlite, system-clipboard, trash, which │
│ installed_plugins │ │
╰────────────────────┴─────────────────────────────────────────────────╯
```
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
After this is merged I would like to write docs somewhere for scripts
writer to advise using these members instead of the string values. Where
should I put said docs ?
# Description
This PR improves the `nu --lsp` tooltips by using nu code blocks around
the examples and a few other places.
This is what it looks like in Zed.
Here it is in Helix.
This coloring is far from perfect, but it's what the tree-sitter-nu
queries generate.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
in order to change the style of the _serialized_ NUON data,
`nuon::to_nuon` takes three mutually exclusive arguments, `raw: bool`,
`tabs: Option<usize>` and `indent: Option<usize>` 🤔
this begs to use an enumeration with all possible alternatives, right?
this PR changes the signature of `nuon::to_nuon` to use `nuon::ToStyle`
which has three variants
- `Raw`: no newlines
- `Tabs(n: usize)`: newlines and `n` tabulations as indent
- `Spaces(n: usize)`: newlines and `n` spaces as indent
# User-Facing Changes
the signature of `nuon::to_nuon` changes from
```rust
to_nuon(
input: &Value,
raw: bool,
tabs: Option<usize>,
indent: Option<usize>,
span: Option<Span>,
) -> Result<String, ShellError>
```
to
```rust
to_nuon(
input: &Value,
style: ToStyle,
span: Option<Span>
) -> Result<String, ShellError>
```
# Tests + Formatting
# After Submitting
# Description
Close: #12514
# User-Facing Changes
`^ls | skip 1` will raise an error
```nushell
❯ ^ls | skip 1
Error: nu:🐚:only_supports_this_input_type
× Input type not supported.
╭─[entry #1:1:2]
1 │ ^ls | skip 1
· ─┬ ──┬─
· │ ╰── only list, binary or range input data is supported
· ╰── input type: raw data
╰────
```
# Tests + Formatting
Sorry I can't add it because of the issue:
https://github.com/nushell/nushell/issues/12558
# After Submitting
Nan
# Description
In conflict with the documentation, the msgpack serializer for plugins
is actually using the compact format, which doesn't name struct fields,
and is instead dependent on their ordering, rendering them as tuples.
This is not a good idea for a robust protocol even if it makes the
serialization and deserialization faster.
I expect this to have some impact on performance, but I think the
robustness is probably worth it.
Deserialization always accepts either format, so this shouldn't cause
too many incompatibilities.
# User-Facing Changes
This does technically change the protocol, but it makes it reflect the
documentation. It shouldn't break deserialization, so plugins shouldn't
necessarily need a recompile.
Performance is likely worse and I should benchmark the difference.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
The polars dtype command is largerly redundant since the introduction of
the schema command. The schema command also has the added benefit that
it's output can be used as a parameter to other schema commands:
```nushell
[[a b]; [5 6] [5 7]] | polars into-df -s ($df | polars schema
```
# User-Facing Changes
`polars dtypes` has been removed. Users should use `polars schema`
instead.
Co-authored-by: Jack Wright <jack.wright@disqo.com>
# Description
playing with the NUON format in Rust code in some plugins, we agreed
with the team it was a great time to create a standalone NUON format to
allow Rust devs to use this Nushell file format.
> **Note**
> this PR almost copy-pastes the code from
`nu_commands/src/formats/from/nuon.rs` and
`nu_commands/src/formats/to/nuon.rs` to `nuon/src/from.rs` and
`nuon/src/to.rs`, with minor tweaks to make then standalone functions,
e.g. remove the rest of the command implementations
### TODO
- [x] add tests
- [x] add documentation
# User-Facing Changes
devs will have access to a new crate, `nuon`, and two functions,
`from_nuon` and `to_nuon`
```rust
from_nuon(
input: &str,
span: Option<Span>,
) -> Result<Value, ShellError>
```
```rust
to_nuon(
input: &Value,
raw: bool,
tabs: Option<usize>,
indent: Option<usize>,
span: Option<Span>,
) -> Result<String, ShellError>
```
# Tests + Formatting
i've basically taken all the tests from
`crates/nu-command/tests/format_conversions/nuon.rs` and converted them
to use `from_nuon` and `to_nuon` instead of Nushell commands
- i've created a `nuon_end_to_end` to run both conversions with an
optional middle value to check that all is fine
> **Note**
> the `nuon::tests::read_code_should_fail_rather_than_panic` test does
give different results locally and in the CI...
> i've left it ignored with comments to help future us :)
# After Submitting
mention that in the release notes for sure!!
# Description
As suggested by @fdncred.
It's neat that this is possible, but the particularly useful part of
this is that we can actually
test it because it doesn't have any external dependencies, unlike the
python plugin.
Right now this just implements exactly the same behavior as the python
plugin, but we could have it
exercise a few more things.
Also fixes a couple of bugs:
- `.nu` plugins were not run with `nu --stdin`, so they couldn't take
input.
- `register` couldn't be called if `--no-config-file` was set, because
it would error on trying to
update the plugin file.
# User-Facing Changes
- `nu_plugin_nu_example` plugin added.
- `register` now works in `--no-config-file` mode.
# Tests + Formatting
Tests added for `nu_plugin_nu_example`.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] Add the version bump to the release script just like for python
# Description
EngineState now tracks the script currently running, instead of the
parent directory of the script. This also provides an easy way to expose
the current running script to the user (Issue #12195).
Similarly, StateWorkingSet now tracks scripts instead of directories.
`parsed_module_files` and `currently_parsed_pwd` are merged into one
variable, `scripts`, which acts like a stack for tracking the current
running script (which is on the top of the stack).
Circular import check is added for `source` operations, in addition to
module import. A simple testcase is added for circular source.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
It shouldn't have any user facing changes.
Should close#10833 — though I'd imagine that should have already been
closed.
# Description
Very minor tweak, but it was quite noticeable when using Zellij which
relies on OSC 2 to set pane titles. Before the change:
Note that the default `Pane #1` is still showing for the untouched
shell, but running a command like `htop` or `ls` correctly sets the
title during / afterwards.
After this PR:
There are now no-longer any unset titles — even if the shell hasn't been
touched.
**As an aside:** I feel quite strongly that (at least OSC 2) shell
integration should be enabled by default, as it is for every other Linux
shell I've used, but I'm not sure which issues that caused that the
default config refers to? Which terminals are broken by shell
integration, and could some of the shell integrations be turned on by
default after splitting things into sub-options as suggested in #11301 ?
# User-Facing Changes
You'll just have shell integrations working from right after the shell
has been launched, instead of needing to run something first.
# Tests + Formatting
Not quite sure how to test this one? Are there any other tests that
currently exist for shell integration? I couldn't quite track them
down...
# After Submitting
Let me know if you think this needs any user-facing docs changes!
# Description
This PR adds the ability to set metadata. This is especially useful for
activating LS_COLORS when using table literals.
You can also set the filepath metadata, although I'm not really user how
useful this is. We may end up removing this option entirely.
```nushell
❯ "crates" | metadata set --datasource-filepath $'(pwd)/crates' | metadata
╭────────┬───────────────────────────────────╮
│ source │ /Users/fdncred/src/nushell/crates │
╰────────┴───────────────────────────────────╯
```
No file paths are checked. You could also do this.
```nushell
❯ "crates" | metadata set --datasource-filepath $'a/b/c/d/crates' | metadata
╭────────┬────────────────╮
│ source │ a/b/c/d/crates │
╰────────┴────────────────╯
```
The command name and parameter names are still WIP. We could change
them.
There are currently 3 kinds of metadata in nushell.
```rust
pub enum DataSource {
Ls,
HtmlThemes,
FilePath(PathBuf),
}
```
I've skipped adding `HtmlThemes` because it seems to be specific to our
`to html` command only.
I had previously changed NuLazyFrame::collect to set the NuDataFrame's
from_lazy field to false to prevent conversion back to a lazy frame. It
appears there are cases where this should happen. Instead, I am only
setting from_lazy=false inside the `polars collect` command.
[Related discord
message](https://discord.com/channels/601130461678272522/1227612017171501136/1230600465159421993)
Co-authored-by: Jack Wright <jack.wright@disqo.com>
# Description
Adds a `Box` around the `ImportPattern` in `Expr` which decreases the
size of `Expr` from 152 to 64 bytes (and `Expression` from 216 to 128
bytes). This seems to speed up parsing a little bit according to the
benchmarks (main is top, PR is bottom):
```
benchmarks fastest │ slowest │ median │ mean │ samples │ iters
benchmarks fastest │ slowest │ median │ mean │ samples │ iters
├─ parser_benchmarks │ │ │ │ │
├─ parser_benchmarks │ │ │ │ │
│ ├─ parse_default_config_file 2.287 ms │ 4.532 ms │ 2.311 ms │ 2.437 ms │ 100 │ 100
│ ├─ parse_default_config_file 2.255 ms │ 2.781 ms │ 2.281 ms │ 2.312 ms │ 100 │ 100
│ ╰─ parse_default_env_file 421.8 µs │ 824.6 µs │ 494.3 µs │ 527.5 µs │ 100 │ 100
│ ╰─ parse_default_env_file 402 µs │ 486.6 µs │ 414.8 µs │ 416.2 µs │ 100 │ 100
```
# Description
Remove unused/effect-less features, make sure we show all relevant
features in `version`
# User-Facing Changes
- **Remove unused feature `wasi`**
- will cause failure to build should you enable it. Otherwise no effect
- **Include feat `system-clipboard` in `version`**
# Description
This PR adds a `ListItem` enum to our set of AST types. It encodes the
two possible expressions inside of list expression: a singular item or a
spread. This is similar to the existing `RecordItem` enum. Adding
`ListItem` allows us to remove the existing `Expr::Spread` case which
was previously used for list spreads. As a consequence, this guarantees
(via the type system) that spreads can only ever occur inside lists,
records, or as command args.
This PR also does a little bit of cleanup in relevant parser code.
# Description
Duration can not be negative, and an underflow causes a panic.
This should fix#12539 as from what I can tell that bug was caused in
`nu-explore:📟:events` from subtracting durations, but I figured
this might be more widespread, and saturating to zero generally makes
sense.
I also added the relevant clippy lint to try to prevent this from
happening in the future. I can't think of a reason we would ever want to
subtract durations without checking first.
cc @fdncred
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
The implementation of this function had a few issues before:
- It didn't check that the `cp` pointer is actually ahead of the `start`
pointer, so `len` could potentially underflow and wrap around, which
would be a violation of memory safety
- It used `Vec::from_raw_parts` even though the buffer is borrowed, not
owned. Although `std::mem::forget` is used later to ensure the
destructor doesn't run, there is a risk that the destructor would run if
a panic happened during `String::from_utf8_unchecked`, which would lead
to a `free()` of a pointer we don't own
Bumps [rmp-serde](https://github.com/3Hren/msgpack-rust) from 1.1.2 to
1.2.0.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/3Hren/msgpack-rust/commits/rmp-serde/v1.2.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This is good practice as all our iterators will never return a value
after reaching `None`
The benefit should be minimal as only `Iterator::fuse` is directly
specialized and itself rarely used (sometimes in `itertools` adaptors)
Thus it is mostly a documentation thing
# Description
When a closure if provided to `group-by`, errors that occur in the
closure are currently ignored. That is, `group-by` will fall back and
use the `"error"` key if an error occurs. For example, the code snippet
below will group all `ls` entries under the `"error"` column.
```nushell
ls | group-by { get nope }
```
This PR changes `group-by` to instead bubble up any errors triggered
inside the closure. In addition, this PR also does some refactoring and
cleanup inside `group-by`.
# User-Facing Changes
Errors are now returned from the closure provided to `group-by` instead
of falling back to the `"error"` group/key.
# Description
If a panic happens during a plugin call, because it always happens
outside of the main thread, it currently just hangs Nushell because the
plugin stays running without ever producing a response to the call.
This adds a panic handler that calls `exit(1)` after the unwind finishes
to the plugin runner. The panic error is still printed to stderr as
always, and waiting for the unwind to finish helps to ensure that
anything on the stack with `Drop` behavior that needed to run still
runs, at least on that thread.
# User-Facing Changes
Panics now look like this, which is what they looked like before the
plugin behavior was moved to a separate thread:
```
thread 'plugin runner (primary)' panicked at crates/nu_plugin_example/src/commands/main.rs:45:9:
Test panic
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: nu:🐚:plugin_failed_to_decode
× Plugin failed to decode: Failed to receive response to plugin call
```
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
A little refactor that use `working_set.error` rather than
`working_set.parse_errors.push`, which is reported here:
https://github.com/nushell/nushell/pull/12238
> Inconsistent error reporting. Usage of both working_set.error() and
working_set.parse_errors.push(). Using ParseError::Expected for an
invalid variable name when there's ParseError::VariableNotValid (from
parser.rs:5237). Checking variable names manually when there's
is_variable() (from parser.rs:2905).
# User-Facing Changes
NaN
# Tests + Formatting
Done
# Description
Remove a couple of legacy fields (`input_type`, `output_type`), and
`var_id` which is optional and not required for deserialization.
I think until I document this in the plugin protocol ref, most people
will probably be using this example to get started, so it should be as
correct as possible
# After Submitting
- [ ] TODO: document `Signature` in plugin protocol reference
# Description
This is just some cleanup. I moved to_pipeline_data and to_cache_value
to the CustomValueSupport trait, where I should've put them to begin
with.
Co-authored-by: Jack Wright <jack.wright@disqo.com>
# Description
Work for #7149
- **Error `with-env` given uneven count in list form**
- **Fix `with-env` `CantConvert` to record**
- **Error `with-env` when given protected env vars**
- **Deprecate list/table input of vars to `with-env`**
- **Remove examples for deprecated input**
# User-Facing Changes
## Deprecation of the following forms
```
> with-env [MYENV "my env value"] { $env.MYENV }
my env value
> with-env [X Y W Z] { $env.X }
Y
> with-env [[X W]; [Y Z]] { $env.W }
Z
```
## recommended standardized form
```
# Set by key-value record
> with-env {X: "Y", W: "Z"} { [$env.X $env.W] }
╭───┬───╮
│ 0 │ Y │
│ 1 │ Z │
╰───┴───╯
```
## (Side effect) Repeated definitions in an env shorthand are now
disallowed
```
> FOO=bar FOO=baz $env
Error: nu:🐚:column_defined_twice
× Record field or table column used twice: FOO
╭─[entry #1:1:1]
1 │ FOO=bar FOO=baz $env
· ─┬─ ─┬─
· │ ╰── field redefined here
· ╰── field first defined here
╰────
```
# Description
@maxim-uvarov discovered the following error:
```
> [[a b]; [6 2] [1 4] [4 1]] | polars into-lazy | polars sort-by a | polars unique --subset [a]
Error: × Error using as series
╭─[entry #1:1:68]
1 │ [[a b]; [6 2] [1 4] [4 1]] | polars into-lazy | polars sort-by a | polars unique --subset [a]
· ──────┬──────
· ╰── dataframe has more than one column
╰────
```
During investigation, I discovered the root cause was that the lazy frame was incorrectly converted back to a eager dataframe. In order to keep this from happening, I explicitly set that the dataframe did not come from an eager frame. This causes the conversion logic to not attempt to convert the dataframe later in the pipeline.
---------
Co-authored-by: Jack Wright <jack.wright@disqo.com>
# Description
Fixes#12520
# User-Facing Changes
Breaking change:
Any operation parsing input with `PWD` to set the environment will now
fail with `ShellError::AutomaticEnvVarSetManually`
Furthermore transactions containing the special env-vars will be
rejected before executing any modifications. Prevoiusly this was
changing valid variables before while leaving valid variables after the
violation untouched.
## `PWD` handling.
Now failing
```
{PWD: "/trolling"} | load-env
```
already failing
```
load-env {PWD: "/trolling"}
```
## Error management
```
> load-env {MY_VAR1: foo, PWD: "/trolling", MY_VAR2: bar}
Error: nu:🐚:automatic_env_var_set_manually
× PWD cannot be set manually.
╭─[entry #1:1:2]
1 │ load-env {MY_VAR1: foo, PWD: "/trolling", MY_VAR2: bar}
· ────┬───
· ╰── cannot set 'PWD' manually
╰────
help: The environment variable 'PWD' is set automatically by Nushell and cannot be set manually.
```
### Before:
```
> $env.MY_VAR1
foo
> $env.MY_VAR2
Error: nu:🐚:name_not_found
....
```
### After:
```
> $env.MY_VAR1
Error: nu:🐚:name_not_found
....
> $env.MY_VAR2
Error: nu:🐚:name_not_found
....
```
# After Submitting
We need to check if any integrations rely on this hack.
# Description
Adds support for running plugins using local socket communication
instead of stdio. This will be an optional thing that not all plugins
have to support.
This frees up stdio for use to make plugins that use stdio to create
terminal UIs, cc @amtoine, @fdncred.
This uses the [`interprocess`](https://crates.io/crates/interprocess)
crate (298 stars, MIT license, actively maintained), which seems to be
the best option for cross-platform local socket support in Rust. On
Windows, a local socket name is provided. On Unixes, it's a path. The
socket name is kept to a relatively small size because some operating
systems have pretty strict limits on the whole path (~100 chars), so on
macOS for example we prefer `/tmp/nu.{pid}.{hash64}.sock` where the hash
includes the plugin filename and timestamp to be unique enough.
This also adds an API for moving plugins in and out of the foreground
group, which is relevant for Unixes where direct terminal control
depends on that.
TODO:
- [x] Generate local socket path according to OS conventions
- [x] Add support for passing `--local-socket` to the plugin executable
instead of `--stdio`, and communicating over that instead
- [x] Test plugins that were broken, including
[amtoine/nu_plugin_explore](https://github.com/amtoine/nu_plugin_explore)
- [x] Automatically upgrade to using local sockets when supported,
falling back if it doesn't work, transparently to the user without any
visible error messages
- Added protocol feature: `LocalSocket`
- [x] Reset preferred mode to `None` on `register`
- [x] Allow plugins to detect whether they're running on a local socket
and can use stdio freely, so that TUI plugins can just produce an error
message otherwise
- Implemented via `EngineInterface::is_using_stdio()`
- [x] Clean up foreground state when plugin command exits on the engine
side too, not just whole plugin
- [x] Make sure tests for failure cases work as intended
- `nu_plugin_stress_internals` added
# User-Facing Changes
- TUI plugins work
- Non-Rust plugins could optionally choose to use this
- This might behave differently, so will need to test it carefully
across different operating systems
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] Document local socket option in plugin contrib docs
- [ ] Document how to do a terminal UI plugin in plugin contrib docs
- [ ] Document: `EnterForeground` engine call
- [ ] Document: `LeaveForeground` engine call
- [ ] Document: `LocalSocket` protocol feature
# Description
In the plugin protocol, I had used `#[serde(untagged)]` on the `Stream`
variant to make it smaller and include all of the stream messages at the
top level, but unfortunately this causes serde to make really unhelpful
errors if anything fails to decode anywhere:
```
Error: nu:🐚:plugin_failed_to_decode
× Plugin failed to decode: data did not match any variant of untagged enum PluginOutput
```
If you are trying to develop something using the plugin protocol
directly, this error is incredibly unhelpful. Even as a user, this
basically just says 'something is wrong'. With this change, the errors
are much better:
```
Error: nu:🐚:plugin_failed_to_decode
× Plugin failed to decode: unknown variant `PipelineDatra`, expected one of `Error`, `Signature`, `Ordering`, `PipelineData` at line 2 column 37
```
The only downside is it means I have to duplicate all of the
`StreamMessage` variants manually, but there's only 4 of them and
they're small.
This doesn't actually change the protocol at all - everything is still
identical on the wire.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
This adds a `SharedCow` type as a transparent copy-on-write pointer that
clones to unique on mutate.
As an initial test, the `Record` within `Value::Record` is shared.
There are some pretty big wins for performance. I'll post benchmark
results in a comment. The biggest winner is nested access, as that would
have cloned the records for each cell path follow before and it doesn't
have to anymore.
The reusability of the `SharedCow` type is nice and I think it could be
used to clean up the previous work I did with `Arc` in `EngineState`.
It's meant to be a mostly transparent clone-on-write that just clones on
`.to_mut()` or `.into_owned()` if there are actually multiple
references, but avoids cloning if the reference is unique.
# User-Facing Changes
- `Value::Record` field is a different type (plugin authors)
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] use for `EngineState`
- [ ] use for `Value::List`
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
* Fixes#12482
* Initial PR failed due to CI issues at the time. Subsequent rebase
failed, so creating new PR.
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Use https://github.com/ndjson/ndjson-spec for help links instead of
former spam site
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Link changed for `help to ndjson` and `help from ndjson`.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Added a method for getting the base value for a PluginCustomValue.
cc: @devyn
---------
Co-authored-by: Jack Wright <jack.wright@disqo.com>
# Description
From @maxim-uvarov's
[post](https://discord.com/channels/601130461678272522/1227612017171501136/1228656319704203375).
When calling `to-lazy` back to back in a pipeline, an error should not
occur:
```
> [[a b]; [6 2] [1 4] [4 1]] | polars into-lazy | polars into-lazy
Error: nu:🐚:cant_convert
× Can't convert to NuDataFrame.
╭─[entry #1:1:30]
1 │ [[a b]; [6 2] [1 4] [4 1]] | polars into-lazy | polars into-lazy
· ────────┬───────
· ╰── can't convert NuLazyFrameCustomValue to NuDataFrame
╰────
```
This pull request ensures that custom value's of NuLazyFrameCustomValue are properly converted when passed in.
Co-authored-by: Jack Wright <jack.wright@disqo.com>
# Description
Close: #12147Close: #11796
About the change: it make pattern handling into a function:
`ls_for_one_pattern`(for ls), `du_for_one_pattern`(for du). Then
iterates on user input pattern, call these core function, and chaining
these iterator to one pipelinedata.
# Description
- Refactors `first` and `last` using `Vec::truncate` and `Vec::drain`.
- `std::mem::take` was also used to eliminate a few `Value` clones.
- The `NeedsPositiveValue` error now uses the span of the `rows`
argument instead of the call head span.
- `last` now errors on an empty stream to match `first` which does
error.
- Made metadata preservation more consistent.
# User-Facing Changes
Breaking change: `last` now errors on an empty stream to match `first`
which does error.
# Description
@maxim-uvarov discovered an issue with the current implementation. When
executing [[index a]; [1 1]] | polars into-df, a plugin_failed_to_decode
error occurs. This happens because a Record is created with two columns
named "index" as an index column is added during conversion. This pull
request addresses the problem by not adding an index column if there is
already a column named "index" in the dataframe.
---------
Co-authored-by: Jack Wright <jack.wright@disqo.com>
# Description
All polars commands that output a file were not handling relative paths
correctly.
A command like
``` [[a b]; [6 2] [1 4] [4 1]] | polars into-df | polars to-parquet foo.json```
was outputting the foo.json to the directory of the plugin executable.
This pull request pulls in nu-path and using it for resolving the file paths.
Related discussion
https://discord.com/channels/601130461678272522/1227612017171501136/1227889870358183966
# User-Facing Changes
None
# Tests + Formatting
Done, added tests for each of the polars to-* commands.
---------
Co-authored-by: Jack Wright <jack.wright@disqo.com>
# Description
The `kill` command returns a stream with a single value. This PR changes
it to simply return the value.
# User-Facing Changes
Technically a breaking change.
# Description
Refactors `drop` using `Vec::truncate` and adds a `NeedsPositiveValue`
error.
# User-Facing Changes
Breaking change: `drop` now errors if the number of rows/columns is
negative.
# Description
This PR just adds better logging for shell_integration and tweaks the
ansi escapes so they're closer to where the action happens. I also added
some perf log entries to help better understand plugin file load and
eval performance.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
`polars ls` is already different that `dfr ls`. Currently it just shows
the cache key, columns, rows, and type. I have added:
- creation time
- size
- span contents
- span start and end
<img width="1471" alt="Screenshot 2024-04-10 at 17 27 06"
src="https://github.com/nushell/nushell/assets/56345/545918b7-7c96-4c25-bc01-b9e2b659a408">
# Tests + Formatting
Done
Co-authored-by: Jack Wright <jack.wright@disqo.com>
related to
- https://github.com/nushell/nushell/pull/12196
# Description
while i'm 100% okey with the original intent behind
https://github.com/nushell/nushell/pull/12196, i think the PR did
introduce two unintended things:
- extra parentheses that make the `log.nu` module look like Lisp lol
- a renaming of the `NU_LOG_LEVEL` environment variable to
`NU_log-level`. this breaks previous usage of `std log` and, as it's not
mentionned at all in the PR, i thought it was not intentional 😋
# User-Facing Changes
users can now control `std log` with `$env.NU_LOG_LEVEL`
# Tests + Formatting
the "log" tests have been fixed as well.
# After Submitting
# Description
Fixes: #11996
After this change `let t = timeit ^ls` will list current directory to
stdout.
```
❯ let t = timeit ^ls
CODE_OF_CONDUCT.md Cargo.lock Cross.toml README.md aaa benches devdocs here11 scripts target toolkit.nu wix
CONTRIBUTING.md Cargo.toml LICENSE a.txt assets crates docker rust-toolchain.toml src tests typos.toml
```
If user don't want such behavior, he can redirect the stdout to `std
null-stream` easily
```
> use std
> let t = timeit { ^ls o> (std null-device) }
```
# User-Facing Changes
NaN
# Tests + Formatting
Done
# After Submitting
Nan
---------
Co-authored-by: Ian Manske <ian.manske@pm.me>
# Description
This is an attempt to isolate the unit tests from whatever might be in
the user's config. If the
user's config is broken in some way or incompatible with this version
(for example, especially if
there are plugins that aren't built for this version), tests can
spuriously fail.
This makes tests more reliably pass the same way they would on CI even
if the user has config, and
should also make them run faster.
I think this is _good enough_, but I still think we should have a
specific config dir env variable for nushell specifically (rather than
having to use `XDG_CONFIG_HOME`, which would mess with other things) and
then we can just have `nu-test-support` set that to a temporary dir
containing the shipped default config files.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
I spent a while trying to come up with a good name for what is currently
`IoStream`. Looking back, this name is not the best, because it:
1. Implies that it is a stream, when it all it really does is specify
the output destination for a stream/pipeline.
2. Implies that it handles input and output, when it really only handles
output.
So, this PR renames `IoStream` to `OutDest` instead, which should be
more clear.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Fixes#7849, #11465 based on @kubouch's suggestion in
https://github.com/nushell/nushell/issues/11465#issuecomment-1883847806.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Can source files relative to `env.nu` or `config.nu` like in #6150.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Adds test that previously failed.
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
I have `nu` set as my shell in my editor, which allows me to easily pipe
selections of text to things like `str pascal-case` or even more complex
string operation pipelines, which I find super handy. However, the only
annoying thing is that I pretty much always have to add `| print -n` at
the end, because `nu` adds a newline when it prints the resulting value.
This adds a `--no-newline` option to stop that from happening, and then
you don't need to pipe to `print -n` anymore, you can just have your
shell command for your editor contain that flag.
# User-Facing Changes
- Add `--no-newline` command line option
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
This allows plugins to view the source code of spans.
Requested by @ayax79 for implementing `polars ls`. Note that this won't
really help you find the location of the span. I'm planning to add
another engine call that will return information more similar to what
shows up in the miette diagnostics, with filename / line number / some
context, but I'll want to refactor some of the existing logic to make
that happen, so it was easier to just do this first. I hope this is
enough to at least have something somewhat useful show up for `polars
ls`.
# User-Facing Changes
- Example plugin: added `example view span` command
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] Add to plugin protocol reference
# Description
This PR tries to be a bit more precise with the repl logging when
starting nushell with `nu --log-level debug`. It adds a few more `perf`
lines and changes some of the text of others.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
Changed `export` for `import`
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
`help stor import` showed a help string that was probably copy-pasted
from `stor export`
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
# User-Facing Changes
Now `help stor import` shows a correct description of the operation that
it is doing
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Edits the `echo` help text to mention the `print` command.
---------
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
# Description
@ayax79 added `nu-cmd-lang` as a dep for `nu-plugin-test-support` in
order to get access to `let`. Since we have the dep anyway now, we might
as well just add all of the lang commands - there aren't very many of
them and it would be less confusing than only `let` working.
# User-Facing Changes
- Can use some more core nu language features in plugin tests, like
loops and `do`
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] Might need to change something about the plugin testing section of
the book, since I think it says something about there only being the
plugin command itself available
# Description
This decouples the serialized representation of `Record` from its
internal implementation. It now gets treated as a map type in `serde`.
This has several benefits:
- more efficient representation (not showing inner fields)
- human readable e.g. as a JSON record
- no breaking changes when refactoring the `Record` internals in the
future (see #12326, or potential introduction of `indexmap::IndexMap`
for large N)
- we now deny the creation of invalid records a non-cooperating plugin
could produce
- guaranteed key-value correspondence
- checking for unique keys
# Breaking change to the plugin protocol:
Now expects a record/map directly as the `Record.val` field instead of a
serialization of it.
# Description
The `let` command is needed for many example tests. This pull request
adds the `let` command to the EngineState of Test Plugin.
cc: @devyn
# User-Facing Changes
No user changes. Plugin tests can now have examples with the let
keyword.
Co-authored-by: Jack Wright <jack.wright@disqo.com>
# Description
This closes (nushell#10591)
The Command encode's help text says that utf-16le and utf-16be encodings
are not supported, however you could still use these encodings and they
didn't work properly, since they returned the bytes UTF-8 encoded:
```bash
"䆺ש" | encode utf-16
Length: 5 (0x5) bytes | printable whitespace ascii_other non_ascii
00000000: e4 86 ba d7 a9 ×××××
```
# User-Facing Changes
The Command encode's help text was updated and now when trying to encode with utf-16le and utf-16be returns an error:
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
Currently, `Range` is a struct with a `from`, `to`, and `incr` field,
which are all type `Value`. This PR changes `Range` to be an enum over
`IntRange` and `FloatRange` for better type safety / stronger compile
time guarantees.
Fixes: #11778Fixes: #11777Fixes: #11776Fixes: #11775Fixes: #11774Fixes: #11773Fixes: #11769.
# User-Facing Changes
Hopefully none, besides bug fixes.
Although, the `serde` representation might have changed.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
Resolves#11756.
Resolves#12346.
As per description, shell no longer hangs:
```
~/CodingProjects/nushell> [1 2 3] | select (-2)
Error: nu:🐚:cant_convert
× Can't convert to cell path.
╭─[entry #1:1:18]
1 │ [1 2 3] | select (-2)
· ──┬─
· ╰── can't convert negative number to cell path
╰────
```
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Added relevant test 🚀
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
Possibly support `get` `get`ting negative numbers, as per #12346
discussion. Alternatively, we can consider adding a cellpath for
negative indexing?
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
I was playing around with auto-cd and realised it didn't check for
permissions before cd'ing. This PR fixes that.
```
~/CodingProjects/nushell> /root
Error: nu:🐚:io_error
× I/O error
help: Cannot change directory to /root: You are neither the owner, in the group, nor the super user and do not have permission
```
This PR also refactors some of the filesystem utilities to nu-utils,
specifically the permissions checking and users.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
This speeds up writing messages to the plugin, because otherwise every
individual piece of the messages (not even the entire message) is
written with one syscall, leading to a lot of back and forth with the
kernel.
I learned this by running `strace` to debug something and saw a ton of
`write()` calls.
```nushell
# Before
1..10 | each { timeit { example seq 1 10000 | example sum } } | math avg
269ms 779µs 149ns
# After
> 1..10 | each { timeit { example seq 1 10000 | example sum } } | math avg
39ms 636µs 643ns
```
# User-Facing Changes
- Performance improvement
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
Requested by @ayax79. This makes the custom value behavior more correct,
by calling the methods on the plugin to handle the custom values in
examples rather than the methods on the custom values themselves. This
helps for handle-type custom values (like what he's doing with
dataframes).
- Equality checking in `PluginTest::test_examples()` changed to use
`PluginInterface::custom_value_partial_cmp()`
- Base value rendering for `PluginSignature` changed to use
`Plugin::custom_value_to_base_value()`
- Had to be moved closer to `serve_plugin` for this reason, so the test
for writing signatures containing custom values was removed
- That behavior should still be tested to some degree, since if custom
values are not handled, signatures will fail to parse, so all of the
other tests won't work.
# User-Facing Changes
- `Record::sort_cols()` method added to share functionality required by
`PartialCmp`, and it might also be slightly faster
- Otherwise, everything should mostly be the same but better. Plugins
that don't implement special handling for custom values will still work
the same way, because the default implementation is just a pass-through
to the `CustomValue` methods.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
Because the plugin interface reader thread can be responsible for
sending a drop notification, it's possible for it to end up in a
deadlock where it's waiting for the response to the drop notification
call.
I decided that the best way to address this is to just discard the
response and not wait for it. It's not really important to synchronize
with the response to `Dropped`, so this is probably faster anyway.
cc @ayax79, this is your issue where polars is getting stuck
# User-Facing Changes
- A bug fix
- Custom value plugin: `custom-value handle update` command
# Tests + Formatting
Tried to add a test with a long pipeline with a lot of drops and run it
over and over to reproduce the deadlock.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
fixes#12361
Looking at the condition, `TRASH_SUPPORTED && (trash || (rm_always_trash
&& !permanent))`, this code path seems only to run when `--trash` is
enabled and `--permanent` is disabled.
This suggests that the `--trash` suggestion is a mistake and should have
suggested `--permanent`.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Resolves#11800.
```
~/CodingProjects/nushell> def "url expand" [$urls:any = []]: [string -> string, list -> table] {
::: let urls = ($in | default $urls)
::: def expand-link [] {
::: http head --redirect-mode manual $in | where name == location | get value.0
::: }
::: match ($urls | describe) {
::: string => { $urls | expand-link }
::: $type if ($type =~ list) => { $urls | wrap link | insert expanded {|url| $url.link | expand-link}}
::: }
::: }; view source "url expand"
def "url expand" [ $urls: any = [] ]: [string -> string, list<any> -> table] {
let urls = ($in | default $urls)
def expand-link [] {
http head --redirect-mode manual $in | where name == location | get value.0
}
match ($urls | describe) {
string => { $urls | expand-link }
$type if ($type =~ list) => { $urls | wrap link | insert expanded {|url| $url.link | expand-link}}
}
}
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
`view source` now
- adds quotes to commands with spaces
- shows default argument values
- shows type signatures
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
The error message when using `dfr open --type` shows an outdated list of
supported formats.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
User is now informed that jsonl and avro formats are supported.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Done.
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
No doc changes.
# Description
This fixes#12391.
nushell/nushell@87c5f6e455 accidentally introduced a bug where the path
was not being properly
expanded according to the cwd. This makes both 'touch' and 'mkdir' use
globs just like the rest of
the commands to preserve tilde behavior while still expanding the paths
properly.
This doesn't actually expand the globs. Should it?
# User-Facing Changes
- Restore behavior of `mkdir`, `touch`
- Help text now says they can take globs, but they won't actually expand
them, maybe this should be changed
# Tests + Formatting
Regression tests added.
# After Submitting
This is severe enough and should be included in the point release.
# Description
Fixes#12382, where overlay changes from hooks were not preserved into
the global state. This was due to creating child stacks for hooks, when
the global stack should have been used instead.
# Description
This keeps plugin custom values that have requested drop notification
around during the lifetime of a plugin call / stream by sending them to
a channel that gets persisted during the lifetime of the call.
Before this change, it was very likely that the drop notification would
be sent before the plugin ever had a chance to handle the value it
received.
Tests have been added to make sure this works - see the `custom_values`
plugin.
cc @ayax79
# User-Facing Changes
This is basically just a bugfix, just a slightly big one.
However, I did add an `as_mut_any()` function for custom values, to
avoid having to clone them. This is a breaking change.
Some platforms don't support the `system-clipboard` feature, notably
termux on android.
The default config currently contained references to `reedline` events
that are only available with the feature enabled (#12179). This thus
broke the out of the box config for those users.
For now be more defensive about this and only enable default events. Add
the alternative as commented out code you can quickly enable.
## Tested with:
```
cargo run --no-default-features --features default-no-clipboard -- --config crates/nu-utils/src/sample_config/default_config.nu
```
- [x] `cargo hack` feature flag compatibility run
- [x] reedline released and pinned
- [x] `nu-plugin-test-support` added to release script
- [x] dependency tree checked
- [x] release notes
# Description
Fixes how the directory permissions are calculated in `mkdir`. Instead
of subtraction, the umask is actually used as a mask via negation
followed by bitwise and with the default mode. This matches how [uucore
calculates](cac7155fba/src/uu/mkdir/src/mkdir.rs (L61))
the mode.
# Description
This shrinks `Record`'s size in half and and allows you to include it in
`Value` without growing the size.
Changing the `Record` internals may have slightly different performance
characteristics as the cache locality changes on lookups (if you
directly need the value, it should be closer, but in other cases may
blow up the cache line budget)
Also different perf characteristics on creation expected.
`Record::from_raw_cols_vals` now probably worse.
## Benchmarking
Comparison with the main branch (boxed Record) revealed no significant
change to the creation but an improvement when accessing larger N.
The fact that this was more pronounced for nested access (still cloning
before nushell/nushell#12325) leads to the conclusion that this may
still be dominated by the smaller clone necessary for a 24-byte `Record`
over the previous 48 bytes.
# User-Facing Changes
Reduced memory usage
# Description
Currently `into bits` will try to coerce a `date`/`Value::Date` into a
string with a locale/timezone specific behavior (See #12268).
To resolve the ambiguity, remove the support for `date` entirely.
# User-Facing Changes
`date now | into bits` will now fail.
Instead you can use `... | format date '%c' | into bits` or any more
specific explicit choices to achieve the same behavior.
As `into bits` has minimal uses (and only pulled out of `extra` with
#12140), this doesn't warrant a deprecation.
# Description
Where possible, this PR replaces usages of raw `libc` bindings to
instead use safe interfaces from the `nix` crate. Where not possible,
the `libc` version reexported through `nix` was used instead of having a
separate `libc` dependency.
# Description
This changes the interface for plugins to always represent errors as
`LabeledError`s. This is good for altlang plugins, as it would suck for
them to have to implement and track `ShellError`. We save a lot of
generated code from the `ShellError` serde impl too, so `nu` and plugins
get to have a smaller binary size.
Reduces the release binary size by 1.2 MiB on my build configuration.
# User-Facing Changes
- Changes plugin protocol. `ShellError` no longer serialized.
- `ShellError` serialize output is different
- `ShellError` no longer deserializes to exactly the same value as
serialized
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] Document in plugin protocol reference
Those allocations are all small and insignificant in the grand scheme of
things and the optimizer may be able to resolve some of those but better
to be nice anyways.
Primarily inspired by the new
[`clippy::assigning_clones`](https://rust-lang.github.io/rust-clippy/master/index.html#/assigning_clones)
- **Avoid reallocs with `clone_from` in `nu-parser`**
- **Avoid realloc on assignment in `Stack`**
- **Fix `clippy::assigning_clones` in `nu-cli`**
- **Reuse allocations in `nu-explore` if possible**
# Description
This clone is not necessary and tanks the performance of deep nested
access.
As soon as we found the value, we know we discard the old value, so can
`std::mem::take` the inner (`impl Default for Value` to the rescue)
We may be able to further optimize this but not having to clone the
value is vital.
# Description
This pr is addressing feedback from
https://github.com/nushell/nushell/pull/12277#issuecomment-2027246752
Currently I think it's fine to replace `--legacy` flag with `--guess`
one. Only use `guess_width` algorithm if `--guess` is provided.
# User-Facing Changes
So it won't be a breaking change to previous version.
# Description
In #10232, the allowed input types were changed to be stricter, only
allowing records with types that can easily map onto sqlite equivalents.
Unfortunately, null was left out of the accepted input types, which
makes inserting rows with null values impossible.
This change fixes that by accepting null values as input.
One caveat of this is that when the command is creating a new table, it
uses the first row to infer an appropriate sqlite schema. If the first
row contains a null value, then it is impossible to tell which type this
column is supposed to have.
Throwing a hard error seems undesirable from a UX perspective, but
guessing can lead to a potentially useless database if we guess wrong.
So as a compromise, for null columns, we will assume the sqlite type is
TEXT and print a warning so the user knows. For the time being, if users
can't avoid a first row with null values, but also wants the right
schema, they are advised to create their table before running `into
sqlite`.
A future PR can add the ability to explicitly specify a schema.
Fixes#12225
# Tests + Formatting
* Tests added to cover expected behavior around insertion of null values
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Closes#12253.
Exposes the option as "recursion_limit" under config.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
The config file now has a new option!
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
Nothing else...? Do let me know if there's something I've missed!
# Description
Fixes `open --raw file o> out.txt` and other instances where
`PipelineData::ExternalStream` is created from sources that are not
external commands.
# Description
This PR adds a few more `trace!()` and `perf()` statements that allowed
a deeper understanding of the nushell startup process when used with `nu
-n --no-std-lib --log-level trace`.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Again avoid uses of the `Record` internals, so we are free to change the
data layout
- **Don't use internals of `Record` in `into sqlite`**
- **Don't use internals of `Record` in `to xml`**
Remaining: `rename`
# User-Facing Changes
None
# Description
This PR reverts sqlparser to 0.39.0. It should stay here until we can
get polars updated so that we don't have to have two versions of
sqlparser.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
The second `Value` is redundant and will consume five extra bytes on
each transmission of a custom value to/from a plugin.
# User-Facing Changes
This is a breaking change to the plugin protocol.
The [example in the protocol
reference](https://www.nushell.sh/contributor-book/plugin_protocol_reference.html#value)
becomes
```json
{
"Custom": {
"val": {
"type": "PluginCustomValue",
"name": "database",
"data": [36, 190, 127, 40, 12, 3, 46, 83],
"notify_on_drop": true
},
"span": {
"start": 320,
"end": 340
}
}
}
```
instead of
```json
{
"CustomValue": {
...
}
}
```
# After Submitting
Update plugin protocol reference
# Description
Now we only use `nix 0.28.0`
Achieved by
- updating `ctrlc` to `3.4.4`
- updating `wl-clipboard-rs` to `0.8.1`
- update our own dependency on `nix` from `0.27` to `0.28`
- required fixing uses of `nix::unistd::{tcgetpgrp,tcsetpgrp}`
- now requires an I/O safe file descriptor
- fake one pointing to `libc::STDIN_FILENO` (we were only accessing
`0` previously, dito for fish)
# User-Facing Changes
Better compile times and less to download as source dependencies
# Description
This is something that was discussed in the core team meeting last
Wednesday. @ayax79 is building `nu-plugin-polars` with all of the
dataframe commands into a plugin, and there are a lot of them, so it
would help to make the API more similar. At the same time, I think the
`Command` API is just better anyway. I don't think the difference is
justified, and the types for core commands have the benefit of requiring
less `.into()` because they often don't own their data
- Broke `signature()` up into `name()`, `usage()`, `extra_usage()`,
`search_terms()`, `examples()`
- `signature()` returns `nu_protocol::Signature`
- `examples()` returns `Vec<nu_protocol::Example>`
- `PluginSignature` and `PluginExample` no longer need to be used by
plugin developers
# User-Facing Changes
Breaking API for plugins yet again 😄
Bumps [ical](https://github.com/Peltoche/ical-rs) from 0.10.0 to 0.11.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/Peltoche/ical-rs/releases">ical's
releases</a>.</em></p>
<blockquote>
<h2>v0.11.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Update the version inside the readme by <a
href="https://github.com/Peltoche"><code>@Peltoche</code></a> in <a
href="https://redirect.github.com/Peltoche/ical-rs/pull/58">Peltoche/ical-rs#58</a></li>
<li>Fix <a
href="https://redirect.github.com/Peltoche/ical-rs/issues/62">#62</a> by
<a href="https://github.com/ddnomad"><code>@ddnomad</code></a> in <a
href="https://redirect.github.com/Peltoche/ical-rs/pull/63">Peltoche/ical-rs#63</a></li>
<li>replaced split_line with a multibyte aware version by <a
href="https://github.com/ronnybremer"><code>@ronnybremer</code></a> in
<a
href="https://redirect.github.com/Peltoche/ical-rs/pull/61">Peltoche/ical-rs#61</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/ddnomad"><code>@ddnomad</code></a> made
their first contribution in <a
href="https://redirect.github.com/Peltoche/ical-rs/pull/63">Peltoche/ical-rs#63</a></li>
<li><a
href="https://github.com/ronnybremer"><code>@ronnybremer</code></a>
made their first contribution in <a
href="https://redirect.github.com/Peltoche/ical-rs/pull/61">Peltoche/ical-rs#61</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/Peltoche/ical-rs/compare/v0.10.0...v0.11.0">https://github.com/Peltoche/ical-rs/compare/v0.10.0...v0.11.0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="c2f6bb3be9"><code>c2f6bb3</code></a>
chore: Release ical version 0.11.0</li>
<li><a
href="e435769c7b"><code>e435769</code></a>
final fix to test for new split_line</li>
<li><a
href="1db49580e1"><code>1db4958</code></a>
fixed incorrect test for new split_line</li>
<li><a
href="248227b08d"><code>248227b</code></a>
added test case with multibyte characters for split_line</li>
<li><a
href="ba696e5c02"><code>ba696e5</code></a>
take 75 chars of the first line and 74 chars of subsequent lines</li>
<li><a
href="28ffa72bb1"><code>28ffa72</code></a>
replaced split_line with a multibyte aware version</li>
<li><a
href="a10a15d571"><code>a10a15d</code></a>
Fix <a
href="https://redirect.github.com/Peltoche/ical-rs/issues/62">#62</a></li>
<li><a
href="7f93147560"><code>7f93147</code></a>
Update the version inside the readme</li>
<li>See full diff in <a
href="https://github.com/Peltoche/ical-rs/compare/v0.10.0...v0.11.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
# Description
When implementing a `Command`, one must also import all the types
present in the function signatures for `Command`. This makes it so that
we often import the same set of types in each command implementation
file. E.g., something like this:
```rust
use nu_protocol::ast::Call;
use nu_protocol::engine::{Command, EngineState, Stack};
use nu_protocol::{
record, Category, Example, IntoInterruptiblePipelineData, IntoPipelineData, PipelineData,
ShellError, Signature, Span, Type, Value,
};
```
This PR adds the `nu_engine::command_prelude` module which contains the
necessary and commonly used types to implement a `Command`:
```rust
// command_prelude.rs
pub use crate::CallExt;
pub use nu_protocol::{
ast::{Call, CellPath},
engine::{Command, EngineState, Stack},
record, Category, Example, IntoInterruptiblePipelineData, IntoPipelineData, IntoSpanned,
PipelineData, Record, ShellError, Signature, Span, Spanned, SyntaxShape, Type, Value,
};
```
This should reduce the boilerplate needed to implement a command and
also gives us a place to track the breadth of the `Command` API. I tried
to be conservative with what went into the prelude modules, since it
might be hard/annoying to remove items from the prelude in the future.
Let me know if something should be included or excluded.
# Description
Binary values passed to `table` may or may not be pretty formatted based
on the output destination. This leads to weird behavior as documented in
#12287. This PR changes `table` to always pretty print binary values.
However, binary values passed to external commands will not be formatted
(this is the existing behavior).
# User-Facing Changes
This is a breaking change. E.g.:
```nushell
0x[8989] | table | cat -
```
used to print raw bytes, but it will now print the pretty formatted
bytes.
# After Submitting
Add to 0.92.0 release notes and update documentation.
# Description
Closes https://github.com/nushell/nushell/issues/12257
This was down to the use of `eval_block` instead of
`eval_block_with_early_return`. We may want to reconsider how we
differentiate between this behavior. We currently need to check all the
remaining commands that can invoke a closure block, if they properly
handle `ShellError::Return` as a passing of a `Value`
- **Add test for `return` in `filter` closure**
- **Fix use of `return` in `filter` closure**
# User-Facing Changes
You can now return a value from a `filter` closure
# Tests + Formatting
Regression test
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Boxes `Record` inside `Value` to reduce memory usage, `Value` goes from
`72` -> `56` bytes after this change.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
@fdncred found another histogram based algorithm to detect columns, and
rewrite it in rust: https://github.com/fdncred/guess-width
I have tested it manually, and it works good with `df`, `docker ps`,
`^ps`. This pr is going to use the algorithm in `detect columns`
Fix: #4183
The pitfall of new algorithm:
1. it may not works well if there isn't too much rows of input
2. it may not works well if the length of value is less than the header
to value, e.g:
```
c1 c2 c3 c4 c5
a b c d e
g h i j k
g a a q d
a v c q q | detect columns
```
In this case, users might need to use ~~`--old`~~ `--legacy` to make it
works well.
# User-Facing Changes
User might need to add ~~`--old`~~ `--legacy` to scripts if they find
`detect columns` in their scripts broken.
# Tests + Formatting
Done
# After Submitting
NaN
# Description
Uses the new `nu-plugin-test-support` crate to test the examples of
commands provided by plugins in the repo.
Also fixed some of the examples to pass.
# User-Facing Changes
- Examples that are more guaranteed to work
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
Fixes#12280.
# Description
This removes the dependency on the `difference` crate, which is
unmaintained, for `nu-plugin-test-support`. The `similar` crate
(Apache-2.0) is used instead, which is a bit larger and more complex,
but still suitable for a dev dep for tests. Also switched to use
`crossterm` for colors, since `similar` doesn't come with any terminal
pretty printing functionality.
# User-Facing Changes
None - output should be identical.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
Cleanup search terms and help usage to be consistent and include
coreutils so people can easily find out which commands are coreutils.
or
```nushell
help commands | where usage =~ coreutils | reject params input_output
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
Hi,
This PR aims at implementing the first iteration for `uname` using
`uutils`. Couple of things:
* Currently my [PR](https://github.com/uutils/coreutils/pull/5921) to
make the required changes is pending in `uutils` repo.
* I guess the number of flags has to be investigated. Still the tests
cover all of them.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- [X] `cargo fmt --all -- --check` to check standard code formatting
(`cargo fmt --all` applies these changes)
- [X] `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used`
to check that you're using the standard code style
- [X] `cargo test --workspace` to check that all tests pass (on Windows
make sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- [X] `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
---------
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
# Description
Fixes: #11887Fixes: #11626
This pr unify the tilde expand behavior over several filesystem relative
commands. It follows the same rule with glob expansion:
| command | result |
| ----------- | ------ |
| ls ~/aaa | expand tilde
| ls "~/aaa" | don't expand tilde
| let f = "~/aaa"; ls $f | don't expand tilde, if you want to: use `ls
($f \| path expand)`
| let f: glob = "~/aaa"; ls $f | expand tilde, they don't expand on
`mkdir`, `touch` comamnd.
Actually I'm not sure for 4th item, currently it's expanding is just
because it followes the same rule with glob expansion.
### About the change
It changes `expand_path_with` to accept a new argument called
`expand_tilde`, if it's true, expand it, if not, just keep it as `~`
itself.
# User-Facing Changes
After this change, `ls "~/aaa"` won't expand tilde.
# Tests + Formatting
Done
# Description
This commit fills in the completion item kind into the
`textDocument/completion` response so that LSP client can present more
information to the user.
It is an improvement in the context of #10794
# User-Facing Changes
Improved information display in editor's intelli-sense menu
# Description
This PR adds a `--params` param to `query db`. This closes#11643.
You can't combine both named and positional parameters, I think this
might be a limitation with rusqlite itself. I tried using named
parameters with indices like `{ ':named': 123, '1': "positional" }` but
that always failed with a rusqlite error. On the flip side, the other
way around works: for something like `VALUES (:named, ?)`, you can treat
both as positional: `-p [hello 123]`.
This PR introduces some very gnarly code repetition in
`prepared_statement_to_nu_list`. I tried, I swear; the compiler wasn't
having any of it, it kept telling me to box my closures and then it said
that the reference lifetimes were incompatible in the match arms. I gave
up and put the mapping code in the match itself, but I'm still not
happy.
Another thing I'm unhappy about: I don't like how you have to put the
`:colon` in named parameters. I think nushell should insert it if it's
[missing](https://www.sqlite.org/lang_expr.html#parameters). But this is
the way [rusqlite
works](https://docs.rs/rusqlite/latest/rusqlite/trait.Params.html#example-named),
so for now, I'll let it be consistent. Just know that it's not really a
blocker, and it isn't a compatibility change to later make `{ colon: 123
}` work, without the quotes and `:`. This would require allocating and
turning our pretty little `&str` into a `String`, though
# User-Facing Changes
Less incentive to leave yourself open to SQL injection with statements
like `query db $"INSERT INTO x VALUES \($unsafe_user_input)"`.
Additionally, the `$""` syntax being annoying with parentheses plays in
our favor, making users even more likely to use ? with `--params`.
# Tests + Formatting
Hehe
# Description
The hover was bugged with 3 backticks. I don't understand how it worked
before, but this apparently now works correctly on my machine. This is
really puzzling. My next step is to make a test to assert this will
break a little less. I fixed it 3 times in the past
# Tests + Formatting
Added a test to be sure this doesn't breaks again 😄 (at least from
nushell/nushell side)
# Description
@WindSoilder [reported on
Discord](https://discord.com/channels/601130461678272522/855947301380947968/1221233630093901834)
that some plugin stream tests have been failing on the CI. It seems to
just be a timing thing (probably due to busy CI), so this extends the
amount of time that we can wait for a condition to be true.
# User-Facing Changes
None
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
There wasn't really a good way to implement a command group style (e.g.
`from`, `query`, etc.) command in the past that just returns the help
text even if `--help` is not passed. This adds a new engine call that
just does that.
This is actually something I ran into before when developing the dbus
plugin, so it's nice to fix it.
# User-Facing Changes
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] Document `GetHelp` engine call in proto
# Description
Adds a `nu-plugin-test-support` crate with an interface that supports
testing plugins.
Unlike in reality, these plugins run in the same process on separate
threads. This will allow
testing aspects of the plugin internal state and handling serialized
plugin custom values easily.
We still serialize their custom values and all of the engine to plugin
logic is still in play, so
from a logical perspective this should still expose any bugs that would
have been caused by that.
The only difference is that it doesn't run in a different process, and
doesn't try to serialize
everything to the final wire format for stdin/stdout.
TODO still:
- [x] Clean up warnings about private types exposed in trait definition
- [x] Automatically deserialize plugin custom values in the result so
they can be inspected
- [x] Automatic plugin examples test function
- [x] Write a bit more documentation
- [x] More tests
- [x] Add MIT License file to new crate
# User-Facing Changes
Plugin developers get a nice way to test their plugins.
# Tests + Formatting
Run the tests with `cargo test -p nu-plugin-test-support --
--show-output` to see some examples of what the failing test output for
examples can look like. I used the `difference` crate (MIT licensed) to
make it look nice.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] Add a section to the book about testing
- [ ] Test some of the example plugins this way
- [ ] Add example tests to nu_plugin_template so plugin developers have
something to start with
# Description
Just a bunch of miscellaneous fixes to the Rust documentation that I
found recently while doing
a pass on some things.
# User-Facing Changes
None
Bumps [base64](https://github.com/marshallpierce/rust-base64) from
0.21.7 to 0.22.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md">base64's
changelog</a>.</em></p>
<blockquote>
<h1>0.22.0</h1>
<ul>
<li><code>DecodeSliceError::OutputSliceTooSmall</code> is now
conservative rather than precise. That is, the error will only occur if
the decoded output <em>cannot</em> fit, meaning that
<code>Engine::decode_slice</code> can now be used with exactly-sized
output slices. As part of this, <code>Engine::internal_decode</code> now
returns <code>DecodeSliceError</code> instead of
<code>DecodeError</code>, but that is not expected to affect any
external callers.</li>
<li><code>DecodeError::InvalidLength</code> now refers specifically to
the <em>number of valid symbols</em> being invalid (i.e. <code>len % 4
== 1</code>), rather than just the number of input bytes. This avoids
confusing scenarios when based on interpretation you could make a case
for either <code>InvalidLength</code> or <code>InvalidByte</code> being
appropriate.</li>
<li>Decoding is somewhat faster (5-10%)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="5d70ba7576"><code>5d70ba7</code></a>
Merge pull request <a
href="https://redirect.github.com/marshallpierce/rust-base64/issues/269">#269</a>
from marshallpierce/mp/decode-precisely</li>
<li><a
href="efb6c006c7"><code>efb6c00</code></a>
Release notes</li>
<li><a
href="2b91084a31"><code>2b91084</code></a>
Add some tests to boost coverage</li>
<li><a
href="9e9c7abe65"><code>9e9c7ab</code></a>
Engine::internal_decode now returns DecodeSliceError</li>
<li><a
href="a8a60f43c5"><code>a8a60f4</code></a>
Decode main loop improvements</li>
<li><a
href="a25be0667c"><code>a25be06</code></a>
Simplify leftover output writes</li>
<li><a
href="9979cc33bb"><code>9979cc3</code></a>
Keep morsels as separate bytes</li>
<li><a
href="37670c5ec2"><code>37670c5</code></a>
Bump dev toolchain version (<a
href="https://redirect.github.com/marshallpierce/rust-base64/issues/268">#268</a>)</li>
<li>See full diff in <a
href="https://github.com/marshallpierce/rust-base64/compare/v0.21.7...v0.22.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [heck](https://github.com/withoutboats/heck) from 0.4.1 to 0.5.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/withoutboats/heck/blob/master/CHANGELOG.md">heck's
changelog</a>.</em></p>
<blockquote>
<h1>0.5.0</h1>
<ul>
<li>Add <code>no_std</code> support.</li>
<li>Remove non-additive <code>unicode</code> feature. The library now
uses <code>char::is_alphanumeric</code>
instead of the <code>unicode-segmentation</code> library to determine
word boundaries in all cases.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/withoutboats/heck/commits">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
you can also mention related issues, PRs or discussions!
-->
- fixes#11014
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
When the `command_not_found` hook is entered, we set an environment
variable for context. If an unknown command is encountered and the
`command_not_found` context environment variable is already present, it
implies a command in the hook closure is also not found. We stop the
recursion right there.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Incorrect `command_not_found` hooks can be caught without panicking.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Tests are passing.
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
With the release of Rust 1.77.0 today we're able to bump the
rust-toolchain for nushell to 1.75.0.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
Manual checks are added to `parse_let`, `parse_mut`, and `parse_const`.
`parse_var_with_opt_type` is also fixed to update `spans_idx` correctly.
Fixes#12125.
It's technically a fix, but I'd rather not merge this directly. I'm
making this PR to bring into attention the code quality of the parser
code. For example:
* Inconsistent usage of `spans_idx`. What is its purpose, and which
parsing functions need it? I suspect it's possible to remove the usage
of `spans_idx` entirely.
* Lacking documentation for top-level functions. What does `mutable`
mean for `parse_var_with_opt_type()`?
* Inconsistent error reporting. Usage of both `working_set.error()` and
`working_set.parse_errors.push()`. Using `ParseError::Expected` for an
invalid variable name when there's `ParseError::VariableNotValid` (from
`parser.rs:5237`). Checking variable names manually when there's
`is_variable()` (from `parser.rs:2905`).
* `span()` is a terrible name for a function that flattens a bunch of
spans into one (from `nu-protocal/src/span.rs:92`). The top-level
comment (`Used when you have a slice of spans of at least size 1`)
doesn't help either.
I've only looked at a small portion of the parser code; I expect there
are a lot more. These issues made it much harder to fix a simple bug
like #12125. I believe we should invest some effort to cleanup the
parser code, which will ease maintainance in the future. I'll willing to
help if there is interest.
# Description
This makes `LabeledError` much more capable of representing close to
everything a `miette::Diagnostic` can, including `ShellError`, and
allows plugins to generate multiple error spans, codes, help, etc.
`LabeledError` is now embeddable within `ShellError` as a transparent
variant.
This could also be used to improve `error make` and `try/catch` to
reflect `LabeledError` exactly in the future.
Also cleaned up some errors in existing plugins.
# User-Facing Changes
Breaking change for plugins. Nicer errors for users.
fixes#11900
# Description
Use `serde_json` instead.
# User-Facing Changes
The problem described in the issue now no longer persists.
No whitespace in the output of `to json --raw`
Output of unicode escape changed to consistent `\uffff`
# Tests + Formatting
I corrected all Tests that were affected by this change.
# Description
@sholderbach left a very helpful review and this just implements the
suggestions he made.
Didn't notice any difference in performance, but there could potentially
be for a long running Nushell session or one that loads a lot of stuff.
I also caught a bug where nu-protocol won't build without `plugin`
because of the previous conditional import. Oops. Fixed.
# User-Facing Changes
`blocks` and `modules` type in `EngineState` changed again. Shouldn't
affect plugins or anything else though really
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
---------
Co-authored-by: sholderbach <sholderbach@users.noreply.github.com>
# Description
Get rid of two parallel `Vec`s in `StateDelta` and `EngineState`, that
also duplicated span information. Use a struct with documenting fields.
Also use `Arc<str>` and `Arc<[u8]>` for the allocations as they are
never modified and cloned often (see #12229 for the first improvement).
This also makes the representation more compact as no capacity is
necessary.
# User-Facing Changes
API breakage on `EngineState`/`StateWorkingSet`/`StateDelta` that should
not really affect plugin authors.
closes#12115
# Description
This fix addresses a bug where the --tabs flag couldn't be utilized due
to improper handling of the tab quantity provided by the user.
Previously, the code mistakenly attempted to convert the tab quantity to
a boolean value, leading to a conversion error. The resolution involves
adjusting the condition clauses to properly validate the presence of the
flag's value. Now, the code checks whether the get_flag() function
returns a value or None associated with the --tabs flag. This adjustment
enables the --tabs flag to function correctly, triggering the
appropriate condition and allowing the conversion to proceed as
expected. Similarly, the fix applies to the --indent flag. Additionally,
a default case was added, and the conversion now works properly without
flags. Two tests were added to validate the corrected behavior of these
flags.
# User-Facing Changes
Now the conversion should work properly instead of displaying an error.
# Tests + Formatting
-🟢 toolkit fmt
-🟢 toolkit clippy
-🟢 toolkit test
-🟢 toolkit test stdlib
To run added tests:
- cargo test --package nu-command --test main --
format_conversions::json::test_tabs_indent_flag
- cargo test --package nu-command --test main --
format_conversions::json::test_indent_flag
# Description
It was a bit ugly that when new `EngineCall`s or response types were
added, they needed to be added to multiple places with redundant code
just to change the types, even if they didn't have any stream content.
This fixes that and locates all of that logic in one place.
# User-Facing Changes
None
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
Improves the accuracy of sleep when the duration is larger than 100ms.
Fixes#12223.
# User-Facing Changes
Sleeping for 150ms should work now.
```nushell
~/nushell> timeit { sleep 150ms } 03/19/2024 10:41:55 AM AM
151ms 344µs 201ns
```
# Description
This makes many of the larger objects in `EngineState` into `Arc`, and
uses `Arc::make_mut` to do clone-on-write if the reference is not
unique. This is generally very cheap, giving us the best of both worlds
- allowing us to mutate without cloning if we have an exclusive
reference, and cloning if we don't.
This started as more of a curiosity for me after remembering that
`Arc::make_mut` exists and can make using `Arc` for mostly immutable
data that sometimes needs to be changed very convenient, and also after
hearing someone complain about memory usage on Discord - this is a
somewhat significant win for that.
The exact objects that were wrapped in `Arc`:
- `files`, `file_contents` - the strings and byte buffers
- `decls` - the whole `Vec`, but mostly to avoid lots of individual
`malloc()` calls on Clone rather than for memory usage
- `blocks` - the blocks themselves, rather than the outer Vec
- `modules` - the modules themselves, rather than the outer Vec
- `env_vars`, `previous_env_vars` - the entire maps
- `config`
The changes required were relatively minimal, but this is a breaking API
change. In particular, blocks are added as Arcs, to allow the parser
cache functionality to work.
With my normal nu config, running on Linux, this saves me about 15 MiB
of process memory usage when running interactively (65 MiB → 50 MiB).
This also makes quick command executions cheaper, particularly since
every REPL loop now involves a clone of the engine state so that we can
recover from a panic. It also reduces memory usage where engine state
needs to be cloned and sent to another thread or kept within an
iterator.
# User-Facing Changes
Shouldn't be any, since it's all internal stuff, but it does change some
public interfaces so it's a breaking change
# Description
As suggested by @WindSoilder, since plugins can now contain both simple
commands that produce `Value` and commands that produce `PipelineData`
without having to choose one or the other for the whole plugin, this
change merges `stream_example` into `example`.
# User-Facing Changes
All of the example plugins are renamed.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] Check nushell/nushell.github.io for any docs that match the
command names changed
[Context on
Discord](https://discord.com/channels/601130461678272522/855947301380947968/1219425984990806207)
# Description
- Rename `CustomValue::value_string()` to `type_name()` to reflect its
usage better.
- Change print behavior to always call `to_base_value()` first, to give
the custom value better control over the output.
- Change `describe --detailed` to show the type name as the subtype,
rather than trying to describe the base value.
- Change custom `Type` to use `type_name()` rather than `typetag_name()`
to make things like `PluginCustomValue` more transparent
One question: should `describe --detailed` still include a description
of the base value somewhere? I'm torn on it, it seems possibly useful
for some things (maybe sqlite databases?), but having `describe -d` not
include the custom type name anywhere felt weird. Another option would
be to add another method to `CustomValue` for info to be displayed in
`describe`, so that it can be more type-specific?
# User-Facing Changes
Everything above has implications for printing and `describe` on custom
values
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
Context: @abusch is working on a semver plugin with custom values and
wants users to be able to convert them back to strings
# Description
This allows `into string` to work on custom values if their base value
representation could be converted into a string with the same rules.
# User-Facing Changes
`into string` works on custom values.
Unfortunately, I couldn't really demo this with an example, because
there aren't any custom values that can be represented that way
included.
# Tests + Formatting
I was able to write a test using the custom values plugin.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
Fixes#12057 where it was pointed out that `export use` takes an
**optional** `members` positional argument whereas `use` takes a
**rest** `members` argument.
# Description
@ayax79 says that the dataframe commands all have dataframe custom
values in their examples, and they're used for tests.
Rather than send the custom values to the engine, if they're in
examples, this change just renders them using `to_base_value()` first.
That way we avoid potentially having to hold onto custom values in
`plugins.nu` that might not be valid indefinitely - as will be the case
for dataframes in particular - but we still avoid forcing plugin writers
to not use custom values in their examples.
# User-Facing Changes
- Custom values usable in plugin examples
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
# Description
`help zip` now reports:
```
other <one_of(any, closure())>: The other input, or closure returning a stream.
```
Thanks to @edhowland for pointing this out ❤️
# User-Facing Changes
- Doc change for zip
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
# Description
We do a lot of visiting contained values in the serialization / validity
functions within `PluginCustomValue` utils. This adds
`Value::recurse_mut()` which wraps up most of that logic into something
that can be reused.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
With the introduction of the system clipboard to nushell, many commands
changed their behavior from using a local cut buffer to the system
clipboard, perhaps surprisingly for many users. (See #11907)
This PR changes most of them back to using the local cut buffer and
introduces three commands (`CutSelectionSystem`, `CopySelectionSystem`
and `PasteSystem`) to explicitly use the system clipboard.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Users who in the meantime already used the system clipboard now default
back to the local clipboard. To be able to use the system clipboard
again they have to append the suffix `system` to their current edit
command specified in their keybindings.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
The commands themselves are tested in `reedline`. The changes introduces
in nushell are minimal and simply forward from a match on the keybinding
name to the command.
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
---------
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
# Description
Adds the `AddEnvVar` plugin call, which allows plugins to set
environment variables in the caller's scope. This is the first engine
call that mutates the caller's stack, and opens the door to more
operations like this if needed.
This also comes with an extra benefit: in doing this, I needed to
refactor how context was handled, and I was able to avoid cloning
`EngineInterface` / `Stack` / `Call` in most cases that plugin calls are
used. They now only need to be cloned if the plugin call returns a
stream. The performance increase is welcome (5.5x faster on `inc`!):
```nushell
# Before
> timeit { 1..100 | each { |i| $"2.0.($i)" | inc -p } }
405ms 941µs 952ns
# After
> timeit { 1..100 | each { |i| $"2.0.($i)" | inc -p } }
73ms 68µs 749ns
```
# User-Facing Changes
- New engine call: `add_env_var()`
- Performance enhancement for plugin calls
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [x] Document env manipulation in plugins guide
- [x] Document `AddEnvVar` in plugin protocol
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
With this change, `mkdir` mirrors coreutils works. Closes#12161
I referred to the implementation of `mkdir` in uutils/coreutils. I add
`uucore` required for implementation to dependencies. Since `uucore` is
already included in dependencies of `uu_mkdir`, I don't think there will
be any additional dependencies.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
- Directories are created according to `umask` except for Windows.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
I add `mkdir` test considering permissions. The test assumes that the
default `umask` is `022`.
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Fixes#12193 where the `$in` value may be null for closures provided to
`insert`.
# User-Facing Changes
The `$in` value will now always be the same as the closure parameter for
`insert`.
[Context on
Discord](https://discord.com/channels/601130461678272522/855947301380947968/1216517833312309419)
# Description
This is a significant breaking change to the plugin API, but one I think
is worthwhile. @ayax79 mentioned on Discord that while trying to start
on a dataframes plugin, he was a little disappointed that more wasn't
provided in terms of code organization for commands, particularly since
there are *a lot* of `dfr` commands.
This change treats plugins more like miniatures of the engine, with
dispatch of the command name being handled inherently, each command
being its own type, and each having their own signature within the trait
impl for the command type rather than having to find a way to centralize
it all into one `Vec`.
For the example plugins that have multiple commands, I definitely like
how this looks a lot better. This encourages doing code organization the
right way and feels very good.
For the plugins that have only one command, it's just a little bit more
boilerplate - but still worth it, in my opinion.
The `Box<dyn PluginCommand<Plugin = Self>>` type in `commands()` is a
little bit hairy, particularly for Rust beginners, but ultimately not so
bad, and it gives the desired flexibility for shared state for a whole
plugin + the individual commands.
# User-Facing Changes
Pretty big breaking change to plugin API, but probably one that's worth
making.
```rust
use nu_plugin::*;
use nu_protocol::{PluginSignature, PipelineData, Type, Value};
struct LowercasePlugin;
struct Lowercase;
// Plugins can now have multiple commands
impl PluginCommand for Lowercase {
type Plugin = LowercasePlugin;
// The signature lives with the command
fn signature(&self) -> PluginSignature {
PluginSignature::build("lowercase")
.usage("Convert each string in a stream to lowercase")
.input_output_type(Type::List(Type::String.into()), Type::List(Type::String.into()))
}
// We also provide SimplePluginCommand which operates on Value like before
fn run(
&self,
plugin: &LowercasePlugin,
engine: &EngineInterface,
call: &EvaluatedCall,
input: PipelineData,
) -> Result<PipelineData, LabeledError> {
let span = call.head;
Ok(input.map(move |value| {
value.as_str()
.map(|string| Value::string(string.to_lowercase(), span))
// Errors in a stream should be returned as values.
.unwrap_or_else(|err| Value::error(err, span))
}, None)?)
}
}
// Plugin now just has a list of commands, and the custom value op stuff still goes here
impl Plugin for LowercasePlugin {
fn commands(&self) -> Vec<Box<dyn PluginCommand<Plugin=Self>>> {
vec![Box::new(Lowercase)]
}
}
fn main() {
serve_plugin(&LowercasePlugin{}, MsgPackSerializer)
}
```
Time this however you like - we're already breaking stuff for 0.92, so
it might be good to do it now, but if it feels like a lot all at once,
it could wait.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] Update examples in the book
- [x] Fix#12088 to match - this change would actually simplify it a
lot, because the methods are currently just duplicated between `Plugin`
and `StreamingPlugin`, but they only need to be on `Plugin` with this
change
# Description
The PR overhauls how IO redirection is handled, allowing more explicit
and fine-grain control over `stdout` and `stderr` output as well as more
efficient IO and piping.
To summarize the changes in this PR:
- Added a new `IoStream` type to indicate the intended destination for a
pipeline element's `stdout` and `stderr`.
- The `stdout` and `stderr` `IoStream`s are stored in the `Stack` and to
avoid adding 6 additional arguments to every eval function and
`Command::run`. The `stdout` and `stderr` streams can be temporarily
overwritten through functions on `Stack` and these functions will return
a guard that restores the original `stdout` and `stderr` when dropped.
- In the AST, redirections are now directly part of a `PipelineElement`
as a `Option<Redirection>` field instead of having multiple different
`PipelineElement` enum variants for each kind of redirection. This
required changes to the parser, mainly in `lite_parser.rs`.
- `Command`s can also set a `IoStream` override/redirection which will
apply to the previous command in the pipeline. This is used, for
example, in `ignore` to allow the previous external command to have its
stdout redirected to `Stdio::null()` at spawn time. In contrast, the
current implementation has to create an os pipe and manually consume the
output on nushell's side. File and pipe redirections (`o>`, `e>`, `e>|`,
etc.) have precedence over overrides from commands.
This PR improves piping and IO speed, partially addressing #10763. Using
the `throughput` command from that issue, this PR gives the following
speedup on my setup for the commands below:
| Command | Before (MB/s) | After (MB/s) | Bash (MB/s) |
| --------------------------- | -------------:| ------------:|
-----------:|
| `throughput o> /dev/null` | 1169 | 52938 | 54305 |
| `throughput \| ignore` | 840 | 55438 | N/A |
| `throughput \| null` | Error | 53617 | N/A |
| `throughput \| rg 'x'` | 1165 | 3049 | 3736 |
| `(throughput) \| rg 'x'` | 810 | 3085 | 3815 |
(Numbers above are the median samples for throughput)
This PR also paves the way to refactor our `ExternalStream` handling in
the various commands. For example, this PR already fixes the following
code:
```nushell
^sh -c 'echo -n "hello "; sleep 0; echo "world"' | find "hello world"
```
This returns an empty list on 0.90.1 and returns a highlighted "hello
world" on this PR.
Since the `stdout` and `stderr` `IoStream`s are available to commands
when they are run, then this unlocks the potential for more convenient
behavior. E.g., the `find` command can disable its ansi highlighting if
it detects that the output `IoStream` is not the terminal. Knowing the
output streams will also allow background job output to be redirected
more easily and efficiently.
# User-Facing Changes
- External commands returned from closures will be collected (in most
cases):
```nushell
1..2 | each {|_| nu -c "print a" }
```
This gives `["a", "a"]` on this PR, whereas this used to print "a\na\n"
and then return an empty list.
```nushell
1..2 | each {|_| nu -c "print -e a" }
```
This gives `["", ""]` and prints "a\na\n" to stderr, whereas this used
to return an empty list and print "a\na\n" to stderr.
- Trailing new lines are always trimmed for external commands when
piping into internal commands or collecting it as a value. (Failure to
decode the output as utf-8 will keep the trailing newline for the last
binary value.) In the current nushell version, the following three code
snippets differ only in parenthesis placement, but they all also have
different outputs:
1. `1..2 | each { ^echo a }`
```
a
a
╭────────────╮
│ empty list │
╰────────────╯
```
2. `1..2 | each { (^echo a) }`
```
╭───┬───╮
│ 0 │ a │
│ 1 │ a │
╰───┴───╯
```
3. `1..2 | (each { ^echo a })`
```
╭───┬───╮
│ 0 │ a │
│ │ │
│ 1 │ a │
│ │ │
╰───┴───╯
```
But in this PR, the above snippets will all have the same output:
```
╭───┬───╮
│ 0 │ a │
│ 1 │ a │
╰───┴───╯
```
- All existing flags on `run-external` are now deprecated.
- File redirections now apply to all commands inside a code block:
```nushell
(nu -c "print -e a"; nu -c "print -e b") e> test.out
```
This gives "a\nb\n" in `test.out` and prints nothing. The same result
would happen when printing to stdout and using a `o>` file redirection.
- External command output will (almost) never be ignored, and ignoring
output must be explicit now:
```nushell
(^echo a; ^echo b)
```
This prints "a\nb\n", whereas this used to print only "b\n". This only
applies to external commands; values and internal commands not in return
position will not print anything (e.g., `(echo a; echo b)` still only
prints "b").
- `complete` now always captures stderr (`do` is not necessary).
# After Submitting
The language guide and other documentation will need to be updated.
