<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Resolves#11800.
```
~/CodingProjects/nushell> def "url expand" [$urls:any = []]: [string -> string, list -> table] {
::: let urls = ($in | default $urls)
::: def expand-link [] {
::: http head --redirect-mode manual $in | where name == location | get value.0
::: }
::: match ($urls | describe) {
::: string => { $urls | expand-link }
::: $type if ($type =~ list) => { $urls | wrap link | insert expanded {|url| $url.link | expand-link}}
::: }
::: }; view source "url expand"
def "url expand" [ $urls: any = [] ]: [string -> string, list<any> -> table] {
let urls = ($in | default $urls)
def expand-link [] {
http head --redirect-mode manual $in | where name == location | get value.0
}
match ($urls | describe) {
string => { $urls | expand-link }
$type if ($type =~ list) => { $urls | wrap link | insert expanded {|url| $url.link | expand-link}}
}
}
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
`view source` now
- adds quotes to commands with spaces
- shows default argument values
- shows type signatures
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
This fixes#12391.
nushell/nushell@87c5f6e455 accidentally introduced a bug where the path
was not being properly
expanded according to the cwd. This makes both 'touch' and 'mkdir' use
globs just like the rest of
the commands to preserve tilde behavior while still expanding the paths
properly.
This doesn't actually expand the globs. Should it?
# User-Facing Changes
- Restore behavior of `mkdir`, `touch`
- Help text now says they can take globs, but they won't actually expand
them, maybe this should be changed
# Tests + Formatting
Regression tests added.
# After Submitting
This is severe enough and should be included in the point release.
# Description
This keeps plugin custom values that have requested drop notification
around during the lifetime of a plugin call / stream by sending them to
a channel that gets persisted during the lifetime of the call.
Before this change, it was very likely that the drop notification would
be sent before the plugin ever had a chance to handle the value it
received.
Tests have been added to make sure this works - see the `custom_values`
plugin.
cc @ayax79
# User-Facing Changes
This is basically just a bugfix, just a slightly big one.
However, I did add an `as_mut_any()` function for custom values, to
avoid having to clone them. This is a breaking change.
- [x] `cargo hack` feature flag compatibility run
- [x] reedline released and pinned
- [x] `nu-plugin-test-support` added to release script
- [x] dependency tree checked
- [x] release notes
# Description
Fixes how the directory permissions are calculated in `mkdir`. Instead
of subtraction, the umask is actually used as a mask via negation
followed by bitwise and with the default mode. This matches how [uucore
calculates](cac7155fba/src/uu/mkdir/src/mkdir.rs (L61))
the mode.
# Description
Where possible, this PR replaces usages of raw `libc` bindings to
instead use safe interfaces from the `nix` crate. Where not possible,
the `libc` version reexported through `nix` was used instead of having a
separate `libc` dependency.
# Description
This pr is addressing feedback from
https://github.com/nushell/nushell/pull/12277#issuecomment-2027246752
Currently I think it's fine to replace `--legacy` flag with `--guess`
one. Only use `guess_width` algorithm if `--guess` is provided.
# User-Facing Changes
So it won't be a breaking change to previous version.
# Description
In #10232, the allowed input types were changed to be stricter, only
allowing records with types that can easily map onto sqlite equivalents.
Unfortunately, null was left out of the accepted input types, which
makes inserting rows with null values impossible.
This change fixes that by accepting null values as input.
One caveat of this is that when the command is creating a new table, it
uses the first row to infer an appropriate sqlite schema. If the first
row contains a null value, then it is impossible to tell which type this
column is supposed to have.
Throwing a hard error seems undesirable from a UX perspective, but
guessing can lead to a potentially useless database if we guess wrong.
So as a compromise, for null columns, we will assume the sqlite type is
TEXT and print a warning so the user knows. For the time being, if users
can't avoid a first row with null values, but also wants the right
schema, they are advised to create their table before running `into
sqlite`.
A future PR can add the ability to explicitly specify a schema.
Fixes#12225
# Tests + Formatting
* Tests added to cover expected behavior around insertion of null values
# Description
Again avoid uses of the `Record` internals, so we are free to change the
data layout
- **Don't use internals of `Record` in `into sqlite`**
- **Don't use internals of `Record` in `to xml`**
Remaining: `rename`
# User-Facing Changes
None
# Description
The second `Value` is redundant and will consume five extra bytes on
each transmission of a custom value to/from a plugin.
# User-Facing Changes
This is a breaking change to the plugin protocol.
The [example in the protocol
reference](https://www.nushell.sh/contributor-book/plugin_protocol_reference.html#value)
becomes
```json
{
"Custom": {
"val": {
"type": "PluginCustomValue",
"name": "database",
"data": [36, 190, 127, 40, 12, 3, 46, 83],
"notify_on_drop": true
},
"span": {
"start": 320,
"end": 340
}
}
}
```
instead of
```json
{
"CustomValue": {
...
}
}
```
# After Submitting
Update plugin protocol reference
# Description
When implementing a `Command`, one must also import all the types
present in the function signatures for `Command`. This makes it so that
we often import the same set of types in each command implementation
file. E.g., something like this:
```rust
use nu_protocol::ast::Call;
use nu_protocol::engine::{Command, EngineState, Stack};
use nu_protocol::{
record, Category, Example, IntoInterruptiblePipelineData, IntoPipelineData, PipelineData,
ShellError, Signature, Span, Type, Value,
};
```
This PR adds the `nu_engine::command_prelude` module which contains the
necessary and commonly used types to implement a `Command`:
```rust
// command_prelude.rs
pub use crate::CallExt;
pub use nu_protocol::{
ast::{Call, CellPath},
engine::{Command, EngineState, Stack},
record, Category, Example, IntoInterruptiblePipelineData, IntoPipelineData, IntoSpanned,
PipelineData, Record, ShellError, Signature, Span, Spanned, SyntaxShape, Type, Value,
};
```
This should reduce the boilerplate needed to implement a command and
also gives us a place to track the breadth of the `Command` API. I tried
to be conservative with what went into the prelude modules, since it
might be hard/annoying to remove items from the prelude in the future.
Let me know if something should be included or excluded.
# Description
Binary values passed to `table` may or may not be pretty formatted based
on the output destination. This leads to weird behavior as documented in
#12287. This PR changes `table` to always pretty print binary values.
However, binary values passed to external commands will not be formatted
(this is the existing behavior).
# User-Facing Changes
This is a breaking change. E.g.:
```nushell
0x[8989] | table | cat -
```
used to print raw bytes, but it will now print the pretty formatted
bytes.
# After Submitting
Add to 0.92.0 release notes and update documentation.
# Description
Closes https://github.com/nushell/nushell/issues/12257
This was down to the use of `eval_block` instead of
`eval_block_with_early_return`. We may want to reconsider how we
differentiate between this behavior. We currently need to check all the
remaining commands that can invoke a closure block, if they properly
handle `ShellError::Return` as a passing of a `Value`
- **Add test for `return` in `filter` closure**
- **Fix use of `return` in `filter` closure**
# User-Facing Changes
You can now return a value from a `filter` closure
# Tests + Formatting
Regression test
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Boxes `Record` inside `Value` to reduce memory usage, `Value` goes from
`72` -> `56` bytes after this change.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
@fdncred found another histogram based algorithm to detect columns, and
rewrite it in rust: https://github.com/fdncred/guess-width
I have tested it manually, and it works good with `df`, `docker ps`,
`^ps`. This pr is going to use the algorithm in `detect columns`
Fix: #4183
The pitfall of new algorithm:
1. it may not works well if there isn't too much rows of input
2. it may not works well if the length of value is less than the header
to value, e.g:
```
c1 c2 c3 c4 c5
a b c d e
g h i j k
g a a q d
a v c q q | detect columns
```
In this case, users might need to use ~~`--old`~~ `--legacy` to make it
works well.
# User-Facing Changes
User might need to add ~~`--old`~~ `--legacy` to scripts if they find
`detect columns` in their scripts broken.
# Tests + Formatting
Done
# After Submitting
NaN
# Description
Cleanup search terms and help usage to be consistent and include
coreutils so people can easily find out which commands are coreutils.
or
```nushell
help commands | where usage =~ coreutils | reject params input_output
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
Hi,
This PR aims at implementing the first iteration for `uname` using
`uutils`. Couple of things:
* Currently my [PR](https://github.com/uutils/coreutils/pull/5921) to
make the required changes is pending in `uutils` repo.
* I guess the number of flags has to be investigated. Still the tests
cover all of them.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- [X] `cargo fmt --all -- --check` to check standard code formatting
(`cargo fmt --all` applies these changes)
- [X] `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used`
to check that you're using the standard code style
- [X] `cargo test --workspace` to check that all tests pass (on Windows
make sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- [X] `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
---------
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
# Description
Fixes: #11887Fixes: #11626
This pr unify the tilde expand behavior over several filesystem relative
commands. It follows the same rule with glob expansion:
| command | result |
| ----------- | ------ |
| ls ~/aaa | expand tilde
| ls "~/aaa" | don't expand tilde
| let f = "~/aaa"; ls $f | don't expand tilde, if you want to: use `ls
($f \| path expand)`
| let f: glob = "~/aaa"; ls $f | expand tilde, they don't expand on
`mkdir`, `touch` comamnd.
Actually I'm not sure for 4th item, currently it's expanding is just
because it followes the same rule with glob expansion.
### About the change
It changes `expand_path_with` to accept a new argument called
`expand_tilde`, if it's true, expand it, if not, just keep it as `~`
itself.
# User-Facing Changes
After this change, `ls "~/aaa"` won't expand tilde.
# Tests + Formatting
Done
# Description
This PR adds a `--params` param to `query db`. This closes#11643.
You can't combine both named and positional parameters, I think this
might be a limitation with rusqlite itself. I tried using named
parameters with indices like `{ ':named': 123, '1': "positional" }` but
that always failed with a rusqlite error. On the flip side, the other
way around works: for something like `VALUES (:named, ?)`, you can treat
both as positional: `-p [hello 123]`.
This PR introduces some very gnarly code repetition in
`prepared_statement_to_nu_list`. I tried, I swear; the compiler wasn't
having any of it, it kept telling me to box my closures and then it said
that the reference lifetimes were incompatible in the match arms. I gave
up and put the mapping code in the match itself, but I'm still not
happy.
Another thing I'm unhappy about: I don't like how you have to put the
`:colon` in named parameters. I think nushell should insert it if it's
[missing](https://www.sqlite.org/lang_expr.html#parameters). But this is
the way [rusqlite
works](https://docs.rs/rusqlite/latest/rusqlite/trait.Params.html#example-named),
so for now, I'll let it be consistent. Just know that it's not really a
blocker, and it isn't a compatibility change to later make `{ colon: 123
}` work, without the quotes and `:`. This would require allocating and
turning our pretty little `&str` into a `String`, though
# User-Facing Changes
Less incentive to leave yourself open to SQL injection with statements
like `query db $"INSERT INTO x VALUES \($unsafe_user_input)"`.
Additionally, the `$""` syntax being annoying with parentheses plays in
our favor, making users even more likely to use ? with `--params`.
# Tests + Formatting
Hehe
# Description
Just a bunch of miscellaneous fixes to the Rust documentation that I
found recently while doing
a pass on some things.
# User-Facing Changes
None
Bumps [base64](https://github.com/marshallpierce/rust-base64) from
0.21.7 to 0.22.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md">base64's
changelog</a>.</em></p>
<blockquote>
<h1>0.22.0</h1>
<ul>
<li><code>DecodeSliceError::OutputSliceTooSmall</code> is now
conservative rather than precise. That is, the error will only occur if
the decoded output <em>cannot</em> fit, meaning that
<code>Engine::decode_slice</code> can now be used with exactly-sized
output slices. As part of this, <code>Engine::internal_decode</code> now
returns <code>DecodeSliceError</code> instead of
<code>DecodeError</code>, but that is not expected to affect any
external callers.</li>
<li><code>DecodeError::InvalidLength</code> now refers specifically to
the <em>number of valid symbols</em> being invalid (i.e. <code>len % 4
== 1</code>), rather than just the number of input bytes. This avoids
confusing scenarios when based on interpretation you could make a case
for either <code>InvalidLength</code> or <code>InvalidByte</code> being
appropriate.</li>
<li>Decoding is somewhat faster (5-10%)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="5d70ba7576"><code>5d70ba7</code></a>
Merge pull request <a
href="https://redirect.github.com/marshallpierce/rust-base64/issues/269">#269</a>
from marshallpierce/mp/decode-precisely</li>
<li><a
href="efb6c006c7"><code>efb6c00</code></a>
Release notes</li>
<li><a
href="2b91084a31"><code>2b91084</code></a>
Add some tests to boost coverage</li>
<li><a
href="9e9c7abe65"><code>9e9c7ab</code></a>
Engine::internal_decode now returns DecodeSliceError</li>
<li><a
href="a8a60f43c5"><code>a8a60f4</code></a>
Decode main loop improvements</li>
<li><a
href="a25be0667c"><code>a25be06</code></a>
Simplify leftover output writes</li>
<li><a
href="9979cc33bb"><code>9979cc3</code></a>
Keep morsels as separate bytes</li>
<li><a
href="37670c5ec2"><code>37670c5</code></a>
Bump dev toolchain version (<a
href="https://redirect.github.com/marshallpierce/rust-base64/issues/268">#268</a>)</li>
<li>See full diff in <a
href="https://github.com/marshallpierce/rust-base64/compare/v0.21.7...v0.22.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
you can also mention related issues, PRs or discussions!
-->
- fixes#11014
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
When the `command_not_found` hook is entered, we set an environment
variable for context. If an unknown command is encountered and the
`command_not_found` context environment variable is already present, it
implies a command in the hook closure is also not found. We stop the
recursion right there.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Incorrect `command_not_found` hooks can be caught without panicking.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Tests are passing.
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
fixes#11900
# Description
Use `serde_json` instead.
# User-Facing Changes
The problem described in the issue now no longer persists.
No whitespace in the output of `to json --raw`
Output of unicode escape changed to consistent `\uffff`
# Tests + Formatting
I corrected all Tests that were affected by this change.
# Description
Get rid of two parallel `Vec`s in `StateDelta` and `EngineState`, that
also duplicated span information. Use a struct with documenting fields.
Also use `Arc<str>` and `Arc<[u8]>` for the allocations as they are
never modified and cloned often (see #12229 for the first improvement).
This also makes the representation more compact as no capacity is
necessary.
# User-Facing Changes
API breakage on `EngineState`/`StateWorkingSet`/`StateDelta` that should
not really affect plugin authors.
closes#12115
# Description
This fix addresses a bug where the --tabs flag couldn't be utilized due
to improper handling of the tab quantity provided by the user.
Previously, the code mistakenly attempted to convert the tab quantity to
a boolean value, leading to a conversion error. The resolution involves
adjusting the condition clauses to properly validate the presence of the
flag's value. Now, the code checks whether the get_flag() function
returns a value or None associated with the --tabs flag. This adjustment
enables the --tabs flag to function correctly, triggering the
appropriate condition and allowing the conversion to proceed as
expected. Similarly, the fix applies to the --indent flag. Additionally,
a default case was added, and the conversion now works properly without
flags. Two tests were added to validate the corrected behavior of these
flags.
# User-Facing Changes
Now the conversion should work properly instead of displaying an error.
# Tests + Formatting
-🟢 toolkit fmt
-🟢 toolkit clippy
-🟢 toolkit test
-🟢 toolkit test stdlib
To run added tests:
- cargo test --package nu-command --test main --
format_conversions::json::test_tabs_indent_flag
- cargo test --package nu-command --test main --
format_conversions::json::test_indent_flag
# Description
Improves the accuracy of sleep when the duration is larger than 100ms.
Fixes#12223.
# User-Facing Changes
Sleeping for 150ms should work now.
```nushell
~/nushell> timeit { sleep 150ms } 03/19/2024 10:41:55 AM AM
151ms 344µs 201ns
```
# Description
This makes many of the larger objects in `EngineState` into `Arc`, and
uses `Arc::make_mut` to do clone-on-write if the reference is not
unique. This is generally very cheap, giving us the best of both worlds
- allowing us to mutate without cloning if we have an exclusive
reference, and cloning if we don't.
This started as more of a curiosity for me after remembering that
`Arc::make_mut` exists and can make using `Arc` for mostly immutable
data that sometimes needs to be changed very convenient, and also after
hearing someone complain about memory usage on Discord - this is a
somewhat significant win for that.
The exact objects that were wrapped in `Arc`:
- `files`, `file_contents` - the strings and byte buffers
- `decls` - the whole `Vec`, but mostly to avoid lots of individual
`malloc()` calls on Clone rather than for memory usage
- `blocks` - the blocks themselves, rather than the outer Vec
- `modules` - the modules themselves, rather than the outer Vec
- `env_vars`, `previous_env_vars` - the entire maps
- `config`
The changes required were relatively minimal, but this is a breaking API
change. In particular, blocks are added as Arcs, to allow the parser
cache functionality to work.
With my normal nu config, running on Linux, this saves me about 15 MiB
of process memory usage when running interactively (65 MiB → 50 MiB).
This also makes quick command executions cheaper, particularly since
every REPL loop now involves a clone of the engine state so that we can
recover from a panic. It also reduces memory usage where engine state
needs to be cloned and sent to another thread or kept within an
iterator.
# User-Facing Changes
Shouldn't be any, since it's all internal stuff, but it does change some
public interfaces so it's a breaking change
[Context on
Discord](https://discord.com/channels/601130461678272522/855947301380947968/1219425984990806207)
# Description
- Rename `CustomValue::value_string()` to `type_name()` to reflect its
usage better.
- Change print behavior to always call `to_base_value()` first, to give
the custom value better control over the output.
- Change `describe --detailed` to show the type name as the subtype,
rather than trying to describe the base value.
- Change custom `Type` to use `type_name()` rather than `typetag_name()`
to make things like `PluginCustomValue` more transparent
One question: should `describe --detailed` still include a description
of the base value somewhere? I'm torn on it, it seems possibly useful
for some things (maybe sqlite databases?), but having `describe -d` not
include the custom type name anywhere felt weird. Another option would
be to add another method to `CustomValue` for info to be displayed in
`describe`, so that it can be more type-specific?
# User-Facing Changes
Everything above has implications for printing and `describe` on custom
values
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
Context: @abusch is working on a semver plugin with custom values and
wants users to be able to convert them back to strings
# Description
This allows `into string` to work on custom values if their base value
representation could be converted into a string with the same rules.
# User-Facing Changes
`into string` works on custom values.
Unfortunately, I couldn't really demo this with an example, because
there aren't any custom values that can be represented that way
included.
# Tests + Formatting
I was able to write a test using the custom values plugin.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
`help zip` now reports:
```
other <one_of(any, closure())>: The other input, or closure returning a stream.
```
Thanks to @edhowland for pointing this out ❤️
# User-Facing Changes
- Doc change for zip
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
With this change, `mkdir` mirrors coreutils works. Closes#12161
I referred to the implementation of `mkdir` in uutils/coreutils. I add
`uucore` required for implementation to dependencies. Since `uucore` is
already included in dependencies of `uu_mkdir`, I don't think there will
be any additional dependencies.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
- Directories are created according to `umask` except for Windows.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
I add `mkdir` test considering permissions. The test assumes that the
default `umask` is `022`.
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Fixes#12193 where the `$in` value may be null for closures provided to
`insert`.
# User-Facing Changes
The `$in` value will now always be the same as the closure parameter for
`insert`.
# Description
The PR overhauls how IO redirection is handled, allowing more explicit
and fine-grain control over `stdout` and `stderr` output as well as more
efficient IO and piping.
To summarize the changes in this PR:
- Added a new `IoStream` type to indicate the intended destination for a
pipeline element's `stdout` and `stderr`.
- The `stdout` and `stderr` `IoStream`s are stored in the `Stack` and to
avoid adding 6 additional arguments to every eval function and
`Command::run`. The `stdout` and `stderr` streams can be temporarily
overwritten through functions on `Stack` and these functions will return
a guard that restores the original `stdout` and `stderr` when dropped.
- In the AST, redirections are now directly part of a `PipelineElement`
as a `Option<Redirection>` field instead of having multiple different
`PipelineElement` enum variants for each kind of redirection. This
required changes to the parser, mainly in `lite_parser.rs`.
- `Command`s can also set a `IoStream` override/redirection which will
apply to the previous command in the pipeline. This is used, for
example, in `ignore` to allow the previous external command to have its
stdout redirected to `Stdio::null()` at spawn time. In contrast, the
current implementation has to create an os pipe and manually consume the
output on nushell's side. File and pipe redirections (`o>`, `e>`, `e>|`,
etc.) have precedence over overrides from commands.
This PR improves piping and IO speed, partially addressing #10763. Using
the `throughput` command from that issue, this PR gives the following
speedup on my setup for the commands below:
| Command | Before (MB/s) | After (MB/s) | Bash (MB/s) |
| --------------------------- | -------------:| ------------:|
-----------:|
| `throughput o> /dev/null` | 1169 | 52938 | 54305 |
| `throughput \| ignore` | 840 | 55438 | N/A |
| `throughput \| null` | Error | 53617 | N/A |
| `throughput \| rg 'x'` | 1165 | 3049 | 3736 |
| `(throughput) \| rg 'x'` | 810 | 3085 | 3815 |
(Numbers above are the median samples for throughput)
This PR also paves the way to refactor our `ExternalStream` handling in
the various commands. For example, this PR already fixes the following
code:
```nushell
^sh -c 'echo -n "hello "; sleep 0; echo "world"' | find "hello world"
```
This returns an empty list on 0.90.1 and returns a highlighted "hello
world" on this PR.
Since the `stdout` and `stderr` `IoStream`s are available to commands
when they are run, then this unlocks the potential for more convenient
behavior. E.g., the `find` command can disable its ansi highlighting if
it detects that the output `IoStream` is not the terminal. Knowing the
output streams will also allow background job output to be redirected
more easily and efficiently.
# User-Facing Changes
- External commands returned from closures will be collected (in most
cases):
```nushell
1..2 | each {|_| nu -c "print a" }
```
This gives `["a", "a"]` on this PR, whereas this used to print "a\na\n"
and then return an empty list.
```nushell
1..2 | each {|_| nu -c "print -e a" }
```
This gives `["", ""]` and prints "a\na\n" to stderr, whereas this used
to return an empty list and print "a\na\n" to stderr.
- Trailing new lines are always trimmed for external commands when
piping into internal commands or collecting it as a value. (Failure to
decode the output as utf-8 will keep the trailing newline for the last
binary value.) In the current nushell version, the following three code
snippets differ only in parenthesis placement, but they all also have
different outputs:
1. `1..2 | each { ^echo a }`
```
a
a
╭────────────╮
│ empty list │
╰────────────╯
```
2. `1..2 | each { (^echo a) }`
```
╭───┬───╮
│ 0 │ a │
│ 1 │ a │
╰───┴───╯
```
3. `1..2 | (each { ^echo a })`
```
╭───┬───╮
│ 0 │ a │
│ │ │
│ 1 │ a │
│ │ │
╰───┴───╯
```
But in this PR, the above snippets will all have the same output:
```
╭───┬───╮
│ 0 │ a │
│ 1 │ a │
╰───┴───╯
```
- All existing flags on `run-external` are now deprecated.
- File redirections now apply to all commands inside a code block:
```nushell
(nu -c "print -e a"; nu -c "print -e b") e> test.out
```
This gives "a\nb\n" in `test.out` and prints nothing. The same result
would happen when printing to stdout and using a `o>` file redirection.
- External command output will (almost) never be ignored, and ignoring
output must be explicit now:
```nushell
(^echo a; ^echo b)
```
This prints "a\nb\n", whereas this used to print only "b\n". This only
applies to external commands; values and internal commands not in return
position will not print anything (e.g., `(echo a; echo b)` still only
prints "b").
- `complete` now always captures stderr (`do` is not necessary).
# After Submitting
The language guide and other documentation will need to be updated.
# Description
There are lots of duplicate test for `cp`, it's because we once have
`old-cp` command.
Today `old-cp` is removed, so there is no need to keep these tests.
Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.8.1 to 1.9.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/rayon-rs/rayon/blob/main/RELEASES.md">rayon's
changelog</a>.</em></p>
<blockquote>
<h1>Release rayon 1.9.0 (2024-02-27)</h1>
<ul>
<li>The new methods
<code>IndexedParallelIterator::by_exponential_blocks</code> and
<code>by_uniform_blocks</code> allow processing items in smaller groups
at a time.</li>
<li>The new <code>iter::walk_tree</code>, <code>walk_tree_prefix</code>,
and <code>walk_tree_postfix</code>
functions enable custom parallel iteration over tree-like
structures.</li>
<li>The new method <code>ParallelIterator::collect_vec_list</code>
returns items as a linked
list of vectors, which is an efficient mode of parallel collection used
by
many of the internal implementations of <code>collect</code>.</li>
<li>The new methods
<code>ParallelSliceMut::par_split_inclusive_mut</code>,
<code>ParallelSlice::par_split_inclusive</code>, and
<code>ParallelString::par_split_inclusive</code> all work like a normal
split but
keeping the separator as part of the left slice.</li>
<li>The new <code>ParallelString::par_split_ascii_whitespace</code>
splits only on ASCII
whitespace, which is faster than including Unicode multi-byte
whitespace.</li>
<li><code>OsString</code> now implements
<code>FromParallelIterator<_></code> and
<code>ParallelExtend<_></code>
for a few item types similar to the standard <code>FromIterator</code>
and <code>Extend</code>.</li>
<li>The internal <code>Pattern</code> trait for string methods is now
implemented for
<code>[char; N]</code> and <code>&[char; N]</code>, matching any of
the given characters.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="dc13cb7875"><code>dc13cb7</code></a>
Merge <a
href="https://redirect.github.com/rayon-rs/rayon/issues/810">#810</a></li>
<li><a
href="67eeea6f2a"><code>67eeea6</code></a>
Release rayon 1.5.0 / rayon-core 1.9.0</li>
<li><a
href="4828f30eef"><code>4828f30</code></a>
Merge <a
href="https://redirect.github.com/rayon-rs/rayon/issues/808">#808</a></li>
<li><a
href="eeb0d1ad5e"><code>eeb0d1a</code></a>
update ci/compat-Cargo.lock</li>
<li><a
href="12f0d202b8"><code>12f0d20</code></a>
Update glium so that rayon-demo runs on Gnome Wayland</li>
<li><a
href="1f069d7710"><code>1f069d7</code></a>
Merge <a
href="https://redirect.github.com/rayon-rs/rayon/issues/807">#807</a></li>
<li><a
href="9691328a5a"><code>9691328</code></a>
Use Iterator::copied</li>
<li><a
href="e81835c074"><code>e81835c</code></a>
Update crossbeam dependencies (requires Rust 1.36)</li>
<li><a
href="5b3d917d6c"><code>5b3d917</code></a>
Merge <a
href="https://redirect.github.com/rayon-rs/rayon/issues/804">#804</a></li>
<li><a
href="5c55033950"><code>5c55033</code></a>
Release rayon 1.4.1</li>
<li>Additional commits viewable in <a
href="https://github.com/rayon-rs/rayon/compare/rayon-core-v1.8.1...rayon-core-v1.9.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
# Description
Adds support for the following operations on plugin custom values, in
addition to `to_base_value` which was already present:
- `follow_path_int()`
- `follow_path_string()`
- `partial_cmp()`
- `operation()`
- `Drop` (notification, if opted into with
`CustomValue::notify_plugin_on_drop`)
There are additionally customizable methods within the `Plugin` and
`StreamingPlugin` traits for implementing these functions in a way that
requires access to the plugin state, as a registered handle model such
as might be used in a dataframes plugin would.
`Value::append` was also changed to handle custom values correctly.
# User-Facing Changes
- Signature of `CustomValue::follow_path_string` and
`CustomValue::follow_path_int` changed to give access to the span of the
custom value itself, useful for some errors.
- Plugins using custom values have to be recompiled because the engine
will try to do custom value operations that aren't supported
- Plugins can do more things 🎉
# Tests + Formatting
Tests were added for all of the new custom values functionality.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] Document protocol reference `CustomValueOp` variants:
- [ ] `FollowPathInt`
- [ ] `FollowPathString`
- [ ] `PartialCmp`
- [ ] `Operation`
- [ ] `Dropped`
- [ ] Document `notify_on_drop` optional field in `PluginCustomValue`
# Description
Fixes some ignored clippy lints.
# User-Facing Changes
Changes some signatures and return types to `&dyn Command` instead of
`&Box<dyn Command`, but I believe this is only an internal change.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
- fixes#12126
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
This pr improves the error message for issue #12126
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
The intended effect of the `extra` feature has been undermined by
introducing the full builds on our release pages and having more
activity on some of the extra commands.
To simplify the feature matrix let's get rid of it and focus our effort
on truly either refining a command to well-specified behavior or
discarding it entirely from the `nu` binary and moving it into plugins.
## Details
- Remove `--features extra` from CI
- Don't explicitly name `extra` in full build wf
- Remove feature extra from build-help scripts
- Update README in `nu-cmd-extra`
- Remove feature `extra`
- Fix previously dead `format pattern` tests
- Relax signature of `to html`
- Fix/ignore `html::test_no_color_flag`
- Remove dead features from `version`
- Refine `to html` type signature
# User-Facing Changes
The commands that were previously only available when building with
`--features extra` will now be available to everyone. This increases the
number of dependencies slightly but has a limited impact on the overall
binary size.
# Tests + Formatting
Some tests that were left in `nu-command` during cratification were dead
because the feature was not passed to `nu-command` and only to
`nu-cmd-lang` for feature-flag mention in `version`.
Those tests have now been either fixed or ignored in one case.
# After Submitting
There may be places in the documentation where we point to `--features
extra` that will now be moot (apart from the generated command help)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
This PR refactors `nu-check` and makes it possible to check module
directories. Also removes the requirement for files to end with .nu: It
was too limiting for module directories and there are executable scripts
[around](https://github.com/nushell/nu_scripts/tree/main/make_release/release-note)
that do not end with .nu, it's a common practice for scripts to omit it.
Other changes are:
* Removed the `--all` flag and heuristic parse because these are
irrelevant now when module syntax is a subset of script syntax (i.e.,
every module can be parsed as script).
* Reduced code duplication and in general tidied up the code
* Replaced unspanned errors with spanned ones.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
* `nu-check` doesn't require files to end with .nu
* can check module directories
* Removed `--all` flag
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
