mirror of
https://github.com/nushell/nushell
synced 2025-01-06 18:29:02 +00:00
9 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Devyn Cairns
|
14b0ff3f05
|
Add --no-newline option to nu (#12410)
# Description I have `nu` set as my shell in my editor, which allows me to easily pipe selections of text to things like `str pascal-case` or even more complex string operation pipelines, which I find super handy. However, the only annoying thing is that I pretty much always have to add `| print -n` at the end, because `nu` adds a newline when it prints the resulting value. This adds a `--no-newline` option to stop that from happening, and then you don't need to pipe to `print -n` anymore, you can just have your shell command for your editor contain that flag. # User-Facing Changes - Add `--no-newline` command line option # Tests + Formatting - 🟢 `toolkit fmt` - 🟢 `toolkit clippy` - 🟢 `toolkit test` - 🟢 `toolkit test stdlib` |
||
Ian Manske
|
c747ec75c9
|
Add command_prelude module (#12291)
# Description
When implementing a `Command`, one must also import all the types
present in the function signatures for `Command`. This makes it so that
we often import the same set of types in each command implementation
file. E.g., something like this:
```rust
use nu_protocol::ast::Call;
use nu_protocol::engine::{Command, EngineState, Stack};
use nu_protocol::{
record, Category, Example, IntoInterruptiblePipelineData, IntoPipelineData, PipelineData,
ShellError, Signature, Span, Type, Value,
};
```
This PR adds the `nu_engine::command_prelude` module which contains the
necessary and commonly used types to implement a `Command`:
```rust
// command_prelude.rs
pub use crate::CallExt;
pub use nu_protocol::{
ast::{Call, CellPath},
engine::{Command, EngineState, Stack},
record, Category, Example, IntoInterruptiblePipelineData, IntoPipelineData, IntoSpanned,
PipelineData, Record, ShellError, Signature, Span, Spanned, SyntaxShape, Type, Value,
};
```
This should reduce the boilerplate needed to implement a command and
also gives us a place to track the breadth of the `Command` API. I tried
to be conservative with what went into the prelude modules, since it
might be hard/annoying to remove items from the prelude in the future.
Let me know if something should be included or excluded.
|
||
|
Ian Manske
|
b6c7656194
|
IO and redirection overhaul (#11934)
# Description The PR overhauls how IO redirection is handled, allowing more explicit and fine-grain control over `stdout` and `stderr` output as well as more efficient IO and piping. To summarize the changes in this PR: - Added a new `IoStream` type to indicate the intended destination for a pipeline element's `stdout` and `stderr`. - The `stdout` and `stderr` `IoStream`s are stored in the `Stack` and to avoid adding 6 additional arguments to every eval function and `Command::run`. The `stdout` and `stderr` streams can be temporarily overwritten through functions on `Stack` and these functions will return a guard that restores the original `stdout` and `stderr` when dropped. - In the AST, redirections are now directly part of a `PipelineElement` as a `Option<Redirection>` field instead of having multiple different `PipelineElement` enum variants for each kind of redirection. This required changes to the parser, mainly in `lite_parser.rs`. - `Command`s can also set a `IoStream` override/redirection which will apply to the previous command in the pipeline. This is used, for example, in `ignore` to allow the previous external command to have its stdout redirected to `Stdio::null()` at spawn time. In contrast, the current implementation has to create an os pipe and manually consume the output on nushell's side. File and pipe redirections (`o>`, `e>`, `e>|`, etc.) have precedence over overrides from commands. This PR improves piping and IO speed, partially addressing #10763. Using the `throughput` command from that issue, this PR gives the following speedup on my setup for the commands below: | Command | Before (MB/s) | After (MB/s) | Bash (MB/s) | | --------------------------- | -------------:| ------------:| -----------:| | `throughput o> /dev/null` | 1169 | 52938 | 54305 | | `throughput \| ignore` | 840 | 55438 | N/A | | `throughput \| null` | Error | 53617 | N/A | | `throughput \| rg 'x'` | 1165 | 3049 | 3736 | | `(throughput) \| rg 'x'` | 810 | 3085 | 3815 | (Numbers above are the median samples for throughput) This PR also paves the way to refactor our `ExternalStream` handling in the various commands. For example, this PR already fixes the following code: ```nushell ^sh -c 'echo -n "hello "; sleep 0; echo "world"' | find "hello world" ``` This returns an empty list on 0.90.1 and returns a highlighted "hello world" on this PR. Since the `stdout` and `stderr` `IoStream`s are available to commands when they are run, then this unlocks the potential for more convenient behavior. E.g., the `find` command can disable its ansi highlighting if it detects that the output `IoStream` is not the terminal. Knowing the output streams will also allow background job output to be redirected more easily and efficiently. # User-Facing Changes - External commands returned from closures will be collected (in most cases): ```nushell 1..2 | each {|_| nu -c "print a" } ``` This gives `["a", "a"]` on this PR, whereas this used to print "a\na\n" and then return an empty list. ```nushell 1..2 | each {|_| nu -c "print -e a" } ``` This gives `["", ""]` and prints "a\na\n" to stderr, whereas this used to return an empty list and print "a\na\n" to stderr. - Trailing new lines are always trimmed for external commands when piping into internal commands or collecting it as a value. (Failure to decode the output as utf-8 will keep the trailing newline for the last binary value.) In the current nushell version, the following three code snippets differ only in parenthesis placement, but they all also have different outputs: 1. `1..2 | each { ^echo a }` ``` a a ╭────────────╮ │ empty list │ ╰────────────╯ ``` 2. `1..2 | each { (^echo a) }` ``` ╭───┬───╮ │ 0 │ a │ │ 1 │ a │ ╰───┴───╯ ``` 3. `1..2 | (each { ^echo a })` ``` ╭───┬───╮ │ 0 │ a │ │ │ │ │ 1 │ a │ │ │ │ ╰───┴───╯ ``` But in this PR, the above snippets will all have the same output: ``` ╭───┬───╮ │ 0 │ a │ │ 1 │ a │ ╰───┴───╯ ``` - All existing flags on `run-external` are now deprecated. - File redirections now apply to all commands inside a code block: ```nushell (nu -c "print -e a"; nu -c "print -e b") e> test.out ``` This gives "a\nb\n" in `test.out` and prints nothing. The same result would happen when printing to stdout and using a `o>` file redirection. - External command output will (almost) never be ignored, and ignoring output must be explicit now: ```nushell (^echo a; ^echo b) ``` This prints "a\nb\n", whereas this used to print only "b\n". This only applies to external commands; values and internal commands not in return position will not print anything (e.g., `(echo a; echo b)` still only prints "b"). - `complete` now always captures stderr (`do` is not necessary). # After Submitting The language guide and other documentation will need to be updated. |
||
Jakub Žádník
|
14d1c67863
|
Debugger experiments (#11441)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR adds a new evaluator path with callbacks to a mutable trait object implementing a Debugger trait. The trait object can do anything, e.g., profiling, code coverage, step debugging. Currently, entering/leaving a block and a pipeline element is marked with callbacks, but more callbacks can be added as necessary. Not all callbacks need to be used by all debuggers; unused ones are simply empty calls. A simple profiler is implemented as a proof of concept. The debugging support is implementing by making `eval_xxx()` functions generic depending on whether we're debugging or not. This has zero computational overhead, but makes the binary slightly larger (see benchmarks below). `eval_xxx()` variants called from commands (like `eval_block_with_early_return()` in `each`) are chosen with a dynamic dispatch for two reasons: to not grow the binary size due to duplicating the code of many commands, and for the fact that it isn't possible because it would make Command trait objects object-unsafe. In the future, I hope it will be possible to allow plugin callbacks such that users would be able to implement their profiler plugins instead of having to recompile Nushell. [DAP](https://microsoft.github.io/debug-adapter-protocol/) would also be interesting to explore. Try `help debug profile`. ## Screenshots Basic output:  To profile with more granularity, increase the profiler depth (you'll see that repeated `is-windows` calls take a large chunk of total time, making it a good candidate for optimizing):  ## Benchmarks ### Binary size Binary size increase vs. main: **+40360 bytes**. _(Both built with `--release --features=extra,dataframe`.)_ ### Time ```nushell # bench_debug.nu use std bench let test = { 1..100 | each { ls | each {|row| $row.name | str length } } | flatten | math avg } print 'debug:' let res2 = bench { debug profile $test } --pretty print $res2 ``` ```nushell # bench_nodebug.nu use std bench let test = { 1..100 | each { ls | each {|row| $row.name | str length } } | flatten | math avg } print 'no debug:' let res1 = bench { do $test } --pretty print $res1 ``` `cargo run --release -- bench_debug.nu` is consistently 1--2 ms slower than `cargo run --release -- bench_nodebug.nu` due to the collection overhead + gathering the report. This is expected. When gathering more stuff, the overhead is obviously higher. `cargo run --release -- bench_nodebug.nu` vs. `nu bench_nodebug.nu` I didn't measure any difference. Both benchmarks report times between 97 and 103 ms randomly, without one being consistently higher than the other. This suggests that at least in this particular case, when not running any debugger, there is no runtime overhead. ## API changes This PR adds a generic parameter to all `eval_xxx` functions that forces you to specify whether you use the debugger. You can resolve it in two ways: * Use a provided helper that will figure it out for you. If you wanted to use `eval_block(&engine_state, ...)`, call `let eval_block = get_eval_block(&engine_state); eval_block(&engine_state, ...)` * If you know you're in an evaluation path that doesn't need debugger support, call `eval_block::<WithoutDebug>(&engine_state, ...)` (this is the case of hooks, for example). I tried to add more explanation in the docstring of `debugger_trait.rs`. ## TODO - [x] Better profiler output to reduce spam of iterative commands like `each` - [x] Resolve `TODO: DEBUG` comments - [x] Resolve unwraps - [x] Add doc comments - [x] Add usage and extra usage for `debug profile`, explaining all columns # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> Hopefully none. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
||
|
Ian Manske
|
68fcd71898
|
Add Value::coerce_str (#11885)
# Description Following #11851, this PR adds one final conversion function for `Value`. `Value::coerce_str` takes a `&Value` and converts it to a `Cow<str>`, creating an owned `String` for types that needed converting. Otherwise, it returns a borrowed `str` for `String` and `Binary` `Value`s which avoids a clone/allocation. Where possible, `coerce_str` and `coerce_into_string` should be used instead of `coerce_string`, since `coerce_string` always allocates a new `String`. |
||
|
Ian Manske
|
1c49ca503a
|
Name the Value conversion functions more clearly (#11851)
# Description This PR renames the conversion functions on `Value` to be more consistent. It follows the Rust [API guidelines](https://rust-lang.github.io/api-guidelines/naming.html#ad-hoc-conversions-follow-as_-to_-into_-conventions-c-conv) for ad-hoc conversions. The conversion functions on `Value` now come in a few forms: - `coerce_{type}` takes a `&Value` and attempts to convert the value to `type` (e.g., `i64` are converted to `f64`). This is the old behavior of some of the `as_{type}` functions -- these functions have simply been renamed to better reflect what they do. - The new `as_{type}` functions take a `&Value` and returns an `Ok` result only if the value is of `type` (no conversion is attempted). The returned value will be borrowed if `type` is non-`Copy`, otherwise an owned value is returned. - `into_{type}` exists for non-`Copy` types, but otherwise does not attempt conversion just like `as_type`. It takes an owned `Value` and always returns an owned result. - `coerce_into_{type}` has the same relationship with `coerce_{type}` as `into_{type}` does with `as_{type}`. - `to_{kind}_string`: conversion to different string formats (debug, abbreviated, etc.). Only two of the old string conversion functions were removed, the rest have been renamed only. - `to_{type}`: other conversion functions. Currently, only `to_path` exists. (And `to_string` through `Display`.) This table summaries the above: | Form | Cost | Input Ownership | Output Ownership | Converts `Value` case/`type` | | ---------------------------- | ----- | --------------- | ---------------- | -------- | | `as_{type}` | Cheap | Borrowed | Borrowed/Owned | No | | `into_{type}` | Cheap | Owned | Owned | No | | `coerce_{type}` | Cheap | Borrowed | Borrowed/Owned | Yes | | `coerce_into_{type}` | Cheap | Owned | Owned | Yes | | `to_{kind}_string` | Expensive | Borrowed | Owned | Yes | | `to_{type}` | Expensive | Borrowed | Owned | Yes | # User-Facing Changes Breaking API change for `Value` in `nu-protocol` which is exposed as part of the plugin API. |
||
WindSoilder
|
5d98a727ca
|
Deprecate --flag: bool in custom command (#11365)
# Description While #11057 is merged, it's hard to tell the difference between `--flag: bool` and `--flag`, and it makes user hard to read custom commands' signature, and hard to use them correctly. After discussion, I think we can deprecate `--flag: bool` usage, and encourage using `--flag` instead. # User-Facing Changes The following code will raise warning message, but don't stop from running. ```nushell ❯ def florb [--dry-run: bool, --another-flag] { "aaa" }; florb Error: × Deprecated: --flag: bool ╭─[entry #7:1:1] 1 │ def florb [--dry-run: bool, --another-flag] { "aaa" }; florb · ──┬─ · ╰── `--flag: bool` is deprecated. Please use `--flag` instead, more info: https://www.nushell.sh/book/custom_commands.html ╰──── aaa ``` cc @kubouch # Tests + Formatting Done # After Submitting - [ ] Add more information under https://www.nushell.sh/book/custom_commands.html to indicate `--dry-run: bool` is not allowed, - [ ] remove `: bool` from custom commands between 0.89 and 0.90 --------- Co-authored-by: Antoine Stevan <44101798+amtoine@users.noreply.github.com> |
||
Stefan Holderbach
|
86cd387439
|
Refactor and fix Config<->Value mechanism (#10896)
# Description Our config exists both as a `Config` struct for internal consumption and as a `Value`. The latter is exposed through `$env.config` and can be both set and read. Thus we have a complex bug-prone mechanism, that reads a `Value` and then tries to plug anything where the value is unrepresentable in `Config` with the correct state from `Config`. The parsing involves therefore mutation of the `Value` in a nested `Record` structure. Previously this was wholy done manually, with indices. To enable deletion for example, things had to be iterated over from the back. Also things were indexed in a bunch of places. This was hard to read and an invitation for bugs. With #10876 we can now use `Record::retain_mut` to traverse the records, modify anything that needs fixing, and drop invalid fields. # Parts: - Error messages now consistently use the correct spans pointing to the problematic value and the paths displayed in some messages are also aligned with the keys used for lookup. - Reconstruction of values has been fixed for: - `table.padding` - `buffer_editor` - `hooks.command_not_found` - `datetime_format` (partial solution) - Fix validation of `table.padding` input so value is not set (and underflows `usize` causing `table` to run forever with negative values) - New proper types for settings. Fully validated enums instead of strings: - `config.edit_mode` -> `EditMode` - Don't fall back to vi-mode on invalid string - `config.table.mode` -> `TableMode` - there is still a fall back to `rounded` if given an invalid `TableMode` as argument to the `nu` binary - `config.completions.algorithm` -> `CompletionAlgorithm` - `config.error_style` -> `ErrorStyle` - don't implicitly fall back to `fancy` when given an invalid value. - This should also shrink the size of `Config` as instead of 4x24 bytes those fields now need only 4x1 bytes in `Config` - Completely removed macros relying on the scope of `Value::into_config` so we can break it up into smaller parts in the future. - Factored everything into smaller files with the types and helpers for particular topics. - `NuCursorShape` now explicitly expresses the `Inherit` setting. conversion to option only happens at the interface to `reedline` |
||
|
Stefan Holderbach
|
57510f2fd2
|
Move CLI related commands to nu-cli (#8832)
# Description
Part of the larger cratification effort.
Moves all `reedline` or shell line editor specific commands to `nu-cli`.
## From `nu-cmd-lang`:
- `commandline`
- This shouldn't have moved there. Doesn't directly depend on reedline
but assumes parts in the engine state that are specific to the use of
reedline or a REPL
## From `nu-command`:
- `keybindings` and subcommands
- `keybindings default`
- `keybindings list`
- `keybindings listen`
- very `reedline` specific
- `history`
- needs `reedline`
- `history session`
## internal use
Instead of having a separate `create_default_context()` that calls
`nu-command`'s `create_default_context()`, I added a `add_cli_context()`
that updates an `EngineState`
# User-Facing Changes
None
## Build time comparison
`cargo build --timings` from a `cargo clean --profile dev`
### total
main: 64 secs
this: 59 secs
### `nu-command` build time
branch | total| codegen | fraction
---|---|---|---
main | 14.0s | 6.2s | (44%)
this | 12.5s | 5.5s | (44%)
`nu-cli` depends on `nu-command` at the moment.
Thus it is built during the code-gen phase of `nu-command` (on 16
virtual cores)
# Tests + Formatting
I removed the `test_example()` facilities for now as we had not run any
of the commands in an `Example` test and importing the right context for
those tests seemed more of a hassle than the duplicated
`test_examples()` implementations in `nu-cmd-lang` and `nu-command`
|
Renamed from crates/nu-cli/src/commands.rs (Browse further)