nushell/crates/nu-plugin/src/plugin/declaration.rs

147 lines
4.6 KiB
Rust
Raw Normal View History

use super::{PluginExecutionCommandContext, PluginIdentity};
use crate::protocol::{CallInfo, EvaluatedCall};
use std::path::{Path, PathBuf};
use std::sync::Arc;
Debugger experiments (#11441) <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR adds a new evaluator path with callbacks to a mutable trait object implementing a Debugger trait. The trait object can do anything, e.g., profiling, code coverage, step debugging. Currently, entering/leaving a block and a pipeline element is marked with callbacks, but more callbacks can be added as necessary. Not all callbacks need to be used by all debuggers; unused ones are simply empty calls. A simple profiler is implemented as a proof of concept. The debugging support is implementing by making `eval_xxx()` functions generic depending on whether we're debugging or not. This has zero computational overhead, but makes the binary slightly larger (see benchmarks below). `eval_xxx()` variants called from commands (like `eval_block_with_early_return()` in `each`) are chosen with a dynamic dispatch for two reasons: to not grow the binary size due to duplicating the code of many commands, and for the fact that it isn't possible because it would make Command trait objects object-unsafe. In the future, I hope it will be possible to allow plugin callbacks such that users would be able to implement their profiler plugins instead of having to recompile Nushell. [DAP](https://microsoft.github.io/debug-adapter-protocol/) would also be interesting to explore. Try `help debug profile`. ## Screenshots Basic output: ![profiler_new](https://github.com/nushell/nushell/assets/25571562/418b9df0-b659-4dcb-b023-2d5fcef2c865) To profile with more granularity, increase the profiler depth (you'll see that repeated `is-windows` calls take a large chunk of total time, making it a good candidate for optimizing): ![profiler_new_m3](https://github.com/nushell/nushell/assets/25571562/636d756d-5d56-460c-a372-14716f65f37f) ## Benchmarks ### Binary size Binary size increase vs. main: **+40360 bytes**. _(Both built with `--release --features=extra,dataframe`.)_ ### Time ```nushell # bench_debug.nu use std bench let test = { 1..100 | each { ls | each {|row| $row.name | str length } } | flatten | math avg } print 'debug:' let res2 = bench { debug profile $test } --pretty print $res2 ``` ```nushell # bench_nodebug.nu use std bench let test = { 1..100 | each { ls | each {|row| $row.name | str length } } | flatten | math avg } print 'no debug:' let res1 = bench { do $test } --pretty print $res1 ``` `cargo run --release -- bench_debug.nu` is consistently 1--2 ms slower than `cargo run --release -- bench_nodebug.nu` due to the collection overhead + gathering the report. This is expected. When gathering more stuff, the overhead is obviously higher. `cargo run --release -- bench_nodebug.nu` vs. `nu bench_nodebug.nu` I didn't measure any difference. Both benchmarks report times between 97 and 103 ms randomly, without one being consistently higher than the other. This suggests that at least in this particular case, when not running any debugger, there is no runtime overhead. ## API changes This PR adds a generic parameter to all `eval_xxx` functions that forces you to specify whether you use the debugger. You can resolve it in two ways: * Use a provided helper that will figure it out for you. If you wanted to use `eval_block(&engine_state, ...)`, call `let eval_block = get_eval_block(&engine_state); eval_block(&engine_state, ...)` * If you know you're in an evaluation path that doesn't need debugger support, call `eval_block::<WithoutDebug>(&engine_state, ...)` (this is the case of hooks, for example). I tried to add more explanation in the docstring of `debugger_trait.rs`. ## TODO - [x] Better profiler output to reduce spam of iterative commands like `each` - [x] Resolve `TODO: DEBUG` comments - [x] Resolve unwraps - [x] Add doc comments - [x] Add usage and extra usage for `debug profile`, explaining all columns # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> Hopefully none. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2024-03-08 18:21:35 +00:00
use nu_engine::{get_eval_block, get_eval_expression};
use nu_protocol::engine::{Command, EngineState, Stack};
Make plugin commands support examples. (#7984) # Description As title, we can't provide examples for plugin commands, this pr would make it possible # User-Facing Changes Take plugin `nu-example-1` as example: ``` ❯ nu-example-1 -h PluginSignature test 1 for plugin. Returns Value::Nothing Usage: > nu-example-1 {flags} <a> <b> (opt) ...(rest) Flags: -h, --help - Display the help message for this command -f, --flag - a flag for the signature -n, --named <String> - named string Parameters: a <int>: required integer value b <string>: required string value (optional) opt <int>: Optional number ...rest <string>: rest value string Examples: running example with an int value and string value > nu-example-1 3 bb ``` The examples session is newly added. ## Basic idea behind these changes when nushell query plugin signatures, plugin just returns it's signature without any examples, so nushell have no idea about the examples of plugin commands. To adding the feature, we just making plugin returns it's signature with examples. Before: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<Signature> ``` After: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<PluginSignature> ``` When writing plugin signature to $nu.plugin-path: Serialize `<PluginSignature>` rather than `<Signature>`, which would enable us to serialize examples to `$nu.plugin-path` ## Shortcoming It's a breaking changes because `Plugin::signature` is changed, and it requires plugin authors to change their code for new signatures. Fortunally it should be easy to change, for rust based plugin, we just need to make a global replace from word `Signature` to word `PluginSignature` in their plugin project. Our content of plugin-path is really large, if one plugin have many examples, it'd results to larger body of $nu.plugin-path, which is not really scale. A solution would be save register information in other binary formats rather than `json`. But I think it'd be another story. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-02-08 22:14:18 +00:00
use nu_protocol::{ast::Call, PluginSignature, Signature};
use nu_protocol::{Example, PipelineData, ShellError, Value};
Plugin api docs (#9452) <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Added comments to support API docs for the `nu-plugin` crate. Removed a few items that I'd expect should only be used internally to Nushell from the documentation and reduced the visibility of some items that did not need to be public. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> There should be no user facing impact. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect -A clippy::result_large_err` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> Standard tests run. Additionally numerous doctests were added to the `nu-plugin` crate. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> No changes to the website necessary.
2023-06-16 14:25:40 +00:00
#[doc(hidden)] // Note: not for plugin authors / only used in nu-parser
#[derive(Clone)]
pub struct PluginDeclaration {
name: String,
Make plugin commands support examples. (#7984) # Description As title, we can't provide examples for plugin commands, this pr would make it possible # User-Facing Changes Take plugin `nu-example-1` as example: ``` ❯ nu-example-1 -h PluginSignature test 1 for plugin. Returns Value::Nothing Usage: > nu-example-1 {flags} <a> <b> (opt) ...(rest) Flags: -h, --help - Display the help message for this command -f, --flag - a flag for the signature -n, --named <String> - named string Parameters: a <int>: required integer value b <string>: required string value (optional) opt <int>: Optional number ...rest <string>: rest value string Examples: running example with an int value and string value > nu-example-1 3 bb ``` The examples session is newly added. ## Basic idea behind these changes when nushell query plugin signatures, plugin just returns it's signature without any examples, so nushell have no idea about the examples of plugin commands. To adding the feature, we just making plugin returns it's signature with examples. Before: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<Signature> ``` After: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<PluginSignature> ``` When writing plugin signature to $nu.plugin-path: Serialize `<PluginSignature>` rather than `<Signature>`, which would enable us to serialize examples to `$nu.plugin-path` ## Shortcoming It's a breaking changes because `Plugin::signature` is changed, and it requires plugin authors to change their code for new signatures. Fortunally it should be easy to change, for rust based plugin, we just need to make a global replace from word `Signature` to word `PluginSignature` in their plugin project. Our content of plugin-path is really large, if one plugin have many examples, it'd results to larger body of $nu.plugin-path, which is not really scale. A solution would be save register information in other binary formats rather than `json`. But I think it'd be another story. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-02-08 22:14:18 +00:00
signature: PluginSignature,
identity: Arc<PluginIdentity>,
}
impl PluginDeclaration {
Make plugin commands support examples. (#7984) # Description As title, we can't provide examples for plugin commands, this pr would make it possible # User-Facing Changes Take plugin `nu-example-1` as example: ``` ❯ nu-example-1 -h PluginSignature test 1 for plugin. Returns Value::Nothing Usage: > nu-example-1 {flags} <a> <b> (opt) ...(rest) Flags: -h, --help - Display the help message for this command -f, --flag - a flag for the signature -n, --named <String> - named string Parameters: a <int>: required integer value b <string>: required string value (optional) opt <int>: Optional number ...rest <string>: rest value string Examples: running example with an int value and string value > nu-example-1 3 bb ``` The examples session is newly added. ## Basic idea behind these changes when nushell query plugin signatures, plugin just returns it's signature without any examples, so nushell have no idea about the examples of plugin commands. To adding the feature, we just making plugin returns it's signature with examples. Before: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<Signature> ``` After: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<PluginSignature> ``` When writing plugin signature to $nu.plugin-path: Serialize `<PluginSignature>` rather than `<Signature>`, which would enable us to serialize examples to `$nu.plugin-path` ## Shortcoming It's a breaking changes because `Plugin::signature` is changed, and it requires plugin authors to change their code for new signatures. Fortunally it should be easy to change, for rust based plugin, we just need to make a global replace from word `Signature` to word `PluginSignature` in their plugin project. Our content of plugin-path is really large, if one plugin have many examples, it'd results to larger body of $nu.plugin-path, which is not really scale. A solution would be save register information in other binary formats rather than `json`. But I think it'd be another story. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-02-08 22:14:18 +00:00
pub fn new(filename: PathBuf, signature: PluginSignature, shell: Option<PathBuf>) -> Self {
Self {
Make plugin commands support examples. (#7984) # Description As title, we can't provide examples for plugin commands, this pr would make it possible # User-Facing Changes Take plugin `nu-example-1` as example: ``` ❯ nu-example-1 -h PluginSignature test 1 for plugin. Returns Value::Nothing Usage: > nu-example-1 {flags} <a> <b> (opt) ...(rest) Flags: -h, --help - Display the help message for this command -f, --flag - a flag for the signature -n, --named <String> - named string Parameters: a <int>: required integer value b <string>: required string value (optional) opt <int>: Optional number ...rest <string>: rest value string Examples: running example with an int value and string value > nu-example-1 3 bb ``` The examples session is newly added. ## Basic idea behind these changes when nushell query plugin signatures, plugin just returns it's signature without any examples, so nushell have no idea about the examples of plugin commands. To adding the feature, we just making plugin returns it's signature with examples. Before: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<Signature> ``` After: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<PluginSignature> ``` When writing plugin signature to $nu.plugin-path: Serialize `<PluginSignature>` rather than `<Signature>`, which would enable us to serialize examples to `$nu.plugin-path` ## Shortcoming It's a breaking changes because `Plugin::signature` is changed, and it requires plugin authors to change their code for new signatures. Fortunally it should be easy to change, for rust based plugin, we just need to make a global replace from word `Signature` to word `PluginSignature` in their plugin project. Our content of plugin-path is really large, if one plugin have many examples, it'd results to larger body of $nu.plugin-path, which is not really scale. A solution would be save register information in other binary formats rather than `json`. But I think it'd be another story. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-02-08 22:14:18 +00:00
name: signature.sig.name.clone(),
signature,
identity: Arc::new(PluginIdentity::new(filename, shell)),
}
}
}
impl Command for PluginDeclaration {
fn name(&self) -> &str {
&self.name
}
fn signature(&self) -> Signature {
Make plugin commands support examples. (#7984) # Description As title, we can't provide examples for plugin commands, this pr would make it possible # User-Facing Changes Take plugin `nu-example-1` as example: ``` ❯ nu-example-1 -h PluginSignature test 1 for plugin. Returns Value::Nothing Usage: > nu-example-1 {flags} <a> <b> (opt) ...(rest) Flags: -h, --help - Display the help message for this command -f, --flag - a flag for the signature -n, --named <String> - named string Parameters: a <int>: required integer value b <string>: required string value (optional) opt <int>: Optional number ...rest <string>: rest value string Examples: running example with an int value and string value > nu-example-1 3 bb ``` The examples session is newly added. ## Basic idea behind these changes when nushell query plugin signatures, plugin just returns it's signature without any examples, so nushell have no idea about the examples of plugin commands. To adding the feature, we just making plugin returns it's signature with examples. Before: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<Signature> ``` After: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<PluginSignature> ``` When writing plugin signature to $nu.plugin-path: Serialize `<PluginSignature>` rather than `<Signature>`, which would enable us to serialize examples to `$nu.plugin-path` ## Shortcoming It's a breaking changes because `Plugin::signature` is changed, and it requires plugin authors to change their code for new signatures. Fortunally it should be easy to change, for rust based plugin, we just need to make a global replace from word `Signature` to word `PluginSignature` in their plugin project. Our content of plugin-path is really large, if one plugin have many examples, it'd results to larger body of $nu.plugin-path, which is not really scale. A solution would be save register information in other binary formats rather than `json`. But I think it'd be another story. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-02-08 22:14:18 +00:00
self.signature.sig.clone()
}
fn usage(&self) -> &str {
Make plugin commands support examples. (#7984) # Description As title, we can't provide examples for plugin commands, this pr would make it possible # User-Facing Changes Take plugin `nu-example-1` as example: ``` ❯ nu-example-1 -h PluginSignature test 1 for plugin. Returns Value::Nothing Usage: > nu-example-1 {flags} <a> <b> (opt) ...(rest) Flags: -h, --help - Display the help message for this command -f, --flag - a flag for the signature -n, --named <String> - named string Parameters: a <int>: required integer value b <string>: required string value (optional) opt <int>: Optional number ...rest <string>: rest value string Examples: running example with an int value and string value > nu-example-1 3 bb ``` The examples session is newly added. ## Basic idea behind these changes when nushell query plugin signatures, plugin just returns it's signature without any examples, so nushell have no idea about the examples of plugin commands. To adding the feature, we just making plugin returns it's signature with examples. Before: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<Signature> ``` After: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<PluginSignature> ``` When writing plugin signature to $nu.plugin-path: Serialize `<PluginSignature>` rather than `<Signature>`, which would enable us to serialize examples to `$nu.plugin-path` ## Shortcoming It's a breaking changes because `Plugin::signature` is changed, and it requires plugin authors to change their code for new signatures. Fortunally it should be easy to change, for rust based plugin, we just need to make a global replace from word `Signature` to word `PluginSignature` in their plugin project. Our content of plugin-path is really large, if one plugin have many examples, it'd results to larger body of $nu.plugin-path, which is not really scale. A solution would be save register information in other binary formats rather than `json`. But I think it'd be another story. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-02-08 22:14:18 +00:00
self.signature.sig.usage.as_str()
}
Show plugin extra usage and search terms (#10952) # Description The `PluginSignature` type supports extra usage but this was not available in `plugin_name --help`. It also supports search terms but these did not appear in `help commands` New behavior show below is the "Extra usage for nu-example-1" line and the "Search terms:" line ``` ❯ nu-example-1 --help PluginSignature test 1 for plugin. Returns Value::Nothing Extra usage for nu-example-1 Search terms: example Usage: > nu-example-1 {flags} <a> <b> (opt) ...(rest) Flags: -h, --help - Display the help message for this command -f, --flag - a flag for the signature -n, --named <String> - named string Parameters: a <int>: required integer value b <string>: required string value opt <int>: Optional number (optional) ...rest <string>: rest value string Examples: running example with an int value and string value > nu-example-1 3 bb ``` Search terms are also available in `help commands`: ``` ❯ help commands | where name == "nu-example-1" | select name search_terms ╭──────────────┬──────────────╮ │ name │ search_terms │ ├──────────────┼──────────────┤ │ nu-example-1 │ example │ ╰──────────────┴──────────────╯ ``` # User-Facing Changes Users can now see plugin extra usage and search terms # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` # After Submitting N/A
2023-11-04 20:12:58 +00:00
fn extra_usage(&self) -> &str {
self.signature.sig.extra_usage.as_str()
}
fn search_terms(&self) -> Vec<&str> {
self.signature
.sig
.search_terms
.iter()
.map(|term| term.as_str())
.collect()
}
Make plugin commands support examples. (#7984) # Description As title, we can't provide examples for plugin commands, this pr would make it possible # User-Facing Changes Take plugin `nu-example-1` as example: ``` ❯ nu-example-1 -h PluginSignature test 1 for plugin. Returns Value::Nothing Usage: > nu-example-1 {flags} <a> <b> (opt) ...(rest) Flags: -h, --help - Display the help message for this command -f, --flag - a flag for the signature -n, --named <String> - named string Parameters: a <int>: required integer value b <string>: required string value (optional) opt <int>: Optional number ...rest <string>: rest value string Examples: running example with an int value and string value > nu-example-1 3 bb ``` The examples session is newly added. ## Basic idea behind these changes when nushell query plugin signatures, plugin just returns it's signature without any examples, so nushell have no idea about the examples of plugin commands. To adding the feature, we just making plugin returns it's signature with examples. Before: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<Signature> ``` After: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<PluginSignature> ``` When writing plugin signature to $nu.plugin-path: Serialize `<PluginSignature>` rather than `<Signature>`, which would enable us to serialize examples to `$nu.plugin-path` ## Shortcoming It's a breaking changes because `Plugin::signature` is changed, and it requires plugin authors to change their code for new signatures. Fortunally it should be easy to change, for rust based plugin, we just need to make a global replace from word `Signature` to word `PluginSignature` in their plugin project. Our content of plugin-path is really large, if one plugin have many examples, it'd results to larger body of $nu.plugin-path, which is not really scale. A solution would be save register information in other binary formats rather than `json`. But I think it'd be another story. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-02-08 22:14:18 +00:00
fn examples(&self) -> Vec<Example> {
let mut res = vec![];
for e in self.signature.examples.iter() {
res.push(Example {
example: &e.example,
description: &e.description,
result: e.result.clone(),
})
}
res
}
fn run(
&self,
engine_state: &EngineState,
stack: &mut Stack,
call: &Call,
input: PipelineData,
) -> Result<PipelineData, ShellError> {
Debugger experiments (#11441) <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR adds a new evaluator path with callbacks to a mutable trait object implementing a Debugger trait. The trait object can do anything, e.g., profiling, code coverage, step debugging. Currently, entering/leaving a block and a pipeline element is marked with callbacks, but more callbacks can be added as necessary. Not all callbacks need to be used by all debuggers; unused ones are simply empty calls. A simple profiler is implemented as a proof of concept. The debugging support is implementing by making `eval_xxx()` functions generic depending on whether we're debugging or not. This has zero computational overhead, but makes the binary slightly larger (see benchmarks below). `eval_xxx()` variants called from commands (like `eval_block_with_early_return()` in `each`) are chosen with a dynamic dispatch for two reasons: to not grow the binary size due to duplicating the code of many commands, and for the fact that it isn't possible because it would make Command trait objects object-unsafe. In the future, I hope it will be possible to allow plugin callbacks such that users would be able to implement their profiler plugins instead of having to recompile Nushell. [DAP](https://microsoft.github.io/debug-adapter-protocol/) would also be interesting to explore. Try `help debug profile`. ## Screenshots Basic output: ![profiler_new](https://github.com/nushell/nushell/assets/25571562/418b9df0-b659-4dcb-b023-2d5fcef2c865) To profile with more granularity, increase the profiler depth (you'll see that repeated `is-windows` calls take a large chunk of total time, making it a good candidate for optimizing): ![profiler_new_m3](https://github.com/nushell/nushell/assets/25571562/636d756d-5d56-460c-a372-14716f65f37f) ## Benchmarks ### Binary size Binary size increase vs. main: **+40360 bytes**. _(Both built with `--release --features=extra,dataframe`.)_ ### Time ```nushell # bench_debug.nu use std bench let test = { 1..100 | each { ls | each {|row| $row.name | str length } } | flatten | math avg } print 'debug:' let res2 = bench { debug profile $test } --pretty print $res2 ``` ```nushell # bench_nodebug.nu use std bench let test = { 1..100 | each { ls | each {|row| $row.name | str length } } | flatten | math avg } print 'no debug:' let res1 = bench { do $test } --pretty print $res1 ``` `cargo run --release -- bench_debug.nu` is consistently 1--2 ms slower than `cargo run --release -- bench_nodebug.nu` due to the collection overhead + gathering the report. This is expected. When gathering more stuff, the overhead is obviously higher. `cargo run --release -- bench_nodebug.nu` vs. `nu bench_nodebug.nu` I didn't measure any difference. Both benchmarks report times between 97 and 103 ms randomly, without one being consistently higher than the other. This suggests that at least in this particular case, when not running any debugger, there is no runtime overhead. ## API changes This PR adds a generic parameter to all `eval_xxx` functions that forces you to specify whether you use the debugger. You can resolve it in two ways: * Use a provided helper that will figure it out for you. If you wanted to use `eval_block(&engine_state, ...)`, call `let eval_block = get_eval_block(&engine_state); eval_block(&engine_state, ...)` * If you know you're in an evaluation path that doesn't need debugger support, call `eval_block::<WithoutDebug>(&engine_state, ...)` (this is the case of hooks, for example). I tried to add more explanation in the docstring of `debugger_trait.rs`. ## TODO - [x] Better profiler output to reduce spam of iterative commands like `each` - [x] Resolve `TODO: DEBUG` comments - [x] Resolve unwraps - [x] Add doc comments - [x] Add usage and extra usage for `debug profile`, explaining all columns # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> Hopefully none. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2024-03-08 18:21:35 +00:00
let eval_block = get_eval_block(engine_state);
let eval_expression = get_eval_expression(engine_state);
// Create the EvaluatedCall to send to the plugin first - it's best for this to fail early,
// before we actually try to run the plugin command
Debugger experiments (#11441) <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR adds a new evaluator path with callbacks to a mutable trait object implementing a Debugger trait. The trait object can do anything, e.g., profiling, code coverage, step debugging. Currently, entering/leaving a block and a pipeline element is marked with callbacks, but more callbacks can be added as necessary. Not all callbacks need to be used by all debuggers; unused ones are simply empty calls. A simple profiler is implemented as a proof of concept. The debugging support is implementing by making `eval_xxx()` functions generic depending on whether we're debugging or not. This has zero computational overhead, but makes the binary slightly larger (see benchmarks below). `eval_xxx()` variants called from commands (like `eval_block_with_early_return()` in `each`) are chosen with a dynamic dispatch for two reasons: to not grow the binary size due to duplicating the code of many commands, and for the fact that it isn't possible because it would make Command trait objects object-unsafe. In the future, I hope it will be possible to allow plugin callbacks such that users would be able to implement their profiler plugins instead of having to recompile Nushell. [DAP](https://microsoft.github.io/debug-adapter-protocol/) would also be interesting to explore. Try `help debug profile`. ## Screenshots Basic output: ![profiler_new](https://github.com/nushell/nushell/assets/25571562/418b9df0-b659-4dcb-b023-2d5fcef2c865) To profile with more granularity, increase the profiler depth (you'll see that repeated `is-windows` calls take a large chunk of total time, making it a good candidate for optimizing): ![profiler_new_m3](https://github.com/nushell/nushell/assets/25571562/636d756d-5d56-460c-a372-14716f65f37f) ## Benchmarks ### Binary size Binary size increase vs. main: **+40360 bytes**. _(Both built with `--release --features=extra,dataframe`.)_ ### Time ```nushell # bench_debug.nu use std bench let test = { 1..100 | each { ls | each {|row| $row.name | str length } } | flatten | math avg } print 'debug:' let res2 = bench { debug profile $test } --pretty print $res2 ``` ```nushell # bench_nodebug.nu use std bench let test = { 1..100 | each { ls | each {|row| $row.name | str length } } | flatten | math avg } print 'no debug:' let res1 = bench { do $test } --pretty print $res1 ``` `cargo run --release -- bench_debug.nu` is consistently 1--2 ms slower than `cargo run --release -- bench_nodebug.nu` due to the collection overhead + gathering the report. This is expected. When gathering more stuff, the overhead is obviously higher. `cargo run --release -- bench_nodebug.nu` vs. `nu bench_nodebug.nu` I didn't measure any difference. Both benchmarks report times between 97 and 103 ms randomly, without one being consistently higher than the other. This suggests that at least in this particular case, when not running any debugger, there is no runtime overhead. ## API changes This PR adds a generic parameter to all `eval_xxx` functions that forces you to specify whether you use the debugger. You can resolve it in two ways: * Use a provided helper that will figure it out for you. If you wanted to use `eval_block(&engine_state, ...)`, call `let eval_block = get_eval_block(&engine_state); eval_block(&engine_state, ...)` * If you know you're in an evaluation path that doesn't need debugger support, call `eval_block::<WithoutDebug>(&engine_state, ...)` (this is the case of hooks, for example). I tried to add more explanation in the docstring of `debugger_trait.rs`. ## TODO - [x] Better profiler output to reduce spam of iterative commands like `each` - [x] Resolve `TODO: DEBUG` comments - [x] Resolve unwraps - [x] Add doc comments - [x] Add usage and extra usage for `debug profile`, explaining all columns # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> Hopefully none. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2024-03-08 18:21:35 +00:00
let evaluated_call =
EvaluatedCall::try_from_call(call, engine_state, stack, eval_expression)?;
Allow plugins to receive configuration from the nushell configuration (#10955) # Description When nushell calls a plugin it now sends a configuration `Value` from the nushell config under `$env.config.plugins.PLUGIN_SHORT_NAME`. This allows plugin authors to read configuration provided by plugin users. The `PLUGIN_SHORT_NAME` must match the registered filename after `nu_plugin_`. If you register `target/debug/nu_plugin_config` the `PLUGIN_NAME` will be `config` and the nushell config will loook like: $env.config = { # ... plugins: { config: [ some values ] } } Configuration may also use a closure which allows passing values from `$env` to a plugin: $env.config = { # ... plugins: { config: {|| $env.some_value } } } This is a breaking change for the plugin API as the `Plugin::run()` function now accepts a new configuration argument which is an `&Option<Value>`. If no configuration was supplied the value is `None`. Plugins compiled after this change should work with older nushell, and will behave as if the configuration was not set. Initially discussed in #10867 # User-Facing Changes * Plugins can read configuration data stored in `$env.config.plugins` * The plugin `CallInfo` now includes a `config` entry, existing plugins will require updates # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` # After Submitting - [ ] Update [Creating a plugin (in Rust)](https://www.nushell.sh/contributor-book/plugins.html#creating-a-plugin-in-rust) [source](https://github.com/nushell/nushell.github.io/blob/main/contributor-book/plugins.md) - [ ] Add "Configuration" section to [Plugins documentation](https://www.nushell.sh/contributor-book/plugins.html)
2024-01-15 08:59:47 +00:00
// Fetch the configuration for a plugin
//
// The `plugin` must match the registered name of a plugin. For
// `register nu_plugin_example` the plugin config lookup uses `"example"`
let config = nu_engine::get_config(engine_state, stack)
.plugins
.get(&self.identity.plugin_name)
.cloned()
Allow plugins to receive configuration from the nushell configuration (#10955) # Description When nushell calls a plugin it now sends a configuration `Value` from the nushell config under `$env.config.plugins.PLUGIN_SHORT_NAME`. This allows plugin authors to read configuration provided by plugin users. The `PLUGIN_SHORT_NAME` must match the registered filename after `nu_plugin_`. If you register `target/debug/nu_plugin_config` the `PLUGIN_NAME` will be `config` and the nushell config will loook like: $env.config = { # ... plugins: { config: [ some values ] } } Configuration may also use a closure which allows passing values from `$env` to a plugin: $env.config = { # ... plugins: { config: {|| $env.some_value } } } This is a breaking change for the plugin API as the `Plugin::run()` function now accepts a new configuration argument which is an `&Option<Value>`. If no configuration was supplied the value is `None`. Plugins compiled after this change should work with older nushell, and will behave as if the configuration was not set. Initially discussed in #10867 # User-Facing Changes * Plugins can read configuration data stored in `$env.config.plugins` * The plugin `CallInfo` now includes a `config` entry, existing plugins will require updates # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` # After Submitting - [ ] Update [Creating a plugin (in Rust)](https://www.nushell.sh/contributor-book/plugins.html#creating-a-plugin-in-rust) [source](https://github.com/nushell/nushell.github.io/blob/main/contributor-book/plugins.md) - [ ] Add "Configuration" section to [Plugins documentation](https://www.nushell.sh/contributor-book/plugins.html)
2024-01-15 08:59:47 +00:00
.map(|value| {
let span = value.span();
match value {
Value::Closure { val, .. } => {
let input = PipelineData::Empty;
let block = engine_state.get_block(val.block_id).clone();
let mut stack = stack.captures_to_stack(val.captures);
match eval_block(engine_state, &mut stack, &block, input, false, false) {
Ok(v) => v.into_value(span),
Err(e) => Value::error(e, call.head),
}
}
_ => value.clone(),
}
});
// We need the current environment variables for `python` based plugins
// Or we'll likely have a problem when a plugin is implemented in a virtual Python environment.
let current_envs = nu_engine::env::env_to_strings(engine_state, stack).unwrap_or_default();
// Start the plugin
let plugin = self.identity.clone().spawn(current_envs).map_err(|err| {
let decl = engine_state.get_decl(call.decl_id);
ShellError::GenericError {
error: format!("Unable to spawn plugin for `{}`", decl.name()),
msg: err.to_string(),
span: Some(call.head),
help: None,
inner: vec![],
}
})?;
// Create the context to execute in
let context = Arc::new(PluginExecutionCommandContext::new(
engine_state,
stack,
call,
));
plugin.run(
CallInfo {
name: self.name.clone(),
call: evaluated_call,
input,
config,
},
context,
)
}
fn is_plugin(&self) -> Option<(&Path, Option<&Path>)> {
Some((&self.identity.filename, self.identity.shell.as_deref()))
}
}