nushell/crates/nu-plugin/src/plugin/declaration.rs

181 lines
6.1 KiB
Rust
Raw Normal View History

use crate::EvaluatedCall;
use super::{call_plugin, create_command, get_plugin_encoding};
Add CustomValue support to plugins (#6070) * Skeleton implementation Lots and lots of TODOs * Bootstrap simple CustomValue plugin support test * Create nu_plugin_custom_value * Skeleton for nu_plugin_custom_values * Return a custom value from plugin * Encode CustomValues from plugin calls as PluginResponse::PluginData * Add new PluginCall variant CollapseCustomValue * Handle CollapseCustomValue plugin calls * Add CallInput::Data variant to CallInfo inputs * Handle CallInfo with CallInput::Data plugin calls * Send CallInput::Data if Value is PluginCustomValue from plugin calls * Remove unnecessary boxing of plugins CallInfo * Add fields needed to collapse PluginCustomValue to it * Document PluginCustomValue and its purpose * Impl collapsing using plugin calls in PluginCustomValue::to_base_value * Implement proper typetag based deserialization for CoolCustomValue * Test demonstrating that passing back a custom value to plugin works * Added a failing test for describing plugin CustomValues * Support describe for PluginCustomValues - Add name to PluginResponse::PluginData - Also turn it into a struct for clarity - Add name to PluginCustomValue - Return name field from PluginCustomValue * Demonstrate that plugins can create and handle multiple CustomValues * Add bincode to nu-plugin dependencies This is for demonstration purposes, any schemaless binary seralization format will work. I picked bincode since it's the most popular for Rust but there are defintely better options out there for this usecase * serde_json::Value -> Vec<u8> * Update capnp schema for new CallInfo.input field * Move call_input capnp serialization and deserialization into new file * Deserialize Value's span from Value itself instead of passing call.head I am not sure if this was correct and I am breaking it or if it was a bug, I don't fully understand how nu creates and uses Spans. What should reuse spans and what should recreate new ones? But yeah it felt weird that the Value's Span was being ignored since in the json serializer just uses the Value's Span * Add call_info value round trip test * Add capnp CallInput::Data serialization and deserialization support * Add CallInfo::CollapseCustomValue to capnp schema * Add capnp PluginCall::CollapseCustomValue serialization and deserialization support * Add PluginResponse::PluginData to capnp schema * Add capnp PluginResponse::PluginData serialization and deserialization support * Switch plugins::custom_values tests to capnp Both json and capnp would work now! Sadly I can't choose both at the same time :( * Add missing JsonSerializer round trip tests * Handle plugin returning PluginData as a response to CollapseCustomValue * Refactor plugin calling into a reusable function Many less levels of indentation now! * Export PluginData from nu_plugin So plugins can create their very own serve_plugin with whatever CustomValue behavior they may desire * Error if CustomValue cannot be handled by Plugin
2022-07-25 16:32:56 +00:00
use crate::protocol::{
CallInfo, CallInput, PluginCall, PluginCustomValue, PluginData, PluginResponse,
};
use std::path::{Path, PathBuf};
use nu_protocol::engine::{Command, EngineState, Stack};
Make plugin commands support examples. (#7984) # Description As title, we can't provide examples for plugin commands, this pr would make it possible # User-Facing Changes Take plugin `nu-example-1` as example: ``` ❯ nu-example-1 -h PluginSignature test 1 for plugin. Returns Value::Nothing Usage: > nu-example-1 {flags} <a> <b> (opt) ...(rest) Flags: -h, --help - Display the help message for this command -f, --flag - a flag for the signature -n, --named <String> - named string Parameters: a <int>: required integer value b <string>: required string value (optional) opt <int>: Optional number ...rest <string>: rest value string Examples: running example with an int value and string value > nu-example-1 3 bb ``` The examples session is newly added. ## Basic idea behind these changes when nushell query plugin signatures, plugin just returns it's signature without any examples, so nushell have no idea about the examples of plugin commands. To adding the feature, we just making plugin returns it's signature with examples. Before: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<Signature> ``` After: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<PluginSignature> ``` When writing plugin signature to $nu.plugin-path: Serialize `<PluginSignature>` rather than `<Signature>`, which would enable us to serialize examples to `$nu.plugin-path` ## Shortcoming It's a breaking changes because `Plugin::signature` is changed, and it requires plugin authors to change their code for new signatures. Fortunally it should be easy to change, for rust based plugin, we just need to make a global replace from word `Signature` to word `PluginSignature` in their plugin project. Our content of plugin-path is really large, if one plugin have many examples, it'd results to larger body of $nu.plugin-path, which is not really scale. A solution would be save register information in other binary formats rather than `json`. But I think it'd be another story. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-02-08 22:14:18 +00:00
use nu_protocol::{ast::Call, PluginSignature, Signature};
use nu_protocol::{Example, PipelineData, ShellError, Value};
Plugin api docs (#9452) <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Added comments to support API docs for the `nu-plugin` crate. Removed a few items that I'd expect should only be used internally to Nushell from the documentation and reduced the visibility of some items that did not need to be public. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> There should be no user facing impact. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect -A clippy::result_large_err` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> Standard tests run. Additionally numerous doctests were added to the `nu-plugin` crate. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> No changes to the website necessary.
2023-06-16 14:25:40 +00:00
#[doc(hidden)] // Note: not for plugin authors / only used in nu-parser
#[derive(Clone)]
pub struct PluginDeclaration {
name: String,
Make plugin commands support examples. (#7984) # Description As title, we can't provide examples for plugin commands, this pr would make it possible # User-Facing Changes Take plugin `nu-example-1` as example: ``` ❯ nu-example-1 -h PluginSignature test 1 for plugin. Returns Value::Nothing Usage: > nu-example-1 {flags} <a> <b> (opt) ...(rest) Flags: -h, --help - Display the help message for this command -f, --flag - a flag for the signature -n, --named <String> - named string Parameters: a <int>: required integer value b <string>: required string value (optional) opt <int>: Optional number ...rest <string>: rest value string Examples: running example with an int value and string value > nu-example-1 3 bb ``` The examples session is newly added. ## Basic idea behind these changes when nushell query plugin signatures, plugin just returns it's signature without any examples, so nushell have no idea about the examples of plugin commands. To adding the feature, we just making plugin returns it's signature with examples. Before: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<Signature> ``` After: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<PluginSignature> ``` When writing plugin signature to $nu.plugin-path: Serialize `<PluginSignature>` rather than `<Signature>`, which would enable us to serialize examples to `$nu.plugin-path` ## Shortcoming It's a breaking changes because `Plugin::signature` is changed, and it requires plugin authors to change their code for new signatures. Fortunally it should be easy to change, for rust based plugin, we just need to make a global replace from word `Signature` to word `PluginSignature` in their plugin project. Our content of plugin-path is really large, if one plugin have many examples, it'd results to larger body of $nu.plugin-path, which is not really scale. A solution would be save register information in other binary formats rather than `json`. But I think it'd be another story. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-02-08 22:14:18 +00:00
signature: PluginSignature,
filename: PathBuf,
shell: Option<PathBuf>,
}
impl PluginDeclaration {
Make plugin commands support examples. (#7984) # Description As title, we can't provide examples for plugin commands, this pr would make it possible # User-Facing Changes Take plugin `nu-example-1` as example: ``` ❯ nu-example-1 -h PluginSignature test 1 for plugin. Returns Value::Nothing Usage: > nu-example-1 {flags} <a> <b> (opt) ...(rest) Flags: -h, --help - Display the help message for this command -f, --flag - a flag for the signature -n, --named <String> - named string Parameters: a <int>: required integer value b <string>: required string value (optional) opt <int>: Optional number ...rest <string>: rest value string Examples: running example with an int value and string value > nu-example-1 3 bb ``` The examples session is newly added. ## Basic idea behind these changes when nushell query plugin signatures, plugin just returns it's signature without any examples, so nushell have no idea about the examples of plugin commands. To adding the feature, we just making plugin returns it's signature with examples. Before: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<Signature> ``` After: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<PluginSignature> ``` When writing plugin signature to $nu.plugin-path: Serialize `<PluginSignature>` rather than `<Signature>`, which would enable us to serialize examples to `$nu.plugin-path` ## Shortcoming It's a breaking changes because `Plugin::signature` is changed, and it requires plugin authors to change their code for new signatures. Fortunally it should be easy to change, for rust based plugin, we just need to make a global replace from word `Signature` to word `PluginSignature` in their plugin project. Our content of plugin-path is really large, if one plugin have many examples, it'd results to larger body of $nu.plugin-path, which is not really scale. A solution would be save register information in other binary formats rather than `json`. But I think it'd be another story. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-02-08 22:14:18 +00:00
pub fn new(filename: PathBuf, signature: PluginSignature, shell: Option<PathBuf>) -> Self {
Self {
Make plugin commands support examples. (#7984) # Description As title, we can't provide examples for plugin commands, this pr would make it possible # User-Facing Changes Take plugin `nu-example-1` as example: ``` ❯ nu-example-1 -h PluginSignature test 1 for plugin. Returns Value::Nothing Usage: > nu-example-1 {flags} <a> <b> (opt) ...(rest) Flags: -h, --help - Display the help message for this command -f, --flag - a flag for the signature -n, --named <String> - named string Parameters: a <int>: required integer value b <string>: required string value (optional) opt <int>: Optional number ...rest <string>: rest value string Examples: running example with an int value and string value > nu-example-1 3 bb ``` The examples session is newly added. ## Basic idea behind these changes when nushell query plugin signatures, plugin just returns it's signature without any examples, so nushell have no idea about the examples of plugin commands. To adding the feature, we just making plugin returns it's signature with examples. Before: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<Signature> ``` After: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<PluginSignature> ``` When writing plugin signature to $nu.plugin-path: Serialize `<PluginSignature>` rather than `<Signature>`, which would enable us to serialize examples to `$nu.plugin-path` ## Shortcoming It's a breaking changes because `Plugin::signature` is changed, and it requires plugin authors to change their code for new signatures. Fortunally it should be easy to change, for rust based plugin, we just need to make a global replace from word `Signature` to word `PluginSignature` in their plugin project. Our content of plugin-path is really large, if one plugin have many examples, it'd results to larger body of $nu.plugin-path, which is not really scale. A solution would be save register information in other binary formats rather than `json`. But I think it'd be another story. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-02-08 22:14:18 +00:00
name: signature.sig.name.clone(),
signature,
filename,
shell,
}
}
}
impl Command for PluginDeclaration {
fn name(&self) -> &str {
&self.name
}
fn signature(&self) -> Signature {
Make plugin commands support examples. (#7984) # Description As title, we can't provide examples for plugin commands, this pr would make it possible # User-Facing Changes Take plugin `nu-example-1` as example: ``` ❯ nu-example-1 -h PluginSignature test 1 for plugin. Returns Value::Nothing Usage: > nu-example-1 {flags} <a> <b> (opt) ...(rest) Flags: -h, --help - Display the help message for this command -f, --flag - a flag for the signature -n, --named <String> - named string Parameters: a <int>: required integer value b <string>: required string value (optional) opt <int>: Optional number ...rest <string>: rest value string Examples: running example with an int value and string value > nu-example-1 3 bb ``` The examples session is newly added. ## Basic idea behind these changes when nushell query plugin signatures, plugin just returns it's signature without any examples, so nushell have no idea about the examples of plugin commands. To adding the feature, we just making plugin returns it's signature with examples. Before: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<Signature> ``` After: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<PluginSignature> ``` When writing plugin signature to $nu.plugin-path: Serialize `<PluginSignature>` rather than `<Signature>`, which would enable us to serialize examples to `$nu.plugin-path` ## Shortcoming It's a breaking changes because `Plugin::signature` is changed, and it requires plugin authors to change their code for new signatures. Fortunally it should be easy to change, for rust based plugin, we just need to make a global replace from word `Signature` to word `PluginSignature` in their plugin project. Our content of plugin-path is really large, if one plugin have many examples, it'd results to larger body of $nu.plugin-path, which is not really scale. A solution would be save register information in other binary formats rather than `json`. But I think it'd be another story. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-02-08 22:14:18 +00:00
self.signature.sig.clone()
}
fn usage(&self) -> &str {
Make plugin commands support examples. (#7984) # Description As title, we can't provide examples for plugin commands, this pr would make it possible # User-Facing Changes Take plugin `nu-example-1` as example: ``` ❯ nu-example-1 -h PluginSignature test 1 for plugin. Returns Value::Nothing Usage: > nu-example-1 {flags} <a> <b> (opt) ...(rest) Flags: -h, --help - Display the help message for this command -f, --flag - a flag for the signature -n, --named <String> - named string Parameters: a <int>: required integer value b <string>: required string value (optional) opt <int>: Optional number ...rest <string>: rest value string Examples: running example with an int value and string value > nu-example-1 3 bb ``` The examples session is newly added. ## Basic idea behind these changes when nushell query plugin signatures, plugin just returns it's signature without any examples, so nushell have no idea about the examples of plugin commands. To adding the feature, we just making plugin returns it's signature with examples. Before: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<Signature> ``` After: ``` 1. get signature ----------------> Nushell ------------------ Plugin <----------------- 2. returns Vec<PluginSignature> ``` When writing plugin signature to $nu.plugin-path: Serialize `<PluginSignature>` rather than `<Signature>`, which would enable us to serialize examples to `$nu.plugin-path` ## Shortcoming It's a breaking changes because `Plugin::signature` is changed, and it requires plugin authors to change their code for new signatures. Fortunally it should be easy to change, for rust based plugin, we just need to make a global replace from word `Signature` to word `PluginSignature` in their plugin project. Our content of plugin-path is really large, if one plugin have many examples, it'd results to larger body of $nu.plugin-path, which is not really scale. A solution would be save register information in other binary formats rather than `json`. But I think it'd be another story. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-02-08 22:14:18 +00:00
self.signature.sig.usage.as_str()
}
fn examples(&self) -> Vec<Example> {
let mut res = vec![];
for e in self.signature.examples.iter() {
res.push(Example {
example: &e.example,
description: &e.description,
result: e.result.clone(),
})
}
res
}
fn run(
&self,
engine_state: &EngineState,
stack: &mut Stack,
call: &Call,
input: PipelineData,
) -> Result<PipelineData, ShellError> {
// Call the command with self path
// Decode information from plugin
// Create PipelineData
let source_file = Path::new(&self.filename);
let mut plugin_cmd = create_command(source_file, &self.shell);
// We need the current environment variables for `python` based plugins
// Or we'll likely have a problem when a plugin is implemented in a virtual Python environment.
let current_envs = nu_engine::env::env_to_strings(engine_state, stack).unwrap_or_default();
plugin_cmd.envs(current_envs);
let mut child = plugin_cmd.spawn().map_err(|err| {
let decl = engine_state.get_decl(call.decl_id);
ShellError::GenericError(
format!("Unable to spawn plugin for {}", decl.name()),
format!("{err}"),
Some(call.head),
None,
Vec::new(),
)
})?;
let input = input.into_value(call.head);
Add CustomValue support to plugins (#6070) * Skeleton implementation Lots and lots of TODOs * Bootstrap simple CustomValue plugin support test * Create nu_plugin_custom_value * Skeleton for nu_plugin_custom_values * Return a custom value from plugin * Encode CustomValues from plugin calls as PluginResponse::PluginData * Add new PluginCall variant CollapseCustomValue * Handle CollapseCustomValue plugin calls * Add CallInput::Data variant to CallInfo inputs * Handle CallInfo with CallInput::Data plugin calls * Send CallInput::Data if Value is PluginCustomValue from plugin calls * Remove unnecessary boxing of plugins CallInfo * Add fields needed to collapse PluginCustomValue to it * Document PluginCustomValue and its purpose * Impl collapsing using plugin calls in PluginCustomValue::to_base_value * Implement proper typetag based deserialization for CoolCustomValue * Test demonstrating that passing back a custom value to plugin works * Added a failing test for describing plugin CustomValues * Support describe for PluginCustomValues - Add name to PluginResponse::PluginData - Also turn it into a struct for clarity - Add name to PluginCustomValue - Return name field from PluginCustomValue * Demonstrate that plugins can create and handle multiple CustomValues * Add bincode to nu-plugin dependencies This is for demonstration purposes, any schemaless binary seralization format will work. I picked bincode since it's the most popular for Rust but there are defintely better options out there for this usecase * serde_json::Value -> Vec<u8> * Update capnp schema for new CallInfo.input field * Move call_input capnp serialization and deserialization into new file * Deserialize Value's span from Value itself instead of passing call.head I am not sure if this was correct and I am breaking it or if it was a bug, I don't fully understand how nu creates and uses Spans. What should reuse spans and what should recreate new ones? But yeah it felt weird that the Value's Span was being ignored since in the json serializer just uses the Value's Span * Add call_info value round trip test * Add capnp CallInput::Data serialization and deserialization support * Add CallInfo::CollapseCustomValue to capnp schema * Add capnp PluginCall::CollapseCustomValue serialization and deserialization support * Add PluginResponse::PluginData to capnp schema * Add capnp PluginResponse::PluginData serialization and deserialization support * Switch plugins::custom_values tests to capnp Both json and capnp would work now! Sadly I can't choose both at the same time :( * Add missing JsonSerializer round trip tests * Handle plugin returning PluginData as a response to CollapseCustomValue * Refactor plugin calling into a reusable function Many less levels of indentation now! * Export PluginData from nu_plugin So plugins can create their very own serve_plugin with whatever CustomValue behavior they may desire * Error if CustomValue cannot be handled by Plugin
2022-07-25 16:32:56 +00:00
let input = match input {
Value::CustomValue { val, span } => {
match val.as_any().downcast_ref::<PluginCustomValue>() {
Some(plugin_data) if plugin_data.filename == self.filename => {
CallInput::Data(PluginData {
data: plugin_data.data.clone(),
span,
})
}
_ => {
let custom_value_name = val.value_string();
return Err(ShellError::GenericError(
format!(
"Plugin {} can not handle the custom value {}",
self.name, custom_value_name
),
format!("custom value {custom_value_name}"),
Add CustomValue support to plugins (#6070) * Skeleton implementation Lots and lots of TODOs * Bootstrap simple CustomValue plugin support test * Create nu_plugin_custom_value * Skeleton for nu_plugin_custom_values * Return a custom value from plugin * Encode CustomValues from plugin calls as PluginResponse::PluginData * Add new PluginCall variant CollapseCustomValue * Handle CollapseCustomValue plugin calls * Add CallInput::Data variant to CallInfo inputs * Handle CallInfo with CallInput::Data plugin calls * Send CallInput::Data if Value is PluginCustomValue from plugin calls * Remove unnecessary boxing of plugins CallInfo * Add fields needed to collapse PluginCustomValue to it * Document PluginCustomValue and its purpose * Impl collapsing using plugin calls in PluginCustomValue::to_base_value * Implement proper typetag based deserialization for CoolCustomValue * Test demonstrating that passing back a custom value to plugin works * Added a failing test for describing plugin CustomValues * Support describe for PluginCustomValues - Add name to PluginResponse::PluginData - Also turn it into a struct for clarity - Add name to PluginCustomValue - Return name field from PluginCustomValue * Demonstrate that plugins can create and handle multiple CustomValues * Add bincode to nu-plugin dependencies This is for demonstration purposes, any schemaless binary seralization format will work. I picked bincode since it's the most popular for Rust but there are defintely better options out there for this usecase * serde_json::Value -> Vec<u8> * Update capnp schema for new CallInfo.input field * Move call_input capnp serialization and deserialization into new file * Deserialize Value's span from Value itself instead of passing call.head I am not sure if this was correct and I am breaking it or if it was a bug, I don't fully understand how nu creates and uses Spans. What should reuse spans and what should recreate new ones? But yeah it felt weird that the Value's Span was being ignored since in the json serializer just uses the Value's Span * Add call_info value round trip test * Add capnp CallInput::Data serialization and deserialization support * Add CallInfo::CollapseCustomValue to capnp schema * Add capnp PluginCall::CollapseCustomValue serialization and deserialization support * Add PluginResponse::PluginData to capnp schema * Add capnp PluginResponse::PluginData serialization and deserialization support * Switch plugins::custom_values tests to capnp Both json and capnp would work now! Sadly I can't choose both at the same time :( * Add missing JsonSerializer round trip tests * Handle plugin returning PluginData as a response to CollapseCustomValue * Refactor plugin calling into a reusable function Many less levels of indentation now! * Export PluginData from nu_plugin So plugins can create their very own serve_plugin with whatever CustomValue behavior they may desire * Error if CustomValue cannot be handled by Plugin
2022-07-25 16:32:56 +00:00
Some(span),
None,
Vec::new(),
));
}
}
}
LazyRecord (#7619) This is an attempt to implement a new `Value::LazyRecord` variant for performance reasons. `LazyRecord` is like a regular `Record`, but it's possible to access individual columns without evaluating other columns. I've implemented `LazyRecord` for the special `$nu` variable; accessing `$nu` is relatively slow because of all the information in `scope`, and [`$nu` accounts for about 2/3 of Nu's startup time on Linux](https://github.com/nushell/nushell/issues/6677#issuecomment-1364618122). ### Benchmarks I ran some benchmarks on my desktop (Linux, 12900K) and the results are very pleasing. Nu's time to start up and run a command (`cargo build --release; hyperfine 'target/release/nu -c "echo \"Hello, world!\""' --shell=none --warmup 10`) goes from **8.8ms to 3.2ms, about 2.8x faster**. Tests are also much faster! Running `cargo nextest` (with our very slow `proptest` tests disabled) goes from **7.2s to 4.4s (1.6x faster)**, because most tests involve launching a new instance of Nu. ### Design (updated) I've added a new `LazyRecord` trait and added a `Value` variant wrapping those trait objects, much like `CustomValue`. `LazyRecord` implementations must implement these 2 functions: ```rust // All column names fn column_names(&self) -> Vec<&'static str>; // Get 1 specific column value fn get_column_value(&self, column: &str) -> Result<Value, ShellError>; ``` ### Serializability `Value` variants must implement `Serializable` and `Deserializable`, which poses some problems because I want to use unserializable things like `EngineState` in `LazyRecord`s. To work around this, I basically lie to the type system: 1. Add `#[typetag::serde(tag = "type")]` to `LazyRecord` to make it serializable 2. Any unserializable fields in `LazyRecord` implementations get marked with `#[serde(skip)]` 3. At the point where a `LazyRecord` normally would get serialized and sent to a plugin, I instead collect it into a regular `Value::Record` (which can be serialized)
2023-01-19 03:27:26 +00:00
Value::LazyRecord { val, .. } => CallInput::Value(val.collect()?),
Add CustomValue support to plugins (#6070) * Skeleton implementation Lots and lots of TODOs * Bootstrap simple CustomValue plugin support test * Create nu_plugin_custom_value * Skeleton for nu_plugin_custom_values * Return a custom value from plugin * Encode CustomValues from plugin calls as PluginResponse::PluginData * Add new PluginCall variant CollapseCustomValue * Handle CollapseCustomValue plugin calls * Add CallInput::Data variant to CallInfo inputs * Handle CallInfo with CallInput::Data plugin calls * Send CallInput::Data if Value is PluginCustomValue from plugin calls * Remove unnecessary boxing of plugins CallInfo * Add fields needed to collapse PluginCustomValue to it * Document PluginCustomValue and its purpose * Impl collapsing using plugin calls in PluginCustomValue::to_base_value * Implement proper typetag based deserialization for CoolCustomValue * Test demonstrating that passing back a custom value to plugin works * Added a failing test for describing plugin CustomValues * Support describe for PluginCustomValues - Add name to PluginResponse::PluginData - Also turn it into a struct for clarity - Add name to PluginCustomValue - Return name field from PluginCustomValue * Demonstrate that plugins can create and handle multiple CustomValues * Add bincode to nu-plugin dependencies This is for demonstration purposes, any schemaless binary seralization format will work. I picked bincode since it's the most popular for Rust but there are defintely better options out there for this usecase * serde_json::Value -> Vec<u8> * Update capnp schema for new CallInfo.input field * Move call_input capnp serialization and deserialization into new file * Deserialize Value's span from Value itself instead of passing call.head I am not sure if this was correct and I am breaking it or if it was a bug, I don't fully understand how nu creates and uses Spans. What should reuse spans and what should recreate new ones? But yeah it felt weird that the Value's Span was being ignored since in the json serializer just uses the Value's Span * Add call_info value round trip test * Add capnp CallInput::Data serialization and deserialization support * Add CallInfo::CollapseCustomValue to capnp schema * Add capnp PluginCall::CollapseCustomValue serialization and deserialization support * Add PluginResponse::PluginData to capnp schema * Add capnp PluginResponse::PluginData serialization and deserialization support * Switch plugins::custom_values tests to capnp Both json and capnp would work now! Sadly I can't choose both at the same time :( * Add missing JsonSerializer round trip tests * Handle plugin returning PluginData as a response to CollapseCustomValue * Refactor plugin calling into a reusable function Many less levels of indentation now! * Export PluginData from nu_plugin So plugins can create their very own serve_plugin with whatever CustomValue behavior they may desire * Error if CustomValue cannot be handled by Plugin
2022-07-25 16:32:56 +00:00
value => CallInput::Value(value),
};
Add CustomValue support to plugins (#6070) * Skeleton implementation Lots and lots of TODOs * Bootstrap simple CustomValue plugin support test * Create nu_plugin_custom_value * Skeleton for nu_plugin_custom_values * Return a custom value from plugin * Encode CustomValues from plugin calls as PluginResponse::PluginData * Add new PluginCall variant CollapseCustomValue * Handle CollapseCustomValue plugin calls * Add CallInput::Data variant to CallInfo inputs * Handle CallInfo with CallInput::Data plugin calls * Send CallInput::Data if Value is PluginCustomValue from plugin calls * Remove unnecessary boxing of plugins CallInfo * Add fields needed to collapse PluginCustomValue to it * Document PluginCustomValue and its purpose * Impl collapsing using plugin calls in PluginCustomValue::to_base_value * Implement proper typetag based deserialization for CoolCustomValue * Test demonstrating that passing back a custom value to plugin works * Added a failing test for describing plugin CustomValues * Support describe for PluginCustomValues - Add name to PluginResponse::PluginData - Also turn it into a struct for clarity - Add name to PluginCustomValue - Return name field from PluginCustomValue * Demonstrate that plugins can create and handle multiple CustomValues * Add bincode to nu-plugin dependencies This is for demonstration purposes, any schemaless binary seralization format will work. I picked bincode since it's the most popular for Rust but there are defintely better options out there for this usecase * serde_json::Value -> Vec<u8> * Update capnp schema for new CallInfo.input field * Move call_input capnp serialization and deserialization into new file * Deserialize Value's span from Value itself instead of passing call.head I am not sure if this was correct and I am breaking it or if it was a bug, I don't fully understand how nu creates and uses Spans. What should reuse spans and what should recreate new ones? But yeah it felt weird that the Value's Span was being ignored since in the json serializer just uses the Value's Span * Add call_info value round trip test * Add capnp CallInput::Data serialization and deserialization support * Add CallInfo::CollapseCustomValue to capnp schema * Add capnp PluginCall::CollapseCustomValue serialization and deserialization support * Add PluginResponse::PluginData to capnp schema * Add capnp PluginResponse::PluginData serialization and deserialization support * Switch plugins::custom_values tests to capnp Both json and capnp would work now! Sadly I can't choose both at the same time :( * Add missing JsonSerializer round trip tests * Handle plugin returning PluginData as a response to CollapseCustomValue * Refactor plugin calling into a reusable function Many less levels of indentation now! * Export PluginData from nu_plugin So plugins can create their very own serve_plugin with whatever CustomValue behavior they may desire * Error if CustomValue cannot be handled by Plugin
2022-07-25 16:32:56 +00:00
let plugin_call = PluginCall::CallInfo(CallInfo {
name: self.name.clone(),
call: EvaluatedCall::try_from_call(call, engine_state, stack)?,
input,
});
let encoding = {
let stdout_reader = match &mut child.stdout {
Some(out) => out,
None => {
return Err(ShellError::PluginFailedToLoad(
"Plugin missing stdout reader".into(),
))
}
};
get_plugin_encoding(stdout_reader)?
};
let response = call_plugin(&mut child, plugin_call, &encoding, call.head).map_err(|err| {
let decl = engine_state.get_decl(call.decl_id);
ShellError::GenericError(
format!("Unable to decode call for {}", decl.name()),
err.to_string(),
Some(call.head),
None,
Vec::new(),
)
});
Add CustomValue support to plugins (#6070) * Skeleton implementation Lots and lots of TODOs * Bootstrap simple CustomValue plugin support test * Create nu_plugin_custom_value * Skeleton for nu_plugin_custom_values * Return a custom value from plugin * Encode CustomValues from plugin calls as PluginResponse::PluginData * Add new PluginCall variant CollapseCustomValue * Handle CollapseCustomValue plugin calls * Add CallInput::Data variant to CallInfo inputs * Handle CallInfo with CallInput::Data plugin calls * Send CallInput::Data if Value is PluginCustomValue from plugin calls * Remove unnecessary boxing of plugins CallInfo * Add fields needed to collapse PluginCustomValue to it * Document PluginCustomValue and its purpose * Impl collapsing using plugin calls in PluginCustomValue::to_base_value * Implement proper typetag based deserialization for CoolCustomValue * Test demonstrating that passing back a custom value to plugin works * Added a failing test for describing plugin CustomValues * Support describe for PluginCustomValues - Add name to PluginResponse::PluginData - Also turn it into a struct for clarity - Add name to PluginCustomValue - Return name field from PluginCustomValue * Demonstrate that plugins can create and handle multiple CustomValues * Add bincode to nu-plugin dependencies This is for demonstration purposes, any schemaless binary seralization format will work. I picked bincode since it's the most popular for Rust but there are defintely better options out there for this usecase * serde_json::Value -> Vec<u8> * Update capnp schema for new CallInfo.input field * Move call_input capnp serialization and deserialization into new file * Deserialize Value's span from Value itself instead of passing call.head I am not sure if this was correct and I am breaking it or if it was a bug, I don't fully understand how nu creates and uses Spans. What should reuse spans and what should recreate new ones? But yeah it felt weird that the Value's Span was being ignored since in the json serializer just uses the Value's Span * Add call_info value round trip test * Add capnp CallInput::Data serialization and deserialization support * Add CallInfo::CollapseCustomValue to capnp schema * Add capnp PluginCall::CollapseCustomValue serialization and deserialization support * Add PluginResponse::PluginData to capnp schema * Add capnp PluginResponse::PluginData serialization and deserialization support * Switch plugins::custom_values tests to capnp Both json and capnp would work now! Sadly I can't choose both at the same time :( * Add missing JsonSerializer round trip tests * Handle plugin returning PluginData as a response to CollapseCustomValue * Refactor plugin calling into a reusable function Many less levels of indentation now! * Export PluginData from nu_plugin So plugins can create their very own serve_plugin with whatever CustomValue behavior they may desire * Error if CustomValue cannot be handled by Plugin
2022-07-25 16:32:56 +00:00
let pipeline_data = match response {
Ok(PluginResponse::Value(value)) => {
Ok(PipelineData::Value(value.as_ref().clone(), None))
}
Add CustomValue support to plugins (#6070) * Skeleton implementation Lots and lots of TODOs * Bootstrap simple CustomValue plugin support test * Create nu_plugin_custom_value * Skeleton for nu_plugin_custom_values * Return a custom value from plugin * Encode CustomValues from plugin calls as PluginResponse::PluginData * Add new PluginCall variant CollapseCustomValue * Handle CollapseCustomValue plugin calls * Add CallInput::Data variant to CallInfo inputs * Handle CallInfo with CallInput::Data plugin calls * Send CallInput::Data if Value is PluginCustomValue from plugin calls * Remove unnecessary boxing of plugins CallInfo * Add fields needed to collapse PluginCustomValue to it * Document PluginCustomValue and its purpose * Impl collapsing using plugin calls in PluginCustomValue::to_base_value * Implement proper typetag based deserialization for CoolCustomValue * Test demonstrating that passing back a custom value to plugin works * Added a failing test for describing plugin CustomValues * Support describe for PluginCustomValues - Add name to PluginResponse::PluginData - Also turn it into a struct for clarity - Add name to PluginCustomValue - Return name field from PluginCustomValue * Demonstrate that plugins can create and handle multiple CustomValues * Add bincode to nu-plugin dependencies This is for demonstration purposes, any schemaless binary seralization format will work. I picked bincode since it's the most popular for Rust but there are defintely better options out there for this usecase * serde_json::Value -> Vec<u8> * Update capnp schema for new CallInfo.input field * Move call_input capnp serialization and deserialization into new file * Deserialize Value's span from Value itself instead of passing call.head I am not sure if this was correct and I am breaking it or if it was a bug, I don't fully understand how nu creates and uses Spans. What should reuse spans and what should recreate new ones? But yeah it felt weird that the Value's Span was being ignored since in the json serializer just uses the Value's Span * Add call_info value round trip test * Add capnp CallInput::Data serialization and deserialization support * Add CallInfo::CollapseCustomValue to capnp schema * Add capnp PluginCall::CollapseCustomValue serialization and deserialization support * Add PluginResponse::PluginData to capnp schema * Add capnp PluginResponse::PluginData serialization and deserialization support * Switch plugins::custom_values tests to capnp Both json and capnp would work now! Sadly I can't choose both at the same time :( * Add missing JsonSerializer round trip tests * Handle plugin returning PluginData as a response to CollapseCustomValue * Refactor plugin calling into a reusable function Many less levels of indentation now! * Export PluginData from nu_plugin So plugins can create their very own serve_plugin with whatever CustomValue behavior they may desire * Error if CustomValue cannot be handled by Plugin
2022-07-25 16:32:56 +00:00
Ok(PluginResponse::PluginData(name, plugin_data)) => Ok(PipelineData::Value(
Value::CustomValue {
val: Box::new(PluginCustomValue {
name,
data: plugin_data.data,
filename: self.filename.clone(),
shell: self.shell.clone(),
source: engine_state.get_decl(call.decl_id).name().to_owned(),
}),
span: plugin_data.span,
},
None,
)),
Ok(PluginResponse::Error(err)) => Err(err.into()),
Ok(PluginResponse::Signature(..)) => Err(ShellError::GenericError(
"Plugin missing value".into(),
"Received a signature from plugin instead of value".into(),
Some(call.head),
None,
Vec::new(),
Add CustomValue support to plugins (#6070) * Skeleton implementation Lots and lots of TODOs * Bootstrap simple CustomValue plugin support test * Create nu_plugin_custom_value * Skeleton for nu_plugin_custom_values * Return a custom value from plugin * Encode CustomValues from plugin calls as PluginResponse::PluginData * Add new PluginCall variant CollapseCustomValue * Handle CollapseCustomValue plugin calls * Add CallInput::Data variant to CallInfo inputs * Handle CallInfo with CallInput::Data plugin calls * Send CallInput::Data if Value is PluginCustomValue from plugin calls * Remove unnecessary boxing of plugins CallInfo * Add fields needed to collapse PluginCustomValue to it * Document PluginCustomValue and its purpose * Impl collapsing using plugin calls in PluginCustomValue::to_base_value * Implement proper typetag based deserialization for CoolCustomValue * Test demonstrating that passing back a custom value to plugin works * Added a failing test for describing plugin CustomValues * Support describe for PluginCustomValues - Add name to PluginResponse::PluginData - Also turn it into a struct for clarity - Add name to PluginCustomValue - Return name field from PluginCustomValue * Demonstrate that plugins can create and handle multiple CustomValues * Add bincode to nu-plugin dependencies This is for demonstration purposes, any schemaless binary seralization format will work. I picked bincode since it's the most popular for Rust but there are defintely better options out there for this usecase * serde_json::Value -> Vec<u8> * Update capnp schema for new CallInfo.input field * Move call_input capnp serialization and deserialization into new file * Deserialize Value's span from Value itself instead of passing call.head I am not sure if this was correct and I am breaking it or if it was a bug, I don't fully understand how nu creates and uses Spans. What should reuse spans and what should recreate new ones? But yeah it felt weird that the Value's Span was being ignored since in the json serializer just uses the Value's Span * Add call_info value round trip test * Add capnp CallInput::Data serialization and deserialization support * Add CallInfo::CollapseCustomValue to capnp schema * Add capnp PluginCall::CollapseCustomValue serialization and deserialization support * Add PluginResponse::PluginData to capnp schema * Add capnp PluginResponse::PluginData serialization and deserialization support * Switch plugins::custom_values tests to capnp Both json and capnp would work now! Sadly I can't choose both at the same time :( * Add missing JsonSerializer round trip tests * Handle plugin returning PluginData as a response to CollapseCustomValue * Refactor plugin calling into a reusable function Many less levels of indentation now! * Export PluginData from nu_plugin So plugins can create their very own serve_plugin with whatever CustomValue behavior they may desire * Error if CustomValue cannot be handled by Plugin
2022-07-25 16:32:56 +00:00
)),
Err(err) => Err(err),
};
// We need to call .wait() on the child, or we'll risk summoning the zombie horde
let _ = child.wait();
pipeline_data
}
fn is_plugin(&self) -> Option<(&PathBuf, &Option<PathBuf>)> {
Some((&self.filename, &self.shell))
}
}