# Description
The intended effect of the `extra` feature has been undermined by
introducing the full builds on our release pages and having more
activity on some of the extra commands.
To simplify the feature matrix let's get rid of it and focus our effort
on truly either refining a command to well-specified behavior or
discarding it entirely from the `nu` binary and moving it into plugins.
## Details
- Remove `--features extra` from CI
- Don't explicitly name `extra` in full build wf
- Remove feature extra from build-help scripts
- Update README in `nu-cmd-extra`
- Remove feature `extra`
- Fix previously dead `format pattern` tests
- Relax signature of `to html`
- Fix/ignore `html::test_no_color_flag`
- Remove dead features from `version`
- Refine `to html` type signature
# User-Facing Changes
The commands that were previously only available when building with
`--features extra` will now be available to everyone. This increases the
number of dependencies slightly but has a limited impact on the overall
binary size.
# Tests + Formatting
Some tests that were left in `nu-command` during cratification were dead
because the feature was not passed to `nu-command` and only to
`nu-cmd-lang` for feature-flag mention in `version`.
Those tests have now been either fixed or ignored in one case.
# After Submitting
There may be places in the documentation where we point to `--features
extra` that will now be moot (apart from the generated command help)
# Description
This allows plugins to make calls back to the engine to get config,
evaluate closures, and do other things that must be done within the
engine process.
Engine calls can both produce and consume streams as necessary. Closures
passed to plugins can both accept stream input and produce stream output
sent back to the plugin.
Engine calls referring to a plugin call's context can be processed as
long either the response hasn't been received, or the response created
streams that haven't ended yet.
This is a breaking API change for plugins. There are some pretty major
changes to the interface that plugins must implement, including:
1. Plugins now run with `&self` and must be `Sync`. Executing multiple
plugin calls in parallel is supported, and there's a chance that a
closure passed to a plugin could invoke the same plugin. Supporting
state across plugin invocations is left up to the plugin author to do in
whichever way they feel best, but the plugin object itself is still
shared. Even though the engine doesn't run multiple plugin calls through
the same process yet, I still considered it important to break the API
in this way at this stage. We might want to consider an optional
threadpool feature for performance.
2. Plugins take a reference to `EngineInterface`, which can be cloned.
This interface allows plugins to make calls back to the engine,
including for getting config and running closures.
3. Plugins no longer take the `config` parameter. This can be accessed
from the interface via the `.get_plugin_config()` engine call.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Not only does this have plugin protocol changes, it will require plugins
to make some code changes before they will work again. But on the plus
side, the engine call feature is extensible, and we can add more things
to it as needed.
Plugin maintainers will have to change the trait signature at the very
least. If they were using `config`, they will have to call
`engine.get_plugin_config()` instead.
If they were using the mutable reference to the plugin, they will have
to come up with some strategy to work around it (for example, for `Inc`
I just cloned it). This shouldn't be such a big deal at the moment as
it's not like plugins have ever run as daemons with persistent state in
the past, and they don't in this PR either. But I thought it was
important to make the change before we support plugins as daemons, as an
exclusive mutable reference is not compatible with parallel plugin
calls.
I suggest this gets merged sometime *after* the current pending release,
so that we have some time to adjust to the previous plugin protocol
changes that don't require code changes before making ones that do.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
I will document the additional protocol features (`EngineCall`,
`EngineCallResponse`), and constraints on plugin call processing if
engine calls are used - basically, to be aware that an engine call could
result in a nested plugin call, so the plugin should be able to handle
that.
# Description
This PR is just a minor development improvement. While working on
another feature, I noticed that the root crate lists the super useful
`pretty_assertions` in the root crate but doesn't use it in most tests.
With this change `pretty_assertions::assert_eq!` is used instead of
`core::assert_eq!` for better diffs when debugging the tests.
I thought of adding the dependency to other crates but I decided not to
since I didn't want a huge disruptive PR :)
Following up on #7180 with some feature cleanup:
- Move the `database` feature from `plugin` to `default`
- Rename the `database` feature to `sqlite`
- Remove `--features=extra` from a lot of scripts etc.
- No need to specify this, the `extra` feature is now the same as the
default feature set
- Remove the now-redundant 2nd Ubuntu test run
* Skeleton implementation
Lots and lots of TODOs
* Bootstrap simple CustomValue plugin support test
* Create nu_plugin_custom_value
* Skeleton for nu_plugin_custom_values
* Return a custom value from plugin
* Encode CustomValues from plugin calls as PluginResponse::PluginData
* Add new PluginCall variant CollapseCustomValue
* Handle CollapseCustomValue plugin calls
* Add CallInput::Data variant to CallInfo inputs
* Handle CallInfo with CallInput::Data plugin calls
* Send CallInput::Data if Value is PluginCustomValue from plugin calls
* Remove unnecessary boxing of plugins CallInfo
* Add fields needed to collapse PluginCustomValue to it
* Document PluginCustomValue and its purpose
* Impl collapsing using plugin calls in PluginCustomValue::to_base_value
* Implement proper typetag based deserialization for CoolCustomValue
* Test demonstrating that passing back a custom value to plugin works
* Added a failing test for describing plugin CustomValues
* Support describe for PluginCustomValues
- Add name to PluginResponse::PluginData
- Also turn it into a struct for clarity
- Add name to PluginCustomValue
- Return name field from PluginCustomValue
* Demonstrate that plugins can create and handle multiple CustomValues
* Add bincode to nu-plugin dependencies
This is for demonstration purposes, any schemaless binary seralization
format will work. I picked bincode since it's the most popular for Rust
but there are defintely better options out there for this usecase
* serde_json::Value -> Vec<u8>
* Update capnp schema for new CallInfo.input field
* Move call_input capnp serialization and deserialization into new file
* Deserialize Value's span from Value itself instead of passing call.head
I am not sure if this was correct and I am breaking it or if it was a
bug, I don't fully understand how nu creates and uses Spans. What should
reuse spans and what should recreate new ones?
But yeah it felt weird that the Value's Span was being ignored since in
the json serializer just uses the Value's Span
* Add call_info value round trip test
* Add capnp CallInput::Data serialization and deserialization support
* Add CallInfo::CollapseCustomValue to capnp schema
* Add capnp PluginCall::CollapseCustomValue serialization and deserialization support
* Add PluginResponse::PluginData to capnp schema
* Add capnp PluginResponse::PluginData serialization and deserialization support
* Switch plugins::custom_values tests to capnp
Both json and capnp would work now! Sadly I can't choose both at the
same time :(
* Add missing JsonSerializer round trip tests
* Handle plugin returning PluginData as a response to CollapseCustomValue
* Refactor plugin calling into a reusable function
Many less levels of indentation now!
* Export PluginData from nu_plugin
So plugins can create their very own serve_plugin with whatever
CustomValue behavior they may desire
* Error if CustomValue cannot be handled by Plugin
