Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
//! The tests in this file check the soundness of plugin persistence. When a plugin is needed by Nu,
|
|
|
|
//! it is spawned only if it was not already running. Plugins that are spawned are kept running and
|
|
|
|
//! are referenced in the engine state. Plugins can be stopped by the user if desired, but not
|
|
|
|
//! removed.
|
|
|
|
|
|
|
|
use nu_test_support::{nu, nu_with_plugins};
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
fn plugin_list_shows_installed_plugins() {
|
|
|
|
let out = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
|
|
|
plugins: [("nu_plugin_inc"), ("nu_plugin_custom_values")],
|
|
|
|
r#"(plugin list).name | str join ','"#
|
|
|
|
);
|
Make `plugin list` read state from plugin registry file as well (#14085)
# Description
[Context on
Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583)
**This is a breaking change, due to the removal of `is_running`.**
Some users find the `plugin list` command confusing, because it doesn't
show anything different after running `plugin add` or `plugin rm`. This
modifies the `plugin list` command to also look at the plugin registry
file to give some idea of how the plugins in engine state differ from
those in the plugin registry file.
The following values of `status` are now produced instead of
`is_running`:
- `added`: The plugin is present in the plugin registry file, but not in
the engine.
- `loaded`: The plugin is present both in the plugin registry file and
in the engine, but is not running.
- `running`: The plugin is currently running, and the `pid` column
should contain its process ID.
- `modified`: The plugin state present in the plugin registry file is
different from the state in the engine.
- `removed`: The plugin is still loaded in the engine, but is not
present in the plugin registry file.
- `invalid`: The data in the plugin registry file couldn't be
deserialized, and the plugin most likely needs to be added again.
Example (`commands` omitted):
```
╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮
│ # │ name │ version │ status │ pid │ filename │ shell │
├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤
│ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │
│ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │
│ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │
│ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │
│ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │
│ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │
│ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │
│ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │
│ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │
│ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │
╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯
```
# User-Facing Changes
To `plugin list`:
* **Breaking:** The `is_running` column is removed and replaced with
`status`. Use `status == running` to filter equivalently.
* The `--plugin-config` from other plugin management commands is now
supported.
* Added an `--engine` flag which behaves more or less like before, and
doesn't load the plugin registry file at all.
* Added a `--registry` flag which only checks the plugin registry file.
All plugins appear as `added` since there is no state to compare with.
Because the default is to check both, the `plugin list` command might be
a little bit slower. If you don't need to check the plugin registry
file, the `--engine` flag does not load the plugin registry file at all,
so it should be just as fast as before.
# Tests + Formatting
Added tests for `added` and `removed` statuses. `modified` and `invalid`
are a bit more tricky so I didn't try.
# After Submitting
- [ ] update documentation that references the `plugin list` command
- [ ] release notes
2024-10-17 02:24:45 +00:00
|
|
|
assert_eq!("custom_values,inc", out.out);
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
assert!(out.status.success());
|
2024-06-21 11:27:09 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
fn plugin_list_shows_installed_plugin_version() {
|
|
|
|
let out = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
|
|
|
plugin: ("nu_plugin_inc"),
|
|
|
|
r#"(plugin list).version.0"#
|
|
|
|
);
|
|
|
|
assert_eq!(env!("CARGO_PKG_VERSION"), out.out);
|
|
|
|
assert!(out.status.success());
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
fn plugin_keeps_running_after_calling_it() {
|
|
|
|
let out = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
|
|
|
plugin: ("nu_plugin_inc"),
|
|
|
|
r#"
|
|
|
|
plugin stop inc
|
Make `plugin list` read state from plugin registry file as well (#14085)
# Description
[Context on
Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583)
**This is a breaking change, due to the removal of `is_running`.**
Some users find the `plugin list` command confusing, because it doesn't
show anything different after running `plugin add` or `plugin rm`. This
modifies the `plugin list` command to also look at the plugin registry
file to give some idea of how the plugins in engine state differ from
those in the plugin registry file.
The following values of `status` are now produced instead of
`is_running`:
- `added`: The plugin is present in the plugin registry file, but not in
the engine.
- `loaded`: The plugin is present both in the plugin registry file and
in the engine, but is not running.
- `running`: The plugin is currently running, and the `pid` column
should contain its process ID.
- `modified`: The plugin state present in the plugin registry file is
different from the state in the engine.
- `removed`: The plugin is still loaded in the engine, but is not
present in the plugin registry file.
- `invalid`: The data in the plugin registry file couldn't be
deserialized, and the plugin most likely needs to be added again.
Example (`commands` omitted):
```
╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮
│ # │ name │ version │ status │ pid │ filename │ shell │
├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤
│ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │
│ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │
│ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │
│ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │
│ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │
│ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │
│ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │
│ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │
│ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │
│ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │
╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯
```
# User-Facing Changes
To `plugin list`:
* **Breaking:** The `is_running` column is removed and replaced with
`status`. Use `status == running` to filter equivalently.
* The `--plugin-config` from other plugin management commands is now
supported.
* Added an `--engine` flag which behaves more or less like before, and
doesn't load the plugin registry file at all.
* Added a `--registry` flag which only checks the plugin registry file.
All plugins appear as `added` since there is no state to compare with.
Because the default is to check both, the `plugin list` command might be
a little bit slower. If you don't need to check the plugin registry
file, the `--engine` flag does not load the plugin registry file at all,
so it should be just as fast as before.
# Tests + Formatting
Added tests for `added` and `removed` statuses. `modified` and `invalid`
are a bit more tricky so I didn't try.
# After Submitting
- [ ] update documentation that references the `plugin list` command
- [ ] release notes
2024-10-17 02:24:45 +00:00
|
|
|
(plugin list).0.status == running | print
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
print ";"
|
|
|
|
"2.0.0" | inc -m | ignore
|
Make `plugin list` read state from plugin registry file as well (#14085)
# Description
[Context on
Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583)
**This is a breaking change, due to the removal of `is_running`.**
Some users find the `plugin list` command confusing, because it doesn't
show anything different after running `plugin add` or `plugin rm`. This
modifies the `plugin list` command to also look at the plugin registry
file to give some idea of how the plugins in engine state differ from
those in the plugin registry file.
The following values of `status` are now produced instead of
`is_running`:
- `added`: The plugin is present in the plugin registry file, but not in
the engine.
- `loaded`: The plugin is present both in the plugin registry file and
in the engine, but is not running.
- `running`: The plugin is currently running, and the `pid` column
should contain its process ID.
- `modified`: The plugin state present in the plugin registry file is
different from the state in the engine.
- `removed`: The plugin is still loaded in the engine, but is not
present in the plugin registry file.
- `invalid`: The data in the plugin registry file couldn't be
deserialized, and the plugin most likely needs to be added again.
Example (`commands` omitted):
```
╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮
│ # │ name │ version │ status │ pid │ filename │ shell │
├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤
│ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │
│ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │
│ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │
│ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │
│ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │
│ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │
│ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │
│ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │
│ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │
│ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │
╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯
```
# User-Facing Changes
To `plugin list`:
* **Breaking:** The `is_running` column is removed and replaced with
`status`. Use `status == running` to filter equivalently.
* The `--plugin-config` from other plugin management commands is now
supported.
* Added an `--engine` flag which behaves more or less like before, and
doesn't load the plugin registry file at all.
* Added a `--registry` flag which only checks the plugin registry file.
All plugins appear as `added` since there is no state to compare with.
Because the default is to check both, the `plugin list` command might be
a little bit slower. If you don't need to check the plugin registry
file, the `--engine` flag does not load the plugin registry file at all,
so it should be just as fast as before.
# Tests + Formatting
Added tests for `added` and `removed` statuses. `modified` and `invalid`
are a bit more tricky so I didn't try.
# After Submitting
- [ ] update documentation that references the `plugin list` command
- [ ] release notes
2024-10-17 02:24:45 +00:00
|
|
|
(plugin list).0.status == running | print
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
"#
|
|
|
|
);
|
|
|
|
assert_eq!(
|
|
|
|
"false;true", out.out,
|
Make `plugin list` read state from plugin registry file as well (#14085)
# Description
[Context on
Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583)
**This is a breaking change, due to the removal of `is_running`.**
Some users find the `plugin list` command confusing, because it doesn't
show anything different after running `plugin add` or `plugin rm`. This
modifies the `plugin list` command to also look at the plugin registry
file to give some idea of how the plugins in engine state differ from
those in the plugin registry file.
The following values of `status` are now produced instead of
`is_running`:
- `added`: The plugin is present in the plugin registry file, but not in
the engine.
- `loaded`: The plugin is present both in the plugin registry file and
in the engine, but is not running.
- `running`: The plugin is currently running, and the `pid` column
should contain its process ID.
- `modified`: The plugin state present in the plugin registry file is
different from the state in the engine.
- `removed`: The plugin is still loaded in the engine, but is not
present in the plugin registry file.
- `invalid`: The data in the plugin registry file couldn't be
deserialized, and the plugin most likely needs to be added again.
Example (`commands` omitted):
```
╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮
│ # │ name │ version │ status │ pid │ filename │ shell │
├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤
│ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │
│ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │
│ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │
│ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │
│ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │
│ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │
│ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │
│ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │
│ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │
│ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │
╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯
```
# User-Facing Changes
To `plugin list`:
* **Breaking:** The `is_running` column is removed and replaced with
`status`. Use `status == running` to filter equivalently.
* The `--plugin-config` from other plugin management commands is now
supported.
* Added an `--engine` flag which behaves more or less like before, and
doesn't load the plugin registry file at all.
* Added a `--registry` flag which only checks the plugin registry file.
All plugins appear as `added` since there is no state to compare with.
Because the default is to check both, the `plugin list` command might be
a little bit slower. If you don't need to check the plugin registry
file, the `--engine` flag does not load the plugin registry file at all,
so it should be just as fast as before.
# Tests + Formatting
Added tests for `added` and `removed` statuses. `modified` and `invalid`
are a bit more tricky so I didn't try.
# After Submitting
- [ ] update documentation that references the `plugin list` command
- [ ] release notes
2024-10-17 02:24:45 +00:00
|
|
|
"plugin list didn't show status = running"
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
);
|
|
|
|
assert!(out.status.success());
|
|
|
|
}
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
fn plugin_process_exits_after_stop() {
|
|
|
|
let out = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
|
|
|
plugin: ("nu_plugin_inc"),
|
|
|
|
r#"
|
|
|
|
"2.0.0" | inc -m | ignore
|
2024-04-26 11:24:57 +00:00
|
|
|
sleep 500ms
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
let pid = (plugin list).0.pid
|
2024-03-13 11:43:17 +00:00
|
|
|
if (ps | where pid == $pid | is-empty) {
|
|
|
|
error make {
|
|
|
|
msg: "plugin process not running initially"
|
|
|
|
}
|
|
|
|
}
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
plugin stop inc
|
2024-03-13 11:43:17 +00:00
|
|
|
let start = (date now)
|
|
|
|
mut cond = true
|
|
|
|
while $cond {
|
|
|
|
sleep 100ms
|
|
|
|
$cond = (
|
|
|
|
(ps | where pid == $pid | is-not-empty) and
|
|
|
|
((date now) - $start) < 5sec
|
|
|
|
)
|
|
|
|
}
|
|
|
|
((date now) - $start) | into int
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
"#
|
|
|
|
);
|
2024-03-13 11:43:17 +00:00
|
|
|
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
assert!(out.status.success());
|
2024-03-13 11:43:17 +00:00
|
|
|
|
|
|
|
let nanos = out.out.parse::<i64>().expect("not a number");
|
|
|
|
assert!(
|
|
|
|
nanos < 5_000_000_000,
|
|
|
|
"not stopped after more than 5 seconds: {nanos} ns"
|
|
|
|
);
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
}
|
|
|
|
|
2024-04-24 11:28:45 +00:00
|
|
|
#[test]
|
|
|
|
fn plugin_stop_can_find_by_filename() {
|
|
|
|
let result = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
|
|
|
plugin: ("nu_plugin_inc"),
|
|
|
|
r#"plugin stop (plugin list | where name == inc).0.filename"#
|
|
|
|
);
|
|
|
|
assert!(result.status.success());
|
|
|
|
assert!(result.err.is_empty());
|
|
|
|
}
|
|
|
|
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
#[test]
|
|
|
|
fn plugin_process_exits_when_nushell_exits() {
|
|
|
|
let out = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
|
|
|
plugin: ("nu_plugin_inc"),
|
|
|
|
r#"
|
|
|
|
"2.0.0" | inc -m | ignore
|
|
|
|
(plugin list).0.pid | print
|
|
|
|
"#
|
|
|
|
);
|
|
|
|
assert!(!out.out.is_empty());
|
|
|
|
assert!(out.status.success());
|
|
|
|
|
|
|
|
let pid = out.out.parse::<u32>().expect("failed to parse pid");
|
|
|
|
|
|
|
|
// use nu to check if process exists
|
|
|
|
assert_eq!(
|
|
|
|
"0",
|
2024-04-26 11:24:57 +00:00
|
|
|
nu!(format!("sleep 500ms; ps | where pid == {pid} | length")).out,
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
"plugin process {pid} is still running"
|
|
|
|
);
|
|
|
|
}
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
fn plugin_commands_run_without_error() {
|
|
|
|
let out = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
|
|
|
plugins: [
|
|
|
|
("nu_plugin_inc"),
|
2024-03-19 17:36:46 +00:00
|
|
|
("nu_plugin_example"),
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
("nu_plugin_custom_values"),
|
|
|
|
],
|
|
|
|
r#"
|
|
|
|
"2.0.0" | inc -m | ignore
|
2024-03-19 17:36:46 +00:00
|
|
|
example seq 1 10 | ignore
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
custom-value generate | ignore
|
|
|
|
"#
|
|
|
|
);
|
|
|
|
assert!(out.err.is_empty());
|
|
|
|
assert!(out.status.success());
|
|
|
|
}
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
fn plugin_commands_run_multiple_times_without_error() {
|
|
|
|
let out = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
|
|
|
plugins: [
|
|
|
|
("nu_plugin_inc"),
|
2024-03-19 17:36:46 +00:00
|
|
|
("nu_plugin_example"),
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
("nu_plugin_custom_values"),
|
|
|
|
],
|
|
|
|
r#"
|
|
|
|
["2.0.0" "2.1.0" "2.2.0"] | each { inc -m } | print
|
2024-03-19 17:36:46 +00:00
|
|
|
example seq 1 10 | ignore
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
custom-value generate | ignore
|
2024-03-19 17:36:46 +00:00
|
|
|
example seq 1 20 | ignore
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
custom-value generate2 | ignore
|
|
|
|
"#
|
|
|
|
);
|
|
|
|
assert!(out.err.is_empty());
|
|
|
|
assert!(out.status.success());
|
|
|
|
}
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
fn multiple_plugin_commands_run_with_the_same_plugin_pid() {
|
|
|
|
let out = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
|
|
|
plugin: ("nu_plugin_custom_values"),
|
|
|
|
r#"
|
|
|
|
custom-value generate | ignore
|
|
|
|
(plugin list).0.pid | print
|
|
|
|
print ";"
|
|
|
|
custom-value generate2 | ignore
|
|
|
|
(plugin list).0.pid | print
|
|
|
|
"#
|
|
|
|
);
|
|
|
|
assert!(out.status.success());
|
|
|
|
|
|
|
|
let pids: Vec<&str> = out.out.split(';').collect();
|
|
|
|
assert_eq!(2, pids.len());
|
|
|
|
assert_eq!(pids[0], pids[1]);
|
|
|
|
}
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
fn plugin_pid_changes_after_stop_then_run_again() {
|
|
|
|
let out = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
|
|
|
plugin: ("nu_plugin_custom_values"),
|
|
|
|
r#"
|
|
|
|
custom-value generate | ignore
|
|
|
|
(plugin list).0.pid | print
|
|
|
|
print ";"
|
|
|
|
plugin stop custom_values
|
|
|
|
custom-value generate2 | ignore
|
|
|
|
(plugin list).0.pid | print
|
|
|
|
"#
|
|
|
|
);
|
|
|
|
assert!(out.status.success());
|
|
|
|
|
|
|
|
let pids: Vec<&str> = out.out.split(';').collect();
|
|
|
|
assert_eq!(2, pids.len());
|
|
|
|
assert_ne!(pids[0], pids[1]);
|
|
|
|
}
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
fn custom_values_can_still_be_passed_to_plugin_after_stop() {
|
|
|
|
let out = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
|
|
|
plugin: ("nu_plugin_custom_values"),
|
|
|
|
r#"
|
|
|
|
let cv = custom-value generate
|
|
|
|
plugin stop custom_values
|
|
|
|
$cv | custom-value update
|
|
|
|
"#
|
|
|
|
);
|
|
|
|
assert!(!out.out.is_empty());
|
|
|
|
assert!(out.err.is_empty());
|
|
|
|
assert!(out.status.success());
|
|
|
|
}
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
fn custom_values_can_still_be_collapsed_after_stop() {
|
|
|
|
// print causes a collapse (ToBaseValue) call.
|
|
|
|
let out = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
|
|
|
plugin: ("nu_plugin_custom_values"),
|
|
|
|
r#"
|
|
|
|
let cv = custom-value generate
|
|
|
|
plugin stop custom_values
|
|
|
|
$cv | print
|
|
|
|
"#
|
|
|
|
);
|
|
|
|
assert!(!out.out.is_empty());
|
|
|
|
assert!(out.err.is_empty());
|
|
|
|
assert!(out.status.success());
|
|
|
|
}
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
fn plugin_gc_can_be_configured_to_stop_plugins_immediately() {
|
|
|
|
// I know the test is to stop "immediately", but if we actually check immediately it could
|
2024-03-13 11:43:17 +00:00
|
|
|
// lead to a race condition. Using 100ms sleep just because with contention we don't really
|
|
|
|
// know for sure how long this could take
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
let out = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
|
|
|
plugin: ("nu_plugin_inc"),
|
|
|
|
r#"
|
|
|
|
$env.config.plugin_gc = { default: { stop_after: 0sec } }
|
|
|
|
"2.3.0" | inc -M
|
2024-03-13 11:43:17 +00:00
|
|
|
sleep 100ms
|
Make `plugin list` read state from plugin registry file as well (#14085)
# Description
[Context on
Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583)
**This is a breaking change, due to the removal of `is_running`.**
Some users find the `plugin list` command confusing, because it doesn't
show anything different after running `plugin add` or `plugin rm`. This
modifies the `plugin list` command to also look at the plugin registry
file to give some idea of how the plugins in engine state differ from
those in the plugin registry file.
The following values of `status` are now produced instead of
`is_running`:
- `added`: The plugin is present in the plugin registry file, but not in
the engine.
- `loaded`: The plugin is present both in the plugin registry file and
in the engine, but is not running.
- `running`: The plugin is currently running, and the `pid` column
should contain its process ID.
- `modified`: The plugin state present in the plugin registry file is
different from the state in the engine.
- `removed`: The plugin is still loaded in the engine, but is not
present in the plugin registry file.
- `invalid`: The data in the plugin registry file couldn't be
deserialized, and the plugin most likely needs to be added again.
Example (`commands` omitted):
```
╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮
│ # │ name │ version │ status │ pid │ filename │ shell │
├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤
│ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │
│ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │
│ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │
│ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │
│ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │
│ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │
│ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │
│ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │
│ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │
│ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │
╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯
```
# User-Facing Changes
To `plugin list`:
* **Breaking:** The `is_running` column is removed and replaced with
`status`. Use `status == running` to filter equivalently.
* The `--plugin-config` from other plugin management commands is now
supported.
* Added an `--engine` flag which behaves more or less like before, and
doesn't load the plugin registry file at all.
* Added a `--registry` flag which only checks the plugin registry file.
All plugins appear as `added` since there is no state to compare with.
Because the default is to check both, the `plugin list` command might be
a little bit slower. If you don't need to check the plugin registry
file, the `--engine` flag does not load the plugin registry file at all,
so it should be just as fast as before.
# Tests + Formatting
Added tests for `added` and `removed` statuses. `modified` and `invalid`
are a bit more tricky so I didn't try.
# After Submitting
- [ ] update documentation that references the `plugin list` command
- [ ] release notes
2024-10-17 02:24:45 +00:00
|
|
|
(plugin list | where name == inc).0.status == running
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
"#
|
|
|
|
);
|
|
|
|
assert!(out.status.success());
|
|
|
|
assert_eq!("false", out.out, "with config as default");
|
|
|
|
|
|
|
|
let out = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
|
|
|
plugin: ("nu_plugin_inc"),
|
|
|
|
r#"
|
|
|
|
$env.config.plugin_gc = {
|
|
|
|
plugins: {
|
|
|
|
inc: { stop_after: 0sec }
|
|
|
|
}
|
|
|
|
}
|
|
|
|
"2.3.0" | inc -M
|
2024-03-13 11:43:17 +00:00
|
|
|
sleep 100ms
|
Make `plugin list` read state from plugin registry file as well (#14085)
# Description
[Context on
Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583)
**This is a breaking change, due to the removal of `is_running`.**
Some users find the `plugin list` command confusing, because it doesn't
show anything different after running `plugin add` or `plugin rm`. This
modifies the `plugin list` command to also look at the plugin registry
file to give some idea of how the plugins in engine state differ from
those in the plugin registry file.
The following values of `status` are now produced instead of
`is_running`:
- `added`: The plugin is present in the plugin registry file, but not in
the engine.
- `loaded`: The plugin is present both in the plugin registry file and
in the engine, but is not running.
- `running`: The plugin is currently running, and the `pid` column
should contain its process ID.
- `modified`: The plugin state present in the plugin registry file is
different from the state in the engine.
- `removed`: The plugin is still loaded in the engine, but is not
present in the plugin registry file.
- `invalid`: The data in the plugin registry file couldn't be
deserialized, and the plugin most likely needs to be added again.
Example (`commands` omitted):
```
╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮
│ # │ name │ version │ status │ pid │ filename │ shell │
├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤
│ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │
│ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │
│ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │
│ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │
│ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │
│ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │
│ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │
│ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │
│ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │
│ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │
╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯
```
# User-Facing Changes
To `plugin list`:
* **Breaking:** The `is_running` column is removed and replaced with
`status`. Use `status == running` to filter equivalently.
* The `--plugin-config` from other plugin management commands is now
supported.
* Added an `--engine` flag which behaves more or less like before, and
doesn't load the plugin registry file at all.
* Added a `--registry` flag which only checks the plugin registry file.
All plugins appear as `added` since there is no state to compare with.
Because the default is to check both, the `plugin list` command might be
a little bit slower. If you don't need to check the plugin registry
file, the `--engine` flag does not load the plugin registry file at all,
so it should be just as fast as before.
# Tests + Formatting
Added tests for `added` and `removed` statuses. `modified` and `invalid`
are a bit more tricky so I didn't try.
# After Submitting
- [ ] update documentation that references the `plugin list` command
- [ ] release notes
2024-10-17 02:24:45 +00:00
|
|
|
(plugin list | where name == inc).0.status == running
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
"#
|
|
|
|
);
|
|
|
|
assert!(out.status.success());
|
|
|
|
assert_eq!("false", out.out, "with inc-specific config");
|
|
|
|
}
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
fn plugin_gc_can_be_configured_to_stop_plugins_after_delay() {
|
|
|
|
let out = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
|
|
|
plugin: ("nu_plugin_inc"),
|
|
|
|
r#"
|
|
|
|
$env.config.plugin_gc = { default: { stop_after: 50ms } }
|
|
|
|
"2.3.0" | inc -M
|
2024-03-11 11:01:48 +00:00
|
|
|
let start = (date now)
|
|
|
|
mut cond = true
|
|
|
|
while $cond {
|
|
|
|
sleep 100ms
|
|
|
|
$cond = (
|
Make `plugin list` read state from plugin registry file as well (#14085)
# Description
[Context on
Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583)
**This is a breaking change, due to the removal of `is_running`.**
Some users find the `plugin list` command confusing, because it doesn't
show anything different after running `plugin add` or `plugin rm`. This
modifies the `plugin list` command to also look at the plugin registry
file to give some idea of how the plugins in engine state differ from
those in the plugin registry file.
The following values of `status` are now produced instead of
`is_running`:
- `added`: The plugin is present in the plugin registry file, but not in
the engine.
- `loaded`: The plugin is present both in the plugin registry file and
in the engine, but is not running.
- `running`: The plugin is currently running, and the `pid` column
should contain its process ID.
- `modified`: The plugin state present in the plugin registry file is
different from the state in the engine.
- `removed`: The plugin is still loaded in the engine, but is not
present in the plugin registry file.
- `invalid`: The data in the plugin registry file couldn't be
deserialized, and the plugin most likely needs to be added again.
Example (`commands` omitted):
```
╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮
│ # │ name │ version │ status │ pid │ filename │ shell │
├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤
│ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │
│ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │
│ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │
│ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │
│ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │
│ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │
│ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │
│ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │
│ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │
│ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │
╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯
```
# User-Facing Changes
To `plugin list`:
* **Breaking:** The `is_running` column is removed and replaced with
`status`. Use `status == running` to filter equivalently.
* The `--plugin-config` from other plugin management commands is now
supported.
* Added an `--engine` flag which behaves more or less like before, and
doesn't load the plugin registry file at all.
* Added a `--registry` flag which only checks the plugin registry file.
All plugins appear as `added` since there is no state to compare with.
Because the default is to check both, the `plugin list` command might be
a little bit slower. If you don't need to check the plugin registry
file, the `--engine` flag does not load the plugin registry file at all,
so it should be just as fast as before.
# Tests + Formatting
Added tests for `added` and `removed` statuses. `modified` and `invalid`
are a bit more tricky so I didn't try.
# After Submitting
- [ ] update documentation that references the `plugin list` command
- [ ] release notes
2024-10-17 02:24:45 +00:00
|
|
|
(plugin list | where name == inc).0.status == running and
|
2024-03-11 11:01:48 +00:00
|
|
|
((date now) - $start) < 5sec
|
|
|
|
)
|
|
|
|
}
|
|
|
|
((date now) - $start) | into int
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
"#
|
|
|
|
);
|
|
|
|
assert!(out.status.success());
|
2024-03-11 11:01:48 +00:00
|
|
|
let nanos = out.out.parse::<i64>().expect("not a number");
|
|
|
|
assert!(
|
|
|
|
nanos < 5_000_000_000,
|
|
|
|
"with config as default: more than 5 seconds: {nanos} ns"
|
|
|
|
);
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
|
|
|
|
let out = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
|
|
|
plugin: ("nu_plugin_inc"),
|
|
|
|
r#"
|
|
|
|
$env.config.plugin_gc = {
|
|
|
|
plugins: {
|
|
|
|
inc: { stop_after: 50ms }
|
|
|
|
}
|
|
|
|
}
|
|
|
|
"2.3.0" | inc -M
|
2024-03-11 11:01:48 +00:00
|
|
|
let start = (date now)
|
|
|
|
mut cond = true
|
|
|
|
while $cond {
|
|
|
|
sleep 100ms
|
|
|
|
$cond = (
|
Make `plugin list` read state from plugin registry file as well (#14085)
# Description
[Context on
Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583)
**This is a breaking change, due to the removal of `is_running`.**
Some users find the `plugin list` command confusing, because it doesn't
show anything different after running `plugin add` or `plugin rm`. This
modifies the `plugin list` command to also look at the plugin registry
file to give some idea of how the plugins in engine state differ from
those in the plugin registry file.
The following values of `status` are now produced instead of
`is_running`:
- `added`: The plugin is present in the plugin registry file, but not in
the engine.
- `loaded`: The plugin is present both in the plugin registry file and
in the engine, but is not running.
- `running`: The plugin is currently running, and the `pid` column
should contain its process ID.
- `modified`: The plugin state present in the plugin registry file is
different from the state in the engine.
- `removed`: The plugin is still loaded in the engine, but is not
present in the plugin registry file.
- `invalid`: The data in the plugin registry file couldn't be
deserialized, and the plugin most likely needs to be added again.
Example (`commands` omitted):
```
╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮
│ # │ name │ version │ status │ pid │ filename │ shell │
├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤
│ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │
│ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │
│ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │
│ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │
│ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │
│ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │
│ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │
│ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │
│ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │
│ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │
╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯
```
# User-Facing Changes
To `plugin list`:
* **Breaking:** The `is_running` column is removed and replaced with
`status`. Use `status == running` to filter equivalently.
* The `--plugin-config` from other plugin management commands is now
supported.
* Added an `--engine` flag which behaves more or less like before, and
doesn't load the plugin registry file at all.
* Added a `--registry` flag which only checks the plugin registry file.
All plugins appear as `added` since there is no state to compare with.
Because the default is to check both, the `plugin list` command might be
a little bit slower. If you don't need to check the plugin registry
file, the `--engine` flag does not load the plugin registry file at all,
so it should be just as fast as before.
# Tests + Formatting
Added tests for `added` and `removed` statuses. `modified` and `invalid`
are a bit more tricky so I didn't try.
# After Submitting
- [ ] update documentation that references the `plugin list` command
- [ ] release notes
2024-10-17 02:24:45 +00:00
|
|
|
(plugin list | where name == inc).0.status == running and
|
2024-03-11 11:01:48 +00:00
|
|
|
((date now) - $start) < 5sec
|
|
|
|
)
|
|
|
|
}
|
|
|
|
((date now) - $start) | into int
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
"#
|
|
|
|
);
|
|
|
|
assert!(out.status.success());
|
2024-03-11 11:01:48 +00:00
|
|
|
let nanos = out.out.parse::<i64>().expect("not a number");
|
|
|
|
assert!(
|
|
|
|
nanos < 5_000_000_000,
|
|
|
|
"with inc-specific config: more than 5 seconds: {nanos} ns"
|
|
|
|
);
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
fn plugin_gc_can_be_configured_as_disabled() {
|
|
|
|
let out = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
|
|
|
plugin: ("nu_plugin_inc"),
|
|
|
|
r#"
|
|
|
|
$env.config.plugin_gc = { default: { enabled: false, stop_after: 0sec } }
|
|
|
|
"2.3.0" | inc -M
|
Make `plugin list` read state from plugin registry file as well (#14085)
# Description
[Context on
Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583)
**This is a breaking change, due to the removal of `is_running`.**
Some users find the `plugin list` command confusing, because it doesn't
show anything different after running `plugin add` or `plugin rm`. This
modifies the `plugin list` command to also look at the plugin registry
file to give some idea of how the plugins in engine state differ from
those in the plugin registry file.
The following values of `status` are now produced instead of
`is_running`:
- `added`: The plugin is present in the plugin registry file, but not in
the engine.
- `loaded`: The plugin is present both in the plugin registry file and
in the engine, but is not running.
- `running`: The plugin is currently running, and the `pid` column
should contain its process ID.
- `modified`: The plugin state present in the plugin registry file is
different from the state in the engine.
- `removed`: The plugin is still loaded in the engine, but is not
present in the plugin registry file.
- `invalid`: The data in the plugin registry file couldn't be
deserialized, and the plugin most likely needs to be added again.
Example (`commands` omitted):
```
╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮
│ # │ name │ version │ status │ pid │ filename │ shell │
├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤
│ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │
│ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │
│ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │
│ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │
│ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │
│ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │
│ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │
│ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │
│ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │
│ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │
╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯
```
# User-Facing Changes
To `plugin list`:
* **Breaking:** The `is_running` column is removed and replaced with
`status`. Use `status == running` to filter equivalently.
* The `--plugin-config` from other plugin management commands is now
supported.
* Added an `--engine` flag which behaves more or less like before, and
doesn't load the plugin registry file at all.
* Added a `--registry` flag which only checks the plugin registry file.
All plugins appear as `added` since there is no state to compare with.
Because the default is to check both, the `plugin list` command might be
a little bit slower. If you don't need to check the plugin registry
file, the `--engine` flag does not load the plugin registry file at all,
so it should be just as fast as before.
# Tests + Formatting
Added tests for `added` and `removed` statuses. `modified` and `invalid`
are a bit more tricky so I didn't try.
# After Submitting
- [ ] update documentation that references the `plugin list` command
- [ ] release notes
2024-10-17 02:24:45 +00:00
|
|
|
(plugin list | where name == inc).0.status == running
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
"#
|
|
|
|
);
|
|
|
|
assert!(out.status.success());
|
|
|
|
assert_eq!("true", out.out, "with config as default");
|
|
|
|
|
|
|
|
let out = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
|
|
|
plugin: ("nu_plugin_inc"),
|
|
|
|
r#"
|
|
|
|
$env.config.plugin_gc = {
|
|
|
|
default: { enabled: true, stop_after: 0sec }
|
|
|
|
plugins: {
|
|
|
|
inc: { enabled: false, stop_after: 0sec }
|
|
|
|
}
|
|
|
|
}
|
|
|
|
"2.3.0" | inc -M
|
Make `plugin list` read state from plugin registry file as well (#14085)
# Description
[Context on
Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583)
**This is a breaking change, due to the removal of `is_running`.**
Some users find the `plugin list` command confusing, because it doesn't
show anything different after running `plugin add` or `plugin rm`. This
modifies the `plugin list` command to also look at the plugin registry
file to give some idea of how the plugins in engine state differ from
those in the plugin registry file.
The following values of `status` are now produced instead of
`is_running`:
- `added`: The plugin is present in the plugin registry file, but not in
the engine.
- `loaded`: The plugin is present both in the plugin registry file and
in the engine, but is not running.
- `running`: The plugin is currently running, and the `pid` column
should contain its process ID.
- `modified`: The plugin state present in the plugin registry file is
different from the state in the engine.
- `removed`: The plugin is still loaded in the engine, but is not
present in the plugin registry file.
- `invalid`: The data in the plugin registry file couldn't be
deserialized, and the plugin most likely needs to be added again.
Example (`commands` omitted):
```
╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮
│ # │ name │ version │ status │ pid │ filename │ shell │
├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤
│ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │
│ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │
│ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │
│ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │
│ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │
│ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │
│ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │
│ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │
│ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │
│ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │
╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯
```
# User-Facing Changes
To `plugin list`:
* **Breaking:** The `is_running` column is removed and replaced with
`status`. Use `status == running` to filter equivalently.
* The `--plugin-config` from other plugin management commands is now
supported.
* Added an `--engine` flag which behaves more or less like before, and
doesn't load the plugin registry file at all.
* Added a `--registry` flag which only checks the plugin registry file.
All plugins appear as `added` since there is no state to compare with.
Because the default is to check both, the `plugin list` command might be
a little bit slower. If you don't need to check the plugin registry
file, the `--engine` flag does not load the plugin registry file at all,
so it should be just as fast as before.
# Tests + Formatting
Added tests for `added` and `removed` statuses. `modified` and `invalid`
are a bit more tricky so I didn't try.
# After Submitting
- [ ] update documentation that references the `plugin list` command
- [ ] release notes
2024-10-17 02:24:45 +00:00
|
|
|
(plugin list | where name == inc).0.status == running
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
"#
|
|
|
|
);
|
|
|
|
assert!(out.status.success());
|
|
|
|
assert_eq!("true", out.out, "with inc-specific config");
|
|
|
|
}
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
fn plugin_gc_can_be_disabled_by_plugin() {
|
|
|
|
let out = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
|
|
|
plugin: ("nu_plugin_example"),
|
|
|
|
r#"
|
2024-03-19 17:36:46 +00:00
|
|
|
example disable-gc
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
$env.config.plugin_gc = { default: { stop_after: 0sec } }
|
2024-03-19 17:36:46 +00:00
|
|
|
example one 1 foo | ignore # ensure we've run the plugin with the new config
|
2024-03-13 11:43:17 +00:00
|
|
|
sleep 100ms
|
Make `plugin list` read state from plugin registry file as well (#14085)
# Description
[Context on
Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583)
**This is a breaking change, due to the removal of `is_running`.**
Some users find the `plugin list` command confusing, because it doesn't
show anything different after running `plugin add` or `plugin rm`. This
modifies the `plugin list` command to also look at the plugin registry
file to give some idea of how the plugins in engine state differ from
those in the plugin registry file.
The following values of `status` are now produced instead of
`is_running`:
- `added`: The plugin is present in the plugin registry file, but not in
the engine.
- `loaded`: The plugin is present both in the plugin registry file and
in the engine, but is not running.
- `running`: The plugin is currently running, and the `pid` column
should contain its process ID.
- `modified`: The plugin state present in the plugin registry file is
different from the state in the engine.
- `removed`: The plugin is still loaded in the engine, but is not
present in the plugin registry file.
- `invalid`: The data in the plugin registry file couldn't be
deserialized, and the plugin most likely needs to be added again.
Example (`commands` omitted):
```
╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮
│ # │ name │ version │ status │ pid │ filename │ shell │
├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤
│ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │
│ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │
│ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │
│ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │
│ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │
│ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │
│ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │
│ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │
│ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │
│ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │
╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯
```
# User-Facing Changes
To `plugin list`:
* **Breaking:** The `is_running` column is removed and replaced with
`status`. Use `status == running` to filter equivalently.
* The `--plugin-config` from other plugin management commands is now
supported.
* Added an `--engine` flag which behaves more or less like before, and
doesn't load the plugin registry file at all.
* Added a `--registry` flag which only checks the plugin registry file.
All plugins appear as `added` since there is no state to compare with.
Because the default is to check both, the `plugin list` command might be
a little bit slower. If you don't need to check the plugin registry
file, the `--engine` flag does not load the plugin registry file at all,
so it should be just as fast as before.
# Tests + Formatting
Added tests for `added` and `removed` statuses. `modified` and `invalid`
are a bit more tricky so I didn't try.
# After Submitting
- [ ] update documentation that references the `plugin list` command
- [ ] release notes
2024-10-17 02:24:45 +00:00
|
|
|
(plugin list | where name == example).0.status == running
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
"#
|
|
|
|
);
|
|
|
|
assert!(out.status.success());
|
|
|
|
assert_eq!("true", out.out);
|
|
|
|
}
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
fn plugin_gc_does_not_stop_plugin_while_stream_output_is_active() {
|
|
|
|
let out = nu_with_plugins!(
|
|
|
|
cwd: ".",
|
2024-03-19 17:36:46 +00:00
|
|
|
plugin: ("nu_plugin_example"),
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
r#"
|
|
|
|
$env.config.plugin_gc = { default: { stop_after: 10ms } }
|
|
|
|
# This would exceed the configured time
|
2024-03-19 17:36:46 +00:00
|
|
|
example seq 1 500 | each { |n| sleep 1ms; $n } | length | print
|
Keep plugins persistently running in the background (#12064)
# Description
This PR uses the new plugin protocol to intelligently keep plugin
processes running in the background for further plugin calls.
Running plugins can be seen by running the new `plugin list` command,
and stopped by running the new `plugin stop` command.
This is an enhancement for the performance of plugins, as starting new
plugin processes has overhead, especially for plugins in languages that
take a significant amount of time on startup. It also enables plugins
that have persistent state between commands, making the migration of
features like dataframes and `stor` to plugins possible.
Plugins are automatically stopped by the new plugin garbage collector,
configurable with `$env.config.plugin_gc`:
```nushell
$env.config.plugin_gc = {
# Configuration for plugin garbage collection
default: {
enabled: true # true to enable stopping of inactive plugins
stop_after: 10sec # how long to wait after a plugin is inactive to stop it
}
plugins: {
# alternate configuration for specific plugins, by name, for example:
#
# gstat: {
# enabled: false
# }
}
}
```
If garbage collection is enabled, plugins will be stopped after
`stop_after` passes after they were last active. Plugins are counted as
inactive if they have no running plugin calls. Reading the stream from
the response of a plugin call is still considered to be activity, but if
a plugin holds on to a stream but the call ends without an active
streaming response, it is not counted as active even if it is reading
it. Plugins can explicitly disable the GC as appropriate with
`engine.set_gc_disabled(true)`.
The `version` command now lists plugin names rather than plugin
commands. The list of plugin commands is accessible via `plugin list`.
Recommend doing this together with #12029, because it will likely force
plugin developers to do the right thing with mutability and lead to less
unexpected behavior when running plugins nested / in parallel.
# User-Facing Changes
- new command: `plugin list`
- new command: `plugin stop`
- changed command: `version` (now lists plugin names, rather than
commands)
- new config: `$env.config.plugin_gc`
- Plugins will keep running and be reused, at least for the configured
GC period
- Plugins that used mutable state in weird ways like `inc` did might
misbehave until fixed
- Plugins can disable GC if they need to
- Had to change plugin signature to accept `&EngineInterface` so that
the GC disable feature works. #12029 does this anyway, and I'm expecting
(resolvable) conflicts with that
# Tests + Formatting
- :green_circle: `toolkit fmt`
- :green_circle: `toolkit clippy`
- :green_circle: `toolkit test`
- :green_circle: `toolkit test stdlib`
Because there is some specific OS behavior required for plugins to not
respond to Ctrl-C directly, I've developed against and tested on both
Linux and Windows to ensure that works properly.
# After Submitting
I think this probably needs to be in the book somewhere
2024-03-09 23:10:22 +00:00
|
|
|
"#
|
|
|
|
);
|
|
|
|
assert!(out.status.success());
|
|
|
|
assert_eq!("500", out.out);
|
|
|
|
}
|