nushell/crates/nu-cmd-extra
Devyn Cairns cf321ab510
Make EngineState clone cheaper with Arc on all of the heavy objects (#12229)
# Description
This makes many of the larger objects in `EngineState` into `Arc`, and
uses `Arc::make_mut` to do clone-on-write if the reference is not
unique. This is generally very cheap, giving us the best of both worlds
- allowing us to mutate without cloning if we have an exclusive
reference, and cloning if we don't.

This started as more of a curiosity for me after remembering that
`Arc::make_mut` exists and can make using `Arc` for mostly immutable
data that sometimes needs to be changed very convenient, and also after
hearing someone complain about memory usage on Discord - this is a
somewhat significant win for that.

The exact objects that were wrapped in `Arc`:

- `files`, `file_contents` - the strings and byte buffers
- `decls` - the whole `Vec`, but mostly to avoid lots of individual
`malloc()` calls on Clone rather than for memory usage
- `blocks` - the blocks themselves, rather than the outer Vec
- `modules` - the modules themselves, rather than the outer Vec
- `env_vars`, `previous_env_vars` - the entire maps
- `config`

The changes required were relatively minimal, but this is a breaking API
change. In particular, blocks are added as Arcs, to allow the parser
cache functionality to work.

With my normal nu config, running on Linux, this saves me about 15 MiB
of process memory usage when running interactively (65 MiB → 50 MiB).

This also makes quick command executions cheaper, particularly since
every REPL loop now involves a clone of the engine state so that we can
recover from a panic. It also reduces memory usage where engine state
needs to be cloned and sent to another thread or kept within an
iterator.

# User-Facing Changes
Shouldn't be any, since it's all internal stuff, but it does change some
public interfaces so it's a breaking change
2024-03-19 19:07:00 +01:00
..
assets REFACTOR: move the 0% commands to nu-cmd-extra (#9404) 2023-07-06 08:31:31 -07:00
src Make EngineState clone cheaper with Arc on all of the heavy objects (#12229) 2024-03-19 19:07:00 +01:00
tests Remove feat extra and include in default (#12140) 2024-03-10 17:29:02 +01:00
Cargo.toml Introduce workspace dependencies (#12043) 2024-03-07 14:40:31 -08:00
LICENSE nu-cmd-extra crate infrastructure in place with the Bits command as the model for adding other commands (#9327) 2023-06-01 10:46:16 -07:00
README.md Remove feat extra and include in default (#12140) 2024-03-10 17:29:02 +01:00

nu-cmd-extra

The commands in this crate are the extra commands of Nushell. These commands are not in a state to be guaranteed to be part of the 1.0 API; meaning that there is no guarantee longer term that these commands will be around into the future.

For a while we did exclude them behind the --features extra compile time flag, meaning that the default release did not contain them. As we (the Nushell team) shipped a full build including both extra and dataframe for some time, we chose to sunset the extra feature but keep the commands in this crate for now. In the future the commands may be moved to more topical crates or discarded into plugins.