nushell/crates
Devyn Cairns cf321ab510
Make EngineState clone cheaper with Arc on all of the heavy objects (#12229)
# Description
This makes many of the larger objects in `EngineState` into `Arc`, and
uses `Arc::make_mut` to do clone-on-write if the reference is not
unique. This is generally very cheap, giving us the best of both worlds
- allowing us to mutate without cloning if we have an exclusive
reference, and cloning if we don't.

This started as more of a curiosity for me after remembering that
`Arc::make_mut` exists and can make using `Arc` for mostly immutable
data that sometimes needs to be changed very convenient, and also after
hearing someone complain about memory usage on Discord - this is a
somewhat significant win for that.

The exact objects that were wrapped in `Arc`:

- `files`, `file_contents` - the strings and byte buffers
- `decls` - the whole `Vec`, but mostly to avoid lots of individual
`malloc()` calls on Clone rather than for memory usage
- `blocks` - the blocks themselves, rather than the outer Vec
- `modules` - the modules themselves, rather than the outer Vec
- `env_vars`, `previous_env_vars` - the entire maps
- `config`

The changes required were relatively minimal, but this is a breaking API
change. In particular, blocks are added as Arcs, to allow the parser
cache functionality to work.

With my normal nu config, running on Linux, this saves me about 15 MiB
of process memory usage when running interactively (65 MiB → 50 MiB).

This also makes quick command executions cheaper, particularly since
every REPL loop now involves a clone of the engine state so that we can
recover from a panic. It also reduces memory usage where engine state
needs to be cloned and sent to another thread or kept within an
iterator.

# User-Facing Changes
Shouldn't be any, since it's all internal stuff, but it does change some
public interfaces so it's a breaking change
2024-03-19 19:07:00 +01:00
..
nu-cli Make EngineState clone cheaper with Arc on all of the heavy objects (#12229) 2024-03-19 19:07:00 +01:00
nu-cmd-base Make EngineState clone cheaper with Arc on all of the heavy objects (#12229) 2024-03-19 19:07:00 +01:00
nu-cmd-dataframe Make custom value type handling more consistent (#12230) 2024-03-19 11:09:59 +01:00
nu-cmd-extra Make EngineState clone cheaper with Arc on all of the heavy objects (#12229) 2024-03-19 19:07:00 +01:00
nu-cmd-lang Make EngineState clone cheaper with Arc on all of the heavy objects (#12229) 2024-03-19 19:07:00 +01:00
nu-color-config IO and redirection overhaul (#11934) 2024-03-14 15:51:55 -05:00
nu-command Make EngineState clone cheaper with Arc on all of the heavy objects (#12229) 2024-03-19 19:07:00 +01:00
nu-engine Make EngineState clone cheaper with Arc on all of the heavy objects (#12229) 2024-03-19 19:07:00 +01:00
nu-explore Make EngineState clone cheaper with Arc on all of the heavy objects (#12229) 2024-03-19 19:07:00 +01:00
nu-glob Fix ignored clippy lints (#12160) 2024-03-11 19:46:04 +01:00
nu-json remove repetitive word (#12117) 2024-03-08 15:29:20 +08:00
nu-lsp Make EngineState clone cheaper with Arc on all of the heavy objects (#12229) 2024-03-19 19:07:00 +01:00
nu-parser Make EngineState clone cheaper with Arc on all of the heavy objects (#12229) 2024-03-19 19:07:00 +01:00
nu-path Use XDG_CONFIG_HOME before default config directory (#12118) 2024-03-11 06:15:46 -05:00
nu-plugin Fix broken build: replace value_string() straggler (#12237) 2024-03-19 07:52:49 -05:00
nu-pretty-hex Introduce workspace dependencies (#12043) 2024-03-07 14:40:31 -08:00
nu-protocol Make EngineState clone cheaper with Arc on all of the heavy objects (#12229) 2024-03-19 19:07:00 +01:00
nu-std IO and redirection overhaul (#11934) 2024-03-14 15:51:55 -05:00
nu-system Bump windows from 0.52.0 to 0.54.0 (#12037) 2024-03-07 16:36:28 -08:00
nu-table Introduce workspace dependencies (#12043) 2024-03-07 14:40:31 -08:00
nu-term-grid Bump version to 0.91.1 (#12085) 2024-03-06 23:08:14 +01:00
nu-test-support Update tests Playground (#12134) 2024-03-08 20:31:21 -08:00
nu-utils Use the system clipboard only for explicit copy/paste operations. Addresses issue 11907 (#12179) 2024-03-15 08:59:21 -05:00
nu_plugin_custom_values Make custom value type handling more consistent (#12230) 2024-03-19 11:09:59 +01:00
nu_plugin_example Merge stream_example into example plugin and clean up names (#12234) 2024-03-19 12:36:46 -05:00
nu_plugin_formats Reorganize plugin API around commands (#12170) 2024-03-14 16:40:02 -05:00
nu_plugin_gstat Reorganize plugin API around commands (#12170) 2024-03-14 16:40:02 -05:00
nu_plugin_inc Reorganize plugin API around commands (#12170) 2024-03-14 16:40:02 -05:00
nu_plugin_python Improve the error message for a plugin version mismatch (#12122) 2024-03-08 06:04:22 -06:00
nu_plugin_query Reorganize plugin API around commands (#12170) 2024-03-14 16:40:02 -05:00
README.md Remove old nushell/merge engine-q 2022-02-07 14:54:06 -05:00

Nushell core libraries and plugins

These sub-crates form both the foundation for Nu and a set of plugins which extend Nu with additional functionality.

Foundational libraries are split into two kinds of crates:

  • Core crates - those crates that work together to build the Nushell language engine
  • Support crates - a set of crates that support the engine with additional features like JSON support, ANSI support, and more.

Plugins are likewise also split into two types:

  • Core plugins - plugins that provide part of the default experience of Nu, including access to the system properties, processes, and web-connectivity features.
  • Extra plugins - these plugins run a wide range of different capabilities like working with different file types, charting, viewing binary data, and more.