nushell/src/main.rs

459 lines
15 KiB
Rust
Raw Normal View History

mod command;
2022-01-18 08:48:28 +00:00
mod config_files;
Add IDE support (#8745) # Description This adds a set of new flags on the `nu` binary intended for use in IDEs. Here is the set of supported functionality so far: * goto-def - go to the definition of a variable or custom command * type hints - see the inferred type of variables * check - see the errors in the document (currently only one error is supported) * hover - get information about the variable or custom command * complete - get a completion list at the current position # User-Facing Changes No changes to the REPL experience. This only impacts the IDE scenario. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-utils/standard_library/tests.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --------- Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2023-04-05 19:34:47 +00:00
mod ide;
2022-01-18 08:48:28 +00:00
mod logger;
mod run;
mod signals;
#[cfg(unix)]
mod terminal;
mod test_bins;
2021-08-10 18:57:08 +00:00
Changes global allocator to mimalloc, improving performance. (#9415) # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> this PR makes nushell use mimalloc as the default allocator, this has the benefit of reducing startup time on my machine. `17%` on linux and `22%` on windows, when testing using hyperfine. the overhead to compile seem to be quite small, aswell as the increase of binary size quite small on linux the binary went from `33.1mb` to `33.2mb` linux ![image](https://github.com/nushell/nushell/assets/17986183/ba5379b4-2c08-483a-a9ff-a9d8524d2943) windows ![image](https://github.com/nushell/nushell/assets/17986183/fda5090f-96a9-48d1-ada4-617694b9d880) # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect -A clippy::result_large_err` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2023-06-14 22:27:12 +00:00
#[cfg(feature = "mimalloc")]
#[global_allocator]
static GLOBAL: mimalloc::MiMalloc = mimalloc::MiMalloc;
use crate::{
command::parse_commandline_args,
config_files::set_config_path,
logger::{configure, logger},
};
use command::gather_commandline_args;
2024-03-28 16:27:12 +00:00
use log::{trace, Level};
2022-01-18 08:48:28 +00:00
use miette::Result;
use nu_cli::gather_parent_env_vars;
use nu_cmd_base::util::get_init_cwd;
Integrated Language Server (#10723) # Description This commit integrates a language server into nushell so that IDEs don't have to convert CLI option back and forth. - fixes https://github.com/nushell/vscode-nushell-lang/issues/117 - fixes https://github.com/jokeyrhyme/nuls/issues/8 Tracking tasks - [x] [textDocument/hover](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_hover) -> `nu --ide-hover` - [x] [textDocument/completion](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_completion) -> `nu --ide-complete` - [x] [textDocument/definition](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_definition) -> `nu --ide-goto-def` - ~~[ ] [textDocument/didChange](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_didChange), [textDocument/didClose](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_didClose), and [textDocument/didOpen](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_didOpen)~~ (will be done in a follow-up PR) - ~~[ ] [textDocument/inlayHint](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_inlayHint) -> `nu --ide-check`~~ (will be done in a follow-up PR) - ~~[ ] [textDocument/publishDiagnostics](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_publishDiagnostics) -> `nu --ide-check`~~ (will be done in a follow-up PR) - ~~[ ] [workspace/configuration](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#workspace_configuration)~~ (will be done in a follow-up PR) - ~~[ ] [workspace/didChangeConfiguration](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#workspace_didChangeConfiguration)~~ (will be done in a follow-up PR) # User-Facing Changes The command line options `--lsp` will start a LSP server. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2023-11-02 15:18:57 +00:00
use nu_lsp::LanguageServer;
Fix #12416 by canonicalizing XDG_CONFIG_HOME before comparing to config_dir() (#12420) <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Fixes #12416. Turns out that in src/main.rs, `XDG_CONFIG_HOME` wasn't being canonicalized before being compared to `nu_path::config_dir()` to check if `XDG_CONFIG_HOME` was set to an invalid value. This has been rectified now. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> Setting `XDG_CONFIG_HOME` to a symlink should work now. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> I manually tested it and the error has disappeared: New behavior (this branch): ![image](https://github.com/nushell/nushell/assets/45539777/062d1cc5-551c-431c-b138-d3da8de018bd) Old behavior (main): ![image](https://github.com/nushell/nushell/assets/45539777/22c4b5a3-3fd0-4ab6-9cf0-ae25488645ba) Thanks to a pointer from Devyn, I've now added tests to make sure the `xdg_config_home_invalid` error doesn't pop up when `XDG_CONFIG_HOME` is a symlink (and does when it's actually invalid). Turns out two of the tests in `test_config_path` tried modifying `XDG_CONFIG_HOME` using `playground.with_env` but used `nu!`, so the subprocess didn't actually use the modified value of `XDG_CONFIG_HOME`. When I added them, I was unaware that the `.with_env` didn't actually modify the environment. This has now been rectified. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2024-04-06 14:09:03 +00:00
use nu_path::canonicalize_with;
2023-09-01 06:18:55 +00:00
use nu_protocol::{
Replace `ExternalStream` with new `ByteStream` type (#12774) # Description This PR introduces a `ByteStream` type which is a `Read`-able stream of bytes. Internally, it has an enum over three different byte stream sources: ```rust pub enum ByteStreamSource { Read(Box<dyn Read + Send + 'static>), File(File), Child(ChildProcess), } ``` This is in comparison to the current `RawStream` type, which is an `Iterator<Item = Vec<u8>>` and has to allocate for each read chunk. Currently, `PipelineData::ExternalStream` serves a weird dual role where it is either external command output or a wrapper around `RawStream`. `ByteStream` makes this distinction more clear (via `ByteStreamSource`) and replaces `PipelineData::ExternalStream` in this PR: ```rust pub enum PipelineData { Empty, Value(Value, Option<PipelineMetadata>), ListStream(ListStream, Option<PipelineMetadata>), ByteStream(ByteStream, Option<PipelineMetadata>), } ``` The PR is relatively large, but a decent amount of it is just repetitive changes. This PR fixes #7017, fixes #10763, and fixes #12369. This PR also improves performance when piping external commands. Nushell should, in most cases, have competitive pipeline throughput compared to, e.g., bash. | Command | Before (MB/s) | After (MB/s) | Bash (MB/s) | | -------------------------------------------------- | -------------:| ------------:| -----------:| | `throughput \| rg 'x'` | 3059 | 3744 | 3739 | | `throughput \| nu --testbin relay o> /dev/null` | 3508 | 8087 | 8136 | # User-Facing Changes - This is a breaking change for the plugin communication protocol, because the `ExternalStreamInfo` was replaced with `ByteStreamInfo`. Plugins now only have to deal with a single input stream, as opposed to the previous three streams: stdout, stderr, and exit code. - The output of `describe` has been changed for external/byte streams. - Temporary breaking change: `bytes starts-with` no longer works with byte streams. This is to keep the PR smaller, and `bytes ends-with` already does not work on byte streams. - If a process core dumped, then instead of having a `Value::Error` in the `exit_code` column of the output returned from `complete`, it now is a `Value::Int` with the negation of the signal number. # After Submitting - Update docs and book as necessary - Release notes (e.g., plugin protocol changes) - Adapt/convert commands to work with byte streams (high priority is `str length`, `bytes starts-with`, and maybe `bytes ends-with`). - Refactor the `tee` code, Devyn has already done some work on this. --------- Co-authored-by: Devyn Cairns <devyn.cairns@gmail.com>
2024-05-16 14:11:18 +00:00
engine::EngineState, report_error_new, ByteStream, PipelineData, ShellError, Span, Value,
2023-09-01 06:18:55 +00:00
};
use nu_std::load_standard_library;
add some startup performance metrics (#7851) # Description This PR changes the old performance logging with `Instant` timers. I'm not sure if this is the best way to do it but it does help reveal where time is being spent on startup. This is what it looks like when you launch nushell with `cargo run -- --log-level info`. I'm using the `info` log level exclusively for performance monitoring at this point. ![image](https://user-images.githubusercontent.com/343840/214372903-fdfa9c99-b846-47f3-8faf-bd6ed98df3a9.png) ## After Startup Since you're in the repl, you can continue running commands. Here's the output of `ls`, for instance. ![image](https://user-images.githubusercontent.com/343840/214373035-4d2f6e2d-5c1d-43d3-b997-51d79d496ba3.png) Note that the above screenshots are in debug mode, so they're much slower than release. # User-Facing Changes # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-01-24 20:28:59 +00:00
use nu_utils::utils::perf;
use run::{run_commands, run_file, run_repl};
use signals::ctrlc_protection;
use std::{
Fix #12416 by canonicalizing XDG_CONFIG_HOME before comparing to config_dir() (#12420) <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Fixes #12416. Turns out that in src/main.rs, `XDG_CONFIG_HOME` wasn't being canonicalized before being compared to `nu_path::config_dir()` to check if `XDG_CONFIG_HOME` was set to an invalid value. This has been rectified now. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> Setting `XDG_CONFIG_HOME` to a symlink should work now. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> I manually tested it and the error has disappeared: New behavior (this branch): ![image](https://github.com/nushell/nushell/assets/45539777/062d1cc5-551c-431c-b138-d3da8de018bd) Old behavior (main): ![image](https://github.com/nushell/nushell/assets/45539777/22c4b5a3-3fd0-4ab6-9cf0-ae25488645ba) Thanks to a pointer from Devyn, I've now added tests to make sure the `xdg_config_home_invalid` error doesn't pop up when `XDG_CONFIG_HOME` is a symlink (and does when it's actually invalid). Turns out two of the tests in `test_config_path` tried modifying `XDG_CONFIG_HOME` using `playground.with_env` but used `nu!`, so the subprocess didn't actually use the modified value of `XDG_CONFIG_HOME`. When I added them, I was unaware that the `.with_env` didn't actually modify the environment. This has now been rectified. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2024-04-06 14:09:03 +00:00
path::PathBuf,
str::FromStr,
sync::{atomic::AtomicBool, Arc},
};
2023-06-14 21:12:55 +00:00
fn get_engine_state() -> EngineState {
let engine_state = nu_cmd_lang::create_default_context();
#[cfg(feature = "plugin")]
let engine_state = nu_cmd_plugin::add_plugin_command_context(engine_state);
2023-06-14 21:12:55 +00:00
let engine_state = nu_command::add_shell_command_context(engine_state);
let engine_state = nu_cmd_extra::add_extra_command_context(engine_state);
let engine_state = nu_cli::add_cli_context(engine_state);
nu_explore::add_explore_context(engine_state)
}
fn main() -> Result<()> {
add some startup performance metrics (#7851) # Description This PR changes the old performance logging with `Instant` timers. I'm not sure if this is the best way to do it but it does help reveal where time is being spent on startup. This is what it looks like when you launch nushell with `cargo run -- --log-level info`. I'm using the `info` log level exclusively for performance monitoring at this point. ![image](https://user-images.githubusercontent.com/343840/214372903-fdfa9c99-b846-47f3-8faf-bd6ed98df3a9.png) ## After Startup Since you're in the repl, you can continue running commands. Here's the output of `ls`, for instance. ![image](https://user-images.githubusercontent.com/343840/214373035-4d2f6e2d-5c1d-43d3-b997-51d79d496ba3.png) Note that the above screenshots are in debug mode, so they're much slower than release. # User-Facing Changes # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-01-24 20:28:59 +00:00
let entire_start_time = std::time::Instant::now();
let mut start_time = std::time::Instant::now();
miette::set_panic_hook();
let miette_hook = std::panic::take_hook();
std::panic::set_hook(Box::new(move |x| {
2021-12-04 12:38:21 +00:00
crossterm::terminal::disable_raw_mode().expect("unable to disable raw mode");
miette_hook(x);
}));
2021-09-21 19:37:16 +00:00
Use only $nu.env.PWD for getting the current directory (#587) * Use only $nu.env.PWD for getting current directory Because setting and reading to/from std::env changes the global state shich is problematic if we call `cd` from multiple threads (e.g., in a `par-each` block). With this change, when engine-q starts, it will either inherit existing PWD env var, or create a new one from `std::env::current_dir()`. Otherwise, everything that needs the current directory will get it from `$nu.env.PWD`. Each spawned external command will get its current directory per-process which should be thread-safe. One thing left to do is to patch nu-path for this as well since it uses `std::env::current_dir()` in its expansions. * Rename nu-path functions *_with is not *_relative which should be more descriptive and frees "with" for use in a followup commit. * Clone stack every each iter; Fix some commands Cloning the stack each iteration of `each` makes sure we're not reusing PWD between iterations. Some fixes in commands to make them use the new PWD. * Post-rebase cleanup, fmt, clippy * Change back _relative to _with in nu-path funcs Didn't use the idea I had for the new "_with". * Remove leftover current_dir from rebase * Add cwd sync at merge_delta() This makes sure the parser and completer always have up-to-date cwd. * Always pass absolute path to glob in ls * Do not allow PWD a relative path; Allow recovery Makes it possible to recover PWD by proceeding with the REPL cycle. * Clone stack in each also for byte/string stream * (WIP) Start moving env variables to engine state * (WIP) Move env vars to engine state (ugly) Quick and dirty code. * (WIP) Remove unused mut and args; Fmt * (WIP) Fix dataframe tests * (WIP) Fix missing args after rebase * (WIP) Clone only env vars, not the whole stack * (WIP) Add env var clone to `for` loop as well * Minor edits * Refactor merge_delta() to include stack merging. Less error-prone than doing it manually. * Clone env for each `update` command iteration * Mark env var hidden only when found in eng. state * Fix clippt warnings * Add TODO about env var reading * Do not clone empty environment in loops * Remove extra cwd collection * Split current_dir() into str and path; Fix autocd * Make completions respect PWD env var
2022-01-04 22:30:34 +00:00
// Get initial current working directory.
let init_cwd = get_init_cwd();
2023-06-14 21:12:55 +00:00
let mut engine_state = get_engine_state();
2021-07-17 06:31:34 +00:00
// Custom additions
let delta = {
let mut working_set = nu_protocol::engine::StateWorkingSet::new(&engine_state);
working_set.add_decl(Box::new(nu_cli::NuHighlight));
working_set.add_decl(Box::new(nu_cli::Print));
working_set.render()
};
if let Err(err) = engine_state.merge_delta(delta) {
report_error_new(&engine_state, &err);
}
2021-10-28 04:13:10 +00:00
let ctrlc = Arc::new(AtomicBool::new(false));
// TODO: make this conditional in the future
ctrlc_protection(&mut engine_state, &ctrlc);
Add nu lib dirs default (#11248) # Description This PR is kind of two PRs in one because they were dependent on each other. PR1 - https://github.com/nushell/nushell/commit/3de58d4dc2d441bf853c7ed7c7a456ab7645fe93 with update https://github.com/nushell/nushell/pull/11248/commits/7fcdb242d96c1a32f3deb30594d86b39f5b795ee - This follows our mantra of having everything with defaults written in nushell rust code. So, that if you run without a config, you get the same behavior as with the default config/env files. This sets NU_LIB_DIRS to $nu.config-path/scripts and sets NU_PLUGIN_DIRS to $nu.config-path/plugins. PR2 - https://github.com/nushell/nushell/commit/0e8ac876fd266632b755d27f25858f3c941605ce - The benchmarks have been broke for some time and we didn't notice it. This PR fixes that. It's dependent on PR1 because it was throwing errors because PWD needed to be set to a valid folder and `$nu` did not exist based on how the benchmark was setup. I've tested the benchmarks and they run without error now and I've also launched nushell as `nu -n --no-std-lib` and the env vars exist. closes #11236 # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2023-12-07 14:13:50 +00:00
// Begin: Default NU_LIB_DIRS, NU_PLUGIN_DIRS
// Set default NU_LIB_DIRS and NU_PLUGIN_DIRS here before the env.nu is processed. If
// the env.nu file exists, these values will be overwritten, if it does not exist, or
// there is an error reading it, these values will be used.
let nushell_config_path = if let Some(mut path) = nu_path::config_dir() {
path.push("nushell");
path
} else {
// Not really sure what to default this to if nu_path::config_dir() returns None
std::path::PathBuf::new()
};
Use XDG_CONFIG_HOME before default config directory (#12118) <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> Closes #12103 # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> As described in #12103, this PR makes Nushell use `XDG_CONFIG_HOME` as the config directory if it exists. Otherwise, it uses the old behavior, which was to use `dirs_next::config_dir()`. Edit: We discussed choosing between `XDG_CONFIG_HOME` and the default config directory in Discord and decided against it, at least for now. <s>@kubouch also suggested letting users choose between `XDG_CONFIG_HOME` and the default config directory if config files aren't found on startup and `XDG_CONFIG_HOME` is set to a value different from the default config directory</s> On Windows and MacOS, if the `XDG_CONFIG_HOME` variable is set but `XDG_CONFIG_HOME` is either empty or doesn't exist *and* the old config directory is non-empty, Nushell will issue a warning on startup saying that it won't move files from the old config directory to the new one. To do this, I had to add a `nu_path::config_dir_old()` function. I assume that at some point, we will remove the warning message and the function can be removed too. Alternatively, instead of having that function there, `main.rs` could directly call `dirs_next::config_dir()`. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> When `$env.XDG_CONFIG_HOME` is set to an absolute path, Nushell will use `$"($env.XDG_CONFIG_HOME)/nushell"` as its config directory (previously, this only worked on Linux). To use `App Data\Roaming` (Windows) or `Library/Application Support` (MacOS) instead (the old behavior), one can either leave `XDG_CONFIG_HOME` unset or set it to an empty string. If `XDG_CONFIG_HOME` is set, but to a non-absolute/invalid path, Nushell will report an error on startup and use the default config directory instead: ![image](https://github.com/nushell/nushell/assets/45539777/a434fe04-b7c8-4e95-b50c-80628008ad08) On Windows and MacOS, if the `XDG_CONFIG_HOME` variable is set but `XDG_CONFIG_HOME` is either empty or doesn't exist *and* the old config directory is non-empty, Nushell will issue a warning on startup saying that it won't move files from the old config directory to the new one. ![image](https://github.com/nushell/nushell/assets/45539777/1686cc17-4083-4c12-aecf-1d832460ca57) # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> The existing config path tests have been modified to use `XDG_CONFIG_HOME` to change the config directory on all OSes, not just Linux. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> The documentation will have to be updated to note that Nushell uses `XDG_CONFIG_HOME` now. As @fdncred pointed out, it's possible for people to set `XDG_CONFIG_HOME` to, say, `~/.config/nushell` rather than `~/.config`, so the documentation could warn about that mistake.
2024-03-11 11:15:46 +00:00
if let Ok(xdg_config_home) = std::env::var("XDG_CONFIG_HOME") {
if !xdg_config_home.is_empty() {
Fix #12416 by canonicalizing XDG_CONFIG_HOME before comparing to config_dir() (#12420) <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Fixes #12416. Turns out that in src/main.rs, `XDG_CONFIG_HOME` wasn't being canonicalized before being compared to `nu_path::config_dir()` to check if `XDG_CONFIG_HOME` was set to an invalid value. This has been rectified now. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> Setting `XDG_CONFIG_HOME` to a symlink should work now. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> I manually tested it and the error has disappeared: New behavior (this branch): ![image](https://github.com/nushell/nushell/assets/45539777/062d1cc5-551c-431c-b138-d3da8de018bd) Old behavior (main): ![image](https://github.com/nushell/nushell/assets/45539777/22c4b5a3-3fd0-4ab6-9cf0-ae25488645ba) Thanks to a pointer from Devyn, I've now added tests to make sure the `xdg_config_home_invalid` error doesn't pop up when `XDG_CONFIG_HOME` is a symlink (and does when it's actually invalid). Turns out two of the tests in `test_config_path` tried modifying `XDG_CONFIG_HOME` using `playground.with_env` but used `nu!`, so the subprocess didn't actually use the modified value of `XDG_CONFIG_HOME`. When I added them, I was unaware that the `.with_env` didn't actually modify the environment. This has now been rectified. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2024-04-06 14:09:03 +00:00
if nushell_config_path
!= canonicalize_with(&xdg_config_home, &init_cwd)
.unwrap_or(PathBuf::from(&xdg_config_home))
.join("nushell")
{
Use XDG_CONFIG_HOME before default config directory (#12118) <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> Closes #12103 # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> As described in #12103, this PR makes Nushell use `XDG_CONFIG_HOME` as the config directory if it exists. Otherwise, it uses the old behavior, which was to use `dirs_next::config_dir()`. Edit: We discussed choosing between `XDG_CONFIG_HOME` and the default config directory in Discord and decided against it, at least for now. <s>@kubouch also suggested letting users choose between `XDG_CONFIG_HOME` and the default config directory if config files aren't found on startup and `XDG_CONFIG_HOME` is set to a value different from the default config directory</s> On Windows and MacOS, if the `XDG_CONFIG_HOME` variable is set but `XDG_CONFIG_HOME` is either empty or doesn't exist *and* the old config directory is non-empty, Nushell will issue a warning on startup saying that it won't move files from the old config directory to the new one. To do this, I had to add a `nu_path::config_dir_old()` function. I assume that at some point, we will remove the warning message and the function can be removed too. Alternatively, instead of having that function there, `main.rs` could directly call `dirs_next::config_dir()`. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> When `$env.XDG_CONFIG_HOME` is set to an absolute path, Nushell will use `$"($env.XDG_CONFIG_HOME)/nushell"` as its config directory (previously, this only worked on Linux). To use `App Data\Roaming` (Windows) or `Library/Application Support` (MacOS) instead (the old behavior), one can either leave `XDG_CONFIG_HOME` unset or set it to an empty string. If `XDG_CONFIG_HOME` is set, but to a non-absolute/invalid path, Nushell will report an error on startup and use the default config directory instead: ![image](https://github.com/nushell/nushell/assets/45539777/a434fe04-b7c8-4e95-b50c-80628008ad08) On Windows and MacOS, if the `XDG_CONFIG_HOME` variable is set but `XDG_CONFIG_HOME` is either empty or doesn't exist *and* the old config directory is non-empty, Nushell will issue a warning on startup saying that it won't move files from the old config directory to the new one. ![image](https://github.com/nushell/nushell/assets/45539777/1686cc17-4083-4c12-aecf-1d832460ca57) # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> The existing config path tests have been modified to use `XDG_CONFIG_HOME` to change the config directory on all OSes, not just Linux. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> The documentation will have to be updated to note that Nushell uses `XDG_CONFIG_HOME` now. As @fdncred pointed out, it's possible for people to set `XDG_CONFIG_HOME` to, say, `~/.config/nushell` rather than `~/.config`, so the documentation could warn about that mistake.
2024-03-11 11:15:46 +00:00
report_error_new(
&engine_state,
&ShellError::InvalidXdgConfig {
xdg: xdg_config_home,
default: nushell_config_path.display().to_string(),
},
);
} else if let Some(old_config) = nu_path::config_dir_old().map(|p| p.join("nushell")) {
let xdg_config_empty = nushell_config_path
.read_dir()
.map_or(true, |mut dir| dir.next().is_none());
let old_config_empty = old_config
.read_dir()
.map_or(true, |mut dir| dir.next().is_none());
if !old_config_empty && xdg_config_empty {
eprintln!(
"WARNING: XDG_CONFIG_HOME has been set but {} is empty.\n",
nushell_config_path.display(),
);
eprintln!(
"Nushell will not move your configuration files from {}",
old_config.display()
);
}
}
}
}
Add nu lib dirs default (#11248) # Description This PR is kind of two PRs in one because they were dependent on each other. PR1 - https://github.com/nushell/nushell/commit/3de58d4dc2d441bf853c7ed7c7a456ab7645fe93 with update https://github.com/nushell/nushell/pull/11248/commits/7fcdb242d96c1a32f3deb30594d86b39f5b795ee - This follows our mantra of having everything with defaults written in nushell rust code. So, that if you run without a config, you get the same behavior as with the default config/env files. This sets NU_LIB_DIRS to $nu.config-path/scripts and sets NU_PLUGIN_DIRS to $nu.config-path/plugins. PR2 - https://github.com/nushell/nushell/commit/0e8ac876fd266632b755d27f25858f3c941605ce - The benchmarks have been broke for some time and we didn't notice it. This PR fixes that. It's dependent on PR1 because it was throwing errors because PWD needed to be set to a valid folder and `$nu` did not exist based on how the benchmark was setup. I've tested the benchmarks and they run without error now and I've also launched nushell as `nu -n --no-std-lib` and the env vars exist. closes #11236 # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2023-12-07 14:13:50 +00:00
let mut default_nu_lib_dirs_path = nushell_config_path.clone();
default_nu_lib_dirs_path.push("scripts");
engine_state.add_env_var(
"NU_LIB_DIRS".to_string(),
Value::test_list(vec![Value::test_string(
default_nu_lib_dirs_path.to_string_lossy(),
)]),
Add nu lib dirs default (#11248) # Description This PR is kind of two PRs in one because they were dependent on each other. PR1 - https://github.com/nushell/nushell/commit/3de58d4dc2d441bf853c7ed7c7a456ab7645fe93 with update https://github.com/nushell/nushell/pull/11248/commits/7fcdb242d96c1a32f3deb30594d86b39f5b795ee - This follows our mantra of having everything with defaults written in nushell rust code. So, that if you run without a config, you get the same behavior as with the default config/env files. This sets NU_LIB_DIRS to $nu.config-path/scripts and sets NU_PLUGIN_DIRS to $nu.config-path/plugins. PR2 - https://github.com/nushell/nushell/commit/0e8ac876fd266632b755d27f25858f3c941605ce - The benchmarks have been broke for some time and we didn't notice it. This PR fixes that. It's dependent on PR1 because it was throwing errors because PWD needed to be set to a valid folder and `$nu` did not exist based on how the benchmark was setup. I've tested the benchmarks and they run without error now and I've also launched nushell as `nu -n --no-std-lib` and the env vars exist. closes #11236 # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2023-12-07 14:13:50 +00:00
);
let mut default_nu_plugin_dirs_path = nushell_config_path;
default_nu_plugin_dirs_path.push("plugins");
engine_state.add_env_var(
"NU_PLUGIN_DIRS".to_string(),
Value::test_list(vec![Value::test_string(
default_nu_plugin_dirs_path.to_string_lossy(),
)]),
Add nu lib dirs default (#11248) # Description This PR is kind of two PRs in one because they were dependent on each other. PR1 - https://github.com/nushell/nushell/commit/3de58d4dc2d441bf853c7ed7c7a456ab7645fe93 with update https://github.com/nushell/nushell/pull/11248/commits/7fcdb242d96c1a32f3deb30594d86b39f5b795ee - This follows our mantra of having everything with defaults written in nushell rust code. So, that if you run without a config, you get the same behavior as with the default config/env files. This sets NU_LIB_DIRS to $nu.config-path/scripts and sets NU_PLUGIN_DIRS to $nu.config-path/plugins. PR2 - https://github.com/nushell/nushell/commit/0e8ac876fd266632b755d27f25858f3c941605ce - The benchmarks have been broke for some time and we didn't notice it. This PR fixes that. It's dependent on PR1 because it was throwing errors because PWD needed to be set to a valid folder and `$nu` did not exist based on how the benchmark was setup. I've tested the benchmarks and they run without error now and I've also launched nushell as `nu -n --no-std-lib` and the env vars exist. closes #11236 # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2023-12-07 14:13:50 +00:00
);
// End: Default NU_LIB_DIRS, NU_PLUGIN_DIRS
Add `stor` family of commands (#11170) # Description This PR adds the `stor` family of commands. These commands are meant to create, open, insert, update, delete, reset data in an in-memory sqlite database. This is really an experiment to see how creatively we can use an in-memory database. ``` Usage: > stor Subcommands: stor create - Create a table in the in-memory sqlite database stor delete - Delete a table or specified rows in the in-memory sqlite database stor export - Export the in-memory sqlite database to a sqlite database file stor import - Import a sqlite database file into the in-memory sqlite database stor insert - Insert information into a specified table in the in-memory sqlite database stor open - Opens the in-memory sqlite database stor reset - Reset the in-memory database by dropping all tables stor update - Update information in a specified table in the in-memory sqlite database Flags: -h, --help - Display the help message for this command Input/output types: ╭─#─┬──input──┬─output─╮ │ 0 │ nothing │ string │ ╰───┴─────────┴────────╯ ``` ### Examples ## stor create ```nushell ❯ stor create --table-name nudb --columns {bool1: bool, int1: int, float1: float, str1: str, datetime1: datetime} ╭──────┬────────────────╮ │ nudb │ [list 0 items] │ ╰──────┴────────────────╯ ``` ## stor insert ```nushell ❯ stor insert --table-name nudb --data-record {bool1: true, int1: 2, float1: 1.1, str1: fdncred, datetime1: 2023-04-17} ╭──────┬───────────────╮ │ nudb │ [table 1 row] │ ╰──────┴───────────────╯ ``` ## stor open ```nushell ❯ stor open | table -e ╭──────┬────────────────────────────────────────────────────────────────────╮ │ │ ╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮ │ │ nudb │ │ 0 │ 1 │ 1 │ 2 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │ │ │ │ ╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯ │ ╰──────┴────────────────────────────────────────────────────────────────────╯ ``` ## stor update ```nushell ❯ stor update --table-name nudb --update-record {str1: toby datetime1: 2021-04-17} --where-clause "bool1 = 1" ╭──────┬───────────────╮ │ nudb │ [table 1 row] │ ╰──────┴───────────────╯ ❯ stor open | table -e ╭──────┬─────────────────────────────────────────────────────────────────╮ │ │ ╭─#─┬id─┬bool1┬int1┬float1┬─str1─┬─────────datetime1──────────╮ │ │ nudb │ │ 0 │ 1 │ 1 │ 2 │ 1.10 │ toby │ 2021-04-17 00:00:00 +00:00 │ │ │ │ ╰───┴───┴─────┴────┴──────┴──────┴────────────────────────────╯ │ ╰──────┴─────────────────────────────────────────────────────────────────╯ ``` ## insert another row ```nushell ❯ stor insert --table-name nudb --data-record {bool1: true, int1: 5, float1: 1.1, str1: fdncred, datetime1: 2023-04-17} ╭──────┬────────────────╮ │ nudb │ [table 2 rows] │ ╰──────┴────────────────╯ ❯ stor open | table -e ╭──────┬────────────────────────────────────────────────────────────────────╮ │ │ ╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮ │ │ nudb │ │ 0 │ 1 │ 1 │ 2 │ 1.10 │ toby │ 2021-04-17 00:00:00 +00:00 │ │ │ │ │ 1 │ 2 │ 1 │ 5 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │ │ │ │ ╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯ │ ╰──────┴────────────────────────────────────────────────────────────────────╯ ``` ## stor delete (specific row(s)) ```nushell ❯ stor delete --table-name nudb --where-clause "int1 == 5" ╭──────┬───────────────╮ │ nudb │ [table 1 row] │ ╰──────┴───────────────╯ ``` ## insert multiple tables ```nushell ❯ stor create --table-name nudb1 --columns {bool1: bool, int1: int, float1: float, str1: str, datetime1: datetime} ╭───────┬────────────────╮ │ nudb │ [table 1 row] │ │ nudb1 │ [list 0 items] │ ╰───────┴────────────────╯ ❯ stor insert --table-name nudb1 --data-record {bool1: true, int1: 2, float1: 1.1, str1: fdncred, datetime1: 2023-04-17} ╭───────┬───────────────╮ │ nudb │ [table 1 row] │ │ nudb1 │ [table 1 row] │ ╰───────┴───────────────╯ ❯ stor create --table-name nudb2 --columns {bool1: bool, int1: int, float1: float, str1: str, datetime1: datetime} ╭───────┬────────────────╮ │ nudb │ [table 1 row] │ │ nudb1 │ [table 1 row] │ │ nudb2 │ [list 0 items] │ ╰───────┴────────────────╯ ❯ stor insert --table-name nudb2 --data-record {bool1: true, int1: 2, float1: 1.1, str1: fdncred, datetime1: 2023-04-17} ╭───────┬───────────────╮ │ nudb │ [table 1 row] │ │ nudb1 │ [table 1 row] │ │ nudb2 │ [table 1 row] │ ╰───────┴───────────────╯ ``` ## stor delete (specific table) ```nushell ❯ stor delete --table-name nudb1 ╭───────┬───────────────╮ │ nudb │ [table 1 row] │ │ nudb2 │ [table 1 row] │ ╰───────┴───────────────╯ ``` ## stor reset (all tables are deleted) ```nushell ❯ stor reset ``` ## stor export ```nushell ❯ stor export --file-name nudb.sqlite3 ╭──────┬───────────────╮ │ nudb │ [table 1 row] │ ╰──────┴───────────────╯ ❯ open nudb.sqlite3 | table -e ╭──────┬────────────────────────────────────────────────────────────────────╮ │ │ ╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮ │ │ nudb │ │ 0 │ 1 │ 1 │ 5 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │ │ │ │ ╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯ │ ╰──────┴────────────────────────────────────────────────────────────────────╯ ❯ open nudb.sqlite3 | schema | table -e ╭────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ │ ╭──────┬──────────────────────────────────────────────────────────────────────────────────────────────────╮ │ │ tables │ │ │ ╭───────────────┬──────────────────────────────────────────────────────────────────────────────╮ │ │ │ │ │ nudb │ │ │ ╭─#─┬─cid─┬───name────┬─────type─────┬─notnull─┬───────default────────┬─pk─╮ │ │ │ │ │ │ │ │ columns │ │ 0 │ 0 │ id │ INTEGER │ 1 │ │ 1 │ │ │ │ │ │ │ │ │ │ │ 1 │ 1 │ bool1 │ BOOLEAN │ 0 │ │ 0 │ │ │ │ │ │ │ │ │ │ │ 2 │ 2 │ int1 │ INTEGER │ 0 │ │ 0 │ │ │ │ │ │ │ │ │ │ │ 3 │ 3 │ float1 │ REAL │ 0 │ │ 0 │ │ │ │ │ │ │ │ │ │ │ 4 │ 4 │ str1 │ VARCHAR(255) │ 0 │ │ 0 │ │ │ │ │ │ │ │ │ │ │ 5 │ 5 │ datetime1 │ DATETIME │ 0 │ STRFTIME('%Y-%m-%d │ 0 │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ %H:%M:%f', 'NOW') │ │ │ │ │ │ │ │ │ │ │ ╰─#─┴─cid─┴───name────┴─────type─────┴─notnull─┴───────default────────┴─pk─╯ │ │ │ │ │ │ │ │ constraints │ [list 0 items] │ │ │ │ │ │ │ │ foreign_keys │ [list 0 items] │ │ │ │ │ │ │ │ indexes │ [list 0 items] │ │ │ │ │ │ │ ╰───────────────┴──────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ ╰──────┴──────────────────────────────────────────────────────────────────────────────────────────────────╯ │ ╰────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ ``` ## Using with `query db` ```nushell ❯ stor open | query db "select * from nudb" ╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮ │ 0 │ 1 │ 1 │ 5 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │ ╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯ ``` ## stor import ```nushell ❯ stor open # note, nothing is returned. there is nothing in memory, atm. ❯ stor import --file-name nudb.sqlite3 ╭──────┬───────────────╮ │ nudb │ [table 1 row] │ ╰──────┴───────────────╯ ❯ stor open | table -e ╭──────┬────────────────────────────────────────────────────────────────────╮ │ │ ╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮ │ │ nudb │ │ 0 │ 1 │ 1 │ 5 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │ │ │ │ ╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯ │ ╰──────┴────────────────────────────────────────────────────────────────────╯ ``` TODO: - [x] `stor export` - Export a fully formed sqlite db file. - [x] `stor import` - Imports a specified sqlite db file. - [x] Perhaps feature-gate it with the sqlite feature - [x] Update `query db` to work with the in-memory database - [x] Remove `open --in-memory` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2023-11-29 16:02:46 +00:00
// This is the real secret sauce to having an in-memory sqlite db. You must
// start a connection to the memory database in main so it will exist for the
// lifetime of the program. If it's created with how MEMORY_DB is defined
// you'll be able to access this open connection from anywhere in the program
// by using the identical connection string.
#[cfg(feature = "sqlite")]
let db = nu_command::open_connection_in_memory_custom()?;
#[cfg(feature = "sqlite")]
db.last_insert_rowid();
let (args_to_nushell, script_name, args_to_script) = gather_commandline_args();
let parsed_nu_cli_args = parse_commandline_args(&args_to_nushell.join(" "), &mut engine_state)
.unwrap_or_else(|_| std::process::exit(1));
// keep this condition in sync with the branches at the end
engine_state.is_interactive = parsed_nu_cli_args.interactive_shell.is_some()
|| (parsed_nu_cli_args.testbin.is_none()
&& parsed_nu_cli_args.commands.is_none()
&& script_name.is_empty());
engine_state.is_login = parsed_nu_cli_args.login_shell.is_some();
engine_state.history_enabled = parsed_nu_cli_args.no_history.is_none();
let use_color = engine_state.get_config().use_ansi_coloring;
if let Some(level) = parsed_nu_cli_args
.log_level
.as_ref()
.map(|level| level.item.clone())
{
add some startup performance metrics (#7851) # Description This PR changes the old performance logging with `Instant` timers. I'm not sure if this is the best way to do it but it does help reveal where time is being spent on startup. This is what it looks like when you launch nushell with `cargo run -- --log-level info`. I'm using the `info` log level exclusively for performance monitoring at this point. ![image](https://user-images.githubusercontent.com/343840/214372903-fdfa9c99-b846-47f3-8faf-bd6ed98df3a9.png) ## After Startup Since you're in the repl, you can continue running commands. Here's the output of `ls`, for instance. ![image](https://user-images.githubusercontent.com/343840/214373035-4d2f6e2d-5c1d-43d3-b997-51d79d496ba3.png) Note that the above screenshots are in debug mode, so they're much slower than release. # User-Facing Changes # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-01-24 20:28:59 +00:00
let level = if Level::from_str(&level).is_ok() {
level
} else {
eprintln!(
"ERROR: log library did not recognize log level '{level}', using default 'info'"
);
"info".to_string()
};
let target = parsed_nu_cli_args
.log_target
.as_ref()
.map(|target| target.item.clone())
add some startup performance metrics (#7851) # Description This PR changes the old performance logging with `Instant` timers. I'm not sure if this is the best way to do it but it does help reveal where time is being spent on startup. This is what it looks like when you launch nushell with `cargo run -- --log-level info`. I'm using the `info` log level exclusively for performance monitoring at this point. ![image](https://user-images.githubusercontent.com/343840/214372903-fdfa9c99-b846-47f3-8faf-bd6ed98df3a9.png) ## After Startup Since you're in the repl, you can continue running commands. Here's the output of `ls`, for instance. ![image](https://user-images.githubusercontent.com/343840/214373035-4d2f6e2d-5c1d-43d3-b997-51d79d496ba3.png) Note that the above screenshots are in debug mode, so they're much slower than release. # User-Facing Changes # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-01-24 20:28:59 +00:00
.unwrap_or_else(|| "stderr".to_string());
logger(|builder| configure(&level, &target, builder))?;
// info!("start logging {}:{}:{}", file!(), line!(), column!());
perf(
"start logging",
start_time,
file!(),
line!(),
column!(),
use_color,
);
add some startup performance metrics (#7851) # Description This PR changes the old performance logging with `Instant` timers. I'm not sure if this is the best way to do it but it does help reveal where time is being spent on startup. This is what it looks like when you launch nushell with `cargo run -- --log-level info`. I'm using the `info` log level exclusively for performance monitoring at this point. ![image](https://user-images.githubusercontent.com/343840/214372903-fdfa9c99-b846-47f3-8faf-bd6ed98df3a9.png) ## After Startup Since you're in the repl, you can continue running commands. Here's the output of `ls`, for instance. ![image](https://user-images.githubusercontent.com/343840/214373035-4d2f6e2d-5c1d-43d3-b997-51d79d496ba3.png) Note that the above screenshots are in debug mode, so they're much slower than release. # User-Facing Changes # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-01-24 20:28:59 +00:00
}
start_time = std::time::Instant::now();
set_config_path(
&mut engine_state,
&init_cwd,
"config.nu",
"config-path",
parsed_nu_cli_args.config_file.as_ref(),
);
set_config_path(
&mut engine_state,
&init_cwd,
"env.nu",
"env-path",
parsed_nu_cli_args.env_file.as_ref(),
);
perf(
"set_config_path",
start_time,
file!(),
line!(),
column!(),
use_color,
);
#[cfg(unix)]
{
start_time = std::time::Instant::now();
terminal::acquire(engine_state.is_interactive);
perf(
"acquire_terminal",
start_time,
file!(),
line!(),
column!(),
use_color,
);
}
2024-03-28 16:27:12 +00:00
start_time = std::time::Instant::now();
Adds multi-file support to IDE support (#8857) # Description This adds multi-file support to the in-progress IDE support. The main new features are a `-I` flag that allows you to add a new source search path when starting up the nu binary, and fixes for the current IDE support to support spans in other files. This needs accompanying fixes to the vscode/lsp implementation to pass along the project directory via `-I`. UPDATE: Marking this draft until we have a means to test this. # User-Facing Changes _(List of all changes that impact the user experience here. This helps us keep track of breaking changes.)_ # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-04-12 17:36:29 +00:00
if let Some(include_path) = &parsed_nu_cli_args.include_path {
let span = include_path.span;
let vals: Vec<_> = include_path
.item
.split('\x1e') // \x1e is the record separator character (a character that is unlikely to appear in a path)
Move Value to helpers, separate span call (#10121) # Description As part of the refactor to split spans off of Value, this moves to using helper functions to create values, and using `.span()` instead of matching span out of Value directly. Hoping to get a few more helping hands to finish this, as there are a lot of commands to update :) # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> --------- Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com> Co-authored-by: WindSoilder <windsoilder@outlook.com>
2023-09-03 14:27:29 +00:00
.map(|x| Value::string(x.trim().to_string(), span))
.collect();
Move Value to helpers, separate span call (#10121) # Description As part of the refactor to split spans off of Value, this moves to using helper functions to create values, and using `.span()` instead of matching span out of Value directly. Hoping to get a few more helping hands to finish this, as there are a lot of commands to update :) # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> --------- Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com> Co-authored-by: WindSoilder <windsoilder@outlook.com>
2023-09-03 14:27:29 +00:00
engine_state.add_env_var("NU_LIB_DIRS".into(), Value::list(vals, span));
Adds multi-file support to IDE support (#8857) # Description This adds multi-file support to the in-progress IDE support. The main new features are a `-I` flag that allows you to add a new source search path when starting up the nu binary, and fixes for the current IDE support to support spans in other files. This needs accompanying fixes to the vscode/lsp implementation to pass along the project directory via `-I`. UPDATE: Marking this draft until we have a means to test this. # User-Facing Changes _(List of all changes that impact the user experience here. This helps us keep track of breaking changes.)_ # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-04-12 17:36:29 +00:00
}
2024-03-28 16:27:12 +00:00
perf(
"NU_LIB_DIRS setup",
start_time,
file!(),
line!(),
column!(),
use_color,
);
Adds multi-file support to IDE support (#8857) # Description This adds multi-file support to the in-progress IDE support. The main new features are a `-I` flag that allows you to add a new source search path when starting up the nu binary, and fixes for the current IDE support to support spans in other files. This needs accompanying fixes to the vscode/lsp implementation to pass along the project directory via `-I`. UPDATE: Marking this draft until we have a means to test this. # User-Facing Changes _(List of all changes that impact the user experience here. This helps us keep track of breaking changes.)_ # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-04-12 17:36:29 +00:00
start_time = std::time::Instant::now();
// First, set up env vars as strings only
gather_parent_env_vars(&mut engine_state, &init_cwd);
perf(
"gather env vars",
start_time,
file!(),
line!(),
column!(),
use_color,
);
engine_state.add_env_var(
"NU_VERSION".to_string(),
Value::string(env!("CARGO_PKG_VERSION"), Span::unknown()),
);
if parsed_nu_cli_args.no_std_lib.is_none() {
load_standard_library(&mut engine_state)?;
}
Add IDE support (#8745) # Description This adds a set of new flags on the `nu` binary intended for use in IDEs. Here is the set of supported functionality so far: * goto-def - go to the definition of a variable or custom command * type hints - see the inferred type of variables * check - see the errors in the document (currently only one error is supported) * hover - get information about the variable or custom command * complete - get a completion list at the current position # User-Facing Changes No changes to the REPL experience. This only impacts the IDE scenario. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-utils/standard_library/tests.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --------- Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2023-04-05 19:34:47 +00:00
// IDE commands
if let Some(ide_goto_def) = parsed_nu_cli_args.ide_goto_def {
ide::goto_def(&mut engine_state, &script_name, &ide_goto_def);
return Ok(());
} else if let Some(ide_hover) = parsed_nu_cli_args.ide_hover {
ide::hover(&mut engine_state, &script_name, &ide_hover);
return Ok(());
} else if let Some(ide_complete) = parsed_nu_cli_args.ide_complete {
Adds multi-file support to IDE support (#8857) # Description This adds multi-file support to the in-progress IDE support. The main new features are a `-I` flag that allows you to add a new source search path when starting up the nu binary, and fixes for the current IDE support to support spans in other files. This needs accompanying fixes to the vscode/lsp implementation to pass along the project directory via `-I`. UPDATE: Marking this draft until we have a means to test this. # User-Facing Changes _(List of all changes that impact the user experience here. This helps us keep track of breaking changes.)_ # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-04-12 17:36:29 +00:00
let cwd = std::env::current_dir().expect("Could not get current working directory.");
engine_state.add_env_var("PWD".into(), Value::test_string(cwd.to_string_lossy()));
Add IDE support (#8745) # Description This adds a set of new flags on the `nu` binary intended for use in IDEs. Here is the set of supported functionality so far: * goto-def - go to the definition of a variable or custom command * type hints - see the inferred type of variables * check - see the errors in the document (currently only one error is supported) * hover - get information about the variable or custom command * complete - get a completion list at the current position # User-Facing Changes No changes to the REPL experience. This only impacts the IDE scenario. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-utils/standard_library/tests.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --------- Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2023-04-05 19:34:47 +00:00
ide::complete(Arc::new(engine_state), &script_name, &ide_complete);
return Ok(());
} else if let Some(max_errors) = parsed_nu_cli_args.ide_check {
ide::check(&mut engine_state, &script_name, &max_errors);
Add IDE support (#8745) # Description This adds a set of new flags on the `nu` binary intended for use in IDEs. Here is the set of supported functionality so far: * goto-def - go to the definition of a variable or custom command * type hints - see the inferred type of variables * check - see the errors in the document (currently only one error is supported) * hover - get information about the variable or custom command * complete - get a completion list at the current position # User-Facing Changes No changes to the REPL experience. This only impacts the IDE scenario. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-utils/standard_library/tests.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --------- Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2023-04-05 19:34:47 +00:00
add `--ide-ast` for a simplistic ast for editors (#8995) # Description This is WIP. This generates a simplistic AST so that we can call it from the vscode side for semantic token highlighting. The output is a minified version of this screenshot. ![image](https://user-images.githubusercontent.com/343840/234354668-872d6267-9946-4b92-8a13-4fed45b4513a.png) The script ``` def test [arg] { print $arg for i in (seq 1 10) { echo $i } } ``` The simplistic AST ```json [{"content":"def","index":0,"shape":"shape_internalcall","span":{"end":15,"start":12}},{"content":"test","index":1,"shape":"shape_string","span":{"end":20,"start":16}},{"content":"[arg]","index":2,"shape":"shape_signature","span":{"end":26,"start":21}},{"content":"{\r\n ","index":3,"shape":"shape_closure","span":{"end":32,"start":27}},{"content":"print","index":4,"shape":"shape_internalcall","span":{"end":37,"start":32}},{"content":"$arg","index":5,"shape":"shape_variable","span":{"end":42,"start":38}},{"content":"for","index":6,"shape":"shape_internalcall","span":{"end":49,"start":46}},{"content":"i","index":7,"shape":"shape_vardecl","span":{"end":51,"start":50}},{"content":"in","index":8,"shape":"shape_keyword","span":{"end":54,"start":52}},{"content":"(","index":9,"shape":"shape_block","span":{"end":56,"start":55}},{"content":"seq","index":10,"shape":"shape_internalcall","span":{"end":59,"start":56}},{"content":"1","index":11,"shape":"shape_int","span":{"end":61,"start":60}},{"content":"10","index":12,"shape":"shape_int","span":{"end":64,"start":62}},{"content":")","index":13,"shape":"shape_block","span":{"end":65,"start":64}},{"content":"{\r\n ","index":14,"shape":"shape_block","span":{"end":73,"start":66}},{"content":"echo","index":15,"shape":"shape_internalcall","span":{"end":77,"start":73}},{"content":"$i","index":16,"shape":"shape_variable","span":{"end":80,"start":78}},{"content":"\r\n }","index":17,"shape":"shape_block","span":{"end":85,"start":80}},{"content":"\r\n}","index":18,"shape":"shape_closure","span":{"end":88,"start":85}}] ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2023-04-28 13:51:51 +00:00
return Ok(());
} else if parsed_nu_cli_args.ide_ast.is_some() {
ide::ast(&mut engine_state, &script_name);
Add IDE support (#8745) # Description This adds a set of new flags on the `nu` binary intended for use in IDEs. Here is the set of supported functionality so far: * goto-def - go to the definition of a variable or custom command * type hints - see the inferred type of variables * check - see the errors in the document (currently only one error is supported) * hover - get information about the variable or custom command * complete - get a completion list at the current position # User-Facing Changes No changes to the REPL experience. This only impacts the IDE scenario. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-utils/standard_library/tests.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --------- Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2023-04-05 19:34:47 +00:00
return Ok(());
}
add some startup performance metrics (#7851) # Description This PR changes the old performance logging with `Instant` timers. I'm not sure if this is the best way to do it but it does help reveal where time is being spent on startup. This is what it looks like when you launch nushell with `cargo run -- --log-level info`. I'm using the `info` log level exclusively for performance monitoring at this point. ![image](https://user-images.githubusercontent.com/343840/214372903-fdfa9c99-b846-47f3-8faf-bd6ed98df3a9.png) ## After Startup Since you're in the repl, you can continue running commands. Here's the output of `ls`, for instance. ![image](https://user-images.githubusercontent.com/343840/214373035-4d2f6e2d-5c1d-43d3-b997-51d79d496ba3.png) Note that the above screenshots are in debug mode, so they're much slower than release. # User-Facing Changes # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-01-24 20:28:59 +00:00
start_time = std::time::Instant::now();
if let Some(testbin) = &parsed_nu_cli_args.testbin {
// Call out to the correct testbin
match testbin.item.as_str() {
"echo_env" => test_bins::echo_env(true),
"echo_env_stderr" => test_bins::echo_env(false),
"echo_env_stderr_fail" => test_bins::echo_env_and_fail(false),
"echo_env_mixed" => test_bins::echo_env_mixed(),
"cococo" => test_bins::cococo(),
"meow" => test_bins::meow(),
"meowb" => test_bins::meowb(),
"relay" => test_bins::relay(),
"iecho" => test_bins::iecho(),
"fail" => test_bins::fail(),
"nonu" => test_bins::nonu(),
"chop" => test_bins::chop(),
"repeater" => test_bins::repeater(),
special-case ExternalStream in bytes starts-with (#8203) # Description `bytes starts-with` converts the input into a `Value` before running .starts_with to find if the binary matches. This has two side effects: it makes the code simpler, only dealing in whole values, and simplifying a lot of input pipeline handling and value transforming it would otherwise have to do. _Especially_ in the presence of a cell path to drill into. It also makes buffers the entire input into memory, which can take up a lot of memory when dealing with large files, especially if you only want to check the first few bytes (like for a magic number). This PR adds a special branch on PipelineData::ExternalStream with a streaming version of starts_with. # User-Facing Changes Opening large files and running bytes starts-with on them will not take a long time. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # Drawbacks Streaming checking is more complicated, and there may be bugs. I tested it with multiple chunks with string data and binary data and it seems to work alright up to 8k and over bytes, though. The existing `operate` method still exists because the way it handles cell paths and values is complicated. This causes some "code duplication", or at least some intent duplication, between the value code and the streaming code. This might be worthwhile considering the performance gains (approaching infinity on larger inputs). Another thing to consider is that my ExternalStream branch considers string data as valid input. The operate branch only parses Binary values, so it would fail. `open` is kind of unpredictable on whether it returns string data or binary data, even when passing `--raw`. I think this can be a problem but not really one I'm trying to tackle in this PR, so, it's worth considering.
2023-02-26 14:17:44 +00:00
"repeat_bytes" => test_bins::repeat_bytes(),
"nu_repl" => test_bins::nu_repl(),
pipe binary data to external commands (#8058) Fixes #7615 # Description When calling external commands, we create a table from the pipeline data to handle external commands expecting paginated input. When a binary value is made into a table, we convert the vector of bytes representing the binary bytes into a pretty formatted string. This results in the pretty formatted string being sent to external commands instead of the actual binary bytes. By checking whether the stdout of the call is being redirected, we can decide whether to send the raw binary bytes or the pretty formatted output when creating a table command. # User-Facing Changes When passing binary values to external commands, the external command will receive the actual bytes instead of the pretty printed string. Use cases that don't involve piping a binary value into an external command are unchanged. ![new_behavior](https://user-images.githubusercontent.com/32406734/218349172-24cd12f2-d563-4957-bdf1-6aa804b174b2.png) # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: cargo fmt --all -- --check to check standard code formatting (cargo fmt --all applies these changes) cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect to check that you're using the standard code style cargo test --workspace to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-02-24 20:39:52 +00:00
"input_bytes_length" => test_bins::input_bytes_length(),
_ => std::process::exit(1),
}
std::process::exit(0)
}
perf(
"run test_bins",
start_time,
file!(),
line!(),
column!(),
use_color,
);
add some startup performance metrics (#7851) # Description This PR changes the old performance logging with `Instant` timers. I'm not sure if this is the best way to do it but it does help reveal where time is being spent on startup. This is what it looks like when you launch nushell with `cargo run -- --log-level info`. I'm using the `info` log level exclusively for performance monitoring at this point. ![image](https://user-images.githubusercontent.com/343840/214372903-fdfa9c99-b846-47f3-8faf-bd6ed98df3a9.png) ## After Startup Since you're in the repl, you can continue running commands. Here's the output of `ls`, for instance. ![image](https://user-images.githubusercontent.com/343840/214373035-4d2f6e2d-5c1d-43d3-b997-51d79d496ba3.png) Note that the above screenshots are in debug mode, so they're much slower than release. # User-Facing Changes # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-01-24 20:28:59 +00:00
start_time = std::time::Instant::now();
let input = if let Some(redirect_stdin) = &parsed_nu_cli_args.redirect_stdin {
2024-03-28 16:27:12 +00:00
trace!("redirecting stdin");
Replace `ExternalStream` with new `ByteStream` type (#12774) # Description This PR introduces a `ByteStream` type which is a `Read`-able stream of bytes. Internally, it has an enum over three different byte stream sources: ```rust pub enum ByteStreamSource { Read(Box<dyn Read + Send + 'static>), File(File), Child(ChildProcess), } ``` This is in comparison to the current `RawStream` type, which is an `Iterator<Item = Vec<u8>>` and has to allocate for each read chunk. Currently, `PipelineData::ExternalStream` serves a weird dual role where it is either external command output or a wrapper around `RawStream`. `ByteStream` makes this distinction more clear (via `ByteStreamSource`) and replaces `PipelineData::ExternalStream` in this PR: ```rust pub enum PipelineData { Empty, Value(Value, Option<PipelineMetadata>), ListStream(ListStream, Option<PipelineMetadata>), ByteStream(ByteStream, Option<PipelineMetadata>), } ``` The PR is relatively large, but a decent amount of it is just repetitive changes. This PR fixes #7017, fixes #10763, and fixes #12369. This PR also improves performance when piping external commands. Nushell should, in most cases, have competitive pipeline throughput compared to, e.g., bash. | Command | Before (MB/s) | After (MB/s) | Bash (MB/s) | | -------------------------------------------------- | -------------:| ------------:| -----------:| | `throughput \| rg 'x'` | 3059 | 3744 | 3739 | | `throughput \| nu --testbin relay o> /dev/null` | 3508 | 8087 | 8136 | # User-Facing Changes - This is a breaking change for the plugin communication protocol, because the `ExternalStreamInfo` was replaced with `ByteStreamInfo`. Plugins now only have to deal with a single input stream, as opposed to the previous three streams: stdout, stderr, and exit code. - The output of `describe` has been changed for external/byte streams. - Temporary breaking change: `bytes starts-with` no longer works with byte streams. This is to keep the PR smaller, and `bytes ends-with` already does not work on byte streams. - If a process core dumped, then instead of having a `Value::Error` in the `exit_code` column of the output returned from `complete`, it now is a `Value::Int` with the negation of the signal number. # After Submitting - Update docs and book as necessary - Release notes (e.g., plugin protocol changes) - Adapt/convert commands to work with byte streams (high priority is `str length`, `bytes starts-with`, and maybe `bytes ends-with`). - Refactor the `tee` code, Devyn has already done some work on this. --------- Co-authored-by: Devyn Cairns <devyn.cairns@gmail.com>
2024-05-16 14:11:18 +00:00
PipelineData::ByteStream(ByteStream::stdin(redirect_stdin.span)?, None)
} else {
2024-03-28 16:27:12 +00:00
trace!("not redirecting stdin");
PipelineData::empty()
};
perf(
"redirect stdin",
start_time,
file!(),
line!(),
column!(),
use_color,
);
2024-03-28 16:27:12 +00:00
start_time = std::time::Instant::now();
2023-09-01 06:18:55 +00:00
// Set up the $nu constant before evaluating config files (need to have $nu available in them)
engine_state.generate_nu_constant();
2024-03-28 16:27:12 +00:00
perf(
"create_nu_constant",
start_time,
file!(),
line!(),
column!(),
use_color,
);
2023-09-01 06:18:55 +00:00
Deprecate `register` and add `plugin use` (#12607) # Description Adds a new keyword, `plugin use`. Unlike `register`, this merely loads the signatures from the plugin cache file. The file is configurable with the `--plugin-config` option either to `nu` or to `plugin use` itself, just like the other `plugin` family of commands. At the REPL, one might do this to replace `register`: ```nushell > plugin add ~/.cargo/bin/nu_plugin_foo > plugin use foo ``` This will not work in a script, because `plugin use` is a keyword and `plugin add` does not evaluate at parse time (intentionally). This means we no longer run random binaries during parse. The `--plugins` option has been added to allow running `nu` with certain plugins in one step. This is used especially for the `nu_with_plugins!` test macro, but I'd imagine is generally useful. The only weird quirk is that it has to be a list, and we don't really do this for any of our other CLI args at the moment. `register` now prints a deprecation parse warning. This should fix #11923, as we now have a complete alternative to `register`. # User-Facing Changes - Add `plugin use` command - Deprecate `register` - Add `--plugins` option to `nu` to replace a common use of `register` # Tests + Formatting I think I've tested it thoroughly enough and every existing test passes. Testing nu CLI options and alternate config files is a little hairy and I wish there were some more generic helpers for this, so this will go on my TODO list for refactoring. - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` # After Submitting - [ ] Update plugins sections of book - [ ] Release notes
2024-04-23 11:37:50 +00:00
#[cfg(feature = "plugin")]
if let Some(plugins) = &parsed_nu_cli_args.plugins {
Split the plugin crate (#12563) # Description This breaks `nu-plugin` up into four crates: - `nu-plugin-protocol`: just the type definitions for the protocol, no I/O. If someone wanted to wire up something more bare metal, maybe for async I/O, they could use this. - `nu-plugin-core`: the shared stuff between engine/plugin. Less stable interface. - `nu-plugin-engine`: everything required for the engine to talk to plugins. Less stable interface. - `nu-plugin`: everything required for the plugin to talk to the engine, what plugin developers use. Should be the most stable interface. No changes are made to the interface exposed by `nu-plugin` - it should all still be there. Re-exports from `nu-plugin-protocol` or `nu-plugin-core` are used as required. Plugins shouldn't ever have to use those crates directly. This should be somewhat faster to compile as `nu-plugin-engine` and `nu-plugin` can compile in parallel, and the engine doesn't need `nu-plugin` and plugins don't need `nu-plugin-engine` (except for test support), so that should reduce what needs to be compiled too. The only significant change here other than splitting stuff up was to break the `source` out of `PluginCustomValue` and create a new `PluginCustomValueWithSource` type that contains that instead. One bonus of that is we get rid of the option and it's now more type-safe, but it also means that the logic for that stuff (actually running the plugin for custom value ops) can live entirely within the `nu-plugin-engine` crate. # User-Facing Changes - New crates. - Added `local-socket` feature for `nu` to try to make it possible to compile without that support if needed. # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib`
2024-04-27 17:08:12 +00:00
use nu_plugin_engine::{GetPlugin, PluginDeclaration};
Deprecate `register` and add `plugin use` (#12607) # Description Adds a new keyword, `plugin use`. Unlike `register`, this merely loads the signatures from the plugin cache file. The file is configurable with the `--plugin-config` option either to `nu` or to `plugin use` itself, just like the other `plugin` family of commands. At the REPL, one might do this to replace `register`: ```nushell > plugin add ~/.cargo/bin/nu_plugin_foo > plugin use foo ``` This will not work in a script, because `plugin use` is a keyword and `plugin add` does not evaluate at parse time (intentionally). This means we no longer run random binaries during parse. The `--plugins` option has been added to allow running `nu` with certain plugins in one step. This is used especially for the `nu_with_plugins!` test macro, but I'd imagine is generally useful. The only weird quirk is that it has to be a list, and we don't really do this for any of our other CLI args at the moment. `register` now prints a deprecation parse warning. This should fix #11923, as we now have a complete alternative to `register`. # User-Facing Changes - Add `plugin use` command - Deprecate `register` - Add `--plugins` option to `nu` to replace a common use of `register` # Tests + Formatting I think I've tested it thoroughly enough and every existing test passes. Testing nu CLI options and alternate config files is a little hairy and I wish there were some more generic helpers for this, so this will go on my TODO list for refactoring. - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` # After Submitting - [ ] Update plugins sections of book - [ ] Release notes
2024-04-23 11:37:50 +00:00
use nu_protocol::{engine::StateWorkingSet, ErrSpan, PluginIdentity};
// Load any plugins specified with --plugins
start_time = std::time::Instant::now();
let mut working_set = StateWorkingSet::new(&engine_state);
for plugin_filename in plugins {
// Make sure the plugin filenames are canonicalized
let filename = canonicalize_with(&plugin_filename.item, &init_cwd)
.err_span(plugin_filename.span)
.map_err(ShellError::from)?;
let identity = PluginIdentity::new(&filename, None)
.err_span(plugin_filename.span)
.map_err(ShellError::from)?;
// Create the plugin and add it to the working set
Split the plugin crate (#12563) # Description This breaks `nu-plugin` up into four crates: - `nu-plugin-protocol`: just the type definitions for the protocol, no I/O. If someone wanted to wire up something more bare metal, maybe for async I/O, they could use this. - `nu-plugin-core`: the shared stuff between engine/plugin. Less stable interface. - `nu-plugin-engine`: everything required for the engine to talk to plugins. Less stable interface. - `nu-plugin`: everything required for the plugin to talk to the engine, what plugin developers use. Should be the most stable interface. No changes are made to the interface exposed by `nu-plugin` - it should all still be there. Re-exports from `nu-plugin-protocol` or `nu-plugin-core` are used as required. Plugins shouldn't ever have to use those crates directly. This should be somewhat faster to compile as `nu-plugin-engine` and `nu-plugin` can compile in parallel, and the engine doesn't need `nu-plugin` and plugins don't need `nu-plugin-engine` (except for test support), so that should reduce what needs to be compiled too. The only significant change here other than splitting stuff up was to break the `source` out of `PluginCustomValue` and create a new `PluginCustomValueWithSource` type that contains that instead. One bonus of that is we get rid of the option and it's now more type-safe, but it also means that the logic for that stuff (actually running the plugin for custom value ops) can live entirely within the `nu-plugin-engine` crate. # User-Facing Changes - New crates. - Added `local-socket` feature for `nu` to try to make it possible to compile without that support if needed. # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib`
2024-04-27 17:08:12 +00:00
let plugin = nu_plugin_engine::add_plugin_to_working_set(&mut working_set, &identity)?;
Deprecate `register` and add `plugin use` (#12607) # Description Adds a new keyword, `plugin use`. Unlike `register`, this merely loads the signatures from the plugin cache file. The file is configurable with the `--plugin-config` option either to `nu` or to `plugin use` itself, just like the other `plugin` family of commands. At the REPL, one might do this to replace `register`: ```nushell > plugin add ~/.cargo/bin/nu_plugin_foo > plugin use foo ``` This will not work in a script, because `plugin use` is a keyword and `plugin add` does not evaluate at parse time (intentionally). This means we no longer run random binaries during parse. The `--plugins` option has been added to allow running `nu` with certain plugins in one step. This is used especially for the `nu_with_plugins!` test macro, but I'd imagine is generally useful. The only weird quirk is that it has to be a list, and we don't really do this for any of our other CLI args at the moment. `register` now prints a deprecation parse warning. This should fix #11923, as we now have a complete alternative to `register`. # User-Facing Changes - Add `plugin use` command - Deprecate `register` - Add `--plugins` option to `nu` to replace a common use of `register` # Tests + Formatting I think I've tested it thoroughly enough and every existing test passes. Testing nu CLI options and alternate config files is a little hairy and I wish there were some more generic helpers for this, so this will go on my TODO list for refactoring. - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` # After Submitting - [ ] Update plugins sections of book - [ ] Release notes
2024-04-23 11:37:50 +00:00
// Spawn the plugin to get its signatures, and then add the commands to the working set
for signature in plugin.clone().get_plugin(None)?.get_signature()? {
let decl = PluginDeclaration::new(plugin.clone(), signature);
working_set.add_decl(Box::new(decl));
}
}
engine_state.merge_delta(working_set.render())?;
perf(
"load plugins specified in --plugins",
start_time,
file!(),
line!(),
column!(),
use_color,
)
}
start_time = std::time::Instant::now();
if parsed_nu_cli_args.lsp {
perf(
"lsp starting",
start_time,
file!(),
line!(),
column!(),
use_color,
);
if parsed_nu_cli_args.no_config_file.is_none() {
let mut stack = nu_protocol::engine::Stack::new();
config_files::setup_config(
&mut engine_state,
&mut stack,
#[cfg(feature = "plugin")]
parsed_nu_cli_args.plugin_file,
parsed_nu_cli_args.config_file,
parsed_nu_cli_args.env_file,
false,
);
}
Replace `ExternalStream` with new `ByteStream` type (#12774) # Description This PR introduces a `ByteStream` type which is a `Read`-able stream of bytes. Internally, it has an enum over three different byte stream sources: ```rust pub enum ByteStreamSource { Read(Box<dyn Read + Send + 'static>), File(File), Child(ChildProcess), } ``` This is in comparison to the current `RawStream` type, which is an `Iterator<Item = Vec<u8>>` and has to allocate for each read chunk. Currently, `PipelineData::ExternalStream` serves a weird dual role where it is either external command output or a wrapper around `RawStream`. `ByteStream` makes this distinction more clear (via `ByteStreamSource`) and replaces `PipelineData::ExternalStream` in this PR: ```rust pub enum PipelineData { Empty, Value(Value, Option<PipelineMetadata>), ListStream(ListStream, Option<PipelineMetadata>), ByteStream(ByteStream, Option<PipelineMetadata>), } ``` The PR is relatively large, but a decent amount of it is just repetitive changes. This PR fixes #7017, fixes #10763, and fixes #12369. This PR also improves performance when piping external commands. Nushell should, in most cases, have competitive pipeline throughput compared to, e.g., bash. | Command | Before (MB/s) | After (MB/s) | Bash (MB/s) | | -------------------------------------------------- | -------------:| ------------:| -----------:| | `throughput \| rg 'x'` | 3059 | 3744 | 3739 | | `throughput \| nu --testbin relay o> /dev/null` | 3508 | 8087 | 8136 | # User-Facing Changes - This is a breaking change for the plugin communication protocol, because the `ExternalStreamInfo` was replaced with `ByteStreamInfo`. Plugins now only have to deal with a single input stream, as opposed to the previous three streams: stdout, stderr, and exit code. - The output of `describe` has been changed for external/byte streams. - Temporary breaking change: `bytes starts-with` no longer works with byte streams. This is to keep the PR smaller, and `bytes ends-with` already does not work on byte streams. - If a process core dumped, then instead of having a `Value::Error` in the `exit_code` column of the output returned from `complete`, it now is a `Value::Int` with the negation of the signal number. # After Submitting - Update docs and book as necessary - Release notes (e.g., plugin protocol changes) - Adapt/convert commands to work with byte streams (high priority is `str length`, `bytes starts-with`, and maybe `bytes ends-with`). - Refactor the `tee` code, Devyn has already done some work on this. --------- Co-authored-by: Devyn Cairns <devyn.cairns@gmail.com>
2024-05-16 14:11:18 +00:00
LanguageServer::initialize_stdio_connection()?.serve_requests(engine_state, ctrlc)?
} else if let Some(commands) = parsed_nu_cli_args.commands.clone() {
run_commands(
&mut engine_state,
parsed_nu_cli_args,
use_color,
&commands,
input,
entire_start_time,
);
} else if !script_name.is_empty() {
run_file(
&mut engine_state,
parsed_nu_cli_args,
use_color,
script_name,
args_to_script,
input,
);
} else {
Replace `ExternalStream` with new `ByteStream` type (#12774) # Description This PR introduces a `ByteStream` type which is a `Read`-able stream of bytes. Internally, it has an enum over three different byte stream sources: ```rust pub enum ByteStreamSource { Read(Box<dyn Read + Send + 'static>), File(File), Child(ChildProcess), } ``` This is in comparison to the current `RawStream` type, which is an `Iterator<Item = Vec<u8>>` and has to allocate for each read chunk. Currently, `PipelineData::ExternalStream` serves a weird dual role where it is either external command output or a wrapper around `RawStream`. `ByteStream` makes this distinction more clear (via `ByteStreamSource`) and replaces `PipelineData::ExternalStream` in this PR: ```rust pub enum PipelineData { Empty, Value(Value, Option<PipelineMetadata>), ListStream(ListStream, Option<PipelineMetadata>), ByteStream(ByteStream, Option<PipelineMetadata>), } ``` The PR is relatively large, but a decent amount of it is just repetitive changes. This PR fixes #7017, fixes #10763, and fixes #12369. This PR also improves performance when piping external commands. Nushell should, in most cases, have competitive pipeline throughput compared to, e.g., bash. | Command | Before (MB/s) | After (MB/s) | Bash (MB/s) | | -------------------------------------------------- | -------------:| ------------:| -----------:| | `throughput \| rg 'x'` | 3059 | 3744 | 3739 | | `throughput \| nu --testbin relay o> /dev/null` | 3508 | 8087 | 8136 | # User-Facing Changes - This is a breaking change for the plugin communication protocol, because the `ExternalStreamInfo` was replaced with `ByteStreamInfo`. Plugins now only have to deal with a single input stream, as opposed to the previous three streams: stdout, stderr, and exit code. - The output of `describe` has been changed for external/byte streams. - Temporary breaking change: `bytes starts-with` no longer works with byte streams. This is to keep the PR smaller, and `bytes ends-with` already does not work on byte streams. - If a process core dumped, then instead of having a `Value::Error` in the `exit_code` column of the output returned from `complete`, it now is a `Value::Int` with the negation of the signal number. # After Submitting - Update docs and book as necessary - Release notes (e.g., plugin protocol changes) - Adapt/convert commands to work with byte streams (high priority is `str length`, `bytes starts-with`, and maybe `bytes ends-with`). - Refactor the `tee` code, Devyn has already done some work on this. --------- Co-authored-by: Devyn Cairns <devyn.cairns@gmail.com>
2024-05-16 14:11:18 +00:00
run_repl(&mut engine_state, parsed_nu_cli_args, entire_start_time)?
}
Replace `ExternalStream` with new `ByteStream` type (#12774) # Description This PR introduces a `ByteStream` type which is a `Read`-able stream of bytes. Internally, it has an enum over three different byte stream sources: ```rust pub enum ByteStreamSource { Read(Box<dyn Read + Send + 'static>), File(File), Child(ChildProcess), } ``` This is in comparison to the current `RawStream` type, which is an `Iterator<Item = Vec<u8>>` and has to allocate for each read chunk. Currently, `PipelineData::ExternalStream` serves a weird dual role where it is either external command output or a wrapper around `RawStream`. `ByteStream` makes this distinction more clear (via `ByteStreamSource`) and replaces `PipelineData::ExternalStream` in this PR: ```rust pub enum PipelineData { Empty, Value(Value, Option<PipelineMetadata>), ListStream(ListStream, Option<PipelineMetadata>), ByteStream(ByteStream, Option<PipelineMetadata>), } ``` The PR is relatively large, but a decent amount of it is just repetitive changes. This PR fixes #7017, fixes #10763, and fixes #12369. This PR also improves performance when piping external commands. Nushell should, in most cases, have competitive pipeline throughput compared to, e.g., bash. | Command | Before (MB/s) | After (MB/s) | Bash (MB/s) | | -------------------------------------------------- | -------------:| ------------:| -----------:| | `throughput \| rg 'x'` | 3059 | 3744 | 3739 | | `throughput \| nu --testbin relay o> /dev/null` | 3508 | 8087 | 8136 | # User-Facing Changes - This is a breaking change for the plugin communication protocol, because the `ExternalStreamInfo` was replaced with `ByteStreamInfo`. Plugins now only have to deal with a single input stream, as opposed to the previous three streams: stdout, stderr, and exit code. - The output of `describe` has been changed for external/byte streams. - Temporary breaking change: `bytes starts-with` no longer works with byte streams. This is to keep the PR smaller, and `bytes ends-with` already does not work on byte streams. - If a process core dumped, then instead of having a `Value::Error` in the `exit_code` column of the output returned from `complete`, it now is a `Value::Int` with the negation of the signal number. # After Submitting - Update docs and book as necessary - Release notes (e.g., plugin protocol changes) - Adapt/convert commands to work with byte streams (high priority is `str length`, `bytes starts-with`, and maybe `bytes ends-with`). - Refactor the `tee` code, Devyn has already done some work on this. --------- Co-authored-by: Devyn Cairns <devyn.cairns@gmail.com>
2024-05-16 14:11:18 +00:00
Ok(())
}