nushell/crates/nu-std/tests/test_util.nu

84 lines
2.2 KiB
Text
Raw Normal View History

use std/util *
Implement annotations support in test runner (#9406) <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description Test runner now uses annotations instead of magic function names to pick up code to run. Additionally skipping tests is now done on annotation level so skipping and unskipping a test no longer requires changes to the test code In order for a function to be picked up by the test runner it needs to meet following criteria: * Needs to be private (all exported functions are ignored) * Needs to contain one of valid annotations (and only the annotation) directly above the definition, all other comments are ignored Following are considered valid annotations: * \# test * \# test-skip * \# before-all * \# before-each * \# after-each * \# after-all # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect -A clippy::result_large_err` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2023-07-02 08:41:33 +00:00
#[test]
def path_add [] {
Improves startup time when using std-lib (#13842) Updated summary for commit [612e0e2](https://github.com/nushell/nushell/pull/13842/commits/612e0e21602f55092bc121bfd07f7c3bf5119e4f) - While folks are welcome to read through the entire comments, the core information is summarized here. # Description This PR drastically improves startup times of Nushell by only parsing a single submodule of the Standard Library that provides the `banner` and `pwd` commands. All other Standard Library commands and submodules are parsed when imported by the user. This cuts startup times by more than 60%. At the moment, we have stopped adding to `std-lib` because every addition adds a small amount to the Nushell startup time. With this change, we should once again be able to allow new functionality to be added to the Standard Library without it impacting `nu` startup times. # User-Facing Changes * Nushell now starts about 60% faster * Breaking change: The `dirs` (Shells) aliases will return a warning message that it will not be auto-loaded in the following release, along with instructions on how to restore it (and disable the message) * The `use std <submodule> *` syntax is available for convenience, but should be avoided in scripts as it parses the entire `std` module and all other submodules and places it in scope. The correct syntax to *just* load a submodule is `use std/<submodule> *` (asterisk optional). The slash is important. This will be documented. * `use std *` can be used for convenience to load all of the library but still incurs the full loading-time. * `std/dirs`: Semi-breaking change. The `dirs` command replaces the `show` command. This is more in line with the directory-stack functionality found in other shells. Existing users will not be impacted by this as the alias (`shells`) remains the same. * Breaking-change: Technically a breaking change, but probably only impacts maintainers of `std`. The virtual path for the standard library has changed. It could previously be imported using its virtual path (and technically, this would have been the correct way to do it): ```nu use NU_STDLIB_VIRTUAL_DIR/std ``` The path is now simply `std/`: ```nu use std ``` All submodules have moved accordingly. # Timings Comparisons below were made: * In a temporary, clean config directory using `$env.XDG_CONFIG_HOME = (mktemp -d)`. * `nu` was run with a release build * `nu` was run one time to generate the default `config.nu` (etc.) files - Otherwise timings would include the user-prompt * The shell was exited and then restarted several times to get timing samples (Note: Old timings based on 0.97 rather than 0.98, but in the range of being accurate) | Scenario | `$nu.startup-time` | | --- | --- | | 0.97.2 ([aaaab8e](https://github.com/nushell/nushell/commit/aaaab8e070c644a87bbd7682099e3fe9e6a4b42a)) Without this PR | 23ms - 24ms | | This PR with deprecated commands | 9ms - <11ms | | This PR after deprecated commands are removed in following release | 8ms - <10ms | | Final PR (remove deprecated), using `--no-std-lib` | 6.1ms to 6.4ms | | Final PR (remove deprecated), using `--no-config-file` | 3.1ms - 3.6ms | | Final PR (remove deprecated), using `--no-config-file --no-std-lib` | 1ms - 1.5ms | *These last two timings point to the opportunity for further optimization (see comment in thread below (will link once I write it).* # Implementation details for future maintenance * `use std banner` is a ridiculously deceptive call. That call parses and imports *all* of `std` into scope. Simply replacing it with `use std/core *` is essentially what saves ~14-15ms. This *only* imports the submodule with the `banner` and `pwd` commands. * From the code-comments, the reason that `NU_STDLIB_VIRTUAL_DIR` was used as a prefix was so that there wouldn't be an issue if a user had a `./std/mod.nu` in the current directory. This does **not** appear to be an issue. After removing the prefix, I tested with both a relative module as well as one in the `$env.NU_LIB_DIRS` path, and in all cases the *internal* `std` still took precedence. * By removing the prefix, users can now `use std` (and variants) without requiring that it already be parsed and in scope. * In the next release, we'll stop autoloading the `dirs` (shells) functionality. While this only costs an additional 1-1.5ms, I think it's better moved to the `config.nu` where the user can optionally remove it. The main reason is its use of aliases (which have also caused issues) - The `n`, `p`, and `g` short-commands are valuable real-estate, and users may want to map these to something else. For this release, there's an `deprecated_dirs` module that is still autoloaded. As with the top-level commands, use of these will give a deprecation warning with instructions on how to handle going forward. To help with this, moved the aliases to their own submodule inside the `dirs` module. * Also sneaks in a small change where the top-level `dirs` command is now the replacement for `dirs show` * Fixed a double-import of `assert` in `dirs.nu` * The `show_banner` step is replaced with simply `banner` rather than re-importing it. * A `virtual_path` may now be referenced with either a forward-slash or a backward-slash on Windows. This allows `use std/<submodule>` to work on all platforms. # Performance side-notes: * Future parsing and/or IR improvements should improve performance even further. * While the existing load time penalty of `std-lib` was not noticeable on many systems, Nushell runs on a wide-variety of hardware and OS platforms. Slower platforms will naturally see a bigger jump in performance here. For users starting multiple Nushell sessions frequently (e.g., `tmux`, Zellij, `screen`, et. al.) it is recommended to keep total startup time (including user configuration) under ~250ms. # Tests + Formatting * All tests are green * Updated tests: - Removed the test that confirmed that `std` was loaded (since we don't). - Removed the `shells` test since it is not autoloaded. Main `dirs.nu` functionality is tested through `stdlib-test`. - Many tests assumed that the library was fully loaded, because it was (even though we didn't intend for it to be). Fixed those tests. - Tests now import only the necessary submodules (e.g., `use std/assert`, rather than `use std assert`) - Some tests *thought* they were loading `std/log`, but were doing so improperly. This was masked by the now-fixed "load-everything-into-scope bug". Local CI would pass due the `$env.NU_LOG_<...>` variables being inherited from the calling process, but would fail in the "clean" GitHub CI environment. These tests have also been fixed. * Added additional tests for the changes # After Submitting Will update the Standard Library doc page
2024-10-03 11:28:22 +00:00
use std/assert
fix: 🐛 handle windows Path casing (#9210) # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> PATH and Path are different (in nushell at least) based on the OS # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> None the command now works as expected # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect -A clippy::result_large_err` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2023-05-17 23:55:46 +00:00
let path_name = if "PATH" in $env { "PATH" } else { "Path" }
with-env {$path_name: []} {
fix: 🐛 handle windows Path casing (#9210) # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> PATH and Path are different (in nushell at least) based on the OS # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> None the command now works as expected # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect -A clippy::result_large_err` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2023-05-17 23:55:46 +00:00
def get_path [] { $env | get $path_name }
fix: 🐛 handle windows Path casing (#9210) # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> PATH and Path are different (in nushell at least) based on the OS # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> None the command now works as expected # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect -A clippy::result_large_err` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2023-05-17 23:55:46 +00:00
assert equal (get_path) []
path add "/foo/"
assert equal (get_path) (["/foo/"] | path expand)
path add "/bar/" "/baz/"
assert equal (get_path) (["/bar/", "/baz/", "/foo/"] | path expand)
remove let-env, focus on mutating $env (#9574) # Description For years, Nushell has used `let-env` to set a single environment variable. As our work on scoping continued, we refined what it meant for a variable to be in scope using `let` but never updated how `let-env` would work. Instead, `let-env` confusingly created mutations to the command's copy of `$env`. So, to help fix the mental model and point people to the right way of thinking about what changing the environment means, this PR removes `let-env` to encourage people to think of it as updating the command's environment variable via mutation. Before: ``` let-env FOO = "BAR" ``` Now: ``` $env.FOO = "BAR" ``` It's also a good reminder that the environment owned by the command is in the `$env` variable rather than global like it is in other shells. # User-Facing Changes BREAKING CHANGE BREAKING CHANGE This completely removes `let-env FOO = "BAR"` so that we can focus on `$env.FOO = "BAR"`. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect -A clippy::result_large_err` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After / Before Submitting integration scripts to update: - :heavy_check_mark: [starship](https://github.com/starship/starship/blob/master/src/init/starship.nu) - :heavy_check_mark: [virtualenv](https://github.com/pypa/virtualenv/blob/main/src/virtualenv/activation/nushell/activate.nu) - :heavy_check_mark: [atuin](https://github.com/ellie/atuin/blob/main/atuin/src/shell/atuin.nu) (PR: https://github.com/ellie/atuin/pull/1080) - :x: [zoxide](https://github.com/ajeetdsouza/zoxide/blob/main/templates/nushell.txt) (PR: https://github.com/ajeetdsouza/zoxide/pull/587) - :heavy_check_mark: [oh-my-posh](https://github.com/JanDeDobbeleer/oh-my-posh/blob/main/src/shell/scripts/omp.nu) (pr: https://github.com/JanDeDobbeleer/oh-my-posh/pull/4011)
2023-06-30 19:57:51 +00:00
load-env {$path_name: []}
path add "foo"
path add "bar" "baz" --append
assert equal (get_path) (["foo", "bar", "baz"] | path expand)
assert equal (path add "fooooo" --ret) (["fooooo", "foo", "bar", "baz"] | path expand)
assert equal (get_path) (["fooooo", "foo", "bar", "baz"] | path expand)
feat: ✨ (stdlib) add os record support to path add (#9238) # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Add "os record" support. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> This don't change how path add works but just adds support for "os records" aka records whose key are at least one of: - linux - macos - windows Check the [test](https://github.com/melMass/nushell/blob/a917f1a9244a8bdc0d2739f9d7c8dd295e899b68/crates/nu-std/tests/test_std.nu#L31-L32) if that's not clear enough # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect -A clippy::result_large_err` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> --------- Co-authored-by: amtoine <stevan.antoine@gmail.com>
2023-05-26 07:24:53 +00:00
remove let-env, focus on mutating $env (#9574) # Description For years, Nushell has used `let-env` to set a single environment variable. As our work on scoping continued, we refined what it meant for a variable to be in scope using `let` but never updated how `let-env` would work. Instead, `let-env` confusingly created mutations to the command's copy of `$env`. So, to help fix the mental model and point people to the right way of thinking about what changing the environment means, this PR removes `let-env` to encourage people to think of it as updating the command's environment variable via mutation. Before: ``` let-env FOO = "BAR" ``` Now: ``` $env.FOO = "BAR" ``` It's also a good reminder that the environment owned by the command is in the `$env` variable rather than global like it is in other shells. # User-Facing Changes BREAKING CHANGE BREAKING CHANGE This completely removes `let-env FOO = "BAR"` so that we can focus on `$env.FOO = "BAR"`. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect -A clippy::result_large_err` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After / Before Submitting integration scripts to update: - :heavy_check_mark: [starship](https://github.com/starship/starship/blob/master/src/init/starship.nu) - :heavy_check_mark: [virtualenv](https://github.com/pypa/virtualenv/blob/main/src/virtualenv/activation/nushell/activate.nu) - :heavy_check_mark: [atuin](https://github.com/ellie/atuin/blob/main/atuin/src/shell/atuin.nu) (PR: https://github.com/ellie/atuin/pull/1080) - :x: [zoxide](https://github.com/ajeetdsouza/zoxide/blob/main/templates/nushell.txt) (PR: https://github.com/ajeetdsouza/zoxide/pull/587) - :heavy_check_mark: [oh-my-posh](https://github.com/JanDeDobbeleer/oh-my-posh/blob/main/src/shell/scripts/omp.nu) (pr: https://github.com/JanDeDobbeleer/oh-my-posh/pull/4011)
2023-06-30 19:57:51 +00:00
load-env {$path_name: []}
let target_paths = {
linux: "foo",
windows: "bar",
macos: "baz",
android: "quux",
}
path add $target_paths
assert equal (get_path) ([($target_paths | get $nu.os-info.name)] | path expand)
Surprising symlink resolution for std `path add` (#13258) # Description The standard library's `path add` function has some surprising side effects that I attempt to address in this PR: 1. Paths added, if they are symbolic links, should not be resolved to their targets. Currently, resolution happens. Imagine the following: ```nu # Some time earlier, perhaps even not by the user, a symlink is created mkdir real-dir ln -s real-dir link-dir # Then, step to now, with link-dir that we want in our PATHS variable use std path add link-dir ``` In the current implementation of `path add`, it is _not_ `link-dir` that will be added, as has been stated in the command. It is instead `real-dir`. This is surprising. Users have the agency to do this resolution if they wish with `path expand` (sans a `--no-symlink` flag): for example, `path add (link-dir | path expand)` In particular, when I was trying to set up [fnm](https://github.com/Schniz/fnm), a Node.js version manager, I was bitten by this fact when `fnm` told me that an expected path had not been added to the PATHS variable. It was looking for the non-resolved link. The user in [this comment](https://github.com/Schniz/fnm/issues/463#issuecomment-1710050737) was likely affected by this too. Shells, such as nushell, can handle path symlinks just fine. Binary lookup is unaffected. Let resolution be opt-in. Lastly, there is some convention already in place for **not** resolving path symlinks in the [default $env.ENV_CONVERSIONS table](https://github.com/nushell/nushell/blob/57452337ff4e228102433e99b4c6000700a9b3b2/crates/nu-utils/src/sample_config/default_env.nu#L65). 2. All existing paths in the path variable should be left untouched. Currently, they are `path expand`-ed (including symbolic link resolution). Path add should mean just that: prepend/append this path. Instead, it currently means that, _plus mutate all other paths in the variable_. Again, users have the agency to do this with something like `$env.PATH = $env.PATH | split row (char esep) | path expand`. 3. Minorly, I update documentation on running tests in `crates/nu-std/CONTRIBUTING.md`. The offered command to run the standard library test suite was no longer functional. Thanks to @weirdan in [this Discord conversation](https://discord.com/channels/601130461678272522/614593951969574961/1256029201119576147) for the context. # User-Facing Changes (Written from the perspective of release notes) - The standard library's `path add` function no longer resolves symlinks in either the newly added paths, nor the other paths already in the variable. # Tests + Formatting A test for the changes working correctly has been added to `crates/nu-std/tests/test_std.nu` under the test named `path_add_expand`. You can quickly verify this new test and the existing `path add` test with the following command: ```nu cargo run -- -c 'use crates/nu-std/testing.nu; NU_LOG_LEVEL=INFO testing run-tests --path crates/nu-std --test path_add' ``` All commands suggested in the issue template have been run and complete without error. # After Submitting I'll add a release note to [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged.
2024-06-28 23:11:48 +00:00
load-env {$path_name: [$"(["/foo", "/bar"] | path expand | str join (char esep))"]}
path add "~/foo"
assert equal (get_path) (["~/foo", "/foo", "/bar"] | path expand)
}
}
Surprising symlink resolution for std `path add` (#13258) # Description The standard library's `path add` function has some surprising side effects that I attempt to address in this PR: 1. Paths added, if they are symbolic links, should not be resolved to their targets. Currently, resolution happens. Imagine the following: ```nu # Some time earlier, perhaps even not by the user, a symlink is created mkdir real-dir ln -s real-dir link-dir # Then, step to now, with link-dir that we want in our PATHS variable use std path add link-dir ``` In the current implementation of `path add`, it is _not_ `link-dir` that will be added, as has been stated in the command. It is instead `real-dir`. This is surprising. Users have the agency to do this resolution if they wish with `path expand` (sans a `--no-symlink` flag): for example, `path add (link-dir | path expand)` In particular, when I was trying to set up [fnm](https://github.com/Schniz/fnm), a Node.js version manager, I was bitten by this fact when `fnm` told me that an expected path had not been added to the PATHS variable. It was looking for the non-resolved link. The user in [this comment](https://github.com/Schniz/fnm/issues/463#issuecomment-1710050737) was likely affected by this too. Shells, such as nushell, can handle path symlinks just fine. Binary lookup is unaffected. Let resolution be opt-in. Lastly, there is some convention already in place for **not** resolving path symlinks in the [default $env.ENV_CONVERSIONS table](https://github.com/nushell/nushell/blob/57452337ff4e228102433e99b4c6000700a9b3b2/crates/nu-utils/src/sample_config/default_env.nu#L65). 2. All existing paths in the path variable should be left untouched. Currently, they are `path expand`-ed (including symbolic link resolution). Path add should mean just that: prepend/append this path. Instead, it currently means that, _plus mutate all other paths in the variable_. Again, users have the agency to do this with something like `$env.PATH = $env.PATH | split row (char esep) | path expand`. 3. Minorly, I update documentation on running tests in `crates/nu-std/CONTRIBUTING.md`. The offered command to run the standard library test suite was no longer functional. Thanks to @weirdan in [this Discord conversation](https://discord.com/channels/601130461678272522/614593951969574961/1256029201119576147) for the context. # User-Facing Changes (Written from the perspective of release notes) - The standard library's `path add` function no longer resolves symlinks in either the newly added paths, nor the other paths already in the variable. # Tests + Formatting A test for the changes working correctly has been added to `crates/nu-std/tests/test_std.nu` under the test named `path_add_expand`. You can quickly verify this new test and the existing `path add` test with the following command: ```nu cargo run -- -c 'use crates/nu-std/testing.nu; NU_LOG_LEVEL=INFO testing run-tests --path crates/nu-std --test path_add' ``` All commands suggested in the issue template have been run and complete without error. # After Submitting I'll add a release note to [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged.
2024-06-28 23:11:48 +00:00
#[test]
def path_add_expand [] {
Improves startup time when using std-lib (#13842) Updated summary for commit [612e0e2](https://github.com/nushell/nushell/pull/13842/commits/612e0e21602f55092bc121bfd07f7c3bf5119e4f) - While folks are welcome to read through the entire comments, the core information is summarized here. # Description This PR drastically improves startup times of Nushell by only parsing a single submodule of the Standard Library that provides the `banner` and `pwd` commands. All other Standard Library commands and submodules are parsed when imported by the user. This cuts startup times by more than 60%. At the moment, we have stopped adding to `std-lib` because every addition adds a small amount to the Nushell startup time. With this change, we should once again be able to allow new functionality to be added to the Standard Library without it impacting `nu` startup times. # User-Facing Changes * Nushell now starts about 60% faster * Breaking change: The `dirs` (Shells) aliases will return a warning message that it will not be auto-loaded in the following release, along with instructions on how to restore it (and disable the message) * The `use std <submodule> *` syntax is available for convenience, but should be avoided in scripts as it parses the entire `std` module and all other submodules and places it in scope. The correct syntax to *just* load a submodule is `use std/<submodule> *` (asterisk optional). The slash is important. This will be documented. * `use std *` can be used for convenience to load all of the library but still incurs the full loading-time. * `std/dirs`: Semi-breaking change. The `dirs` command replaces the `show` command. This is more in line with the directory-stack functionality found in other shells. Existing users will not be impacted by this as the alias (`shells`) remains the same. * Breaking-change: Technically a breaking change, but probably only impacts maintainers of `std`. The virtual path for the standard library has changed. It could previously be imported using its virtual path (and technically, this would have been the correct way to do it): ```nu use NU_STDLIB_VIRTUAL_DIR/std ``` The path is now simply `std/`: ```nu use std ``` All submodules have moved accordingly. # Timings Comparisons below were made: * In a temporary, clean config directory using `$env.XDG_CONFIG_HOME = (mktemp -d)`. * `nu` was run with a release build * `nu` was run one time to generate the default `config.nu` (etc.) files - Otherwise timings would include the user-prompt * The shell was exited and then restarted several times to get timing samples (Note: Old timings based on 0.97 rather than 0.98, but in the range of being accurate) | Scenario | `$nu.startup-time` | | --- | --- | | 0.97.2 ([aaaab8e](https://github.com/nushell/nushell/commit/aaaab8e070c644a87bbd7682099e3fe9e6a4b42a)) Without this PR | 23ms - 24ms | | This PR with deprecated commands | 9ms - <11ms | | This PR after deprecated commands are removed in following release | 8ms - <10ms | | Final PR (remove deprecated), using `--no-std-lib` | 6.1ms to 6.4ms | | Final PR (remove deprecated), using `--no-config-file` | 3.1ms - 3.6ms | | Final PR (remove deprecated), using `--no-config-file --no-std-lib` | 1ms - 1.5ms | *These last two timings point to the opportunity for further optimization (see comment in thread below (will link once I write it).* # Implementation details for future maintenance * `use std banner` is a ridiculously deceptive call. That call parses and imports *all* of `std` into scope. Simply replacing it with `use std/core *` is essentially what saves ~14-15ms. This *only* imports the submodule with the `banner` and `pwd` commands. * From the code-comments, the reason that `NU_STDLIB_VIRTUAL_DIR` was used as a prefix was so that there wouldn't be an issue if a user had a `./std/mod.nu` in the current directory. This does **not** appear to be an issue. After removing the prefix, I tested with both a relative module as well as one in the `$env.NU_LIB_DIRS` path, and in all cases the *internal* `std` still took precedence. * By removing the prefix, users can now `use std` (and variants) without requiring that it already be parsed and in scope. * In the next release, we'll stop autoloading the `dirs` (shells) functionality. While this only costs an additional 1-1.5ms, I think it's better moved to the `config.nu` where the user can optionally remove it. The main reason is its use of aliases (which have also caused issues) - The `n`, `p`, and `g` short-commands are valuable real-estate, and users may want to map these to something else. For this release, there's an `deprecated_dirs` module that is still autoloaded. As with the top-level commands, use of these will give a deprecation warning with instructions on how to handle going forward. To help with this, moved the aliases to their own submodule inside the `dirs` module. * Also sneaks in a small change where the top-level `dirs` command is now the replacement for `dirs show` * Fixed a double-import of `assert` in `dirs.nu` * The `show_banner` step is replaced with simply `banner` rather than re-importing it. * A `virtual_path` may now be referenced with either a forward-slash or a backward-slash on Windows. This allows `use std/<submodule>` to work on all platforms. # Performance side-notes: * Future parsing and/or IR improvements should improve performance even further. * While the existing load time penalty of `std-lib` was not noticeable on many systems, Nushell runs on a wide-variety of hardware and OS platforms. Slower platforms will naturally see a bigger jump in performance here. For users starting multiple Nushell sessions frequently (e.g., `tmux`, Zellij, `screen`, et. al.) it is recommended to keep total startup time (including user configuration) under ~250ms. # Tests + Formatting * All tests are green * Updated tests: - Removed the test that confirmed that `std` was loaded (since we don't). - Removed the `shells` test since it is not autoloaded. Main `dirs.nu` functionality is tested through `stdlib-test`. - Many tests assumed that the library was fully loaded, because it was (even though we didn't intend for it to be). Fixed those tests. - Tests now import only the necessary submodules (e.g., `use std/assert`, rather than `use std assert`) - Some tests *thought* they were loading `std/log`, but were doing so improperly. This was masked by the now-fixed "load-everything-into-scope bug". Local CI would pass due the `$env.NU_LOG_<...>` variables being inherited from the calling process, but would fail in the "clean" GitHub CI environment. These tests have also been fixed. * Added additional tests for the changes # After Submitting Will update the Standard Library doc page
2024-10-03 11:28:22 +00:00
use std/assert
Surprising symlink resolution for std `path add` (#13258) # Description The standard library's `path add` function has some surprising side effects that I attempt to address in this PR: 1. Paths added, if they are symbolic links, should not be resolved to their targets. Currently, resolution happens. Imagine the following: ```nu # Some time earlier, perhaps even not by the user, a symlink is created mkdir real-dir ln -s real-dir link-dir # Then, step to now, with link-dir that we want in our PATHS variable use std path add link-dir ``` In the current implementation of `path add`, it is _not_ `link-dir` that will be added, as has been stated in the command. It is instead `real-dir`. This is surprising. Users have the agency to do this resolution if they wish with `path expand` (sans a `--no-symlink` flag): for example, `path add (link-dir | path expand)` In particular, when I was trying to set up [fnm](https://github.com/Schniz/fnm), a Node.js version manager, I was bitten by this fact when `fnm` told me that an expected path had not been added to the PATHS variable. It was looking for the non-resolved link. The user in [this comment](https://github.com/Schniz/fnm/issues/463#issuecomment-1710050737) was likely affected by this too. Shells, such as nushell, can handle path symlinks just fine. Binary lookup is unaffected. Let resolution be opt-in. Lastly, there is some convention already in place for **not** resolving path symlinks in the [default $env.ENV_CONVERSIONS table](https://github.com/nushell/nushell/blob/57452337ff4e228102433e99b4c6000700a9b3b2/crates/nu-utils/src/sample_config/default_env.nu#L65). 2. All existing paths in the path variable should be left untouched. Currently, they are `path expand`-ed (including symbolic link resolution). Path add should mean just that: prepend/append this path. Instead, it currently means that, _plus mutate all other paths in the variable_. Again, users have the agency to do this with something like `$env.PATH = $env.PATH | split row (char esep) | path expand`. 3. Minorly, I update documentation on running tests in `crates/nu-std/CONTRIBUTING.md`. The offered command to run the standard library test suite was no longer functional. Thanks to @weirdan in [this Discord conversation](https://discord.com/channels/601130461678272522/614593951969574961/1256029201119576147) for the context. # User-Facing Changes (Written from the perspective of release notes) - The standard library's `path add` function no longer resolves symlinks in either the newly added paths, nor the other paths already in the variable. # Tests + Formatting A test for the changes working correctly has been added to `crates/nu-std/tests/test_std.nu` under the test named `path_add_expand`. You can quickly verify this new test and the existing `path add` test with the following command: ```nu cargo run -- -c 'use crates/nu-std/testing.nu; NU_LOG_LEVEL=INFO testing run-tests --path crates/nu-std --test path_add' ``` All commands suggested in the issue template have been run and complete without error. # After Submitting I'll add a release note to [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged.
2024-06-28 23:11:48 +00:00
# random paths to avoid collision, especially if left dangling on failure
let real_dir = $nu.temp-path | path join $"real-dir-(random chars)"
let link_dir = $nu.temp-path | path join $"link-dir-(random chars)"
mkdir $real_dir
let path_name = if $nu.os-info.family == 'windows' {
mklink /D $link_dir $real_dir
"Path"
} else {
ln -s $real_dir $link_dir | ignore
"PATH"
}
with-env {$path_name: []} {
def get_path [] { $env | get $path_name }
path add $link_dir
Surprising symlink resolution for std `path add` (#13258) # Description The standard library's `path add` function has some surprising side effects that I attempt to address in this PR: 1. Paths added, if they are symbolic links, should not be resolved to their targets. Currently, resolution happens. Imagine the following: ```nu # Some time earlier, perhaps even not by the user, a symlink is created mkdir real-dir ln -s real-dir link-dir # Then, step to now, with link-dir that we want in our PATHS variable use std path add link-dir ``` In the current implementation of `path add`, it is _not_ `link-dir` that will be added, as has been stated in the command. It is instead `real-dir`. This is surprising. Users have the agency to do this resolution if they wish with `path expand` (sans a `--no-symlink` flag): for example, `path add (link-dir | path expand)` In particular, when I was trying to set up [fnm](https://github.com/Schniz/fnm), a Node.js version manager, I was bitten by this fact when `fnm` told me that an expected path had not been added to the PATHS variable. It was looking for the non-resolved link. The user in [this comment](https://github.com/Schniz/fnm/issues/463#issuecomment-1710050737) was likely affected by this too. Shells, such as nushell, can handle path symlinks just fine. Binary lookup is unaffected. Let resolution be opt-in. Lastly, there is some convention already in place for **not** resolving path symlinks in the [default $env.ENV_CONVERSIONS table](https://github.com/nushell/nushell/blob/57452337ff4e228102433e99b4c6000700a9b3b2/crates/nu-utils/src/sample_config/default_env.nu#L65). 2. All existing paths in the path variable should be left untouched. Currently, they are `path expand`-ed (including symbolic link resolution). Path add should mean just that: prepend/append this path. Instead, it currently means that, _plus mutate all other paths in the variable_. Again, users have the agency to do this with something like `$env.PATH = $env.PATH | split row (char esep) | path expand`. 3. Minorly, I update documentation on running tests in `crates/nu-std/CONTRIBUTING.md`. The offered command to run the standard library test suite was no longer functional. Thanks to @weirdan in [this Discord conversation](https://discord.com/channels/601130461678272522/614593951969574961/1256029201119576147) for the context. # User-Facing Changes (Written from the perspective of release notes) - The standard library's `path add` function no longer resolves symlinks in either the newly added paths, nor the other paths already in the variable. # Tests + Formatting A test for the changes working correctly has been added to `crates/nu-std/tests/test_std.nu` under the test named `path_add_expand`. You can quickly verify this new test and the existing `path add` test with the following command: ```nu cargo run -- -c 'use crates/nu-std/testing.nu; NU_LOG_LEVEL=INFO testing run-tests --path crates/nu-std --test path_add' ``` All commands suggested in the issue template have been run and complete without error. # After Submitting I'll add a release note to [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged.
2024-06-28 23:11:48 +00:00
assert equal (get_path) ([$link_dir])
}
rm $real_dir $link_dir
}
#[test]
def repeat_things [] {
Improves startup time when using std-lib (#13842) Updated summary for commit [612e0e2](https://github.com/nushell/nushell/pull/13842/commits/612e0e21602f55092bc121bfd07f7c3bf5119e4f) - While folks are welcome to read through the entire comments, the core information is summarized here. # Description This PR drastically improves startup times of Nushell by only parsing a single submodule of the Standard Library that provides the `banner` and `pwd` commands. All other Standard Library commands and submodules are parsed when imported by the user. This cuts startup times by more than 60%. At the moment, we have stopped adding to `std-lib` because every addition adds a small amount to the Nushell startup time. With this change, we should once again be able to allow new functionality to be added to the Standard Library without it impacting `nu` startup times. # User-Facing Changes * Nushell now starts about 60% faster * Breaking change: The `dirs` (Shells) aliases will return a warning message that it will not be auto-loaded in the following release, along with instructions on how to restore it (and disable the message) * The `use std <submodule> *` syntax is available for convenience, but should be avoided in scripts as it parses the entire `std` module and all other submodules and places it in scope. The correct syntax to *just* load a submodule is `use std/<submodule> *` (asterisk optional). The slash is important. This will be documented. * `use std *` can be used for convenience to load all of the library but still incurs the full loading-time. * `std/dirs`: Semi-breaking change. The `dirs` command replaces the `show` command. This is more in line with the directory-stack functionality found in other shells. Existing users will not be impacted by this as the alias (`shells`) remains the same. * Breaking-change: Technically a breaking change, but probably only impacts maintainers of `std`. The virtual path for the standard library has changed. It could previously be imported using its virtual path (and technically, this would have been the correct way to do it): ```nu use NU_STDLIB_VIRTUAL_DIR/std ``` The path is now simply `std/`: ```nu use std ``` All submodules have moved accordingly. # Timings Comparisons below were made: * In a temporary, clean config directory using `$env.XDG_CONFIG_HOME = (mktemp -d)`. * `nu` was run with a release build * `nu` was run one time to generate the default `config.nu` (etc.) files - Otherwise timings would include the user-prompt * The shell was exited and then restarted several times to get timing samples (Note: Old timings based on 0.97 rather than 0.98, but in the range of being accurate) | Scenario | `$nu.startup-time` | | --- | --- | | 0.97.2 ([aaaab8e](https://github.com/nushell/nushell/commit/aaaab8e070c644a87bbd7682099e3fe9e6a4b42a)) Without this PR | 23ms - 24ms | | This PR with deprecated commands | 9ms - <11ms | | This PR after deprecated commands are removed in following release | 8ms - <10ms | | Final PR (remove deprecated), using `--no-std-lib` | 6.1ms to 6.4ms | | Final PR (remove deprecated), using `--no-config-file` | 3.1ms - 3.6ms | | Final PR (remove deprecated), using `--no-config-file --no-std-lib` | 1ms - 1.5ms | *These last two timings point to the opportunity for further optimization (see comment in thread below (will link once I write it).* # Implementation details for future maintenance * `use std banner` is a ridiculously deceptive call. That call parses and imports *all* of `std` into scope. Simply replacing it with `use std/core *` is essentially what saves ~14-15ms. This *only* imports the submodule with the `banner` and `pwd` commands. * From the code-comments, the reason that `NU_STDLIB_VIRTUAL_DIR` was used as a prefix was so that there wouldn't be an issue if a user had a `./std/mod.nu` in the current directory. This does **not** appear to be an issue. After removing the prefix, I tested with both a relative module as well as one in the `$env.NU_LIB_DIRS` path, and in all cases the *internal* `std` still took precedence. * By removing the prefix, users can now `use std` (and variants) without requiring that it already be parsed and in scope. * In the next release, we'll stop autoloading the `dirs` (shells) functionality. While this only costs an additional 1-1.5ms, I think it's better moved to the `config.nu` where the user can optionally remove it. The main reason is its use of aliases (which have also caused issues) - The `n`, `p`, and `g` short-commands are valuable real-estate, and users may want to map these to something else. For this release, there's an `deprecated_dirs` module that is still autoloaded. As with the top-level commands, use of these will give a deprecation warning with instructions on how to handle going forward. To help with this, moved the aliases to their own submodule inside the `dirs` module. * Also sneaks in a small change where the top-level `dirs` command is now the replacement for `dirs show` * Fixed a double-import of `assert` in `dirs.nu` * The `show_banner` step is replaced with simply `banner` rather than re-importing it. * A `virtual_path` may now be referenced with either a forward-slash or a backward-slash on Windows. This allows `use std/<submodule>` to work on all platforms. # Performance side-notes: * Future parsing and/or IR improvements should improve performance even further. * While the existing load time penalty of `std-lib` was not noticeable on many systems, Nushell runs on a wide-variety of hardware and OS platforms. Slower platforms will naturally see a bigger jump in performance here. For users starting multiple Nushell sessions frequently (e.g., `tmux`, Zellij, `screen`, et. al.) it is recommended to keep total startup time (including user configuration) under ~250ms. # Tests + Formatting * All tests are green * Updated tests: - Removed the test that confirmed that `std` was loaded (since we don't). - Removed the `shells` test since it is not autoloaded. Main `dirs.nu` functionality is tested through `stdlib-test`. - Many tests assumed that the library was fully loaded, because it was (even though we didn't intend for it to be). Fixed those tests. - Tests now import only the necessary submodules (e.g., `use std/assert`, rather than `use std assert`) - Some tests *thought* they were loading `std/log`, but were doing so improperly. This was masked by the now-fixed "load-everything-into-scope bug". Local CI would pass due the `$env.NU_LOG_<...>` variables being inherited from the calling process, but would fail in the "clean" GitHub CI environment. These tests have also been fixed. * Added additional tests for the changes # After Submitting Will update the Standard Library doc page
2024-10-03 11:28:22 +00:00
use std/assert
assert error { "foo" | repeat -1 }
for x in ["foo", [1 2], {a: 1}] {
assert equal ($x | repeat 0) []
assert equal ($x | repeat 1) [$x]
assert equal ($x | repeat 2) [$x $x]
}
}