This adds new pipeline connectors called out> and err> which redirect either stdout or stderr to a file. You can also use out+err> (or err+out>) to redirect both streams into a file.
Alters `all`, `any`, `each while`, `each`, `insert`, `par-each`, `reduce`, `update`, `upsert` and `where`,
so that their blocks take an optional parameter containing the index.
* add signature information when help on one command
* tell user that one command support operated on cell paths
Also, make type output to be more friendly, like `record<>` should just be `record`
And the same to `table<>`, which should be `table`
* simplify code
* don't show signatures for parser keyword
* update comment
* output arg syntax shape as type, so it's the same as describe command
* fix string when no positional args
* update signature body
* update
* add help signature test
* fix arg output format for composed data type like list or record
* fix clippy
* add comment
This adds support for (limited) mutable variables. Mutable variables are created with mut much the same way immutable variables are made with let.
Mutable variables allow mutation via the assignment operator (=).
❯ mut x = 100
❯ $x = 200
❯ print $x
200
Mutable variables are limited in that they're only tended to be used in the local code block. Trying to capture a local variable will result in an error:
❯ mut x = 123; {|| $x }
Error: nu::parser::expected_keyword (link)
× Capture of mutable variable.
The intent of this limitation is to reduce some of the issues with mutable variables in general: namely they make code that's harder to reason about. By reducing the scope that a mutable variable can be used it, we can help create local reasoning about them.
Mutation can occur with fields as well, as in this case:
❯ mut y = {abc: 123}
❯ $y.abc = 456
❯ $y
On a historical note: mutable variables are something that we resisted for quite a long time, leaning as much as we could on the functional style of pipelines and dataflow. That said, we've watched folks struggle to work with reduce as an approximation for patterns that would be trivial to express with local mutation. With that in mind, we're leaning towards the happy path.
* Fix ignore-errors for select
* fix Value::List match
* fix invalid rows
* add tests
* fix ListStream match
* add one more test for ListStream
* add more tests
* tweak words
* fix: fixcd
try to fix
Log: try to fix the bug with can enter a permisson error fold
* change wording
* fat
* fmt
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
* make format filesize more flexible
* make code simpler
* finish refactor on bytes commands
* finish refactor on str commands
* fimplify code
* rename from column_paths to cell_paths
* fix: ✨ "saner" default for mv
fixes#6747
As highlighted in the issue, the default behavior of nu currently
is to overwrite the destination file without notice.
This is not a "standard" expectation users that want this behavior
can create a dedicated alias.
* fix: 📝 edit the comment
* fix: ✨ updated the tests
* fix: 🚧 use --force for case test
* Remove unnecessary `#[allow]` annots
Reduce the number of lint exceptions that are not necessary with the
current state of the code (or more recent toolchain)
* Remove dead code from `FileStructure` in nu-command
* Replace `allow(unused)` with relevant feature switch
* Deal with `needless_collect` with annotations
* Change hack for needless_collect in `from json`
This change obviates the need for `allow(needless_collect)`
Removes a pessimistic allocation for empty strings, but increases
allocation size to `Value`
Probably not really worth it.
* Revert "Deal with `needless_collect` with annotations"
This reverts commit 05aca98445.
The previous state seems to better from a performance perspective as a
`Vec<String>` is lighter weight than `Vec<Value>`
* Filter out empty glob patterns
An empty argument to the "glob" command will now produce an empty result.
Working towards nushell/nushell#6653.
* Run `cargo fmt --all`
Just autoformatted the repo so that CI passes and we have a consistent code
format across modules.
* Treat empty glob argument as error
The glob command will now report an empty string argument as an error instead
of silently ignoring it.
See https://github.com/nushell/nushell/pull/6707#discussion_r993345013.
* Add tests for glob command
Two small tests for the glob command, one to check that the empty string errors
it, and another to sanity check the '*' glob, have been added.
* Rename glob sanity check star test
Co-authored-by: Kyle Anderson <kyle.anderson@uwaterloo.ca>
* remove export_env command
* remove several export env usage in test code
* adjust hiding relative test case
* fix clippy
* adjust tests
* update tests
* unignore these tests to expose ut failed
* using `use` instead of `overlay use` in some tests
* Revert "using `use` instead of `overlay use` in some tests"
This reverts commit 2ae24b24c3.
* Revert "adjust hiding relative test case"
This reverts commit 4369af6d05.
* Bring back module example
* Revert "update tests"
This reverts commit 6ae94ef513.
* Fix tests
* "Fix" a test
* Remove remaining deprecated env functionality
* Re-enable environment hiding for `hide`
To not break virtualenv since the overlay update is not merged yet
* Fix hiding env in `hide` and ignore some tests
Co-authored-by: kubouch <kubouch@gmail.com>
* Copy lev_distance.rs from the rust compiler
* Minor changes to code from rust compiler
* "Did you mean" suggestions: test instrumented to generate markdown report
* Did you mean suggestions: delete test instrumentation
* Fix tests
* Fix test
`foo` has a genuine match: `for`
* Improve tests
* default to if cellpath not found
* fmt
* add test
* fix test
* fix clippy
* move behaviour behind `-i` flag
* prevent any possibility of an unspanned error
* ignore all errors
* seperate testes
* fmt
* Add support for Arrow IPC file format
Add support for Arrow IPC file format to dataframes commands. Support
opening of Arrow IPC-format files with extension '.arrow' or '.ipc' in
the open-df command. Add a 'to arrow' command to write a dataframe to
Arrow IPC format.
* Add unit test for open-df on Arrow
* Add -t flag to open-df command
Add a `--type`/`-t` flag to the `open-df` command, to explicitly specify
the type of file being used. Allowed values are the same at the set of
allowed file extensions.
* Initialize join.rs as a copy of collect.rs
* Evolve StrCollect into StrJoin
* Replace 'str collect' with 'str join' everywhere
git ls-files | lines | par-each { |it| sed -i 's,str collect,str join,g' $it }
* Deprecate 'str collect'
* Revert "Deprecate 'str collect'"
This reverts commit 959d14203e.
* Change `str collect` help message to say that it is deprecated
We cannot remove `str collect` currently (i.e. via
`nu_protocol::ShellError::DeprecatedCommand` since a prominent project
uses the API:
b85542c31c/src/virtualenv/activation/nushell/activate.nu (L43)
Rename `all?`, `any?` and `empty?` to `all`, `any` and `is-empty` for sake of simplicity and consistency.
- More understandable for newcomers, that these commands are no special to others.
- `?` syntax did not really aprove readability. For me it made it worse.
- We can reserve `?` syntax for any other nushell feature.
* Add source-env test for dynamic path
* Use correct module ID for env overlay imports
* Remove parser check from "overlay list"
It would cause unnecessary errors from some inner scope if some
overlay module was also defined in some inner scope.
* Restore Cargo.lock back
* Remove comments
* add new function to remove data at a cellpath; allow reject to use cellpath
* add tests
* fmt
* fix clippt
* get it working properly with lists of records
* fix clippy, hopefully
* fix clippy, hopefully 2
* start working on source-env
* WIP
* Get most tests working, still one to go
* Fix file-relative paths; Report parser error
* Fix merge conflicts; Restore source as deprecated
* Tests: Use source-env; Remove redundant tests
* Fmt
* Respect hidden env vars
* Fix file-relative eval for source-env
* Add file-relative eval to "overlay use"
* Use FILE_PWD only in source-env and "overlay use"
* Ignore new tests for now
This will be another issue
* Throw an error if setting FILE_PWD manually
* Fix source-related test failures
* Fix nu-check to respect FILE_PWD
* Fix corrupted spans in source-env shell errors
* Fix up some references to old source
* Remove deprecation message
* Re-introduce deleted tests
Co-authored-by: kubouch <kubouch@gmail.com>
* Add decimals to int when using `into string --decimals`
* Add tests for `into string` when converting int with `--decimals`
* Apply formatting
* Merge `into_str` test files
* Comment out unused code and add TODOs
* Use decimal separator depending on system locale
* Add test helper to run closure in different locale
* Add tests for int-to-string conversion using different locales
* Add utils function to get system locale
* Add panic message when locking mutex fails
* Catch and resume panic later to prevent Mutex poisoning when test fails
* Move test to `nu-test-support` to keep `nu-utils` free of `nu-*` dependencies
See https://github.com/nushell/nushell/pull/6085#issuecomment-1193131694
* Rename test support fn `with_fake_locale` to `with_locale_override`
* Move `get_system_locale()` to `locale` module
* Allow overriding locale with special env variable (when not in release)
* Use special env var to override locale during testing
* Allow callback to return a value in `with_locale_override()`
* Allow multiple options in `nu!` macro
* Allow to set locale as `nu!` macro option
* Use new `locale` option of `nu!` macro instead of `with_locale_override`
Using the `locale` options does not lock the `LOCALE_OVERRIDE_MUTEX`
mutex in `nu-test-support::locale_override` but instead calls the `nu`
command directly with the `NU_LOCALE_OVERRIDE` environment variable.
This allows for parallel test excecution.
* Fix: Add option identifier for `cwd` in usage of `nu!` macro
* Rely on `Display` trait for formatting `nu!` macro command
- Removed the `DisplayPath` trait
- Implement `Display` for `AbsolutePath`, `RelativePath` and
`AbsoluteFile`
* Default to locale `en_US.UTF-8` for tests when using `nu!` macro
* Add doc comment to `nu!` macro
* Format code using `cargo fmt --all`
* Pass function directly instead of wrapping the call in a closure
https://rust-lang.github.io/rust-clippy/master/index.html#redundant_closure
* Pass function to `or_else()` instead of calling it inside `or()`
https://rust-lang.github.io/rust-clippy/master/index.html#or_fun_call
* Fix: Add option identifier for `cwd` in usage of `nu!` macro
* add -p for path expand, so it doesn't follow symlink
* fix arg name
* rename from no-dereferenct to no-follow-link
* rename from no-follow-link to no-symlink, and change short -p to -n
* follow strict first
* fix
* simplify test
* fix clippy
* fix test on windows
The test was reading the operating system error message which is
dependent on the system locale.
Just test for the `(os error 2)` errorcode instead.
This should support both
unixoid systems and Windows in more locales.
* Fix ls panics when a file or directory not exists
Fixes#6146
Signed-off-by: nibon7 <nibon7@163.com>
* add test
Signed-off-by: nibon7 <nibon7@163.com>
* expand durations to include month, year, decade
* remove commented out fn
* oops, found more debug comments
* tweaked tests for the new way, borrowed heavily from chrono-humanize-rs
* clippy
* grammar
* introduce external command runs to failed error, and implement semicolon relative logic
* ignore test due to semicolon works
* not raise ShellError for external commands
* update comment
* add relative test in for windows
* fix type-o
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
* Fix 5978
* Add unit test for explicit glob
* Format
* Expansion vs none-expansion
* Add unit tests
* Fix format..
* Add debug message for MacOS
* Fix UT on Mac and add tests for windows
* cleanup
* clean up windows test
* single and double qoutes tests
* format...
* Save format.
* Add log to failed windows unit tests
* try `touch` a file
* PS or CMD
* roll back some change
* format
* Remove log and test case
* Add unit test comments
* Fix
Co-authored-by: Frank <v-frankz@microsoft.com>
* Avoid extending the directory without globs in `nu_engine::glob_from`
* avoid joining a `*` to the directory without globs
* remove checks on directory permission and whether it is empty
The previous implemention of `nu_engine::glob_from` will extend the
given directory even if it containes no glob pattern. This commit
overcomes lack of consistency with the function `nu_glob::glob`.
* Add flag -D to ls, to list the directory itself instead of its contents
* add --directory (-d) flag to ls
* correct the difference between the given path and the cwd
* set default path to `.` instead of `./*` when --directory (-d) flag is true
* add comments
* add an example
* add tests
* fmt
* Add all flag
* Make all and moduel flags as mutually exclusive
* Fix new test
* format code...
* tweak words
* another tweak
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
* adds `capture-errors` flag for `do`
* adds `get-type` core command to get type
* fmt
* add tests in example
* fmt
* fix tests
* manually revert previous changes related to `get-type`
* adds method to check for error name using `into string`
* fix clippy
* feat: deprecate `hash base64` command
* feat: extend `decode` and `encode` command families
This commit
- Adds `encode` command family
- Backports `hash base64` features to `encode base64` and `decode base64` subcommands.
- Refactors code a bit and extends tests for encodings
- `decode base64` returns a binary `Value` (that may be decoded into a string using `decode` command)
* feat: add `--binary(-b)` flag to `decode base64`
Default output type is now string, but binary can be requested using this new flag.
* attempts to add `div` math operator
* allows `//` to be used too
* fmt:
* clippy issue
* returns appropriate type
* returns appropriate type 2
* fmt
* ensure consistency; rename to `fdiv`
* Update parser.rs
* Fix `ls` for Windows system files
* Fix non-Windows builds
* Make Clippy happy on non-Windows platforms
* Fix new test on GitHub runners
* Move ls Windows code into its own module
* cp with no dangling link
* add -p to not follow symbolic link
* change comment
* add one more test case to check symblink body after copied
* better help message
* Cleanup - remove old commented code
* Force a / or \ to distinguish between folders and files for cp
* Force a / or \ to distinguish between folders and files for cp
* Remove unneeded code
* Add cp test for checking copy to non existing directory
* Fix warning in test
* Change miette theme based on ANSI config
Use the base ansi colors to simplify the use of the terminal emulator
theming.
Turn of most eye-candy (including unicode) when using
`$config.use_ansi_coloring: false`
Addresses #5582
* Fix error test affected by changed styling
* one step closer to flatten
* integration code is passing, but still need to do one more level flatten for table
* fix flatten
* using match instead of several if let
* make better comment
* fmt code
* better comment
* database commands
* db commands
* filesystem opens sqlite file
* clippy error
* corrected error in ci file
* removes matrix flag from ci
* flax matrix for clippy
* add conditional compile for tests
* add conditional compile for tests
* correct order of command
* correct error msg
* correct typo
* Add timestamp flag to `touch` command
* Add modify flag to `touch` command
* Add date flag to `touch` command
* Remove unnecessary `touch` test and fix tests setups
* Change `touch` flags descriptions
* Update `touch` example
* Add reference flag to `touch` command
* Add access flag to `touch` command
* Add no-create flag to `touch` command
* Replace `unwrap` with `expect`
* Remove panic from BlockCommands run function
Instead of panicing, the run method now returns an error to prevent
nushell from unexpected termination.
* Add ability to open command to run with blocks
The open command tries to parse the content of the file
if there is a command called 'from (file ending)'. This works
fine if the command was 'built in' because the run method doesn't
fail in this case. It did fail on a BlockCommand, though.
This change will first probe if the command contains a block and
evaluate it, if this is the case. If there is no block, it will run
the command the same way as before.
* Add test open files with BlockCommands
* Update open.rs
* Adjust file type on open with BlockCommand parser
Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
* Add streaming support to save for ExternalStream data
Prior to this change, save would collect data from an ExternalStream (data
originating from externals) consuming memory for the full amount of data piped
to it,
This change adds streaming support for ExternalStream allowing saving of
arbitrarily large files and bounding memory usage.
* Remove broken save test
This test passes but not for the right reasons, since this test was
written filename has become a required parameter. The parser outputs
an error but the test still passes as is checking the original un-mutated
file assuming save has re-written the contents.
This change removes the test.
```
running 1 test
=== stderr
Error: nu::parser::missing_positional (https://docs.rs/nu-parser/0.60.0/nu-parser/enum.ParseError.html#variant.MissingPositional)
× Missing required positional argument.
╭─[source:1:1]
1 │ open save_test_1/cargo_sample.toml | save
· ▲
· ╰── missing filename
╰────
help: Usage: save {flags} <filename>
test commands::save::figures_out_intelligently_where_to_write_out_with_metadata ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 515 filtered out; finished in 0.10s
```
* Refactor & fix which
Instead of fetching all definitions / aliases, only show the one that is
visible.
* Fix $nu.scope to show only visible definitions
* Add missing tests file; Rename one which test
* Add different features combinations
* Specify styles manually
* Fix args
* Fix typo
* Let other CI jobs finish if one fails
* Fix unused symbols without plugin feature
* Put "which" tests behind "which" feature
* Add Python virtualenv job
* Oops forgot git command
* Install Nushell in virtualenv tests
* Add names to steps; Test v.env in separate step
* cd into virtualenv
* Do not run on Python 2.7
* Build Nushell after formatting and clippy checks
* Fix "index out of bounds" when input to group-by is empty #4369
* Fix formatting #4369
* Adds test for empty input
Co-authored-by: Ray Henry <ray.henry@thermofisher.com>
* Allow range in 'drop nth'
* Unit tests for drop nth range
* Add range case to the description
* Fix description 2
* format fixes
* Fix example and some refactoring
* clippy fixes
* fix into filesize tests and filesize
* tweaks
* added span back for like the 10th time
* Update filesize.rs
Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
* have save --append create file if not exists
Currently, doing:
echo a | save --raw --append file.txt
will fail if file.txt does not exist. This PR changes that
* test that `save --append` will create new file
* Change path join signature
* Appending now works without flag
* Column path operation is behind a -c flag
* Move column path arg retrieval to a function
Also improves errors
* Fix path join tests
* Propagate column path changes to all path commands
* Update path command examples with columns paths
* Modernize path command examples by removing "echo"
* Improve structured path error message
* Fix typo
* Add subcommand `into filesize`
It's currently not possible to convert a number or a string containing a number
into a filesize. The only way to create an instance of filesize type today is
with a literal in nushell syntax. This commit adds the `into filesize`
subcommand so that file sizes can be created from the outputs of programs
producing numbers or strings, like standard unix tools.
There is a limitation with this - it doesn't currently parse values like `10 MB`
or `10 MiB`, it can only look at the number itself. If the desire is there, more
flexible parsing can be added.
* fixup! Add subcommand `into filesize`
* fixup! Add subcommand `into filesize`
* Allow environment variables to be hidden
This change allows environment variables in Nushell to have a value of
`Nothing`, which can be set by the user by passing `$nothing` to
`let-env` and friends.
Environment variables with a value of Nothing behave as if they are not
set at all. This allows a user to shadow the value of an environment
variable in a parent scope, effectively removing it from their current
scope. This was not possible before, because a scope can not affect its
parent scopes.
This is a workaround for issues like #3920.
Additionally, this allows a user to simultaneously set, change and
remove multiple environment variables via `load-env`. Any environment
variables set to $nothing will be hidden and thus act as if they are
removed. This simplifies working with virtual environments, which rely
on setting multiple environment variables, including PATH, to specific
values, and remove/change them on deactivation.
One surprising behavior is that an environment variable set to $nothing
will act as if it is not set when querying it (via $nu.env.X), but it is
still possible to remove it entirely via `unlet-env`. If the same
environment variable is present in the parent scope, the value in the
parent scope will be visible to the user. This might be surprising
behavior to users who are not familiar with the implementation details.
An additional corner case is the the shorthand form of `with-env` does
not work with this feature. Using `X=$nothing` will set $nu.env.X to the
string "$nothing". The long-form works as expected: `with-env [X
$nothing] {...}`.
* Remove unused import
* Allow all primitives to be convert to strings
Given we can write nu scripts. As the codebase grows, splitting into many smaller nu scripts is necessary.
In general, when we work with paths and files we seem to face quite a few difficulties. Here we just tackle one of them and it involves sourcing
files that also source other nu files and so forth. The current working directory becomes important here and being on a different directory
when sourcing scripts will not work. Mostly because we expand the path on the current working directory and parse the files when a source command
call is done.
For the moment, we introduce a `lib_dirs` configuration value and, unfortunately, introduce a new dependency in `nu-parser` (`nu-data`) to get
a handle of the configuration file to retrieve it. This should give clues and ideas as the new parser engine continues (introduce a way to also know paths)
With this PR we can do the following:
Let's assume we want to write a nu library called `my_library`. We will have the code in a directory called `project`: The file structure will looks like this:
```
project/my_library.nu
project/my_library/hello.nu
project/my_library/name.nu
```
This "pattern" works well, that is, when creating a library have a directory named `my_library` and next to it a `my_library.nu` file. Filling them like this:
```
source my_library/hello.nu
source my_library/name.nu
```
```
def hello [] {
"hello world"
}
```
```
def name [] {
"Nu"
end
```
Assuming this `project` directory is stored at `/path/to/lib/project`, we can do:
```
config set lib_dirs ['path/to/lib/project']
```
Given we have this `lib_dirs` configuration value, we can be anywhere while using Nu and do the following:
```
source my_library.nu
echo (hello) (name)
```
* Allow sourcing paths with emojis
* Add source command tests for emoji paths
* Fmt
* Disable source tests on Windows with illegal paths
* Test sourcing also ASCII and single-quoted paths
We introduce it here and allow it to work with regular lists (tables with no columns) as well as symmetric tables. Say we have two lists and wish to zip them, like so:
```
[0 2 4 6 8] | zip {
[1 3 5 7 9]
} | flatten
───┬───
0 │ 0
1 │ 1
2 │ 2
3 │ 3
4 │ 4
5 │ 5
6 │ 6
7 │ 7
8 │ 8
9 │ 9
───┴───
```
In the case for two tables instead:
```
[[symbol]; ['('] ['['] ['{']] | zip {
[[symbol]; [')'] [']'] ['}']]
} | each {
get symbol | $'($in.0)nushell($in.1)'
}
───┬───────────
0 │ (nushell)
1 │ [nushell]
2 │ {nushell}
───┴───────────
```
* Refactor Hash code to simplify md5 and sha256 implementations
Md5 and Sha256 (and other future digests) require less boilerplate code
now. Error reporting includues the name of the hash again.
* Add missing hash sha256 test
* Fix swapped PATH env var separators
* Support pathvar to manipulate other vars than PATH
* Add tests for pathvar and its subcommands
* Adjust pathvar tests to comply with env quirks
* Make pathvar tests work on non-Windows as well
* Compact the comments in pathvar tests
* Fix last failing test
Co-authored-by: Jakub Žádník <jakub.zadnik@tuni.fi>
* Fix swapped PATH env var separators
* Support pathvar to manipulate other vars than PATH
* Add tests for pathvar and its subcommands
* Fix PATH env name for Windows
Seems like Windows uses PATH as well.
Co-authored-by: Jakub Žádník <jakub.zadnik@tuni.fi>
* Output error when ls into a file without permission
* math sqrt
* added test to check fails when ls into prohibited dir
* fix lint
* math sqrt with tests and doc
* trigger wasm build
* Update filesystem_shell.rs
* Fix Running echo .. starts printing integers forever
* Fixed panic on operations with very large durations
Co-authored-by: Jonathan Turner <jonathandturner@users.noreply.github.com>
