Commit graph

491 commits

Author SHA1 Message Date
Jason Gedge
dcdfa2a866
Improve tests and labeling in FilesystemShell (#1305)
Additional `ls` command tests and better FilesystemShell error and label messages.
2020-02-01 03:34:34 -05:00
Shaurya Shubham
49a1385543
Make tests work from directory names with spaces (#1325) 2020-01-31 22:12:56 -08:00
Shaurya Shubham
dc3370b103
Make a calc command (#1280) 2020-01-29 08:34:36 -05:00
Jason Gedge
32dfb32741 Switch from subprocess crate to the builtin std::process (#1284)
* Switch from subprocess crate to the builtin std::process

* Update external.rs

* Update external.rs

* Update external.rs

Co-authored-by: Jonathan Turner <jonathandturner@users.noreply.github.com>
2020-01-26 16:03:21 +13:00
Borimino
583f27dc41 Added attributes to from-xml command (#1272)
* Added attributes to from-xml command

* Added attributes as their own rows

* Removed unneccesary lifetime declarations

* from-xml now has children and attributes side by side

* Fixed tests and linting

* Fixed lint-problem
2020-01-26 05:16:40 +13:00
Corvus Corax
a5e1372bc2 RM error on bad filename (#1244)
* rm error on bad filename

* De-lint

* Fix error message in test
2020-01-25 08:16:41 +13:00
Jonathan Turner
798a24eda5
Soften restrictions for external parameters (#1277)
* Soften restrictions for external parameters

* Add test
2020-01-25 08:14:49 +13:00
Jonathan Turner
d38a63473b
Improve shelling out (#1273)
Improvements to shelling out
2020-01-24 08:24:31 +13:00
Jonathan Turner
2b37ae3e81
Switch to using subprocess::shell (#1264)
* Switch to using `shell`

Switch to using the shell for subprocess to enable more natural shelling out.

* Update external.rs

* This is a test with .shell() for external

* El pollo loco's PR

* co co co

* Attempt to fix windows

* Fmt

* Less is more?

Co-authored-by: Andrés N. Robalino <andres@androbtech.com>
2020-01-24 05:21:05 +13:00
Andrés N. Robalino
762c798670
It ls test setup rewrite. (#1260) 2020-01-21 22:56:12 -05:00
Yehuda Katz
7efb31a4e4 Restructure and streamline token expansion (#1123)
Restructure and streamline token expansion

The purpose of this commit is to streamline the token expansion code, by
removing aspects of the code that are no longer relevant, removing
pointless duplication, and eliminating the need to pass the same
arguments to `expand_syntax`.

The first big-picture change in this commit is that instead of a handful
of `expand_` functions, which take a TokensIterator and ExpandContext, a
smaller number of methods on the `TokensIterator` do the same job.

The second big-picture change in this commit is fully eliminating the
coloring traits, making coloring a responsibility of the base expansion
implementations. This also means that the coloring tracer is merged into
the expansion tracer, so you can follow a single expansion and see how
the expansion process produced colored tokens.

One side effect of this change is that the expander itself is marginally
more error-correcting. The error correction works by switching from
structured expansion to `BackoffColoringMode` when an unexpected token
is found, which guarantees that all spans of the source are colored, but
may not be the most optimal error recovery strategy.

That said, because `BackoffColoringMode` only extends as far as a
closing delimiter (`)`, `]`, `}`) or pipe (`|`), it does result in
fairly granular correction strategy.

The current code still produces an `Err` (plus a complete list of
colored shapes) from the parsing process if any errors are encountered,
but this could easily be addressed now that the underlying expansion is
error-correcting.

This commit also colors any spans that are syntax errors in red, and
causes the parser to include some additional information about what
tokens were expected at any given point where an error was encountered,
so that completions and hinting could be more robust in the future.

Co-authored-by: Jonathan Turner <jonathandturner@users.noreply.github.com>
Co-authored-by: Andrés N. Robalino <andres@androbtech.com>
2020-01-21 17:45:03 -05:00
Koenraad Verheyden
e059c74a06 Add support for primitive values to sort-by (#1241)
* Remove redundant clone

* Add support for primitive values to sort-by #1238
2020-01-20 08:08:36 +13:00
Jason Gedge
47d987d37f Add ctrl_c to RunnablePerItemContext. (#1239)
Also, this commit makes `ls` a per-item command.

A command that processes things item by item may still take some time to stream
out the results from a single item. For example, `ls` on a directory with a lot
of files could be interrupted in the middle of showing all of these files.
2020-01-19 15:25:07 +13:00
Andrés N. Robalino
29431e73c2
Externals now spawn independently. (#1230)
This commit changes the way we shell out externals when using the `"$it"` argument. Also pipes per row to an external's stdin if no `"$it"` argument is present for external commands. 

Further separation of logic (preparing the external's command arguments, getting the data for piping, emitting values, spawning processes) will give us a better idea for lower level details regarding external commands until we can find the right abstractions for making them more generic and unify within the pipeline calling logic of Nu internal's and external's.
2020-01-16 04:05:53 -05:00
Andrés N. Robalino
78a644da2b
Restrict Nu with a cleaned environment. (#1222) 2020-01-13 23:17:20 -05:00
Jonathan Turner
5fd3191d91
Fix randomly failing test (#1200)
* Fix randomly failing test

* Fix randomly failing test
2020-01-13 06:03:28 +13:00
Andrés N. Robalino
60043df917
Allow ColumnPaths when picking tables. (#1191) 2020-01-11 01:45:09 -05:00
Andrés N. Robalino
6d3a30772d
Get error message improvements. (#1185)
More especific "get" command error messages + Test refactoring.
2020-01-10 10:44:24 -05:00
Jason Gedge
7451414b9e Eliminate ClassifiedInputStream in favour of InputStream. (#1056) 2020-01-07 13:00:01 -08:00
Shaurya Shubham
b574dc6365 Add the from-ods command (#1161)
* Put a sample_data.ods file for testing

This is a copy of the sample_data.xlsx file but in ods format

* Add the from-ods command

Most of the work was doing `rg xlsx` and then copy/paste with light editing

* Add tests for the from-ods command

* Fix failing test

The problem was improper filename sorting in the test `prepares_and_decorates_filesystem_source_files`
2020-01-07 19:35:00 +13:00
Jonathan Turner
339a2de0eb
More ununwraps (#1152)
* More ununwraps

* More ununwraps

* Update completer.rs

* Update completer.rs
2020-01-03 06:51:20 +13:00
Jonathan Turner
72838cc083
Move to using clippy (#1142)
* Clippy fixes

* Finish converting to use clippy

* fix warnings in new master

* fix windows

* fix windows

Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
2019-12-31 20:36:08 +13:00
Ryan Blecher
f37f29b441 Add uniq command (#1132)
* start playing with ways to use the uniq command

* WIP

* Got uniq working, but still need to figure out args issue and add tests

* Add some tests for uniq

* fmt

* remove commented out code

* Add documentation and some additional tests showing uniq values and rows. Also removed args TODO

* add changes that didn't get committed

* whoops, I didn't save the docs correctly...

* fmt

* Add a test for uniq with nested json

* Add another test

* Fix unique-ness when json keys are out of order and make the test json more complicated
2019-12-31 17:05:02 +13:00
Jonathan Turner
10368d7060 UTF8 fix for twitter-reported issue 2019-12-27 19:25:44 +13:00
Jonathan Turner
68a314b5cb UTF8 fix for twitter-reported issue 2019-12-27 19:03:00 +13:00
Jonathan Turner
4322d373e6 More renames 2019-12-18 07:54:39 +13:00
Andrés N. Robalino
5e961815fc can contain a string line or plain string data. 2019-12-16 17:27:36 -05:00
Andrés N. Robalino
4034129dba This commit is the continuing phase of extracting functionality to subcrates. We extract test helpers and begin to change Nu shell's test organization along with it. 2019-12-15 11:34:58 -05:00
Jonathan Turner
550bda477b
Merge pull request #1060 from naufraghi/issues-972-expand-tilde-as-home-in-external-commands
Expand tilde as home in external commands
2019-12-13 08:46:08 -08:00
Matteo Bertini
219b7e64cd Use shellexpand to expand ~ in external commands
Add tests for ~tilde expansion:

- test that "~" is expanded (no more "~" in output)
- ensure that "1~1" is not expanded to "1/home/user1" as it was
  before

Fixes #972

Note: the first test does not check the literal expansion because
the path on Windows is expanded as a Linux path, but the correct
expansion may come for free once `shellexpand` will use the `dirs`
crate too (https://github.com/netvl/shellexpand/issues/3).
2019-12-13 11:54:41 +01:00
Yehuda Katz
57af9b5040 Add Range and start Signature support
This commit contains two improvements:

- Support for a Range syntax (and a corresponding Range value)
- Work towards a signature syntax

Implementing the Range syntax resulted in cleaning up how operators in
the core syntax works. There are now two kinds of infix operators

- tight operators (`.` and `..`)
- loose operators

Tight operators may not be interspersed (`$it.left..$it.right` is a
syntax error). Loose operators require whitespace on both sides of the
operator, and can be arbitrarily interspersed. Precedence is left to
right in the core syntax.

Note that delimited syntax (like `( ... )` or `[ ... ]`) is a single
token node in the core syntax. A single token node can be parsed from
beginning to end in a context-free manner.

The rule for `.` is `<token node>.<member>`. The rule for `..` is
`<token node>..<token node>`.

Loose operators all have the same syntactic rule: `<token
node><space><loose op><space><token node>`.

The second aspect of this pull request is the beginning of support for a
signature syntax. Before implementing signatures, a necessary
prerequisite is for the core syntax to support multi-line programs.

That work establishes a few things:

- `;` and newlines are handled in the core grammar, and both count as
  "separators"
- line comments begin with `#` and continue until the end of the line

In this commit, multi-token productions in the core grammar can use
separators interchangably with spaces. However, I think we will
ultimately want a different rule preventing separators from occurring
before an infix operator, so that the end of a line is always
unambiguous. This would avoid gratuitous differences between modules and
repl usage.

We already effectively have this rule, because otherwise `x<newline> |
y` would be a single pipeline, but of course that wouldn't work.
2019-12-11 16:41:07 -08:00
Jonathan Turner
e2a95c3e1d Move str and inc to core plugins 2019-12-10 13:59:13 +13:00
Jonathan Turner
88f899d341 Move some plugins back to being core shippable plugins 2019-12-10 13:05:40 +13:00
Jonathan Turner
38cbfdb8a9 Remove partial docker plugin. Embed->wrap 2019-12-09 17:41:09 +13:00
Sebastian Jung
bda5db59c8 Merge remote-tracking branch 'upstream/master' into range 2019-12-03 20:23:49 +01:00
Andrés N. Robalino
4526d757b6
Merge pull request #1049 from andrasio/embed-list
embed as column when embedding a list
2019-12-03 02:51:58 -05:00
Andrés N. Robalino
e5405d7f5c embed as column when embedding a list 2019-12-03 02:26:01 -05:00
Sebastian Jung
201506a5ad add tests for range + run rustfmt 2019-12-03 08:24:49 +01:00
Jonathan Turner
efc879b955 Add new line primitive, bump version, allow bare filepaths 2019-12-03 19:44:59 +13:00
Jonathan Turner
cd9d9ad50b improve duration print 2019-12-02 11:02:58 -08:00
Andrés N. Robalino
552272b37e replace and find-replace str plugin additions. 2019-12-02 11:02:58 -08:00
Yehuda Katz
a1e21828d6 Fix tests 2019-12-02 11:02:57 -08:00
Andrés N. Robalino
2fdafa52b1 replace and find-replace str plugin additions. 2019-12-02 11:02:57 -08:00
Andrés N. Robalino
8cedd2ee5b replace and find-replace str plugin additions. 2019-11-26 19:03:22 -05:00
Yehuda Katz
2eae5a2a89 Merge remote-tracking branch 'origin/master' into cleanup-wip 2019-11-25 19:25:12 -08:00
Andrés N. Robalino
83ac65ced3
Merge pull request #997 from bndbsh/operator-contains
Add `=~` and `!~` operators on strings
2019-11-25 18:19:58 -05:00
Andrés N. Robalino
d320ffe742 nth can select more than one row at a time. 2019-11-25 17:16:58 -05:00
Belhorma Bendebiche
fbc6f01cfb Add =~ and !~ operators on strings
`left =~ right` return true if left contains right, using Rust's
`String::contains`. `!~` is the negated version.

A new `apply_operator` function is added which decouples evaluation from
`Value::compare`. This returns a `Value` and opens the door to
implementing `+` for example, though it wouldn't be useful immediately.

The `operator!` macro had to be changed slightly as it would choke on
`~` in arguments.
2019-11-25 15:06:11 -05:00
Yehuda Katz
5fbea31d15 Remove unused Display implementations
After the previous commit, nushell uses PrettyDebug and
PrettyDebugWithSource for our pretty-printed display output.

PrettyDebug produces a structured `pretty.rs` document rather than
writing directly into a fmt::Formatter, and types that implement
`PrettyDebug` have a convenience `display` method that produces a string
(to be used in situations where `Display` is needed for compatibility
with other traits, or where simple rendering is appropriate).
2019-11-25 10:07:20 -08:00
Andrés N. Robalino
06857fbc52 Take all rows having the column present. 2019-11-24 04:35:36 -05:00
Andrés N. Robalino
1c830b5c95 default command introduced. 2019-11-24 04:20:08 -05:00
Andrés N. Robalino
bc2d65cd2e Remove raw data debugging. 2019-11-23 19:16:25 -05:00
Andrés N. Robalino
1a0b339897 compact command introduced. 2019-11-23 19:05:44 -05:00
Andrés N. Robalino
f4d9975dab Clean up feature build flags. 2019-11-22 03:11:36 -05:00
Andrés N. Robalino
6e8b768d79 Requiring at least one member is no longer necessary. 2019-11-22 01:18:06 -05:00
Yehuda Katz
cdb0eeafa2 --no-edit 2019-11-21 14:22:32 -08:00
David Mason
b3c021899c combine functions behind to/from-c/tsv commands
fixes #969, admittedly without a --delimiter alias

moves from_structured_data.rs to from_delimited_data.rs to better
identify its scope and adds to_delimited_data.rs. Now csv and tsv both
use the same code, tsv passes in a fixed '\t' argument where csv passes
in the value of --separator
2019-11-19 16:02:35 +00:00
Jonathan Turner
90aeb700ea Add from_xlsx for importing excel files 2019-11-17 16:18:41 +13:00
Thomas Hartmann
1060ba2206 Fixes --headerless functionality for from-ssv.
Squashed commit of the following:

commit fc59d47a2291461d84e0587fc0fe63af0dc26f9f
Author: Thomas Hartmann <thomas.o.hartmann@gmail.com>
Date:   Tue Nov 12 15:39:38 2019 +0100

    Fixes inconsistencies in output.

commit da4084e9fdd983557b101207b381e333a443e551
Author: Thomas Hartmann <thomas.o.hartmann@gmail.com>
Date:   Tue Nov 12 13:04:10 2019 +0100

    remove unused enum.

commit 7f6a105879c8746786b99fb19bb9f0860c41796a
Author: Thomas Hartmann <thomas.o.hartmann@gmail.com>
Date:   Tue Nov 12 12:58:41 2019 +0100

    Starts refactoring from_ssv.

commit b70ddd169ef0c900e03fb590cb171cc7181528db
Author: Thomas Hartmann <thomas.o.hartmann@gmail.com>
Date:   Tue Nov 12 11:34:06 2019 +0100

    Fixes --headerless for non-aligned columns.

commit 6332778dd26de8d07be77b291124115141479892
Author: Thomas Hartmann <thomas.o.hartmann@gmail.com>
Date:   Tue Nov 12 10:27:35 2019 +0100

    Fixes from-ssv headerless aligned-columns logic.

commit 747d8c812e06349b4a15b8c130721881d86fff98
Author: Thomas Hartmann <thomas.o.hartmann@gmail.com>
Date:   Mon Nov 11 23:53:59 2019 +0100

    fixes unit tests for ssv.

commit c77cb451623b37a7a9742c791a4fc38cad053d3d
Author: Thomas Hartmann <thomas.o.hartmann@gmail.com>
Date:   Mon Nov 11 22:49:21 2019 +0100

    it compiles! one broken test.

commit 08a05964f56cf92507c255057d0aaf2b6dbb6f45
Author: Thomas Hartmann <thomas.o.hartmann@gmail.com>
Date:   Mon Nov 11 18:52:54 2019 +0100

    Backed into a corner. Help.

commit c95ab683025a8007b8a6f8e1659f021a002df584
Author: Thomas Hartmann <thomas.o.hartmann@gmail.com>
Date:   Mon Nov 11 17:30:54 2019 +0100

    broken but on the way
2019-11-12 16:04:55 +01:00
Thomas Hartmann
a3ff5f1246 Updates tests for from tsv, csv, and ssv.
With the proposed changes, these tests now become invalid. If the first line is
to be counted as data, then converting the headers to ints will fail. Removing
the headers and instead treating the first line as data, however, reflects the
new, desired mode of operation.
2019-11-12 16:04:55 +01:00
Andrés N. Robalino
00b3c2036a This is part of on-going work with capabilities when working with
tables and able to work with them for data processing & viewing
purposes. At the moment, certain ways to process said tables we
are able to view a histogram of a given column.

As usage matures, we may find certain core commands that could
be used ergonomically when working with tables on Nu.
2019-11-12 03:39:30 -05:00
Jonathan Turner
62a5250554 Add format command 2019-11-10 13:14:59 +13:00
Jonathan Turner
9043970e97
Merge pull request #943 from drmason13/from_csv-add-separator-arg
Add --separator argument to from_csv
2019-11-09 15:09:32 -08:00
Jonathan Turner
bab58576b4 Rename read to parse 2019-11-10 11:26:44 +13:00
David Mason
15986c598a Add --separator command to from_csv
The command takes a string, checks it is a single character and then
passes it to csv::ReaderBuilder via .delimiter() method as a u8.
2019-11-08 15:06:33 +00:00
Andrés N. Robalino
f966394b63
Merge pull request #888 from andrasio/data-primitives
WIP [data processing]
2019-11-03 16:52:21 -05:00
Andrés N. Robalino
889d2bb378 Isolate feature. 2019-11-03 16:36:47 -05:00
Andrés N. Robalino
d7b768ee9f Fallback internally to String primitives until Member int serialization lands. 2019-11-03 05:38:47 -05:00
Andrés N. Robalino
6ea8e42331 Move column paths to support broader value types. 2019-11-03 05:38:47 -05:00
Andrés N. Robalino
f57489ed92 get command tests already present and move to their own. 2019-11-02 21:05:27 -05:00
Jonathan Turner
e92d4b2ccb Rename add to insert 2019-11-02 14:47:14 +13:00
Andrés N. Robalino
e31ed66610 get :: support fetching rows using numbers in column path. 2019-10-31 14:20:22 -05:00
Andrés N. Robalino
65ae24fbf1 suite in place. 2019-10-31 04:42:18 -05:00
Andrés N. Robalino
b54ce921dd Better error messages. 2019-10-31 04:36:08 -05:00
Jonathan Turner
81691e07c6 Add prepend and append commands 2019-10-30 19:54:06 +13:00
Jonathan Turner
3820fef801 Add a simple read/parse plugin to better handle text data 2019-10-30 11:33:36 +13:00
Andrés N. Robalino
392ff286b2 This commit is ongoing work for making Nu working with data processing
a joy. Fundamentally we embrace functional programming principles for
transforming the dataset from any format picked up by Nu. This table
processing "primitive" commands will build up and make pipelines
composable with data processing capabilities allowing us the valuate,
reduce, and map, the tables as far as even composing this declartively.

On this regard, `split-by` expects some table with grouped data and we
can use it further in interesting ways (Eg. collecting labels for
visualizing the data in charts and/or suit it for a particular chart
of our interest).
2019-10-29 16:04:31 -05:00
Jonathan Turner
571b33a11c
Merge pull request #857 from andrasio/group-by
Can group rows by given column name.
2019-10-23 18:25:52 +13:00
Andrés N. Robalino
f1630da2cc Suggest a column name in case one unknown column is supplied. 2019-10-22 20:10:42 -05:00
Yehuda Katz
6a7c00eaef Finish the job of moving shapes into the stream
This commit should finish the `coloring_in_tokens` feature, which moves
the shape accumulator into the token stream. This allows rollbacks of
the token stream to also roll back any shapes that were added.

This commit also adds a much nicer syntax highlighter trace, which shows
all of the paths the highlighter took to arrive at a particular coloring
output. This change is fairly substantial, but really improves the
understandability of the flow. I intend to update the normal parser with
a similar tracing view.

In general, this change also fleshes out the concept of "atomic" token
stream operations.

A good next step would be to try to make the parser more
error-correcting, using the coloring infrastructure. A follow-up step
would involve merging the parser and highlighter shapes themselves.
2019-10-22 16:19:22 -07:00
Andrés N. Robalino
a317072e4e Cover failure not found files cases. 2019-10-22 08:08:24 -05:00
Andrés N. Robalino
0611f56776 Can group cells by given column name. 2019-10-20 18:42:07 -05:00
notryanb@gmail.com
9eda573a43 filter out the files that have the same size on multiple operating systems 2019-10-18 20:43:37 -04:00
notryanb@gmail.com
4f91d2512a add a test to calculate average of bytes 2019-10-18 20:43:37 -04:00
notryanb@gmail.com
43fbf4345d remove comment and add test for averaging integers 2019-10-18 20:43:37 -04:00
Thomas Hartmann
0d2044e72e Changes flag to minimum-spaces. 2019-10-15 22:05:32 +02:00
Thomas Hartmann
e7b37bee08 Adds filter test for named param. 2019-10-15 20:58:46 +02:00
Andrés N. Robalino
ec2e35ad81 'last' gets last row if no amount desired given. 2019-10-15 05:41:34 -05:00
Andrés N. Robalino
5ed1ed54a6 Move off 'sum' to internal command 'count' for tests. 2019-10-15 05:16:47 -05:00
Andrés N. Robalino
3f60c9d416 'first' gets first row if no amount desired given. 2019-10-15 04:17:55 -05:00
Jonathan Turner
bd6d8189f8
Merge pull request #830 from t-hart/pull-req/from-master
[DRAFT] Adds `from-ssv` command.
2019-10-15 18:28:43 +13:00
Thomas Hartmann
65546646a7 Pull in upstream changes. 2019-10-14 23:05:52 +02:00
Jason Gedge
ee8cd671cb Fix bug with multiple input objects to an external command.
Previously, we would build a command that looked something like this:

  <ex_cmd> "$it" "&&" "<ex_cmd>" "$it"

So that the "&&" and "<ex_cmd>" would also be arguments to the command,
instead of a chained command. This commit builds up a command string
that can be passed to an external shell.
2019-10-14 16:47:12 -04:00
Thomas Hartmann
de1c4e6c88 Implements from-ssv 2019-10-13 22:50:45 +02:00
Thomas Hartmann
648d4865b1 Adds unimplemented module, tests. 2019-10-13 21:15:30 +02:00
Jonathan Turner
193b00764b
Stream support (#812)
* Moves off of draining between filters. Instead, the sink will pull on the stream, and will drain element-wise. This moves the whole stream to being lazy.
* Adds ctrl-c support and connects it into some of the key points where we pull on the stream. If a ctrl-c is detect, we immediately halt pulling on the stream and return to the prompt.
* Moves away from having a SourceMap where anchor locations are stored. Now AnchorLocation is kept directly in the Tag.
* To make this possible, split tag and span. Span is largely used in the parser and is copyable. Tag is now no longer copyable.
2019-10-13 17:12:43 +13:00
Yehuda Katz
1ad9d6f199 Overhaul the expansion system
The main thrust of this (very large) commit is an overhaul of the
expansion system.

The parsing pipeline is:

- Lightly parse the source file for atoms, basic delimiters and pipeline
  structure into a token tree
- Expand the token tree into a HIR (high-level intermediate
  representation) based upon the baseline syntax rules for expressions
  and the syntactic shape of commands.

Somewhat non-traditionally, nu doesn't have an AST at all. It goes
directly from the token tree, which doesn't represent many important
distinctions (like the difference between `hello` and `5KB`) directly
into a high-level representation that doesn't have a direct
correspondence to the source code.

At a high level, nu commands work like macros, in the sense that the
syntactic shape of the invocation of a command depends on the
definition of a command.

However, commands do not have the ability to perform unrestricted
expansions of the token tree. Instead, they describe their arguments in
terms of syntactic shapes, and the expander expands the token tree into
HIR based upon that definition.

For example, the `where` command says that it takes a block as its first
required argument, and the description of the block syntactic shape
expands the syntax `cpu > 10` into HIR that represents
`{ $it.cpu > 10 }`.

This commit overhauls that system so that the syntactic shapes are
described in terms of a few new traits (`ExpandSyntax` and
`ExpandExpression` are the primary ones) that are more composable than
the previous system.

The first big win of this new system is the addition of the `ColumnPath`
shape, which looks like `cpu."max ghz"` or `package.version`.
Previously, while a variable path could look like `$it.cpu."max ghz"`,
the tail of a variable path could not be easily reused in other
contexts. Now, that tail is its own syntactic shape, and it can be used
as part of a command's signature.

This cleans up commands like `inc`, `add` and `edit` as well as
shorthand blocks, which can now look like `| where cpu."max ghz" > 10`
2019-10-10 08:27:51 -07:00
Andrés N. Robalino
837d12decd Filesystem shell can't cd into files. Ever. 2019-09-24 15:34:30 -05:00
Jonathan Turner
5ff94004c6 Add urlencode/urldecode 2019-09-19 16:25:29 +12:00
Andrés N. Robalino
91bea7fb2a
Assert the column is unknown.
did you mean in error messages appear when `get`ing unknown columns. Here we know the column does not exist so we check the exact error message.
2019-09-14 14:53:31 -05:00
Yehuda Katz
19767ad551 Taking another stab at replacing Span with Tag 2019-09-14 11:48:45 -05:00
Jonathan Turner
074a76c9d4 Fix test 2019-09-13 15:48:32 +12:00
Andrés N. Robalino
7838dac689 first and get coverage. 2019-09-12 05:22:58 -05:00
Andrés N. Robalino
c57c0eb371 pass lint checks. 2019-09-12 01:49:01 -05:00
Andrés N. Robalino
b35549adac Removes regex crate dependency. 2019-09-11 22:20:42 -05:00
Maximilian Roos
127381497c
run rustfmt 2019-09-11 10:36:50 -04:00
Andrés N. Robalino
ba8383ae2f to-[csv/tsv] fixes. 2019-09-10 07:00:25 -05:00
Andrés N. Robalino
f61144006f config test harness. 2019-09-10 05:08:01 -05:00
Andrés N. Robalino
f770409a60 cd '-' valueshell implementation and valueshell refactorings. 2019-09-08 05:40:47 -05:00
Andrés N. Robalino
77c2e4200e Filesystem cd refactor/cleanup. 2019-09-08 04:55:49 -05:00
Andrés N. Robalino
df9ff44956 Filesystem change directory coverage. 2019-09-08 03:09:05 -05:00
Andrés N. Robalino
eecda3ecba
Merge pull request #363 from twe4ked/cd-back
Implement `cd -` to return to the previous directory
2019-09-08 01:00:32 -05:00
Odin Dutton
159cf27e39 Implement cd - to return to the last path for the FilesystemShell 2019-09-08 15:10:46 +10:00
Jonathan Turner
9da896ad4e Attempt so simplify classified 2019-09-08 14:00:04 +12:00
Jonathan Turner
84628f298d Finish fixing failing tests. 2019-09-08 13:35:02 +12:00
Patrick Meredith
39fce1191f Fix typo 2019-09-04 13:38:17 -04:00
Patrick Meredith
e8764911cb Add comments for sample.{bson,db} 2019-09-04 13:36:12 -04:00
Jonathan Turner
479f0a566e Covert to_* commands to work on whole table 2019-09-04 18:48:40 +12:00
Patrick Meredith
05e858fa94 Add test 2019-09-03 22:37:26 -04:00
est31
cf0efb811e Trim trailing whitespace and set it in editorconfig 2019-09-03 02:52:52 +02:00
Andrés N. Robalino
9488c41dcd can embed a new field to the table 2019-09-02 00:37:13 -05:00
Andrés N. Robalino
ca0c6eaf58 This commit introduces a basic help feature. We can go to it
with the `help` command to explore and list all commands available.

Enter will also try to see if the location to be entered is an existing
Nu command, if it is it will let you inspect the command under `help`.

This provides baseline needed so we can iterate on it.
2019-08-31 19:06:11 -05:00
Patrick Meredith
b514d93ffd Add Decimal to sample.bson 2019-08-31 19:15:48 +02:00
Patrick Meredith
3d147d1143 Add SQLite support 2019-08-30 20:54:45 -04:00
Andrés N. Robalino
f1e8c433c2 [from/to]tsv support. 2019-08-29 04:02:16 -05:00
Andrés N. Robalino
9810df25b4 Readability improvement. 2019-08-29 01:31:56 -05:00
Andrés N. Robalino
4d2b0f43f5 Fix test. 2019-08-28 21:05:23 -05:00
Andrés N. Robalino
a07817e0e0 cover pipeline helper. 2019-08-28 20:30:51 -05:00
Andrés N. Robalino
55fb1f8dda Yo quiero Playground taconushell. 2019-08-28 19:32:42 -05:00
Yehuda Katz
f82cc4291f Migrate commands_test 2019-08-28 10:58:00 -07:00
Yehuda Katz
abac7cf746 Migrate rm 2019-08-28 10:48:52 -07:00
Yehuda Katz
2c65b2fc2f Migrate mv 2019-08-28 10:28:58 -07:00
Yehuda Katz
21ad06b1e1 Remove unwraps and clean up playground
The original intent of this patch was to remove more unwraps to reduce
panics. I then lost a ton of time to the fact that the playground isn't
in a temp directory (because of permissions issues on Windows).

This commit improves the test facilities to:

- use a tempdir for the playground
- change the playground API so you instantiate it with a block that
  encloses the lifetime of the tempdir
- the block is called with a `dirs` argument that has `dirs.test()` and
  other important directories that we were computing by hand all the time
- the block is also called with a `playground` argument that you can use
  to construct files (it's the same `Playground` as before)
- change the nu! and nu_error! macros to produce output instead of
  taking a variable binding
- change the nu! and nu_error! macros to do the cwd() transformation
  internally
- change the nu! and nu_error! macros to take varargs at the end that
  get interpolated into the running command

I didn't manage to finish porting all of the tests, so a bunch of tests
are currently commented out. That will need to change before we land
this patch.
2019-08-28 10:01:16 -07:00
Andrés N. Robalino
162c8b4274 swept clean quotes 2019-08-27 06:20:22 -05:00
Andrés N. Robalino
b502954558 [from|to]json additions/refactoring. 2019-08-27 06:05:51 -05:00
Jonathan Turner
3750a04cfc
Merge branch 'master' into expand-tilde 2019-08-27 16:23:56 +12:00
Yehuda Katz
34292b282a Add support for ~ expansion
This ended up being a bit of a yak shave. The basic idea in this commit is to
expand `~` in paths, but only in paths.

The way this is accomplished is by doing the expansion inside of the code that
parses literal syntax for `SyntaxType::Path`.

As a quick refresher: every command is entitled to expand its arguments in a
custom way. While this could in theory be used for general-purpose macros,
today the expansion facility is limited to syntactic hints.

For example, the syntax `where cpu > 0` expands under the hood to
`where { $it.cpu > 0 }`. This happens because the first argument to `where`
is defined as a `SyntaxType::Block`, and the parser coerces binary expressions
whose left-hand-side looks like a member into a block when the command is
expecting one.

This is mildly more magical than what most programming languages would do,
but we believe that it makes sense to allow commands to fine-tune the syntax
because of the domain nushell is in (command-line shells).

The syntactic expansions supported by this facility are relatively limited.
For example, we don't allow `$it` to become a bare word, simply because the
command asks for a string in the relevant position. That would quickly
become more confusing than it's worth.

This PR adds a new `SyntaxType` rule: `SyntaxType::Path`. When a command
declares a parameter as a `SyntaxType::Path`, string literals and bare
words passed as an argument to that parameter are processed using the
path expansion rules. Right now, that only means that `~` is expanded into
the home directory, but additional rules are possible in the future.

By restricting this expansion to a syntactic expansion when passed as an
argument to a command expecting a path, we avoid making `~` a generally
reserved character. This will also allow us to give good tab completion
for paths with `~` characters in them when a command is expecting a path.

In order to accomplish the above, this commit changes the parsing functions
to take a `Context` instead of just a `CommandRegistry`. From the perspective
of macro expansion, you can think of the `CommandRegistry` as a dictionary
of in-scope macros, and the `Context` as the compile-time state used in
expansion. This could gain additional functionality over time as we find
more uses for the expansion system.
2019-08-26 21:03:24 -07:00
Patrick Meredith
738675259e Improve test so that it should work on Windows 2019-08-26 21:26:49 -04:00
Patrick Meredith
87a99bbabf Implement to-bson 2019-08-26 20:07:59 -04:00
Dirkjan Ochtman
b77effa434 Fix formatting with cargo fmt 2019-08-26 20:19:05 +02:00
Andrés N. Robalino
8940e57cf3 mark #[test] back. (deleted and reverting it from.. 12e38063) 2019-08-25 21:50:55 -05:00
Andrés N. Robalino
12e3806349
Merge pull request #415 from androbtech/fromto-refact
[from|to]csv additions/refactoring.
2019-08-25 12:56:28 -05:00
Jonathan Turner
dda4a707a7
Merge pull request #421 from ramonsnir/reverse
add reverse
2019-08-26 05:41:07 +12:00
Andrés N. Robalino
0e14ba86ae [from|to]csv additions/refactoring.
Introduced flag to tell `from-to` / `to-csv` whether we want headers parsed and/or written.
2019-08-25 12:32:08 -05:00
Patrick Meredith
3c89cb7e98 Remove test that refuses to pass on Windows (it's just a minor formatting issue) 2019-08-25 12:25:40 -04:00
Ramon Snir
9735c3fcea
add reverse 2019-08-25 12:14:17 -04:00
Patrick Meredith
376809aa2a Normalize strings for bson tests 2019-08-25 11:43:15 -04:00
Patrick Meredith
e9673c31ea Remove redundant test 2019-08-25 09:50:25 -04:00
Patrick Meredith
a3b4d47b4e Finish last few types and add tests 2019-08-25 09:50:25 -04:00
Jonathan Turner
9c4f94fed5
Merge pull request #402 from ijt/add-last-command
Add `last` filter
2019-08-25 13:00:13 +12:00
Issac Trotts
45d514a4cb sort to get test to pass on linux 2019-08-24 17:21:09 -07:00
Issac Trotts
108f66941b add last command 2019-08-24 15:01:30 -07:00
Issac Trotts
af2439e880 add test 2019-08-24 14:56:56 -07:00
Dmitry Starostin
19772f82aa
Add --reverse option for sort-by 2019-08-25 00:11:38 +03:00
Andrés N. Robalino
8b79b28971 mkdir can take multiple directories or multiple directory hierachies and wil create them as required. 2019-08-21 07:07:37 -05:00
Andrés N. Robalino
8c1d4ed91a sidestep 'enter' integration test failure for files. 2019-08-20 08:01:02 -05:00
Andrés N. Robalino
0f28719564 mv -> More organized method definitions. 2019-08-20 07:35:41 -05:00
Andrés N. Robalino
11095860c8 rm fully operational and error surveyd. 2019-08-20 06:22:11 -05:00
Andrés N. Robalino
308323426d Covered enter's ability to enter files other than filesystems. 2019-08-20 03:11:33 -05:00
Jonathan Turner
51ec48f357 Update enter test 2019-08-20 04:40:47 +12:00
Jonathan Turner
953d28ef74 Fix push/pop for shell manager 2019-08-19 20:07:55 +12:00
Andrés N. Robalino
5dd20850b5 Refactoring and unwrap cleanup beginnings. 2019-08-18 20:28:55 -05:00
Yehuda Katz
0dc4b2b686 Add support for external escape valve (^dir)
This commit makes it possible to force nu to treat a command as an external command by prefixing it with `^`. For example `^dir` will force `dir` to run an external command, even if `dir` is also a registered nu command.

This ensures that users don't need to leave nu just because we happened to use a command they need.

This commit adds a new token type for external commands, which, among other things, makes it pretty straight forward to syntax highlight external commands uniquely, and generally to treat them as special.
2019-08-15 15:18:18 -07:00
Andrés N. Robalino
024f0915a3 Shellring traversal and filesystem environment retention per session confirmed. 2019-08-15 02:10:33 -05:00
Andrés N. Robalino
75e3c6f547 copies_expected -> expected. Use expected as test dirname expectation outcomes from now on. 2019-08-14 16:05:35 -05:00
Andrés N. Robalino
121237ee4b Reverted test removal from (dfcbaed) 2019-08-14 15:55:26 -05:00
Andrés N. Robalino
154063013f mv introduced. \¡Viva\! 2019-08-14 15:29:05 -05:00
Andrés N. Robalino
dfcbaed1c6 More descriptive test playground names for commands other than the basic ones. 2019-08-14 15:29:05 -05:00
Andrés N. Robalino
278de0e517 cp refactoring. 2019-08-14 15:29:05 -05:00
Jonathan Turner
6cf3dc92fc Add support for utf16 files 2019-08-12 16:11:42 +12:00
Andrés N. Robalino
fc38603de8 Test sample changed to check behavior with strings on different operating systems. 2019-08-11 13:18:42 -05:00
Andrés N. Robalino
cee2f5207e Str plugin: replace & find-replace. 2019-08-11 12:46:14 -05:00
Andrés N. Robalino
21ab65979f Covered the remaining semver flags for inc plugin. 2019-08-10 06:04:13 -05:00
Andrés N. Robalino
e744237451 Clean up. 2019-08-10 05:56:47 -05:00
Andrés N. Robalino
5372463b37 Extract tests. 2019-08-10 04:40:32 -05:00
Andrés N. Robalino
cb4b2e6116 Inc refactoring. More tests. 2019-08-10 04:38:28 -05:00
Andrés N. Robalino
5bfa2cc4b8 cp wildcard already supported (documented with test) 2019-08-10 04:32:27 -05:00
Jonathan Turner
83030094e0 All tests pass 2019-08-09 17:36:43 +12:00
Jonathan Turner
aadacc2d36 Merge master 2019-08-09 16:51:21 +12:00
Jonathan Turner
b815768166 Fix test 2019-08-09 05:53:28 +12:00
Andrés N. Robalino
ba6d62ea0c mkdir creates intermediary directories as required (the default). --create-all/--deep flag removed. 2019-08-07 14:38:00 -05:00
Andrés N. Robalino
50393bdf42 Make more visible the strategies for figuring out where to copy files. 2019-08-07 13:42:52 -05:00
Andrés N. Robalino
e0bacaaf37 clean up. more cp. mkdir. more test coverage. fixes.
- Introduced mkdir.
- Minor more labelled error improvements.
- Fix to avoid leaking child zombies.
- cp improvements.
- Introduced mkdir.
2019-08-07 13:28:46 -05:00
Andrés N. Robalino
c8b5329c5c mkdir. 2019-08-07 13:24:20 -05:00
Andrés N. Robalino
1b7dd52713 Tests pass. 2019-08-07 13:23:02 -05:00
Andrés N. Robalino
b38d54e033 Dont let Nu childs become zombies. 2019-08-07 13:18:02 -05:00
Yehuda Katz
fc173c46d8 Restructuring 2019-08-02 12:15:07 -07:00
Jonathan Turner
4821707f96 Test for external quoted strings 2019-08-03 04:33:52 +12:00
Andrés N. Robalino
7a76998f07 regular, * wildcard, and ? covered. 2019-08-02 05:31:15 -05:00
Jonathan Turner
aa54995326
Merge pull request #232 from androbtech/unit-tests-wip
Introduced to-int into str plugin and unit tests coverage along with it.
2019-08-02 13:06:30 +12:00
Andrés N. Robalino
1bc1bb6111 Glob paths can fail. Communicates the error if it happens. thanks (jonathandturner) 2019-08-01 16:55:49 -05:00
Andrés N. Robalino
375f311331 Wildcard support adventure starting with rm command. 2019-08-01 16:01:53 -05:00
Andrés N. Robalino
0893f89e89 More readable tests. 2019-08-01 04:28:49 -05:00
Andrés N. Robalino
610a91a658 Test coverage for pull #235 2019-08-01 03:31:41 -05:00
Andrés N. Robalino
a0890b551a strutils can also convert to an integer now. 2019-07-31 23:50:03 -05:00
Andrés N. Robalino
0b8bbd8637 Unit Testing WIP. 2019-07-31 23:46:59 -05:00
Andrés N. Robalino
7c4706ee50 Validation baseline. 2019-07-28 18:34:37 -05:00
Andrés N. Robalino
59dec999b8 string utils plugin baseline. 2019-07-28 02:01:32 -05:00
Jonathan Turner
e4797f8895 Add end_plugin and sum 2019-07-27 06:40:00 +12:00
Yehuda Katz
73deeb69db Clean up lint errors 2019-07-23 21:10:48 -07:00
Yehuda Katz
5a8e041a48 Tests pass! 2019-07-23 15:22:11 -07:00
Jonathan Turner
72384629fd
Merge pull request #205 from androbtech/cp_play
Introduced initial cp functionality.
2019-07-24 04:46:30 +12:00
Jonathan Turner
ec7d49b0d2 Add and edit functionality 2019-07-22 15:52:57 +12:00
Andrés N. Robalino
2da43f4b06 Introduced initial cp functionality. 2019-07-21 21:23:02 -05:00
Andrés N. Robalino
191dacdd8b Introduced conversion to csv command. 2019-07-21 02:08:05 -05:00
Jonathan Turner
c36c2a7b50 Switch from_csv to output rows 2019-07-20 18:44:21 +12:00
Jonathan Turner
08f6d29b79 Fix race condition in test 2019-07-20 13:18:27 +12:00
Andrés N. Robalino
895a1b2d72 Introduced ability to open csv documents. 2019-07-19 15:48:42 -05:00
Andrés N. Robalino
514da5bfa7 Can remove files and directories. 2019-07-18 05:21:24 -05:00
Jonathan Turner
0d6881383d Add some tests 2019-07-18 13:57:58 +12:00
Jonathan Turner
f10225e26b
Merge pull request #190 from jonathandturner/improved_open_url
Improved open url
2019-07-18 07:25:02 +12:00
Jonathan Turner
00b3106f05 Improve open URL. Format and remove warning in tests 2019-07-18 06:56:15 +12:00
Jonathan Turner
227fdddc12 Add yaml roundtrip test. Bump nu version 2019-07-18 05:51:53 +12:00
Andrés N. Robalino
f120ce63f7 Test cases not needed for now. 2019-07-16 16:47:01 -05:00
Andrés N. Robalino
6824c9c7a4 Sample data moved 2019-07-16 14:00:34 -05:00
Andrés N. Robalino
275126d76d Back to passing. Windows test failure needs more investigation. 2019-07-16 06:48:10 -05:00
Andrés N. Robalino
67a0606b6d Different test data for a format Nu does not recognize. 2019-07-16 06:31:53 -05:00
Andrés N. Robalino
c7531d6b3d Windows speaks \r 2019-07-16 05:58:40 -05:00
Andrés N. Robalino
5ca9d307c6 Integration tests refactoring and visibility in them. 2019-07-16 05:33:41 -05:00
Jonathan Turner
2f7f78094d Fix test 2019-07-14 17:09:07 +12:00
Jonathan Turner
dc8545ce10 Add a test for the plugins 2019-07-04 15:18:19 +12:00
Jonathan Turner
160074346f Fix units and add test 2019-06-24 19:59:23 +12:00
Jonathan Turner
9ae9beb94a WIP 2019-06-22 15:43:37 +12:00
Jonathan Turner
b4eb32f1db Touchups to external values 2019-06-19 17:51:24 +12:00
Jonathan Turner
cd9c1bb247
Update lines.txt 2019-06-18 16:44:38 +12:00
Jonathan Turner
2808337112 add lines and skip-while test 2019-06-18 15:45:50 +12:00
Jonathan Turner
1000ec21b5 Add .ini loading/saving 2019-06-16 18:43:40 +12:00
Jonathan Turner
7a332892bd Fixes for windows 2019-06-16 09:41:26 +12:00
Jonathan Turner
36edc059c8 Add missing test 2019-06-16 09:32:57 +12:00
Jonathan Turner
8cdd567b0e Rewrite README and do some touchups 2019-06-15 14:24:13 +12:00
Jonathan Turner
50fe77968f Fix test 2019-06-14 10:52:36 +12:00
Jonathan Turner
2752eb5d9b Update tests 2019-06-11 18:32:42 +12:00
Jonathan Turner
22f9126cf6 merge master and fix conflict 2019-06-11 18:27:19 +12:00
Jonathan Turner
7642541755 Add xml open support 2019-06-11 18:26:03 +12:00
Yehuda Katz
2b876da56f Some function stuff
It's not done, but honestly it's getting too tedious. I will need to
rethink how I'm doing it.
2019-06-10 13:11:55 -07:00
Jonathan Turner
3715da9328 Finish rename 2019-06-05 13:57:16 +12:00
Jonathan Turner
2a20192519 Fix up some warnings and move integration tests to correct dir 2019-06-03 15:48:58 +12:00
Jonathan Turner
0c2be40afc Fix integration tests for Linux 2019-06-03 13:26:29 +12:00
Jonathan Turner
f3d14aad48 Add more tests 2019-06-03 12:37:16 +12:00
Jonathan Turner
6fe9dc0bbf Add missing test file 2019-06-02 18:56:02 +12:00
Jonathan Turner
040c3a5e9c Add first integration test 2019-06-02 15:37:09 +12:00