Commit graph

14 commits

Author SHA1 Message Date
Ian Manske
9996e4a1f8
Shrink the size of Expr (#12610)
# Description
Continuing from #12568, this PR further reduces the size of `Expr` from
64 to 40 bytes. It also reduces `Expression` from 128 to 96 bytes and
`Type` from 32 to 24 bytes.

This was accomplished by:
- for `Expr` with multiple fields (e.g., `Expr::Thing(A, B, C)`),
merging the fields into new AST struct types and then boxing this struct
(e.g. `Expr::Thing(Box<ABC>)`).
- replacing `Vec<T>` with `Box<[T]>` in multiple places. `Expr`s and
`Expression`s should rarely be mutated, if at all, so this optimization
makes sense.

By reducing the size of these types, I didn't notice a large performance
improvement (at least compared to #12568). But this PR does reduce the
memory usage of nushell. My config is somewhat light so I only noticed a
difference of 1.4MiB (38.9MiB vs 37.5MiB).

---------

Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
2024-04-24 15:46:35 +00:00
Oriol
e234f3ea7b
Fix typo in help stor import (#12442)
Changed `export` for `import`

<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
`help stor import` showed a help string that was probably copy-pasted
from `stor export`
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
Now `help stor import` shows a correct description of the operation that
it is doing
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-04-07 08:09:24 -05:00
Stefan Holderbach
e889679d42
Use nightly clippy to kill dead code/fix style (#12334)
- **Remove duplicated imports**
- **Remove unused field in `CompletionOptions`**
- **Remove unused struct in `nu-table`**
- **Clarify generic bounds**
- **Simplify a subtrait bound for `ExactSizeIterator`**
- **Use `unwrap_or_default`**
- **Use `Option` directly instead of empty string**
- **Elide unneeded clone in `to html`**
2024-03-30 09:17:28 +08:00
Stefan Holderbach
b19da158d5
Rename Value::CustomValue to Value::Custom (#12309)
# Description
The second `Value` is redundant and will consume five extra bytes on
each transmission of a custom value to/from a plugin.

# User-Facing Changes
This is a breaking change to the plugin protocol.

The [example in the protocol
reference](https://www.nushell.sh/contributor-book/plugin_protocol_reference.html#value)
becomes

```json
{
  "Custom": {
    "val": {
      "type": "PluginCustomValue",
      "name": "database",
      "data": [36, 190, 127, 40, 12, 3, 46, 83],
      "notify_on_drop": true
    },
    "span": {
      "start": 320,
      "end": 340
    }
  }
}
```

instead of 

```json
{
  "CustomValue": {
    ...
  }
}
```


# After Submitting
Update plugin protocol reference
2024-03-27 22:10:56 +01:00
Ian Manske
c747ec75c9
Add command_prelude module (#12291)
# Description
When implementing a `Command`, one must also import all the types
present in the function signatures for `Command`. This makes it so that
we often import the same set of types in each command implementation
file. E.g., something like this:
```rust
use nu_protocol::ast::Call;
use nu_protocol::engine::{Command, EngineState, Stack};
use nu_protocol::{
    record, Category, Example, IntoInterruptiblePipelineData, IntoPipelineData, PipelineData,
    ShellError, Signature, Span, Type, Value,
};
```

This PR adds the `nu_engine::command_prelude` module which contains the
necessary and commonly used types to implement a `Command`:
```rust
// command_prelude.rs
pub use crate::CallExt;
pub use nu_protocol::{
    ast::{Call, CellPath},
    engine::{Command, EngineState, Stack},
    record, Category, Example, IntoInterruptiblePipelineData, IntoPipelineData, IntoSpanned,
    PipelineData, Record, ShellError, Signature, Span, Spanned, SyntaxShape, Type, Value,
};
```

This should reduce the boilerplate needed to implement a command and
also gives us a place to track the breadth of the `Command` API. I tried
to be conservative with what went into the prelude modules, since it
might be hard/annoying to remove items from the prelude in the future.
Let me know if something should be included or excluded.
2024-03-26 21:17:30 +00:00
Doru
d1a8992590
Initial --params implementation (#12249)
# Description
This PR adds a `--params` param to `query db`. This closes #11643.

You can't combine both named and positional parameters, I think this
might be a limitation with rusqlite itself. I tried using named
parameters with indices like `{ ':named': 123, '1': "positional" }` but
that always failed with a rusqlite error. On the flip side, the other
way around works: for something like `VALUES (:named, ?)`, you can treat
both as positional: `-p [hello 123]`.

This PR introduces some very gnarly code repetition in
`prepared_statement_to_nu_list`. I tried, I swear; the compiler wasn't
having any of it, it kept telling me to box my closures and then it said
that the reference lifetimes were incompatible in the match arms. I gave
up and put the mapping code in the match itself, but I'm still not
happy.

Another thing I'm unhappy about: I don't like how you have to put the
`:colon` in named parameters. I think nushell should insert it if it's
[missing](https://www.sqlite.org/lang_expr.html#parameters). But this is
the way [rusqlite
works](https://docs.rs/rusqlite/latest/rusqlite/trait.Params.html#example-named),
so for now, I'll let it be consistent. Just know that it's not really a
blocker, and it isn't a compatibility change to later make `{ colon: 123
}` work, without the quotes and `:`. This would require allocating and
turning our pretty little `&str` into a `String`, though

# User-Facing Changes
Less incentive to leave yourself open to SQL injection with statements
like `query db $"INSERT INTO x VALUES \($unsafe_user_input)"`.
Additionally, the `$""` syntax being annoying with parentheses plays in
our favor, making users even more likely to use ? with `--params`.

# Tests + Formatting
Hehe
2024-03-24 15:40:21 -05:00
Ian Manske
68fcd71898
Add Value::coerce_str (#11885)
# Description
Following #11851, this PR adds one final conversion function for
`Value`. `Value::coerce_str` takes a `&Value` and converts it to a
`Cow<str>`, creating an owned `String` for types that needed converting.
Otherwise, it returns a borrowed `str` for `String` and `Binary`
`Value`s which avoids a clone/allocation. Where possible, `coerce_str`
and `coerce_into_string` should be used instead of `coerce_string`,
since `coerce_string` always allocates a new `String`.
2024-02-18 17:47:10 +01:00
Ian Manske
1c49ca503a
Name the Value conversion functions more clearly (#11851)
# Description
This PR renames the conversion functions on `Value` to be more consistent.
It follows the Rust [API guidelines](https://rust-lang.github.io/api-guidelines/naming.html#ad-hoc-conversions-follow-as_-to_-into_-conventions-c-conv) for ad-hoc conversions.
The conversion functions on `Value` now come in a few forms:
- `coerce_{type}` takes a `&Value` and attempts to convert the value to
`type` (e.g., `i64` are converted to `f64`). This is the old behavior of
some of the `as_{type}` functions -- these functions have simply been
renamed to better reflect what they do.
- The new `as_{type}` functions take a `&Value` and returns an `Ok`
result only if the value is of `type` (no conversion is attempted). The
returned value will be borrowed if `type` is non-`Copy`, otherwise an
owned value is returned.
- `into_{type}` exists for non-`Copy` types, but otherwise does not
attempt conversion just like `as_type`. It takes an owned `Value` and
always returns an owned result.
- `coerce_into_{type}` has the same relationship with `coerce_{type}` as
`into_{type}` does with `as_{type}`.
- `to_{kind}_string`: conversion to different string formats (debug,
abbreviated, etc.). Only two of the old string conversion functions were
removed, the rest have been renamed only.
- `to_{type}`: other conversion functions. Currently, only `to_path`
exists. (And `to_string` through `Display`.)

This table summaries the above:
| Form | Cost | Input Ownership | Output Ownership | Converts `Value`
case/`type` |
| ---------------------------- | ----- | --------------- |
---------------- | -------- |
| `as_{type}` | Cheap | Borrowed | Borrowed/Owned | No |
| `into_{type}` | Cheap | Owned | Owned | No |
| `coerce_{type}` | Cheap | Borrowed | Borrowed/Owned | Yes |
| `coerce_into_{type}` | Cheap | Owned | Owned | Yes |
| `to_{kind}_string` | Expensive | Borrowed | Owned | Yes |
| `to_{type}` | Expensive | Borrowed | Owned | Yes |

# User-Facing Changes
Breaking API change for `Value` in `nu-protocol` which is exposed as
part of the plugin API.
2024-02-17 18:14:16 +00:00
Stefan Holderbach
543a25599c
Simplify the feature gates for stor commands (#11416)
All of them depend on feature `sqlite` so just conditionally `use` the
parent module.
2023-12-24 13:31:46 +01:00
Hofer-Julian
76482cc1b2
Move stor commands to category Database (#11315)
Fixes #11309
2023-12-13 16:24:16 +01:00
Eric Hodel
ecb3b3a364
Ensure that command usage starts uppercase and ends period (#11278)
# Description

This repeats #8268 to make all command usage strings start with an
uppercase letter and end with a period per #5056

Adds a test to ensure that commands won't regress

Part of #5066

# User-Facing Changes

Command usage is now consistent

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

Automatic documentation updates
2023-12-10 08:28:54 -06:00
Eric Hodel
a95a4505ef
Convert Shellerror::GenericError to named fields (#11230)
# Description

Replace `.to_string()` used in `GenericError` with `.into()` as
`.into()` seems more popular

Replace `Vec::new()` used in `GenericError` with `vec![]` as `vec![]`
seems more popular

(There are so, so many)
2023-12-07 00:40:03 +01:00
Darren Schroeder
e5d9f405db
add some tests for stor create (#11194)
# Description

This is a PR to start adding a few tests to the `stor` commands. It
refactors the `stor create` command so it's easier to write tests.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2023-11-30 15:00:47 -06:00
Darren Schroeder
e290fa0e68
Add stor family of commands (#11170)
# Description

This PR adds the `stor` family of commands. These commands are meant to
create, open, insert, update, delete, reset data in an in-memory sqlite
database. This is really an experiment to see how creatively we can use
an in-memory database.

```
Usage:
  > stor

Subcommands:
  stor create - Create a table in the in-memory sqlite database
  stor delete - Delete a table or specified rows in the in-memory sqlite database
  stor export - Export the in-memory sqlite database to a sqlite database file
  stor import - Import a sqlite database file into the in-memory sqlite database
  stor insert - Insert information into a specified table in the in-memory sqlite database
  stor open - Opens the in-memory sqlite database
  stor reset - Reset the in-memory database by dropping all tables
  stor update - Update information in a specified table in the in-memory sqlite database

Flags:
  -h, --help - Display the help message for this command

Input/output types:
  ╭─#─┬──input──┬─output─╮
  │ 0 │ nothing │ string │
  ╰───┴─────────┴────────╯
```

### Examples

## stor create
```nushell
❯ stor create --table-name nudb --columns {bool1: bool, int1: int, float1: float, str1: str, datetime1: datetime}
╭──────┬────────────────╮
│ nudb │ [list 0 items] │
╰──────┴────────────────╯
```
## stor insert
```nushell
❯ stor insert --table-name nudb --data-record {bool1: true, int1: 2, float1: 1.1, str1: fdncred, datetime1: 2023-04-17} 
╭──────┬───────────────╮
│ nudb │ [table 1 row] │
╰──────┴───────────────╯
```
## stor open
```nushell
❯ stor open | table -e 
╭──────┬────────────────────────────────────────────────────────────────────╮
│      │ ╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮ │
│ nudb │ │ 0 │ 1 │   1 │  2 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │ │
│      │ ╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯ │
╰──────┴────────────────────────────────────────────────────────────────────╯
```
## stor update
```nushell
❯ stor update --table-name nudb --update-record {str1: toby datetime1: 2021-04-17} --where-clause "bool1 = 1"
╭──────┬───────────────╮
│ nudb │ [table 1 row] │
╰──────┴───────────────╯
❯ stor open | table -e
╭──────┬─────────────────────────────────────────────────────────────────╮
│      │ ╭─#─┬id─┬bool1┬int1┬float1┬─str1─┬─────────datetime1──────────╮ │
│ nudb │ │ 0 │ 1 │   1 │  2 │ 1.10 │ toby │ 2021-04-17 00:00:00 +00:00 │ │
│      │ ╰───┴───┴─────┴────┴──────┴──────┴────────────────────────────╯ │
╰──────┴─────────────────────────────────────────────────────────────────╯
```
## insert another row
```nushell
❯ stor insert --table-name nudb --data-record {bool1: true, int1: 5, float1: 1.1, str1: fdncred, datetime1: 2023-04-17} 
╭──────┬────────────────╮
│ nudb │ [table 2 rows] │
╰──────┴────────────────╯
❯ stor open | table -e
╭──────┬────────────────────────────────────────────────────────────────────╮
│      │ ╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮ │
│ nudb │ │ 0 │ 1 │   1 │  2 │ 1.10 │ toby    │ 2021-04-17 00:00:00 +00:00 │ │
│      │ │ 1 │ 2 │   1 │  5 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │ │
│      │ ╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯ │
╰──────┴────────────────────────────────────────────────────────────────────╯
```
## stor delete (specific row(s))
```nushell
❯ stor delete --table-name nudb --where-clause "int1 == 5"
╭──────┬───────────────╮
│ nudb │ [table 1 row] │
╰──────┴───────────────╯
```
## insert multiple tables
```nushell
❯ stor create --table-name nudb1 --columns {bool1: bool, int1: int, float1: float, str1: str, datetime1: datetime}
╭───────┬────────────────╮
│ nudb  │ [table 1 row]  │
│ nudb1 │ [list 0 items] │
╰───────┴────────────────╯
❯ stor insert --table-name nudb1 --data-record {bool1: true, int1: 2, float1: 1.1, str1: fdncred, datetime1: 2023-04-17}
╭───────┬───────────────╮
│ nudb  │ [table 1 row] │
│ nudb1 │ [table 1 row] │
╰───────┴───────────────╯
❯ stor create --table-name nudb2 --columns {bool1: bool, int1: int, float1: float, str1: str, datetime1: datetime}
╭───────┬────────────────╮
│ nudb  │ [table 1 row]  │
│ nudb1 │ [table 1 row]  │
│ nudb2 │ [list 0 items] │
╰───────┴────────────────╯
❯ stor insert --table-name nudb2 --data-record {bool1: true, int1: 2, float1: 1.1, str1: fdncred, datetime1: 2023-04-17}
╭───────┬───────────────╮
│ nudb  │ [table 1 row] │
│ nudb1 │ [table 1 row] │
│ nudb2 │ [table 1 row] │
╰───────┴───────────────╯
```
## stor delete (specific table)
```nushell
❯ stor delete --table-name nudb1
╭───────┬───────────────╮
│ nudb  │ [table 1 row] │
│ nudb2 │ [table 1 row] │
╰───────┴───────────────╯
```
## stor reset (all tables are deleted)
```nushell
❯ stor reset
```
## stor export
```nushell
❯ stor export --file-name nudb.sqlite3
╭──────┬───────────────╮
│ nudb │ [table 1 row] │
╰──────┴───────────────╯
❯ open nudb.sqlite3 | table -e
╭──────┬────────────────────────────────────────────────────────────────────╮
│      │ ╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮ │
│ nudb │ │ 0 │ 1 │   1 │  5 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │ │
│      │ ╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯ │
╰──────┴────────────────────────────────────────────────────────────────────╯
❯ open nudb.sqlite3 | schema | table -e
╭────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│        │ ╭──────┬──────────────────────────────────────────────────────────────────────────────────────────────────╮ │
│ tables │ │      │ ╭───────────────┬──────────────────────────────────────────────────────────────────────────────╮ │ │
│        │ │ nudb │ │               │ ╭─#─┬─cid─┬───name────┬─────type─────┬─notnull─┬───────default────────┬─pk─╮ │ │ │
│        │ │      │ │ columns       │ │ 0 │ 0   │ id        │ INTEGER      │ 1       │                      │ 1  │ │ │ │
│        │ │      │ │               │ │ 1 │ 1   │ bool1     │ BOOLEAN      │ 0       │                      │ 0  │ │ │ │
│        │ │      │ │               │ │ 2 │ 2   │ int1      │ INTEGER      │ 0       │                      │ 0  │ │ │ │
│        │ │      │ │               │ │ 3 │ 3   │ float1    │ REAL         │ 0       │                      │ 0  │ │ │ │
│        │ │      │ │               │ │ 4 │ 4   │ str1      │ VARCHAR(255) │ 0       │                      │ 0  │ │ │ │
│        │ │      │ │               │ │ 5 │ 5   │ datetime1 │ DATETIME     │ 0       │ STRFTIME('%Y-%m-%d   │ 0  │ │ │ │
│        │ │      │ │               │ │   │     │           │              │         │ %H:%M:%f', 'NOW')    │    │ │ │ │
│        │ │      │ │               │ ╰─#─┴─cid─┴───name────┴─────type─────┴─notnull─┴───────default────────┴─pk─╯ │ │ │
│        │ │      │ │ constraints   │ [list 0 items]                                                               │ │ │
│        │ │      │ │ foreign_keys  │ [list 0 items]                                                               │ │ │
│        │ │      │ │ indexes       │ [list 0 items]                                                               │ │ │
│        │ │      │ ╰───────────────┴──────────────────────────────────────────────────────────────────────────────╯ │ │
│        │ ╰──────┴──────────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
```
## Using with `query db`
```nushell
❯ stor open | query db "select * from nudb"
╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮
│ 0 │ 1 │   1 │  5 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │
╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯
```
## stor import
```nushell
❯ stor open
# note, nothing is returned. there is nothing in memory, atm.
❯ stor import --file-name nudb.sqlite3
╭──────┬───────────────╮
│ nudb │ [table 1 row] │
╰──────┴───────────────╯
❯ stor open | table -e 
╭──────┬────────────────────────────────────────────────────────────────────╮
│      │ ╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮ │
│ nudb │ │ 0 │ 1 │   1 │  5 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │ │
│      │ ╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯ │
╰──────┴────────────────────────────────────────────────────────────────────╯
```

TODO:
- [x] `stor export` - Export a fully formed sqlite db file. 
- [x] `stor import` - Imports a specified sqlite db file.
- [x] Perhaps feature-gate it with the sqlite feature
- [x] Update `query db` to work with the in-memory database
- [x] Remove `open --in-memory`

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2023-11-29 08:02:46 -08:00