Add stor family of commands (#11170)

# Description

This PR adds the `stor` family of commands. These commands are meant to
create, open, insert, update, delete, reset data in an in-memory sqlite
database. This is really an experiment to see how creatively we can use
an in-memory database.

```
Usage:
  > stor

Subcommands:
  stor create - Create a table in the in-memory sqlite database
  stor delete - Delete a table or specified rows in the in-memory sqlite database
  stor export - Export the in-memory sqlite database to a sqlite database file
  stor import - Import a sqlite database file into the in-memory sqlite database
  stor insert - Insert information into a specified table in the in-memory sqlite database
  stor open - Opens the in-memory sqlite database
  stor reset - Reset the in-memory database by dropping all tables
  stor update - Update information in a specified table in the in-memory sqlite database

Flags:
  -h, --help - Display the help message for this command

Input/output types:
  ╭─#─┬──input──┬─output─╮
  │ 0 │ nothing │ string │
  ╰───┴─────────┴────────╯
```

### Examples

## stor create
```nushell
❯ stor create --table-name nudb --columns {bool1: bool, int1: int, float1: float, str1: str, datetime1: datetime}
╭──────┬────────────────╮
│ nudb │ [list 0 items] │
╰──────┴────────────────╯
```
## stor insert
```nushell
❯ stor insert --table-name nudb --data-record {bool1: true, int1: 2, float1: 1.1, str1: fdncred, datetime1: 2023-04-17} 
╭──────┬───────────────╮
│ nudb │ [table 1 row] │
╰──────┴───────────────╯
```
## stor open
```nushell
❯ stor open | table -e 
╭──────┬────────────────────────────────────────────────────────────────────╮
│      │ ╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮ │
│ nudb │ │ 0 │ 1 │   1 │  2 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │ │
│      │ ╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯ │
╰──────┴────────────────────────────────────────────────────────────────────╯
```
## stor update
```nushell
❯ stor update --table-name nudb --update-record {str1: toby datetime1: 2021-04-17} --where-clause "bool1 = 1"
╭──────┬───────────────╮
│ nudb │ [table 1 row] │
╰──────┴───────────────╯
❯ stor open | table -e
╭──────┬─────────────────────────────────────────────────────────────────╮
│      │ ╭─#─┬id─┬bool1┬int1┬float1┬─str1─┬─────────datetime1──────────╮ │
│ nudb │ │ 0 │ 1 │   1 │  2 │ 1.10 │ toby │ 2021-04-17 00:00:00 +00:00 │ │
│      │ ╰───┴───┴─────┴────┴──────┴──────┴────────────────────────────╯ │
╰──────┴─────────────────────────────────────────────────────────────────╯
```
## insert another row
```nushell
❯ stor insert --table-name nudb --data-record {bool1: true, int1: 5, float1: 1.1, str1: fdncred, datetime1: 2023-04-17} 
╭──────┬────────────────╮
│ nudb │ [table 2 rows] │
╰──────┴────────────────╯
❯ stor open | table -e
╭──────┬────────────────────────────────────────────────────────────────────╮
│      │ ╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮ │
│ nudb │ │ 0 │ 1 │   1 │  2 │ 1.10 │ toby    │ 2021-04-17 00:00:00 +00:00 │ │
│      │ │ 1 │ 2 │   1 │  5 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │ │
│      │ ╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯ │
╰──────┴────────────────────────────────────────────────────────────────────╯
```
## stor delete (specific row(s))
```nushell
❯ stor delete --table-name nudb --where-clause "int1 == 5"
╭──────┬───────────────╮
│ nudb │ [table 1 row] │
╰──────┴───────────────╯
```
## insert multiple tables
```nushell
❯ stor create --table-name nudb1 --columns {bool1: bool, int1: int, float1: float, str1: str, datetime1: datetime}
╭───────┬────────────────╮
│ nudb  │ [table 1 row]  │
│ nudb1 │ [list 0 items] │
╰───────┴────────────────╯
❯ stor insert --table-name nudb1 --data-record {bool1: true, int1: 2, float1: 1.1, str1: fdncred, datetime1: 2023-04-17}
╭───────┬───────────────╮
│ nudb  │ [table 1 row] │
│ nudb1 │ [table 1 row] │
╰───────┴───────────────╯
❯ stor create --table-name nudb2 --columns {bool1: bool, int1: int, float1: float, str1: str, datetime1: datetime}
╭───────┬────────────────╮
│ nudb  │ [table 1 row]  │
│ nudb1 │ [table 1 row]  │
│ nudb2 │ [list 0 items] │
╰───────┴────────────────╯
❯ stor insert --table-name nudb2 --data-record {bool1: true, int1: 2, float1: 1.1, str1: fdncred, datetime1: 2023-04-17}
╭───────┬───────────────╮
│ nudb  │ [table 1 row] │
│ nudb1 │ [table 1 row] │
│ nudb2 │ [table 1 row] │
╰───────┴───────────────╯
```
## stor delete (specific table)
```nushell
❯ stor delete --table-name nudb1
╭───────┬───────────────╮
│ nudb  │ [table 1 row] │
│ nudb2 │ [table 1 row] │
╰───────┴───────────────╯
```
## stor reset (all tables are deleted)
```nushell
❯ stor reset
```
## stor export
```nushell
❯ stor export --file-name nudb.sqlite3
╭──────┬───────────────╮
│ nudb │ [table 1 row] │
╰──────┴───────────────╯
❯ open nudb.sqlite3 | table -e
╭──────┬────────────────────────────────────────────────────────────────────╮
│      │ ╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮ │
│ nudb │ │ 0 │ 1 │   1 │  5 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │ │
│      │ ╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯ │
╰──────┴────────────────────────────────────────────────────────────────────╯
❯ open nudb.sqlite3 | schema | table -e
╭────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│        │ ╭──────┬──────────────────────────────────────────────────────────────────────────────────────────────────╮ │
│ tables │ │      │ ╭───────────────┬──────────────────────────────────────────────────────────────────────────────╮ │ │
│        │ │ nudb │ │               │ ╭─#─┬─cid─┬───name────┬─────type─────┬─notnull─┬───────default────────┬─pk─╮ │ │ │
│        │ │      │ │ columns       │ │ 0 │ 0   │ id        │ INTEGER      │ 1       │                      │ 1  │ │ │ │
│        │ │      │ │               │ │ 1 │ 1   │ bool1     │ BOOLEAN      │ 0       │                      │ 0  │ │ │ │
│        │ │      │ │               │ │ 2 │ 2   │ int1      │ INTEGER      │ 0       │                      │ 0  │ │ │ │
│        │ │      │ │               │ │ 3 │ 3   │ float1    │ REAL         │ 0       │                      │ 0  │ │ │ │
│        │ │      │ │               │ │ 4 │ 4   │ str1      │ VARCHAR(255) │ 0       │                      │ 0  │ │ │ │
│        │ │      │ │               │ │ 5 │ 5   │ datetime1 │ DATETIME     │ 0       │ STRFTIME('%Y-%m-%d   │ 0  │ │ │ │
│        │ │      │ │               │ │   │     │           │              │         │ %H:%M:%f', 'NOW')    │    │ │ │ │
│        │ │      │ │               │ ╰─#─┴─cid─┴───name────┴─────type─────┴─notnull─┴───────default────────┴─pk─╯ │ │ │
│        │ │      │ │ constraints   │ [list 0 items]                                                               │ │ │
│        │ │      │ │ foreign_keys  │ [list 0 items]                                                               │ │ │
│        │ │      │ │ indexes       │ [list 0 items]                                                               │ │ │
│        │ │      │ ╰───────────────┴──────────────────────────────────────────────────────────────────────────────╯ │ │
│        │ ╰──────┴──────────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
```
## Using with `query db`
```nushell
❯ stor open | query db "select * from nudb"
╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮
│ 0 │ 1 │   1 │  5 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │
╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯
```
## stor import
```nushell
❯ stor open
# note, nothing is returned. there is nothing in memory, atm.
❯ stor import --file-name nudb.sqlite3
╭──────┬───────────────╮
│ nudb │ [table 1 row] │
╰──────┴───────────────╯
❯ stor open | table -e 
╭──────┬────────────────────────────────────────────────────────────────────╮
│      │ ╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮ │
│ nudb │ │ 0 │ 1 │   1 │  5 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │ │
│      │ ╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯ │
╰──────┴────────────────────────────────────────────────────────────────────╯
```

TODO:
- [x] `stor export` - Export a fully formed sqlite db file. 
- [x] `stor import` - Imports a specified sqlite db file.
- [x] Perhaps feature-gate it with the sqlite feature
- [x] Update `query db` to work with the in-memory database
- [x] Remove `open --in-memory`

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

crates/nu-command/Cargo.toml

@@ -73,7 +73,7 @@ rand = "0.8"
 rayon = "1.8"
 regex = "1.9.5"
 roxmltree = "0.18"
-rusqlite = { version = "0.29", features = ["bundled"], optional = true }
+rusqlite = { version = "0.29", features = ["bundled", "backup"], optional = true }
 same-file = "1.0"
 serde = { version = "1.0", features = ["derive"] }
 serde_json = "1.0"

crates/nu-command/src/database/mod.rs

@@ -5,7 +5,7 @@ use commands::add_commands_decls;
 
 pub use values::{
     convert_sqlite_row_to_nu_value, convert_sqlite_value_to_nu_value, open_connection_in_memory,
-    SQLiteDatabase,
+    open_connection_in_memory_custom, SQLiteDatabase, MEMORY_DB,
 };
 
 use nu_protocol::engine::StateWorkingSet;

crates/nu-command/src/database/values/mod.rs

@@ -3,5 +3,5 @@ pub mod sqlite;
 
 pub use sqlite::{
     convert_sqlite_row_to_nu_value, convert_sqlite_value_to_nu_value, open_connection_in_memory,
-    SQLiteDatabase,
+    open_connection_in_memory_custom, SQLiteDatabase, MEMORY_DB,
 };

crates/nu-command/src/database/values/sqlite.rs

@@ -2,9 +2,10 @@ use super::definitions::{
     db_column::DbColumn, db_constraint::DbConstraint, db_foreignkey::DbForeignKey,
     db_index::DbIndex, db_table::DbTable,
 };
-
 use nu_protocol::{CustomValue, PipelineData, Record, ShellError, Span, Spanned, Value};
-use rusqlite::{types::ValueRef, Connection, Row};
+use rusqlite::{
+    types::ValueRef, Connection, DatabaseName, Error as SqliteError, OpenFlags, Row, Statement,
+};
 use serde::{Deserialize, Serialize};
 use std::{
     fs::File,
@@ -14,8 +15,9 @@ use std::{
 };
 
 const SQLITE_MAGIC_BYTES: &[u8] = "SQLite format 3\0".as_bytes();
+pub const MEMORY_DB: &str = "file:memdb1?mode=memory&cache=shared";
 
-#[derive(Debug, Serialize, Deserialize)]
+#[derive(Debug, Clone, Serialize, Deserialize)]
 pub struct SQLiteDatabase {
     // I considered storing a SQLite connection here, but decided against it because
     // 1) YAGNI, 2) it's not obvious how cloning a connection could work, 3) state
@@ -85,13 +87,14 @@ impl SQLiteDatabase {
     }
 
     pub fn into_value(self, span: Span) -> Value {
-        Value::custom_value(Box::new(self), span)
+        let db = Box::new(self);
+        Value::custom_value(db, span)
     }
 
     pub fn query(&self, sql: &Spanned<String>, call_span: Span) -> Result<Value, ShellError> {
-        let db = open_sqlite_db(&self.path, call_span)?;
+        let conn = open_sqlite_db(&self.path, call_span)?;
 
-        let stream = run_sql_query(db, sql, self.ctrlc.clone()).map_err(|e| {
+        let stream = run_sql_query(conn, sql, self.ctrlc.clone()).map_err(|e| {
             ShellError::GenericError(
                 "Failed to query SQLite database".into(),
                 e.to_string(),
@@ -104,11 +107,23 @@ impl SQLiteDatabase {
         Ok(stream)
     }
 
-    pub fn open_connection(&self) -> Result<Connection, rusqlite::Error> {
+    pub fn open_connection(&self) -> Result<Connection, ShellError> {
-        Connection::open(&self.path)
+        if self.path == PathBuf::from(MEMORY_DB) {
+            open_connection_in_memory_custom()
+        } else {
+            Connection::open(&self.path).map_err(|e| {
+                ShellError::GenericError(
+                    "Failed to open SQLite database from open_connection".into(),
+                    e.to_string(),
+                    None,
+                    None,
+                    Vec::new(),
+                )
+            })
+        }
     }
 
-    pub fn get_tables(&self, conn: &Connection) -> Result<Vec<DbTable>, rusqlite::Error> {
+    pub fn get_tables(&self, conn: &Connection) -> Result<Vec<DbTable>, SqliteError> {
         let mut table_names =
             conn.prepare("SELECT name FROM sqlite_master WHERE type = 'table'")?;
         let rows = table_names.query_map([], |row| row.get(0))?;
@@ -128,7 +143,59 @@ impl SQLiteDatabase {
         Ok(tables.into_iter().collect())
     }
 
-    fn get_column_info(&self, row: &Row) -> Result<DbColumn, rusqlite::Error> {
+    pub fn drop_all_tables(&self, conn: &Connection) -> Result<(), SqliteError> {
+        let tables = self.get_tables(conn)?;
+
+        for table in tables {
+            conn.execute(&format!("DROP TABLE {}", table.name), [])?;
+        }
+
+        Ok(())
+    }
+
+    pub fn export_in_memory_database_to_file(
+        &self,
+        conn: &Connection,
+        filename: String,
+    ) -> Result<(), SqliteError> {
+        //vacuum main into 'c:\\temp\\foo.db'
+        conn.execute(&format!("vacuum main into '{}'", filename), [])?;
+
+        Ok(())
+    }
+
+    pub fn backup_database_to_file(
+        &self,
+        conn: &Connection,
+        filename: String,
+    ) -> Result<(), SqliteError> {
+        conn.backup(DatabaseName::Main, Path::new(&filename), None)?;
+        Ok(())
+    }
+
+    pub fn restore_database_from_file(
+        &self,
+        conn: &mut Connection,
+        filename: String,
+    ) -> Result<(), SqliteError> {
+        conn.restore(
+            DatabaseName::Main,
+            Path::new(&filename),
+            Some(|p: rusqlite::backup::Progress| {
+                let percent = if p.pagecount == 0 {
+                    100
+                } else {
+                    (p.pagecount - p.remaining) * 100 / p.pagecount
+                };
+                if percent % 10 == 0 {
+                    log::trace!("Restoring: {} %", percent);
+                }
+            }),
+        )?;
+        Ok(())
+    }
+
+    fn get_column_info(&self, row: &Row) -> Result<DbColumn, SqliteError> {
         let dbc = DbColumn {
             cid: row.get("cid")?,
             name: row.get("name")?,
@@ -144,7 +211,7 @@ impl SQLiteDatabase {
         &self,
         conn: &Connection,
         table: &DbTable,
-    ) -> Result<Vec<DbColumn>, rusqlite::Error> {
+    ) -> Result<Vec<DbColumn>, SqliteError> {
         let mut column_names = conn.prepare(&format!(
             "SELECT * FROM pragma_table_info('{}');",
             table.name
@@ -160,7 +227,7 @@ impl SQLiteDatabase {
         Ok(columns)
     }
 
-    fn get_constraint_info(&self, row: &Row) -> Result<DbConstraint, rusqlite::Error> {
+    fn get_constraint_info(&self, row: &Row) -> Result<DbConstraint, SqliteError> {
         let dbc = DbConstraint {
             name: row.get("index_name")?,
             column_name: row.get("column_name")?,
@@ -173,7 +240,7 @@ impl SQLiteDatabase {
         &self,
         conn: &Connection,
         table: &DbTable,
-    ) -> Result<Vec<DbConstraint>, rusqlite::Error> {
+    ) -> Result<Vec<DbConstraint>, SqliteError> {
         let mut column_names = conn.prepare(&format!(
             "
             SELECT
@@ -202,7 +269,7 @@ impl SQLiteDatabase {
         Ok(constraints)
     }
 
-    fn get_foreign_keys_info(&self, row: &Row) -> Result<DbForeignKey, rusqlite::Error> {
+    fn get_foreign_keys_info(&self, row: &Row) -> Result<DbForeignKey, SqliteError> {
         let dbc = DbForeignKey {
             column_name: row.get("from")?,
             ref_table: row.get("table")?,
@@ -215,7 +282,7 @@ impl SQLiteDatabase {
         &self,
         conn: &Connection,
         table: &DbTable,
-    ) -> Result<Vec<DbForeignKey>, rusqlite::Error> {
+    ) -> Result<Vec<DbForeignKey>, SqliteError> {
         let mut column_names = conn.prepare(&format!(
             "SELECT p.`from`, p.`to`, p.`table` FROM pragma_foreign_key_list('{}') p",
             &table.name
@@ -231,7 +298,7 @@ impl SQLiteDatabase {
         Ok(foreign_keys)
     }
 
-    fn get_index_info(&self, row: &Row) -> Result<DbIndex, rusqlite::Error> {
+    fn get_index_info(&self, row: &Row) -> Result<DbIndex, SqliteError> {
         let dbc = DbIndex {
             name: row.get("index_name")?,
             column_name: row.get("name")?,
@@ -244,7 +311,7 @@ impl SQLiteDatabase {
         &self,
         conn: &Connection,
         table: &DbTable,
-    ) -> Result<Vec<DbIndex>, rusqlite::Error> {
+    ) -> Result<Vec<DbIndex>, SqliteError> {
         let mut column_names = conn.prepare(&format!(
             "
             SELECT
@@ -330,9 +397,11 @@ impl CustomValue for SQLiteDatabase {
     }
 }
 
-pub fn open_sqlite_db(path: &Path, call_span: Span) -> Result<Connection, nu_protocol::ShellError> {
+pub fn open_sqlite_db(path: &Path, call_span: Span) -> Result<Connection, ShellError> {
+    if path.to_string_lossy() == MEMORY_DB {
+        open_connection_in_memory_custom()
+    } else {
         let path = path.to_string_lossy().to_string();
-
         Connection::open(path).map_err(|e| {
             ShellError::GenericError(
                 "Failed to open SQLite database".into(),
@@ -342,13 +411,14 @@ pub fn open_sqlite_db(path: &Path, call_span: Span) -> Result<Connection, nu_pro
                 Vec::new(),
             )
         })
+    }
 }
 
 fn run_sql_query(
     conn: Connection,
     sql: &Spanned<String>,
     ctrlc: Option<Arc<AtomicBool>>,
-) -> Result<Value, rusqlite::Error> {
+) -> Result<Value, SqliteError> {
     let stmt = conn.prepare(&sql.item)?;
     prepared_statement_to_nu_list(stmt, sql.span, ctrlc)
 }
@@ -358,16 +428,16 @@ fn read_single_table(
     table_name: String,
     call_span: Span,
     ctrlc: Option<Arc<AtomicBool>>,
-) -> Result<Value, rusqlite::Error> {
+) -> Result<Value, SqliteError> {
     let stmt = conn.prepare(&format!("SELECT * FROM [{table_name}]"))?;
     prepared_statement_to_nu_list(stmt, call_span, ctrlc)
 }
 
 fn prepared_statement_to_nu_list(
-    mut stmt: rusqlite::Statement,
+    mut stmt: Statement,
     call_span: Span,
     ctrlc: Option<Arc<AtomicBool>>,
-) -> Result<Value, rusqlite::Error> {
+) -> Result<Value, SqliteError> {
     let column_names = stmt
         .column_names()
         .iter()
@@ -403,7 +473,7 @@ fn read_entire_sqlite_db(
     conn: Connection,
     call_span: Span,
     ctrlc: Option<Arc<AtomicBool>>,
-) -> Result<Value, rusqlite::Error> {
+) -> Result<Value, SqliteError> {
     let mut tables = Record::new();
 
     let mut get_table_names =
@@ -455,9 +525,8 @@ pub fn convert_sqlite_value_to_nu_value(value: ValueRef, span: Span) -> Value {
 
 #[cfg(test)]
 mod test {
-    use nu_protocol::record;
-
     use super::*;
+    use nu_protocol::record;
 
     #[test]
     fn can_read_empty_db() {
@@ -532,10 +601,23 @@ mod test {
     }
 }
 
-pub fn open_connection_in_memory() -> Result<Connection, ShellError> {
+pub fn open_connection_in_memory_custom() -> Result<Connection, ShellError> {
-    Connection::open_in_memory().map_err(|err| {
+    let flags = OpenFlags::default();
+    Connection::open_with_flags(MEMORY_DB, flags).map_err(|err| {
         ShellError::GenericError(
-            "Failed to open SQLite connection in memory".into(),
+            "Failed to open SQLite custom connection in memory".into(),
+            err.to_string(),
+            Some(Span::test_data()),
+            None,
+            Vec::new(),
+        )
+    })
+}
+
+pub fn open_connection_in_memory() -> Result<Connection, ShellError> {
+    Connection::open_in_memory().map_err(|err| {
+        ShellError::GenericError(
+            "Failed to open SQLite standard connection in memory".into(),
             err.to_string(),
             Some(Span::test_data()),
             None,

crates/nu-command/src/default_context.rs

@@ -404,6 +404,20 @@ pub fn add_shell_command_context(mut engine_state: EngineState) -> EngineState {
             DateFormat,
         };
 
+        // Stor
+        #[cfg(feature = "sqlite")]
+        bind_command! {
+            Stor,
+            StorCreate,
+            StorDelete,
+            StorExport,
+            StorImport,
+            StorInsert,
+            StorOpen,
+            StorReset,
+            StorUpdate,
+        };
+
         working_set.render()
     };
 

crates/nu-command/src/lib.rs

@@ -23,6 +23,7 @@ mod random;
 mod removed;
 mod shells;
 mod sort_utils;
+mod stor;
 mod strings;
 mod system;
 mod viewers;
@@ -52,6 +53,8 @@ pub use random::*;
 pub use removed::*;
 pub use shells::*;
 pub use sort_utils::*;
+#[cfg(feature = "sqlite")]
+pub use stor::*;
 pub use strings::*;
 pub use system::*;
 pub use viewers::*;

crates/nu-command/src/stor/create.rs

@@ -0,0 +1,152 @@
+use crate::database::{SQLiteDatabase, MEMORY_DB};
+use nu_engine::CallExt;
+use nu_protocol::{
+    ast::Call,
+    engine::{Command, EngineState, Stack},
+    Category, Example, IntoPipelineData, PipelineData, Record, ShellError, Signature, Span,
+    SyntaxShape, Type, Value,
+};
+
+#[derive(Clone)]
+pub struct StorCreate;
+
+impl Command for StorCreate {
+    fn name(&self) -> &str {
+        "stor create"
+    }
+
+    fn signature(&self) -> Signature {
+        Signature::build("stor create")
+            .input_output_types(vec![(Type::Nothing, Type::Table(vec![]))])
+            .required_named(
+                "table-name",
+                SyntaxShape::String,
+                "name of the table you want to create",
+                Some('t'),
+            )
+            .required_named(
+                "columns",
+                SyntaxShape::Record(vec![]),
+                "a record of column names and datatypes",
+                Some('c'),
+            )
+            .allow_variants_without_examples(true)
+            .category(Category::Math)
+    }
+
+    fn usage(&self) -> &str {
+        "Create a table in the in-memory sqlite database"
+    }
+
+    fn search_terms(&self) -> Vec<&str> {
+        vec!["sqlite", "storing", "table"]
+    }
+
+    fn examples(&self) -> Vec<Example> {
+        vec![Example {
+            description: "Create an in-memory sqlite database with specified table name, column names, and column data types",
+            example: "stor create --table-name nudb --columns {bool1: bool, int1: int, float1: float, str1: str, datetime1: datetime}",
+            result: None,
+        }]
+    }
+
+    fn run(
+        &self,
+        engine_state: &EngineState,
+        stack: &mut Stack,
+        call: &Call,
+        _input: PipelineData,
+    ) -> Result<PipelineData, ShellError> {
+        let span = call.head;
+        let table_name: Option<String> = call.get_flag(engine_state, stack, "table-name")?;
+        let columns: Option<Record> = call.get_flag(engine_state, stack, "columns")?;
+        let db = Box::new(SQLiteDatabase::new(std::path::Path::new(MEMORY_DB), None));
+
+        if table_name.is_none() {
+            return Err(ShellError::MissingParameter {
+                param_name: "requires at table name".into(),
+                span,
+            });
+        }
+        let new_table_name = table_name.unwrap_or("table".into());
+        if let Ok(conn) = db.open_connection() {
+            match columns {
+                Some(record) => {
+                    let mut create_stmt = format!(
+                        "CREATE TABLE {} ( id INTEGER NOT NULL PRIMARY KEY, ",
+                        new_table_name
+                    );
+                    for (column_name, column_datatype) in record {
+                        match column_datatype.as_string()?.as_str() {
+                            "int" => {
+                                create_stmt.push_str(&format!("{} INTEGER, ", column_name));
+                            }
+                            "float" => {
+                                create_stmt.push_str(&format!("{} REAL, ", column_name));
+                            }
+                            "str" => {
+                                create_stmt.push_str(&format!("{} VARCHAR(255), ", column_name));
+                            }
+
+                            "bool" => {
+                                create_stmt.push_str(&format!("{} BOOLEAN, ", column_name));
+                            }
+                            "datetime" => {
+                                create_stmt.push_str(&format!(
+                                    "{} DATETIME DEFAULT(STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')), ",
+                                    column_name
+                                ));
+                            }
+
+                            _ => {
+                                return Err(ShellError::UnsupportedInput {
+                                    msg: "unsupported column data type".into(),
+                                    input: format!("{:?}", column_datatype),
+                                    msg_span: column_datatype.span(),
+                                    input_span: column_datatype.span(),
+                                });
+                            }
+                        }
+                    }
+                    if create_stmt.ends_with(", ") {
+                        create_stmt.pop();
+                        create_stmt.pop();
+                    }
+                    create_stmt.push_str(" )");
+
+                    // dbg!(&create_stmt);
+
+                    conn.execute(&create_stmt, []).map_err(|err| {
+                        ShellError::GenericError(
+                            "Failed to open SQLite connection in memory from create".into(),
+                            err.to_string(),
+                            Some(Span::test_data()),
+                            None,
+                            Vec::new(),
+                        )
+                    })?;
+                }
+                None => {
+                    return Err(ShellError::MissingParameter {
+                        param_name: "requires at least one column".into(),
+                        span: call.head,
+                    });
+                }
+            };
+        }
+        // dbg!(db.clone());
+        Ok(Value::custom_value(db, span).into_pipeline_data())
+    }
+}
+
+#[cfg(test)]
+mod test {
+    use super::*;
+
+    #[test]
+    fn test_examples() {
+        use crate::test_examples;
+
+        test_examples(StorCreate {})
+    }
+}

crates/nu-command/src/stor/delete.rs

@@ -0,0 +1,141 @@
+use crate::database::{SQLiteDatabase, MEMORY_DB};
+use nu_engine::CallExt;
+use nu_protocol::{
+    ast::Call,
+    engine::{Command, EngineState, Stack},
+    Category, Example, IntoPipelineData, PipelineData, ShellError, Signature, Span, SyntaxShape,
+    Type, Value,
+};
+
+#[derive(Clone)]
+pub struct StorDelete;
+
+impl Command for StorDelete {
+    fn name(&self) -> &str {
+        "stor delete"
+    }
+
+    fn signature(&self) -> Signature {
+        Signature::build("stor delete")
+            .input_output_types(vec![(Type::Nothing, Type::Table(vec![]))])
+            .required_named(
+                "table-name",
+                SyntaxShape::String,
+                "name of the table you want to insert into",
+                Some('t'),
+            )
+            .named(
+                "where-clause",
+                SyntaxShape::String,
+                "a sql string to use as a where clause without the WHERE keyword",
+                Some('w'),
+            )
+            .allow_variants_without_examples(true)
+            .category(Category::Math)
+    }
+
+    fn usage(&self) -> &str {
+        "Delete a table or specified rows in the in-memory sqlite database"
+    }
+
+    fn search_terms(&self) -> Vec<&str> {
+        vec!["sqlite", "remove", "table", "saving", "drop"]
+    }
+
+    fn examples(&self) -> Vec<Example> {
+        vec![
+            Example {
+                description: "Delete a table from the in-memory sqlite database",
+                example: "stor delete --table-name nudb",
+                result: None,
+            },
+            Example {
+                description:
+                    "Delete some rows from the in-memory sqlite database with a where clause",
+                example: "stor delete --table-name nudb --where-clause \"int1 == 5\"",
+                result: None,
+            },
+        ]
+    }
+
+    fn run(
+        &self,
+        engine_state: &EngineState,
+        stack: &mut Stack,
+        call: &Call,
+        _input: PipelineData,
+    ) -> Result<PipelineData, ShellError> {
+        let span = call.head;
+        // For dropping/deleting an entire table
+        let table_name_opt: Option<String> = call.get_flag(engine_state, stack, "table-name")?;
+
+        // For deleting rows from a table
+        let where_clause_opt: Option<String> =
+            call.get_flag(engine_state, stack, "where-clause")?;
+
+        if table_name_opt.is_none() && where_clause_opt.is_none() {
+            return Err(ShellError::MissingParameter {
+                param_name: "requires at least one of table-name or where-clause".into(),
+                span,
+            });
+        }
+
+        if table_name_opt.is_none() && where_clause_opt.is_some() {
+            return Err(ShellError::MissingParameter {
+                param_name: "using the where-clause requires the use of a table-name".into(),
+                span,
+            });
+        }
+
+        // Open the in-mem database
+        let db = Box::new(SQLiteDatabase::new(std::path::Path::new(MEMORY_DB), None));
+
+        if let Some(new_table_name) = table_name_opt {
+            let where_clause = match where_clause_opt {
+                Some(where_stmt) => where_stmt,
+                None => String::new(),
+            };
+
+            if let Ok(conn) = db.open_connection() {
+                let sql_stmt = if where_clause.is_empty() {
+                    // We're deleting an entire table
+                    format!("DROP TABLE {}", new_table_name)
+                } else {
+                    // We're just deleting some rows
+                    let mut delete_stmt = format!("DELETE FROM {} ", new_table_name);
+
+                    // Yup, this is a bit janky, but I'm not sure a better way to do this without having
+                    // --and and --or flags as well as supporting ==, !=, <>, is null, is not null, etc.
+                    // and other sql syntax. So, for now, just type a sql where clause as a string.
+                    delete_stmt.push_str(&format!("WHERE {}", where_clause));
+                    delete_stmt
+                };
+
+                // dbg!(&sql_stmt);
+                conn.execute(&sql_stmt, []).map_err(|err| {
+                    ShellError::GenericError(
+                        "Failed to open SQLite connection in memory from delete".into(),
+                        err.to_string(),
+                        Some(Span::test_data()),
+                        None,
+                        Vec::new(),
+                    )
+                })?;
+            }
+        }
+        // dbg!(db.clone());
+        Ok(Value::custom_value(db, span).into_pipeline_data())
+    }
+}
+
+#[cfg(test)]
+mod test {
+    use super::*;
+
+    #[test]
+    fn test_examples() {
+        use crate::test_examples;
+
+        test_examples(StorDelete {})
+    }
+}

crates/nu-command/src/stor/export.rs

@@ -0,0 +1,98 @@
+use crate::database::{SQLiteDatabase, MEMORY_DB};
+use nu_engine::CallExt;
+use nu_protocol::{
+    ast::Call,
+    engine::{Command, EngineState, Stack},
+    Category, Example, IntoPipelineData, PipelineData, ShellError, Signature, Span, SyntaxShape,
+    Type, Value,
+};
+
+#[derive(Clone)]
+pub struct StorExport;
+
+impl Command for StorExport {
+    fn name(&self) -> &str {
+        "stor export"
+    }
+
+    fn signature(&self) -> Signature {
+        Signature::build("stor export")
+            .input_output_types(vec![(Type::Nothing, Type::Table(vec![]))])
+            .required_named(
+                "file-name",
+                SyntaxShape::String,
+                "file name to export the sqlite in-memory database to",
+                Some('f'),
+            )
+            .allow_variants_without_examples(true)
+            .category(Category::Math)
+    }
+
+    fn usage(&self) -> &str {
+        "Export the in-memory sqlite database to a sqlite database file"
+    }
+
+    fn search_terms(&self) -> Vec<&str> {
+        vec!["sqlite", "save", "database", "saving", "file"]
+    }
+
+    fn examples(&self) -> Vec<Example> {
+        vec![Example {
+            description: "Export the in-memory sqlite database",
+            example: "stor export --file-name nudb.sqlite",
+            result: None,
+        }]
+    }
+
+    fn run(
+        &self,
+        engine_state: &EngineState,
+        stack: &mut Stack,
+        call: &Call,
+        _input: PipelineData,
+    ) -> Result<PipelineData, ShellError> {
+        let span = call.head;
+        let file_name_opt: Option<String> = call.get_flag(engine_state, stack, "file-name")?;
+        let file_name = match file_name_opt {
+            Some(file_name) => file_name,
+            None => {
+                return Err(ShellError::MissingParameter {
+                    param_name: "please supply a file name with the --file-name parameter".into(),
+                    span,
+                })
+            }
+        };
+
+        // Open the in-mem database
+        let db = Box::new(SQLiteDatabase::new(std::path::Path::new(MEMORY_DB), None));
+
+        if let Ok(conn) = db.open_connection() {
+            // This uses vacuum. I'm not really sure if this is the best way to do this.
+            // I also added backup in the sqlitedatabase impl. If we have problems, we could switch to that.
+            db.export_in_memory_database_to_file(&conn, file_name)
+                .map_err(|err| {
+                    ShellError::GenericError(
+                        "Failed to open SQLite connection in memory from export".into(),
+                        err.to_string(),
+                        Some(Span::test_data()),
+                        None,
+                        Vec::new(),
+                    )
+                })?;
+        }
+        // dbg!(db.clone());
+        Ok(Value::custom_value(db, span).into_pipeline_data())
+    }
+}
+
+#[cfg(test)]
+mod test {
+    use super::*;
+
+    #[test]
+    fn test_examples() {
+        use crate::test_examples;
+
+        test_examples(StorExport {})
+    }
+}

crates/nu-command/src/stor/import.rs

@@ -0,0 +1,96 @@
+use crate::database::{SQLiteDatabase, MEMORY_DB};
+use nu_engine::CallExt;
+use nu_protocol::{
+    ast::Call,
+    engine::{Command, EngineState, Stack},
+    Category, Example, IntoPipelineData, PipelineData, ShellError, Signature, Span, SyntaxShape,
+    Type, Value,
+};
+
+#[derive(Clone)]
+pub struct StorImport;
+
+impl Command for StorImport {
+    fn name(&self) -> &str {
+        "stor import"
+    }
+
+    fn signature(&self) -> Signature {
+        Signature::build("stor import")
+            .input_output_types(vec![(Type::Nothing, Type::Table(vec![]))])
+            .required_named(
+                "file-name",
+                SyntaxShape::String,
+                "file name to export the sqlite in-memory database to",
+                Some('f'),
+            )
+            .allow_variants_without_examples(true)
+            .category(Category::Math)
+    }
+
+    fn usage(&self) -> &str {
+        "Import a sqlite database file into the in-memory sqlite database"
+    }
+
+    fn search_terms(&self) -> Vec<&str> {
+        vec!["sqlite", "open", "database", "restore", "file"]
+    }
+
+    fn examples(&self) -> Vec<Example> {
+        vec![Example {
+            description: "Import a sqlite database file into the in-memory sqlite database",
+            example: "stor import --file-name nudb.sqlite",
+            result: None,
+        }]
+    }
+
+    fn run(
+        &self,
+        engine_state: &EngineState,
+        stack: &mut Stack,
+        call: &Call,
+        _input: PipelineData,
+    ) -> Result<PipelineData, ShellError> {
+        let span = call.head;
+        let file_name_opt: Option<String> = call.get_flag(engine_state, stack, "file-name")?;
+        let file_name = match file_name_opt {
+            Some(file_name) => file_name,
+            None => {
+                return Err(ShellError::MissingParameter {
+                    param_name: "please supply a file name with the --file-name parameter".into(),
+                    span,
+                })
+            }
+        };
+
+        // Open the in-mem database
+        let db = Box::new(SQLiteDatabase::new(std::path::Path::new(MEMORY_DB), None));
+
+        if let Ok(mut conn) = db.open_connection() {
+            db.restore_database_from_file(&mut conn, file_name)
+                .map_err(|err| {
+                    ShellError::GenericError(
+                        "Failed to open SQLite connection in memory from import".into(),
+                        err.to_string(),
+                        Some(Span::test_data()),
+                        None,
+                        Vec::new(),
+                    )
+                })?;
+        }
+        // dbg!(db.clone());
+        Ok(Value::custom_value(db, span).into_pipeline_data())
+    }
+}
+
+#[cfg(test)]
+mod test {
+    use super::*;
+
+    #[test]
+    fn test_examples() {
+        use crate::test_examples;
+
+        test_examples(StorImport {})
+    }
+}

crates/nu-command/src/stor/insert.rs

@@ -0,0 +1,155 @@
+use crate::database::{SQLiteDatabase, MEMORY_DB};
+use nu_engine::CallExt;
+use nu_protocol::{
+    ast::Call,
+    engine::{Command, EngineState, Stack},
+    Category, Example, IntoPipelineData, PipelineData, Record, ShellError, Signature, Span,
+    SyntaxShape, Type, Value,
+};
+
+#[derive(Clone)]
+pub struct StorInsert;
+
+impl Command for StorInsert {
+    fn name(&self) -> &str {
+        "stor insert"
+    }
+
+    fn signature(&self) -> Signature {
+        Signature::build("stor insert")
+            .input_output_types(vec![(Type::Nothing, Type::Table(vec![]))])
+            .required_named(
+                "table-name",
+                SyntaxShape::String,
+                "name of the table you want to insert into",
+                Some('t'),
+            )
+            .required_named(
+                "data-record",
+                SyntaxShape::Record(vec![]),
+                "a record of column names and column values to insert into the specified table",
+                Some('d'),
+            )
+            .allow_variants_without_examples(true)
+            .category(Category::Math)
+    }
+
+    fn usage(&self) -> &str {
+        "Insert information into a specified table in the in-memory sqlite database"
+    }
+
+    fn search_terms(&self) -> Vec<&str> {
+        vec!["sqlite", "storing", "table", "saving"]
+    }
+
+    fn examples(&self) -> Vec<Example> {
+        vec![Example {
+            description: "Insert data the in-memory sqlite database using a data-record of column-name and column-value pairs",
+            example: "stor insert --table-name nudb --data-record {bool1: true, int1: 5, float1: 1.1, str1: fdncred, datetime1: 2023-04-17}",
+            result: None,
+        }]
+    }
+
+    fn run(
+        &self,
+        engine_state: &EngineState,
+        stack: &mut Stack,
+        call: &Call,
+        _input: PipelineData,
+    ) -> Result<PipelineData, ShellError> {
+        let span = call.head;
+        let table_name: Option<String> = call.get_flag(engine_state, stack, "table-name")?;
+        let columns: Option<Record> = call.get_flag(engine_state, stack, "data-record")?;
+        // let config = engine_state.get_config();
+        let db = Box::new(SQLiteDatabase::new(std::path::Path::new(MEMORY_DB), None));
+
+        if table_name.is_none() {
+            return Err(ShellError::MissingParameter {
+                param_name: "requires at table name".into(),
+                span,
+            });
+        }
+        let new_table_name = table_name.unwrap_or("table".into());
+        if let Ok(conn) = db.open_connection() {
+            match columns {
+                Some(record) => {
+                    let mut create_stmt = format!("INSERT INTO {} ( ", new_table_name);
+                    let cols = record.columns();
+                    cols.for_each(|col| {
+                        create_stmt.push_str(&format!("{}, ", col));
+                    });
+                    if create_stmt.ends_with(", ") {
+                        create_stmt.pop();
+                        create_stmt.pop();
+                    }
+
+                    create_stmt.push_str(") VALUES ( ");
+                    let vals = record.values();
+                    vals.for_each(|val| match val {
+                        Value::Int { val, .. } => {
+                            create_stmt.push_str(&format!("{}, ", val));
+                        }
+                        Value::Float { val, .. } => {
+                            create_stmt.push_str(&format!("{}, ", val));
+                        }
+                        Value::String { val, .. } => {
+                            create_stmt.push_str(&format!("'{}', ", val));
+                        }
+                        Value::Date { val, .. } => {
+                            create_stmt.push_str(&format!("'{}', ", val));
+                        }
+                        Value::Bool { val, .. } => {
+                            create_stmt.push_str(&format!("{}, ", val));
+                        }
+                        _ => {
+                            // return Err(ShellError::UnsupportedInput {
+                            //     msg: format!("{} is not a valid datepart, expected one of year, month, day, hour, minute, second, millisecond, microsecond, nanosecond", part.item),
+                            //     input: "value originates from here".to_string(),
+                            //     msg_span: span,
+                            //     input_span: val.span(),
+                            // });
+                        }
+                    });
+                    if create_stmt.ends_with(", ") {
+                        create_stmt.pop();
+                        create_stmt.pop();
+                    }
+
+                    create_stmt.push(')');
+
+                    // dbg!(&create_stmt);
+
+                    conn.execute(&create_stmt, []).map_err(|err| {
+                        ShellError::GenericError(
+                            "Failed to open SQLite connection in memory from insert".into(),
+                            err.to_string(),
+                            Some(Span::test_data()),
+                            None,
+                            Vec::new(),
+                        )
+                    })?;
+                }
+                None => {
+                    return Err(ShellError::MissingParameter {
+                        param_name: "requires at least one column".into(),
+                        span: call.head,
+                    });
+                }
+            };
+        }
+        // dbg!(db.clone());
+        Ok(Value::custom_value(db, span).into_pipeline_data())
+    }
+}
+
+#[cfg(test)]
+mod test {
+    use super::*;
+
+    #[test]
+    fn test_examples() {
+        use crate::test_examples;
+
+        test_examples(StorInsert {})
+    }
+}

crates/nu-command/src/stor/mod.rs

@@ -0,0 +1,35 @@
+#[cfg(feature = "sqlite")]
+mod create;
+#[cfg(feature = "sqlite")]
+mod delete;
+#[cfg(feature = "sqlite")]
+mod export;
+#[cfg(feature = "sqlite")]
+mod import;
+#[cfg(feature = "sqlite")]
+mod insert;
+#[cfg(feature = "sqlite")]
+mod open;
+#[cfg(feature = "sqlite")]
+mod reset;
+mod stor_;
+#[cfg(feature = "sqlite")]
+mod update;
+
+#[cfg(feature = "sqlite")]
+pub use create::StorCreate;
+#[cfg(feature = "sqlite")]
+pub use delete::StorDelete;
+#[cfg(feature = "sqlite")]
+pub use export::StorExport;
+#[cfg(feature = "sqlite")]
+pub use import::StorImport;
+#[cfg(feature = "sqlite")]
+pub use insert::StorInsert;
+#[cfg(feature = "sqlite")]
+pub use open::StorOpen;
+#[cfg(feature = "sqlite")]
+pub use reset::StorReset;
+pub use stor_::Stor;
+#[cfg(feature = "sqlite")]
+pub use update::StorUpdate;

crates/nu-command/src/stor/open.rs

@@ -0,0 +1,78 @@
+use crate::database::{SQLiteDatabase, MEMORY_DB};
+use nu_protocol::{
+    ast::Call,
+    engine::{Command, EngineState, Stack},
+    Category, Example, IntoPipelineData, PipelineData, ShellError, Signature, Type,
+};
+
+#[derive(Clone)]
+pub struct StorOpen;
+
+impl Command for StorOpen {
+    fn name(&self) -> &str {
+        "stor open"
+    }
+
+    fn signature(&self) -> Signature {
+        Signature::build("stor open")
+            .input_output_types(vec![(
+                Type::Nothing,
+                Type::Custom("sqlite-in-memory".into()),
+            )])
+            .allow_variants_without_examples(true)
+            .category(Category::Math)
+    }
+
+    fn usage(&self) -> &str {
+        "Opens the in-memory sqlite database"
+    }
+
+    fn search_terms(&self) -> Vec<&str> {
+        vec!["sqlite", "storing", "access"]
+    }
+
+    fn examples(&self) -> Vec<Example> {
+        vec![Example {
+            description: "Open the in-memory sqlite database",
+            example: "stor open",
+            result: None,
+        }]
+    }
+
+    fn run(
+        &self,
+        _engine_state: &EngineState,
+        _stack: &mut Stack,
+        call: &Call,
+        _input: PipelineData,
+    ) -> Result<PipelineData, ShellError> {
+        // eprintln!("Initializing nudb");
+        // eprintln!("Here's some things to try:");
+        // eprintln!("* stor open | schema | table -e");
+        // eprintln!("* stor open | query db 'insert into nudb (bool1,int1,float1,str1,datetime1) values (2,200,2.0,'str2','1969-04-17T06:00:00-05:00')'");
+        // eprintln!("* stor open | query db 'select * from nudb'");
+        // eprintln!("Now imagine all those examples happening as commands, without sql, in our normal nushell pipelines\n");
+
+        // TODO: Think about adding the following functionality
+        // * stor open --table-name my_table_name
+        //   It returns the output of `select * from my_table_name`
+
+        // Just create an empty database with MEMORY_DB and nothing else
+        let db = Box::new(SQLiteDatabase::new(std::path::Path::new(MEMORY_DB), None));
+
+        // dbg!(db.clone());
+        Ok(db.into_value(call.head).into_pipeline_data())
+    }
+}
+
+#[cfg(test)]
+mod test {
+    use super::*;
+
+    #[test]
+    fn test_examples() {
+        use crate::test_examples;
+
+        test_examples(StorOpen {})
+    }
+}

crates/nu-command/src/stor/reset.rs

@@ -0,0 +1,77 @@
+use crate::database::{SQLiteDatabase, MEMORY_DB};
+use nu_protocol::{
+    ast::Call,
+    engine::{Command, EngineState, Stack},
+    Category, Example, IntoPipelineData, PipelineData, ShellError, Signature, Span, Type, Value,
+};
+
+#[derive(Clone)]
+pub struct StorReset;
+
+impl Command for StorReset {
+    fn name(&self) -> &str {
+        "stor reset"
+    }
+
+    fn signature(&self) -> Signature {
+        Signature::build("stor reset")
+            .input_output_types(vec![(Type::Nothing, Type::Table(vec![]))])
+            .allow_variants_without_examples(true)
+            .category(Category::Math)
+    }
+
+    fn usage(&self) -> &str {
+        "Reset the in-memory database by dropping all tables"
+    }
+
+    fn search_terms(&self) -> Vec<&str> {
+        vec!["sqlite", "remove", "table", "saving", "drop"]
+    }
+
+    fn examples(&self) -> Vec<Example> {
+        vec![Example {
+            description: "Reset the in-memory sqlite database",
+            example: "stor reset",
+            result: None,
+        }]
+    }
+
+    fn run(
+        &self,
+        _engine_state: &EngineState,
+        _stack: &mut Stack,
+        call: &Call,
+        _input: PipelineData,
+    ) -> Result<PipelineData, ShellError> {
+        let span = call.head;
+
+        // Open the in-mem database
+        let db = Box::new(SQLiteDatabase::new(std::path::Path::new(MEMORY_DB), None));
+
+        if let Ok(conn) = db.open_connection() {
+            db.drop_all_tables(&conn).map_err(|err| {
+                ShellError::GenericError(
+                    "Failed to open SQLite connection in memory from reset".into(),
+                    err.to_string(),
+                    Some(Span::test_data()),
+                    None,
+                    Vec::new(),
+                )
+            })?;
+        }
+        // dbg!(db.clone());
+        Ok(Value::custom_value(db, span).into_pipeline_data())
+    }
+}
+
+#[cfg(test)]
+mod test {
+    use super::*;
+
+    #[test]
+    fn test_examples() {
+        use crate::test_examples;
+
+        test_examples(StorReset {})
+    }
+}

crates/nu-command/src/stor/stor_.rs

@@ -0,0 +1,49 @@
+use nu_engine::get_full_help;
+use nu_protocol::{
+    ast::Call,
+    engine::{Command, EngineState, Stack},
+    Category, IntoPipelineData, PipelineData, ShellError, Signature, Type, Value,
+};
+
+#[derive(Clone)]
+pub struct Stor;
+
+impl Command for Stor {
+    fn name(&self) -> &str {
+        "stor"
+    }
+
+    fn signature(&self) -> Signature {
+        Signature::build("stor")
+            .category(Category::Strings)
+            .input_output_types(vec![(Type::Nothing, Type::String)])
+    }
+
+    fn usage(&self) -> &str {
+        "Various commands for working with the in-memory sqlite database."
+    }
+
+    fn extra_usage(&self) -> &str {
+        "You must use one of the following subcommands. Using this command as-is will only produce this help message."
+    }
+
+    fn run(
+        &self,
+        engine_state: &EngineState,
+        stack: &mut Stack,
+        call: &Call,
+        _input: PipelineData,
+    ) -> Result<PipelineData, ShellError> {
+        Ok(Value::string(
+            get_full_help(
+                &Stor.signature(),
+                &Stor.examples(),
+                engine_state,
+                stack,
+                self.is_parser_keyword(),
+            ),
+            call.head,
+        )
+        .into_pipeline_data())
+    }
+}

crates/nu-command/src/stor/update.rs

@@ -0,0 +1,167 @@
+use crate::database::{SQLiteDatabase, MEMORY_DB};
+use nu_engine::CallExt;
+use nu_protocol::{
+    ast::Call,
+    engine::{Command, EngineState, Stack},
+    Category, Example, IntoPipelineData, PipelineData, Record, ShellError, Signature, Span,
+    Spanned, SyntaxShape, Type, Value,
+};
+
+#[derive(Clone)]
+pub struct StorUpdate;
+
+impl Command for StorUpdate {
+    fn name(&self) -> &str {
+        "stor update"
+    }
+
+    fn signature(&self) -> Signature {
+        Signature::build("stor update")
+            .input_output_types(vec![(Type::Nothing, Type::Table(vec![]))])
+            .required_named(
+                "table-name",
+                SyntaxShape::String,
+                "name of the table you want to insert into",
+                Some('t'),
+            )
+            .required_named(
+                "update-record",
+                SyntaxShape::Record(vec![]),
+                "a record of column names and column values to update in the specified table",
+                Some('u'),
+            )
+            .named(
+                "where-clause",
+                SyntaxShape::String,
+                "a sql string to use as a where clause without the WHERE keyword",
+                Some('w'),
+            )
+            .allow_variants_without_examples(true)
+            .category(Category::Math)
+    }
+
+    fn usage(&self) -> &str {
+        "Update information in a specified table in the in-memory sqlite database"
+    }
+
+    fn search_terms(&self) -> Vec<&str> {
+        vec!["sqlite", "storing", "table", "saving", "changing"]
+    }
+
+    fn examples(&self) -> Vec<Example> {
+        vec![
+        Example {
+            description: "Update the in-memory sqlite database",
+            example: "stor update --table-name nudb --update-record {str1: nushell datetime1: 2020-04-17}",
+            result: None,
+        },
+        Example {
+            description: "Update the in-memory sqlite database with a where clause",
+            example: "stor update --table-name nudb --update-record {str1: nushell datetime1: 2020-04-17} --where-clause \"bool1 = 1\"",
+            result: None,
+        },
+        ]
+    }
+
+    fn run(
+        &self,
+        engine_state: &EngineState,
+        stack: &mut Stack,
+        call: &Call,
+        _input: PipelineData,
+    ) -> Result<PipelineData, ShellError> {
+        let span = call.head;
+        let table_name: Option<String> = call.get_flag(engine_state, stack, "table-name")?;
+        let columns: Option<Record> = call.get_flag(engine_state, stack, "update-record")?;
+        let where_clause_opt: Option<Spanned<String>> =
+            call.get_flag(engine_state, stack, "where-clause")?;
+
+        // Open the in-mem database
+        let db = Box::new(SQLiteDatabase::new(std::path::Path::new(MEMORY_DB), None));
+
+        if table_name.is_none() {
+            return Err(ShellError::MissingParameter {
+                param_name: "requires at table name".into(),
+                span,
+            });
+        }
+        let new_table_name = table_name.unwrap_or("table".into());
+        if let Ok(conn) = db.open_connection() {
+            match columns {
+                Some(record) => {
+                    let mut update_stmt = format!("UPDATE {} ", new_table_name);
+
+                    update_stmt.push_str("SET ");
+                    let vals = record.iter();
+                    vals.for_each(|(key, val)| match val {
+                        Value::Int { val, .. } => {
+                            update_stmt.push_str(&format!("{} = {}, ", key, val));
+                        }
+                        Value::Float { val, .. } => {
+                            update_stmt.push_str(&format!("{} = {}, ", key, val));
+                        }
+                        Value::String { val, .. } => {
+                            update_stmt.push_str(&format!("{} = '{}', ", key, val));
+                        }
+                        Value::Date { val, .. } => {
+                            update_stmt.push_str(&format!("{} = '{}', ", key, val));
+                        }
+                        Value::Bool { val, .. } => {
+                            update_stmt.push_str(&format!("{} = {}, ", key, val));
+                        }
+                        _ => {
+                            // return Err(ShellError::UnsupportedInput {
+                            //     msg: format!("{} is not a valid datepart, expected one of year, month, day, hour, minute, second, millisecond, microsecond, nanosecond", part.item),
+                            //     input: "value originates from here".to_string(),
+                            //     msg_span: span,
+                            //     input_span: val.span(),
+                            // });
+                        }
+                    });
+                    if update_stmt.ends_with(", ") {
+                        update_stmt.pop();
+                        update_stmt.pop();
+                    }
+
+                    // Yup, this is a bit janky, but I'm not sure a better way to do this without having
+                    // --and and --or flags as well as supporting ==, !=, <>, is null, is not null, etc.
+                    // and other sql syntax. So, for now, just type a sql where clause as a string.
+                    if let Some(where_clause) = where_clause_opt {
+                        update_stmt.push_str(&format!(" WHERE {}", where_clause.item));
+                    }
+                    // dbg!(&update_stmt);
+
+                    conn.execute(&update_stmt, []).map_err(|err| {
+                        ShellError::GenericError(
+                            "Failed to open SQLite connection in memory from update".into(),
+                            err.to_string(),
+                            Some(Span::test_data()),
+                            None,
+                            Vec::new(),
+                        )
+                    })?;
+                }
+                None => {
+                    return Err(ShellError::MissingParameter {
+                        param_name: "requires at least one column".into(),
+                        span: call.head,
+                    });
+                }
+            };
+        }
+        // dbg!(db.clone());
+        Ok(Value::custom_value(db, span).into_pipeline_data())
+    }
+}
+
+#[cfg(test)]
+mod test {
+    use super::*;
+
+    #[test]
+    fn test_examples() {
+        use crate::test_examples;
+
+        test_examples(StorUpdate {})
+    }
+}

src/main.rs

@@ -80,6 +80,16 @@ fn main() -> Result<()> {
     ctrlc_protection(&mut engine_state, &ctrlc);
     sigquit_protection(&mut engine_state);
 
+    // This is the real secret sauce to having an in-memory sqlite db. You must
+    // start a connection to the memory database in main so it will exist for the
+    // lifetime of the program. If it's created with how MEMORY_DB is defined
+    // you'll be able to access this open connection from anywhere in the program
+    // by using the identical connection string.
+    #[cfg(feature = "sqlite")]
+    let db = nu_command::open_connection_in_memory_custom()?;
+    #[cfg(feature = "sqlite")]
+    db.last_insert_rowid();
+
     let (args_to_nushell, script_name, args_to_script) = gather_commandline_args();
     let parsed_nu_cli_args = parse_commandline_args(&args_to_nushell.join(" "), &mut engine_state)
         .unwrap_or_else(|_| std::process::exit(1));