revert: move to ahash (#9464)
This PR reverts https://github.com/nushell/nushell/pull/9391
We try not to revert PRs like this, though after discussion with the
Nushell team, we decided to revert this one.
The main reason is that Nushell, as a codebase, isn't ready for these
kinds of optimisations. It's in the part of the development cycle where
our main focus should be on improving the algorithms inside of Nushell
itself. Once we have matured our algorithms, then we can look for
opportunities to switch out technologies we're using for alternate
forms.
Much of Nushell still has lots of opportunities for tuning the codebase,
paying down technical debt, and making the codebase generally cleaner
and more robust. This should be the focus. Performance improvements
should flow out of that work.
Said another, optimisation that isn't part of tuning the codebase is
premature at this stage. We need to focus on doing the hard work of
making the engine, parser, etc better.
# User-Facing Changes
Reverts the HashMap -> ahash change.
cc @FilipAndersson245
2023-06-18 03:27:57 +00:00
|
|
|
use std::collections::{HashMap, HashSet};
|
2021-08-15 22:33:34 +00:00
|
|
|
|
2022-05-20 21:49:42 +00:00
|
|
|
use crate::engine::EngineState;
|
|
|
|
use crate::engine::DEFAULT_OVERLAY_NAME;
|
2022-04-18 22:28:01 +00:00
|
|
|
use crate::{ShellError, Span, Value, VarId};
|
2021-08-15 22:33:34 +00:00
|
|
|
|
2022-05-07 19:39:22 +00:00
|
|
|
/// Environment variables per overlay
|
|
|
|
pub type EnvVars = HashMap<String, HashMap<String, Value>>;
|
|
|
|
|
2021-11-02 19:53:48 +00:00
|
|
|
/// A runtime value stack used during evaluation
|
|
|
|
///
|
|
|
|
/// A note on implementation:
|
|
|
|
///
|
|
|
|
/// We previously set up the stack in a traditional way, where stack frames had parents which would
|
|
|
|
/// represent other frames that you might return to when exiting a function.
|
|
|
|
///
|
|
|
|
/// While experimenting with blocks, we found that we needed to have closure captures of variables
|
|
|
|
/// seen outside of the blocks, so that they blocks could be run in a way that was both thread-safe
|
|
|
|
/// and followed the restrictions for closures applied to iterators. The end result left us with
|
|
|
|
/// closure-captured single stack frames that blocks could see.
|
|
|
|
///
|
|
|
|
/// Blocks make up the only scope and stack definition abstraction in Nushell. As a result, we were
|
|
|
|
/// creating closure captures at any point we wanted to have a Block value we could safely evaluate
|
|
|
|
/// in any context. This meant that the parents were going largely unused, with captured variables
|
|
|
|
/// taking their place. The end result is this, where we no longer have separate frames, but instead
|
|
|
|
/// use the Stack as a way of representing the local and closure-captured state.
|
2021-10-25 04:01:02 +00:00
|
|
|
#[derive(Debug, Clone)]
|
2021-10-25 20:04:23 +00:00
|
|
|
pub struct Stack {
|
2021-11-30 06:14:05 +00:00
|
|
|
/// Variables
|
Move variables to var stack (#8604)
# Description
This moves the representation of variables on the stack to a Vec, which
more closely resembles a stack. For small numbers of variables live at
any one point, this tends to be more efficient than a HashMap. Having a
stack-like vector also allows us to remember a stack position,
temporarily push variables on, then quickly drop the stack back to the
original size when we're done. We'll need this capability to allow
matching inside of conditions.
On this mac, a simple run of:
`timeit { mut x = 1; while $x < 1000000 { $x += 1 } }`
Went from 1 sec 86 ms, down to 1 sec 2 ms. Clearly, we have a lot more
ground we can make up in looping speed 😅 but it's nice that for fixing
this to make matching easier, we also get a win in terms of lookup speed
for small numbers of variables.
# User-Facing Changes
Likely users won't (hopefully) see any negative impact and may even see
a small positive impact.
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
2023-03-24 23:56:45 +00:00
|
|
|
pub vars: Vec<(VarId, Value)>,
|
2021-11-30 06:14:05 +00:00
|
|
|
/// Environment variables arranged as a stack to be able to recover values from parent scopes
|
2022-05-07 19:39:22 +00:00
|
|
|
pub env_vars: Vec<EnvVars>,
|
|
|
|
/// Tells which environment variables from engine state are hidden, per overlay.
|
|
|
|
pub env_hidden: HashMap<String, HashSet<String>>,
|
|
|
|
/// List of active overlays
|
|
|
|
pub active_overlays: Vec<String>,
|
2023-01-05 02:38:50 +00:00
|
|
|
pub recursion_count: Box<u64>,
|
2021-08-15 22:33:34 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
impl Stack {
|
|
|
|
pub fn new() -> Stack {
|
2021-10-25 20:04:23 +00:00
|
|
|
Stack {
|
Move variables to var stack (#8604)
# Description
This moves the representation of variables on the stack to a Vec, which
more closely resembles a stack. For small numbers of variables live at
any one point, this tends to be more efficient than a HashMap. Having a
stack-like vector also allows us to remember a stack position,
temporarily push variables on, then quickly drop the stack back to the
original size when we're done. We'll need this capability to allow
matching inside of conditions.
On this mac, a simple run of:
`timeit { mut x = 1; while $x < 1000000 { $x += 1 } }`
Went from 1 sec 86 ms, down to 1 sec 2 ms. Clearly, we have a lot more
ground we can make up in looping speed 😅 but it's nice that for fixing
this to make matching easier, we also get a win in terms of lookup speed
for small numbers of variables.
# User-Facing Changes
Likely users won't (hopefully) see any negative impact and may even see
a small positive impact.
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
2023-03-24 23:56:45 +00:00
|
|
|
vars: vec![],
|
2021-11-30 06:14:05 +00:00
|
|
|
env_vars: vec![],
|
2022-05-07 19:39:22 +00:00
|
|
|
env_hidden: HashMap::new(),
|
|
|
|
active_overlays: vec![DEFAULT_OVERLAY_NAME.to_string()],
|
2023-01-05 02:38:50 +00:00
|
|
|
recursion_count: Box::new(0),
|
2022-01-04 22:30:34 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2022-05-07 19:39:22 +00:00
|
|
|
pub fn with_env(
|
|
|
|
&mut self,
|
|
|
|
env_vars: &[EnvVars],
|
|
|
|
env_hidden: &HashMap<String, HashSet<String>>,
|
|
|
|
) {
|
2022-01-05 22:21:26 +00:00
|
|
|
// Do not clone the environment if it hasn't changed
|
|
|
|
if self.env_vars.iter().any(|scope| !scope.is_empty()) {
|
2022-01-04 22:30:34 +00:00
|
|
|
self.env_vars = env_vars.to_owned();
|
|
|
|
}
|
|
|
|
|
2022-01-05 22:21:26 +00:00
|
|
|
if !self.env_hidden.is_empty() {
|
2022-05-07 19:39:22 +00:00
|
|
|
self.env_hidden = env_hidden.to_owned();
|
2021-10-25 20:04:23 +00:00
|
|
|
}
|
2021-08-15 22:33:34 +00:00
|
|
|
}
|
2021-11-15 23:16:06 +00:00
|
|
|
|
2022-01-29 13:00:48 +00:00
|
|
|
pub fn get_var(&self, var_id: VarId, span: Span) -> Result<Value, ShellError> {
|
Move variables to var stack (#8604)
# Description
This moves the representation of variables on the stack to a Vec, which
more closely resembles a stack. For small numbers of variables live at
any one point, this tends to be more efficient than a HashMap. Having a
stack-like vector also allows us to remember a stack position,
temporarily push variables on, then quickly drop the stack back to the
original size when we're done. We'll need this capability to allow
matching inside of conditions.
On this mac, a simple run of:
`timeit { mut x = 1; while $x < 1000000 { $x += 1 } }`
Went from 1 sec 86 ms, down to 1 sec 2 ms. Clearly, we have a lot more
ground we can make up in looping speed 😅 but it's nice that for fixing
this to make matching easier, we also get a win in terms of lookup speed
for small numbers of variables.
# User-Facing Changes
Likely users won't (hopefully) see any negative impact and may even see
a small positive impact.
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
2023-03-24 23:56:45 +00:00
|
|
|
for (id, val) in &self.vars {
|
|
|
|
if var_id == *id {
|
|
|
|
return Ok(val.clone().with_span(span));
|
|
|
|
}
|
2021-08-15 22:33:34 +00:00
|
|
|
}
|
2021-11-14 19:25:57 +00:00
|
|
|
|
2023-03-06 17:33:09 +00:00
|
|
|
Err(ShellError::VariableNotFoundAtRuntime { span })
|
2021-08-15 22:33:34 +00:00
|
|
|
}
|
|
|
|
|
2022-02-05 14:39:51 +00:00
|
|
|
pub fn get_var_with_origin(&self, var_id: VarId, span: Span) -> Result<Value, ShellError> {
|
Move variables to var stack (#8604)
# Description
This moves the representation of variables on the stack to a Vec, which
more closely resembles a stack. For small numbers of variables live at
any one point, this tends to be more efficient than a HashMap. Having a
stack-like vector also allows us to remember a stack position,
temporarily push variables on, then quickly drop the stack back to the
original size when we're done. We'll need this capability to allow
matching inside of conditions.
On this mac, a simple run of:
`timeit { mut x = 1; while $x < 1000000 { $x += 1 } }`
Went from 1 sec 86 ms, down to 1 sec 2 ms. Clearly, we have a lot more
ground we can make up in looping speed 😅 but it's nice that for fixing
this to make matching easier, we also get a win in terms of lookup speed
for small numbers of variables.
# User-Facing Changes
Likely users won't (hopefully) see any negative impact and may even see
a small positive impact.
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
2023-03-24 23:56:45 +00:00
|
|
|
for (id, val) in &self.vars {
|
|
|
|
if var_id == *id {
|
|
|
|
return Ok(val.clone());
|
|
|
|
}
|
2022-02-05 14:39:51 +00:00
|
|
|
}
|
|
|
|
|
2023-03-06 17:33:09 +00:00
|
|
|
Err(ShellError::VariableNotFoundAtRuntime { span })
|
2022-02-05 14:39:51 +00:00
|
|
|
}
|
|
|
|
|
2021-10-25 04:01:02 +00:00
|
|
|
pub fn add_var(&mut self, var_id: VarId, value: Value) {
|
Move variables to var stack (#8604)
# Description
This moves the representation of variables on the stack to a Vec, which
more closely resembles a stack. For small numbers of variables live at
any one point, this tends to be more efficient than a HashMap. Having a
stack-like vector also allows us to remember a stack position,
temporarily push variables on, then quickly drop the stack back to the
original size when we're done. We'll need this capability to allow
matching inside of conditions.
On this mac, a simple run of:
`timeit { mut x = 1; while $x < 1000000 { $x += 1 } }`
Went from 1 sec 86 ms, down to 1 sec 2 ms. Clearly, we have a lot more
ground we can make up in looping speed 😅 but it's nice that for fixing
this to make matching easier, we also get a win in terms of lookup speed
for small numbers of variables.
# User-Facing Changes
Likely users won't (hopefully) see any negative impact and may even see
a small positive impact.
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
2023-03-24 23:56:45 +00:00
|
|
|
//self.vars.insert(var_id, value);
|
|
|
|
for (id, val) in &mut self.vars {
|
|
|
|
if *id == var_id {
|
|
|
|
*val = value;
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
self.vars.push((var_id, value));
|
|
|
|
}
|
|
|
|
|
|
|
|
pub fn remove_var(&mut self, var_id: VarId) {
|
|
|
|
for (idx, (id, _)) in self.vars.iter().enumerate() {
|
|
|
|
if *id == var_id {
|
|
|
|
self.vars.remove(idx);
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
}
|
2021-08-15 22:33:34 +00:00
|
|
|
}
|
|
|
|
|
2021-12-17 01:04:54 +00:00
|
|
|
pub fn add_env_var(&mut self, var: String, value: Value) {
|
2022-05-07 19:39:22 +00:00
|
|
|
if let Some(last_overlay) = self.active_overlays.last() {
|
|
|
|
if let Some(env_hidden) = self.env_hidden.get_mut(last_overlay) {
|
|
|
|
// if the env var was hidden, let's activate it again
|
|
|
|
env_hidden.remove(&var);
|
|
|
|
}
|
2022-01-04 22:30:34 +00:00
|
|
|
|
2022-05-07 19:39:22 +00:00
|
|
|
if let Some(scope) = self.env_vars.last_mut() {
|
|
|
|
if let Some(env_vars) = scope.get_mut(last_overlay) {
|
|
|
|
env_vars.insert(var, value);
|
|
|
|
} else {
|
2023-06-10 16:41:58 +00:00
|
|
|
scope.insert(last_overlay.into(), [(var, value)].into_iter().collect());
|
2022-05-07 19:39:22 +00:00
|
|
|
}
|
|
|
|
} else {
|
2023-06-10 16:41:58 +00:00
|
|
|
self.env_vars.push(
|
|
|
|
[(last_overlay.into(), [(var, value)].into_iter().collect())]
|
|
|
|
.into_iter()
|
|
|
|
.collect(),
|
|
|
|
);
|
2022-05-07 19:39:22 +00:00
|
|
|
}
|
2021-11-30 06:14:05 +00:00
|
|
|
} else {
|
2022-05-07 19:39:22 +00:00
|
|
|
// TODO: Remove panic
|
|
|
|
panic!("internal error: no active overlay");
|
2021-11-30 06:14:05 +00:00
|
|
|
}
|
2021-08-15 22:33:34 +00:00
|
|
|
}
|
|
|
|
|
2022-05-07 19:39:22 +00:00
|
|
|
pub fn last_overlay_name(&self) -> Result<String, ShellError> {
|
|
|
|
self.active_overlays
|
|
|
|
.last()
|
|
|
|
.cloned()
|
2023-03-06 17:33:09 +00:00
|
|
|
.ok_or_else(|| ShellError::NushellFailed {
|
|
|
|
msg: "No active overlay".into(),
|
|
|
|
})
|
2022-05-07 19:39:22 +00:00
|
|
|
}
|
2022-01-12 04:06:56 +00:00
|
|
|
|
2022-05-07 19:39:22 +00:00
|
|
|
pub fn captures_to_stack(&self, captures: &HashMap<VarId, Value>) -> Stack {
|
2022-01-12 04:06:56 +00:00
|
|
|
// FIXME: this is probably slow
|
2022-05-07 19:39:22 +00:00
|
|
|
let mut env_vars = self.env_vars.clone();
|
|
|
|
env_vars.push(HashMap::new());
|
2022-01-12 04:06:56 +00:00
|
|
|
|
Move variables to var stack (#8604)
# Description
This moves the representation of variables on the stack to a Vec, which
more closely resembles a stack. For small numbers of variables live at
any one point, this tends to be more efficient than a HashMap. Having a
stack-like vector also allows us to remember a stack position,
temporarily push variables on, then quickly drop the stack back to the
original size when we're done. We'll need this capability to allow
matching inside of conditions.
On this mac, a simple run of:
`timeit { mut x = 1; while $x < 1000000 { $x += 1 } }`
Went from 1 sec 86 ms, down to 1 sec 2 ms. Clearly, we have a lot more
ground we can make up in looping speed 😅 but it's nice that for fixing
this to make matching easier, we also get a win in terms of lookup speed
for small numbers of variables.
# User-Facing Changes
Likely users won't (hopefully) see any negative impact and may even see
a small positive impact.
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
2023-03-24 23:56:45 +00:00
|
|
|
// FIXME make this more efficient
|
|
|
|
let mut vars = vec![];
|
|
|
|
for (id, val) in captures {
|
|
|
|
vars.push((*id, val.clone()));
|
|
|
|
}
|
|
|
|
|
2022-05-07 19:39:22 +00:00
|
|
|
Stack {
|
Move variables to var stack (#8604)
# Description
This moves the representation of variables on the stack to a Vec, which
more closely resembles a stack. For small numbers of variables live at
any one point, this tends to be more efficient than a HashMap. Having a
stack-like vector also allows us to remember a stack position,
temporarily push variables on, then quickly drop the stack back to the
original size when we're done. We'll need this capability to allow
matching inside of conditions.
On this mac, a simple run of:
`timeit { mut x = 1; while $x < 1000000 { $x += 1 } }`
Went from 1 sec 86 ms, down to 1 sec 2 ms. Clearly, we have a lot more
ground we can make up in looping speed 😅 but it's nice that for fixing
this to make matching easier, we also get a win in terms of lookup speed
for small numbers of variables.
# User-Facing Changes
Likely users won't (hopefully) see any negative impact and may even see
a small positive impact.
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
2023-03-24 23:56:45 +00:00
|
|
|
vars,
|
2022-05-07 19:39:22 +00:00
|
|
|
env_vars,
|
2023-02-12 17:48:51 +00:00
|
|
|
env_hidden: self.env_hidden.clone(),
|
2022-05-07 19:39:22 +00:00
|
|
|
active_overlays: self.active_overlays.clone(),
|
2023-01-05 02:38:50 +00:00
|
|
|
recursion_count: self.recursion_count.to_owned(),
|
2022-05-07 19:39:22 +00:00
|
|
|
}
|
2022-01-12 04:06:56 +00:00
|
|
|
}
|
|
|
|
|
Recursively export constants from modules (#10049)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
https://github.com/nushell/nushell/pull/9773 introduced constants to
modules and allowed to export them, but only within one level. This PR:
* allows recursive exporting of constants from all submodules
* fixes submodule imports in a list import pattern
* makes sure exported constants are actual constants
Should unblock https://github.com/nushell/nushell/pull/9678
### Example:
```nushell
module spam {
export module eggs {
export module bacon {
export const viking = 'eats'
}
}
}
use spam
print $spam.eggs.bacon.viking # prints 'eats'
use spam [eggs]
print $eggs.bacon.viking # prints 'eats'
use spam eggs bacon viking
print $viking # prints 'eats'
```
### Limitation 1:
Considering the above `spam` module, attempting to get `eggs bacon` from
`spam` module doesn't work directly:
```nushell
use spam [ eggs bacon ] # attempts to load `eggs`, then `bacon`
use spam [ "eggs bacon" ] # obviously wrong name for a constant, but doesn't work also for commands
```
Workaround (for example):
```nushell
use spam eggs
use eggs [ bacon ]
print $bacon.viking # prints 'eats'
```
I'm thinking I'll just leave it in, as you can easily work around this.
It is also a limitation of the import pattern in general, not just
constants.
### Limitation 2:
`overlay use` successfully imports the constants, but `overlay hide`
does not hide them, even though it seems to hide normal variables
successfully. This needs more investigation.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Allows recursive constant exports from submodules.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect -A clippy::result_large_err` to check that
you're using the standard code style
- `cargo test --workspace` to check that all tests pass
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2023-08-20 12:51:35 +00:00
|
|
|
pub fn gather_captures(&self, engine_state: &EngineState, captures: &[VarId]) -> Stack {
|
Move variables to var stack (#8604)
# Description
This moves the representation of variables on the stack to a Vec, which
more closely resembles a stack. For small numbers of variables live at
any one point, this tends to be more efficient than a HashMap. Having a
stack-like vector also allows us to remember a stack position,
temporarily push variables on, then quickly drop the stack back to the
original size when we're done. We'll need this capability to allow
matching inside of conditions.
On this mac, a simple run of:
`timeit { mut x = 1; while $x < 1000000 { $x += 1 } }`
Went from 1 sec 86 ms, down to 1 sec 2 ms. Clearly, we have a lot more
ground we can make up in looping speed 😅 but it's nice that for fixing
this to make matching easier, we also get a win in terms of lookup speed
for small numbers of variables.
# User-Facing Changes
Likely users won't (hopefully) see any negative impact and may even see
a small positive impact.
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
2023-03-24 23:56:45 +00:00
|
|
|
let mut vars = vec![];
|
2021-10-25 20:04:23 +00:00
|
|
|
|
2022-01-29 13:00:48 +00:00
|
|
|
let fake_span = Span::new(0, 0);
|
|
|
|
|
2021-10-25 20:04:23 +00:00
|
|
|
for capture in captures {
|
2021-10-25 21:14:21 +00:00
|
|
|
// Note: this assumes we have calculated captures correctly and that commands
|
|
|
|
// that take in a var decl will manually set this into scope when running the blocks
|
2022-01-29 13:00:48 +00:00
|
|
|
if let Ok(value) = self.get_var(*capture, fake_span) {
|
Move variables to var stack (#8604)
# Description
This moves the representation of variables on the stack to a Vec, which
more closely resembles a stack. For small numbers of variables live at
any one point, this tends to be more efficient than a HashMap. Having a
stack-like vector also allows us to remember a stack position,
temporarily push variables on, then quickly drop the stack back to the
original size when we're done. We'll need this capability to allow
matching inside of conditions.
On this mac, a simple run of:
`timeit { mut x = 1; while $x < 1000000 { $x += 1 } }`
Went from 1 sec 86 ms, down to 1 sec 2 ms. Clearly, we have a lot more
ground we can make up in looping speed 😅 but it's nice that for fixing
this to make matching easier, we also get a win in terms of lookup speed
for small numbers of variables.
# User-Facing Changes
Likely users won't (hopefully) see any negative impact and may even see
a small positive impact.
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
2023-03-24 23:56:45 +00:00
|
|
|
vars.push((*capture, value));
|
Recursively export constants from modules (#10049)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
https://github.com/nushell/nushell/pull/9773 introduced constants to
modules and allowed to export them, but only within one level. This PR:
* allows recursive exporting of constants from all submodules
* fixes submodule imports in a list import pattern
* makes sure exported constants are actual constants
Should unblock https://github.com/nushell/nushell/pull/9678
### Example:
```nushell
module spam {
export module eggs {
export module bacon {
export const viking = 'eats'
}
}
}
use spam
print $spam.eggs.bacon.viking # prints 'eats'
use spam [eggs]
print $eggs.bacon.viking # prints 'eats'
use spam eggs bacon viking
print $viking # prints 'eats'
```
### Limitation 1:
Considering the above `spam` module, attempting to get `eggs bacon` from
`spam` module doesn't work directly:
```nushell
use spam [ eggs bacon ] # attempts to load `eggs`, then `bacon`
use spam [ "eggs bacon" ] # obviously wrong name for a constant, but doesn't work also for commands
```
Workaround (for example):
```nushell
use spam eggs
use eggs [ bacon ]
print $bacon.viking # prints 'eats'
```
I'm thinking I'll just leave it in, as you can easily work around this.
It is also a limitation of the import pattern in general, not just
constants.
### Limitation 2:
`overlay use` successfully imports the constants, but `overlay hide`
does not hide them, even though it seems to hide normal variables
successfully. This needs more investigation.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Allows recursive constant exports from submodules.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect -A clippy::result_large_err` to check that
you're using the standard code style
- `cargo test --workspace` to check that all tests pass
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2023-08-20 12:51:35 +00:00
|
|
|
} else if let Some(const_val) = &engine_state.get_var(*capture).const_val {
|
|
|
|
vars.push((*capture, const_val.clone()));
|
2021-10-25 21:14:21 +00:00
|
|
|
}
|
2021-10-25 20:04:23 +00:00
|
|
|
}
|
2021-10-25 04:01:02 +00:00
|
|
|
|
2022-05-07 19:39:22 +00:00
|
|
|
let mut env_vars = self.env_vars.clone();
|
|
|
|
env_vars.push(HashMap::new());
|
2021-11-04 02:32:35 +00:00
|
|
|
|
2022-05-07 19:39:22 +00:00
|
|
|
Stack {
|
|
|
|
vars,
|
|
|
|
env_vars,
|
2023-02-12 17:48:51 +00:00
|
|
|
env_hidden: self.env_hidden.clone(),
|
2022-05-07 19:39:22 +00:00
|
|
|
active_overlays: self.active_overlays.clone(),
|
2023-01-05 02:38:50 +00:00
|
|
|
recursion_count: self.recursion_count.to_owned(),
|
2022-05-07 19:39:22 +00:00
|
|
|
}
|
2021-08-15 22:33:34 +00:00
|
|
|
}
|
|
|
|
|
2021-11-30 06:14:05 +00:00
|
|
|
/// Flatten the env var scope frames into one frame
|
2022-01-04 22:30:34 +00:00
|
|
|
pub fn get_env_vars(&self, engine_state: &EngineState) -> HashMap<String, Value> {
|
2022-05-07 19:39:22 +00:00
|
|
|
let mut result = HashMap::new();
|
|
|
|
|
|
|
|
for active_overlay in self.active_overlays.iter() {
|
|
|
|
if let Some(env_vars) = engine_state.env_vars.get(active_overlay) {
|
|
|
|
result.extend(
|
|
|
|
env_vars
|
|
|
|
.iter()
|
|
|
|
.filter(|(k, _)| {
|
|
|
|
if let Some(env_hidden) = self.env_hidden.get(active_overlay) {
|
|
|
|
!env_hidden.contains(*k)
|
|
|
|
} else {
|
|
|
|
// nothing has been hidden in this overlay
|
|
|
|
true
|
|
|
|
}
|
|
|
|
})
|
|
|
|
.map(|(k, v)| (k.clone(), v.clone()))
|
|
|
|
.collect::<HashMap<String, Value>>(),
|
|
|
|
);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
result.extend(self.get_stack_env_vars());
|
|
|
|
|
|
|
|
result
|
|
|
|
}
|
|
|
|
|
|
|
|
/// Get flattened environment variables only from the stack
|
|
|
|
pub fn get_stack_env_vars(&self) -> HashMap<String, Value> {
|
|
|
|
let mut result = HashMap::new();
|
2021-11-30 06:14:05 +00:00
|
|
|
|
|
|
|
for scope in &self.env_vars {
|
2022-05-07 19:39:22 +00:00
|
|
|
for active_overlay in self.active_overlays.iter() {
|
|
|
|
if let Some(env_vars) = scope.get(active_overlay) {
|
|
|
|
result.extend(env_vars.clone());
|
|
|
|
}
|
|
|
|
}
|
2021-11-30 06:14:05 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
result
|
2021-09-19 19:29:58 +00:00
|
|
|
}
|
|
|
|
|
2022-05-24 21:22:17 +00:00
|
|
|
/// Get flattened environment variables only from the stack and one overlay
|
|
|
|
pub fn get_stack_overlay_env_vars(&self, overlay_name: &str) -> HashMap<String, Value> {
|
|
|
|
let mut result = HashMap::new();
|
|
|
|
|
|
|
|
for scope in &self.env_vars {
|
|
|
|
if let Some(active_overlay) = self.active_overlays.iter().find(|n| n == &overlay_name) {
|
|
|
|
if let Some(env_vars) = scope.get(active_overlay) {
|
|
|
|
result.extend(env_vars.clone());
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
result
|
|
|
|
}
|
|
|
|
|
2022-02-04 18:02:03 +00:00
|
|
|
/// Same as get_env_vars, but returns only the names as a HashSet
|
|
|
|
pub fn get_env_var_names(&self, engine_state: &EngineState) -> HashSet<String> {
|
2022-05-07 19:39:22 +00:00
|
|
|
let mut result = HashSet::new();
|
|
|
|
|
|
|
|
for active_overlay in self.active_overlays.iter() {
|
|
|
|
if let Some(env_vars) = engine_state.env_vars.get(active_overlay) {
|
|
|
|
result.extend(
|
|
|
|
env_vars
|
|
|
|
.keys()
|
|
|
|
.filter(|k| {
|
|
|
|
if let Some(env_hidden) = self.env_hidden.get(active_overlay) {
|
|
|
|
!env_hidden.contains(*k)
|
|
|
|
} else {
|
|
|
|
// nothing has been hidden in this overlay
|
|
|
|
true
|
|
|
|
}
|
|
|
|
})
|
|
|
|
.cloned()
|
|
|
|
.collect::<HashSet<String>>(),
|
|
|
|
);
|
|
|
|
}
|
|
|
|
}
|
2022-02-04 18:02:03 +00:00
|
|
|
|
|
|
|
for scope in &self.env_vars {
|
2022-05-07 19:39:22 +00:00
|
|
|
for active_overlay in self.active_overlays.iter() {
|
|
|
|
if let Some(env_vars) = scope.get(active_overlay) {
|
|
|
|
result.extend(env_vars.keys().cloned().collect::<HashSet<String>>());
|
|
|
|
}
|
|
|
|
}
|
2022-02-04 18:02:03 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
result
|
|
|
|
}
|
|
|
|
|
2022-01-04 22:30:34 +00:00
|
|
|
pub fn get_env_var(&self, engine_state: &EngineState, name: &str) -> Option<Value> {
|
2021-11-30 06:14:05 +00:00
|
|
|
for scope in self.env_vars.iter().rev() {
|
2022-05-07 19:39:22 +00:00
|
|
|
for active_overlay in self.active_overlays.iter().rev() {
|
|
|
|
if let Some(env_vars) = scope.get(active_overlay) {
|
|
|
|
if let Some(v) = env_vars.get(name) {
|
|
|
|
return Some(v.clone());
|
|
|
|
}
|
|
|
|
}
|
2021-11-30 06:14:05 +00:00
|
|
|
}
|
2021-10-25 04:01:02 +00:00
|
|
|
}
|
2021-11-30 06:14:05 +00:00
|
|
|
|
2022-05-07 19:39:22 +00:00
|
|
|
for active_overlay in self.active_overlays.iter().rev() {
|
|
|
|
let is_hidden = if let Some(env_hidden) = self.env_hidden.get(active_overlay) {
|
|
|
|
env_hidden.contains(name)
|
|
|
|
} else {
|
|
|
|
false
|
|
|
|
};
|
|
|
|
|
|
|
|
if !is_hidden {
|
|
|
|
if let Some(env_vars) = engine_state.env_vars.get(active_overlay) {
|
|
|
|
if let Some(v) = env_vars.get(name) {
|
|
|
|
return Some(v.clone());
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
2022-01-04 22:30:34 +00:00
|
|
|
}
|
2022-05-07 19:39:22 +00:00
|
|
|
|
|
|
|
None
|
2021-10-02 13:10:28 +00:00
|
|
|
}
|
|
|
|
|
2022-02-04 18:02:03 +00:00
|
|
|
pub fn has_env_var(&self, engine_state: &EngineState, name: &str) -> bool {
|
|
|
|
for scope in self.env_vars.iter().rev() {
|
2022-05-07 19:39:22 +00:00
|
|
|
for active_overlay in self.active_overlays.iter().rev() {
|
|
|
|
if let Some(env_vars) = scope.get(active_overlay) {
|
|
|
|
if env_vars.contains_key(name) {
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
}
|
2022-02-04 18:02:03 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2022-05-07 19:39:22 +00:00
|
|
|
for active_overlay in self.active_overlays.iter().rev() {
|
|
|
|
let is_hidden = if let Some(env_hidden) = self.env_hidden.get(active_overlay) {
|
|
|
|
env_hidden.contains(name)
|
|
|
|
} else {
|
|
|
|
false
|
|
|
|
};
|
|
|
|
|
|
|
|
if !is_hidden {
|
|
|
|
if let Some(env_vars) = engine_state.env_vars.get(active_overlay) {
|
|
|
|
if env_vars.contains_key(name) {
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
2022-02-04 18:02:03 +00:00
|
|
|
}
|
2022-05-07 19:39:22 +00:00
|
|
|
|
|
|
|
false
|
2022-02-04 18:02:03 +00:00
|
|
|
}
|
|
|
|
|
2023-02-12 17:48:51 +00:00
|
|
|
pub fn remove_env_var(&mut self, engine_state: &EngineState, name: &str) -> bool {
|
2021-11-30 06:14:05 +00:00
|
|
|
for scope in self.env_vars.iter_mut().rev() {
|
2022-05-07 19:39:22 +00:00
|
|
|
for active_overlay in self.active_overlays.iter().rev() {
|
|
|
|
if let Some(env_vars) = scope.get_mut(active_overlay) {
|
2023-02-12 17:48:51 +00:00
|
|
|
if env_vars.remove(name).is_some() {
|
|
|
|
return true;
|
2022-05-07 19:39:22 +00:00
|
|
|
}
|
|
|
|
}
|
2021-11-30 06:14:05 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2022-05-07 19:39:22 +00:00
|
|
|
for active_overlay in self.active_overlays.iter().rev() {
|
|
|
|
if let Some(env_vars) = engine_state.env_vars.get(active_overlay) {
|
2023-02-12 17:48:51 +00:00
|
|
|
if env_vars.get(name).is_some() {
|
2022-05-07 19:39:22 +00:00
|
|
|
if let Some(env_hidden) = self.env_hidden.get_mut(active_overlay) {
|
|
|
|
env_hidden.insert(name.into());
|
|
|
|
} else {
|
|
|
|
self.env_hidden
|
2023-06-10 16:41:58 +00:00
|
|
|
.insert(active_overlay.into(), [name.into()].into_iter().collect());
|
2022-05-07 19:39:22 +00:00
|
|
|
}
|
|
|
|
|
2023-02-12 17:48:51 +00:00
|
|
|
return true;
|
2022-05-07 19:39:22 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2023-02-12 17:48:51 +00:00
|
|
|
false
|
2022-05-07 19:39:22 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
pub fn has_env_overlay(&self, name: &str, engine_state: &EngineState) -> bool {
|
|
|
|
for scope in self.env_vars.iter().rev() {
|
|
|
|
if scope.contains_key(name) {
|
|
|
|
return true;
|
2021-11-30 06:14:05 +00:00
|
|
|
}
|
2021-08-15 22:33:34 +00:00
|
|
|
}
|
2022-05-07 19:39:22 +00:00
|
|
|
|
|
|
|
engine_state.env_vars.contains_key(name)
|
|
|
|
}
|
|
|
|
|
2022-05-24 21:22:17 +00:00
|
|
|
pub fn is_overlay_active(&self, name: &String) -> bool {
|
|
|
|
self.active_overlays.contains(name)
|
|
|
|
}
|
|
|
|
|
2022-05-07 19:39:22 +00:00
|
|
|
pub fn add_overlay(&mut self, name: String) {
|
|
|
|
self.active_overlays.retain(|o| o != &name);
|
|
|
|
self.active_overlays.push(name);
|
|
|
|
}
|
|
|
|
|
2022-05-24 21:22:17 +00:00
|
|
|
pub fn remove_overlay(&mut self, name: &String) {
|
2022-05-07 19:39:22 +00:00
|
|
|
self.active_overlays.retain(|o| o != name);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
impl Default for Stack {
|
|
|
|
fn default() -> Self {
|
|
|
|
Self::new()
|
2021-08-15 22:33:34 +00:00
|
|
|
}
|
|
|
|
}
|