mirror of
https://github.com/nushell/nushell
synced 2025-01-08 03:09:00 +00:00
d7392f1f3b
# Description This PR adds an internal representation language to Nushell, offering an alternative evaluator based on simple instructions, stream-containing registers, and indexed control flow. The number of registers required is determined statically at compile-time, and the fixed size required is allocated upon entering the block. Each instruction is associated with a span, which makes going backwards from IR instructions to source code very easy. Motivations for IR: 1. **Performance.** By simplifying the evaluation path and making it more cache-friendly and branch predictor-friendly, code that does a lot of computation in Nushell itself can be sped up a decent bit. Because the IR is fairly easy to reason about, we can also implement optimization passes in the future to eliminate and simplify code. 2. **Correctness.** The instructions mostly have very simple and easily-specified behavior, so hopefully engine changes are a little bit easier to reason about, and they can be specified in a more formal way at some point. I have made an effort to document each of the instructions in the docs for the enum itself in a reasonably specific way. Some of the errors that would have happened during evaluation before are now moved to the compilation step instead, because they don't make sense to check during evaluation. 3. **As an intermediate target.** This is a good step for us to bring the [`new-nu-parser`](https://github.com/nushell/new-nu-parser) in at some point, as code generated from new AST can be directly compared to code generated from old AST. If the IR code is functionally equivalent, it will behave the exact same way. 4. **Debugging.** With a little bit more work, we can probably give control over advancing the virtual machine that `IrBlock`s run on to some sort of external driver, making things like breakpoints and single stepping possible. Tools like `view ir` and [`explore ir`](https://github.com/devyn/nu_plugin_explore_ir) make it easier than before to see what exactly is going on with your Nushell code. The goal is to eventually replace the AST evaluator entirely, once we're sure it's working just as well. You can help dogfood this by running Nushell with `$env.NU_USE_IR` set to some value. The environment variable is checked when Nushell starts, so config runs with IR, or it can also be set on a line at the REPL to change it dynamically. It is also checked when running `do` in case within a script you want to just run a specific piece of code with or without IR. # Example ```nushell view ir { |data| mut sum = 0 for n in $data { $sum += $n } $sum } ``` ```gas # 3 registers, 19 instructions, 0 bytes of data 0: load-literal %0, int(0) 1: store-variable var 904, %0 # let 2: drain %0 3: drop %0 4: load-variable %1, var 903 5: iterate %0, %1, end 15 # for, label(1), from(14:) 6: store-variable var 905, %0 7: load-variable %0, var 904 8: load-variable %2, var 905 9: binary-op %0, Math(Plus), %2 10: span %0 11: store-variable var 904, %0 12: load-literal %0, nothing 13: drain %0 14: jump 5 15: drop %0 # label(0), from(5:) 16: drain %0 17: load-variable %0, var 904 18: return %0 ``` # Benchmarks All benchmarks run on a base model Mac Mini M1. ## Iterative Fibonacci sequence This is about as best case as possible, making use of the much faster control flow. Most code will not experience a speed improvement nearly this large. ```nushell def fib [n: int] { mut a = 0 mut b = 1 for _ in 2..=$n { let c = $a + $b $a = $b $b = $c } $b } use std bench bench { 0..50 | each { |n| fib $n } } ``` IR disabled: ``` ╭───────┬─────────────────╮ │ mean │ 1ms 924µs 665ns │ │ min │ 1ms 700µs 83ns │ │ max │ 3ms 450µs 125ns │ │ std │ 395µs 759ns │ │ times │ [list 50 items] │ ╰───────┴─────────────────╯ ``` IR enabled: ``` ╭───────┬─────────────────╮ │ mean │ 452µs 820ns │ │ min │ 427µs 417ns │ │ max │ 540µs 167ns │ │ std │ 17µs 158ns │ │ times │ [list 50 items] │ ╰───────┴─────────────────╯ ``` ![explore ir view](https://github.com/nushell/nushell/assets/10729/d7bccc03-5222-461c-9200-0dce71b83b83) ## [gradient_benchmark_no_check.nu](https://github.com/nushell/nu_scripts/blob/main/benchmarks/gradient_benchmark_no_check.nu) IR disabled: ``` ╭───┬──────────────────╮ │ 0 │ 27ms 929µs 958ns │ │ 1 │ 21ms 153µs 459ns │ │ 2 │ 18ms 639µs 666ns │ │ 3 │ 19ms 554µs 583ns │ │ 4 │ 13ms 383µs 375ns │ │ 5 │ 11ms 328µs 208ns │ │ 6 │ 5ms 659µs 542ns │ ╰───┴──────────────────╯ ``` IR enabled: ``` ╭───┬──────────────────╮ │ 0 │ 22ms 662µs │ │ 1 │ 17ms 221µs 792ns │ │ 2 │ 14ms 786µs 708ns │ │ 3 │ 13ms 876µs 834ns │ │ 4 │ 13ms 52µs 875ns │ │ 5 │ 11ms 269µs 666ns │ │ 6 │ 6ms 942µs 500ns │ ╰───┴──────────────────╯ ``` ## [random-bytes.nu](https://github.com/nushell/nu_scripts/blob/main/benchmarks/random-bytes.nu) I got pretty random results out of this benchmark so I decided not to include it. Not clear why. # User-Facing Changes - IR compilation errors may appear even if the user isn't evaluating with IR. - IR evaluation can be enabled by setting the `NU_USE_IR` environment variable to any value. - New command `view ir` pretty-prints the IR for a block, and `view ir --json` can be piped into an external tool like [`explore ir`](https://github.com/devyn/nu_plugin_explore_ir). # Tests + Formatting All tests are passing with `NU_USE_IR=1`, and I've added some more eval tests to compare the results for some very core operations. I will probably want to add some more so we don't have to always check `NU_USE_IR=1 toolkit test --workspace` on a regular basis. # After Submitting - [ ] release notes - [ ] further documentation of instructions? - [ ] post-release: publish `nu_plugin_explore_ir`
785 lines
28 KiB
Rust
785 lines
28 KiB
Rust
use crate::{
|
|
engine::{
|
|
ArgumentStack, EngineState, ErrorHandlerStack, Redirection, StackCallArgGuard,
|
|
StackCaptureGuard, StackIoGuard, StackOutDest, DEFAULT_OVERLAY_NAME,
|
|
},
|
|
OutDest, ShellError, Span, Value, VarId, ENV_VARIABLE_ID, NU_VARIABLE_ID,
|
|
};
|
|
use std::{
|
|
collections::{HashMap, HashSet},
|
|
fs::File,
|
|
sync::Arc,
|
|
};
|
|
|
|
/// Environment variables per overlay
|
|
pub type EnvVars = HashMap<String, HashMap<String, Value>>;
|
|
|
|
/// A runtime value stack used during evaluation
|
|
///
|
|
/// A note on implementation:
|
|
///
|
|
/// We previously set up the stack in a traditional way, where stack frames had parents which would
|
|
/// represent other frames that you might return to when exiting a function.
|
|
///
|
|
/// While experimenting with blocks, we found that we needed to have closure captures of variables
|
|
/// seen outside of the blocks, so that they blocks could be run in a way that was both thread-safe
|
|
/// and followed the restrictions for closures applied to iterators. The end result left us with
|
|
/// closure-captured single stack frames that blocks could see.
|
|
///
|
|
/// Blocks make up the only scope and stack definition abstraction in Nushell. As a result, we were
|
|
/// creating closure captures at any point we wanted to have a Block value we could safely evaluate
|
|
/// in any context. This meant that the parents were going largely unused, with captured variables
|
|
/// taking their place. The end result is this, where we no longer have separate frames, but instead
|
|
/// use the Stack as a way of representing the local and closure-captured state.
|
|
#[derive(Debug, Clone)]
|
|
pub struct Stack {
|
|
/// Variables
|
|
pub vars: Vec<(VarId, Value)>,
|
|
/// Environment variables arranged as a stack to be able to recover values from parent scopes
|
|
pub env_vars: Vec<EnvVars>,
|
|
/// Tells which environment variables from engine state are hidden, per overlay.
|
|
pub env_hidden: HashMap<String, HashSet<String>>,
|
|
/// List of active overlays
|
|
pub active_overlays: Vec<String>,
|
|
/// Argument stack for IR evaluation
|
|
pub arguments: ArgumentStack,
|
|
/// Error handler stack for IR evaluation
|
|
pub error_handlers: ErrorHandlerStack,
|
|
/// Set true to always use IR mode
|
|
pub use_ir: bool,
|
|
pub recursion_count: u64,
|
|
pub parent_stack: Option<Arc<Stack>>,
|
|
/// Variables that have been deleted (this is used to hide values from parent stack lookups)
|
|
pub parent_deletions: Vec<VarId>,
|
|
pub(crate) out_dest: StackOutDest,
|
|
}
|
|
|
|
impl Default for Stack {
|
|
fn default() -> Self {
|
|
Self::new()
|
|
}
|
|
}
|
|
|
|
impl Stack {
|
|
/// Create a new stack.
|
|
///
|
|
/// stdout and stderr will be set to [`OutDest::Inherit`]. So, if the last command is an external command,
|
|
/// then its output will be forwarded to the terminal/stdio streams.
|
|
///
|
|
/// Use [`Stack::capture`] afterwards if you need to evaluate an expression to a [`Value`](crate::Value)
|
|
/// (as opposed to a [`PipelineData`](crate::PipelineData)).
|
|
pub fn new() -> Self {
|
|
Self {
|
|
vars: Vec::new(),
|
|
env_vars: Vec::new(),
|
|
env_hidden: HashMap::new(),
|
|
active_overlays: vec![DEFAULT_OVERLAY_NAME.to_string()],
|
|
arguments: ArgumentStack::new(),
|
|
error_handlers: ErrorHandlerStack::new(),
|
|
use_ir: false,
|
|
recursion_count: 0,
|
|
parent_stack: None,
|
|
parent_deletions: vec![],
|
|
out_dest: StackOutDest::new(),
|
|
}
|
|
}
|
|
|
|
/// Create a new child stack from a parent.
|
|
///
|
|
/// Changes from this child can be merged back into the parent with
|
|
/// [`Stack::with_changes_from_child`]
|
|
pub fn with_parent(parent: Arc<Stack>) -> Stack {
|
|
Stack {
|
|
// here we are still cloning environment variable-related information
|
|
env_vars: parent.env_vars.clone(),
|
|
env_hidden: parent.env_hidden.clone(),
|
|
active_overlays: parent.active_overlays.clone(),
|
|
arguments: ArgumentStack::new(),
|
|
error_handlers: ErrorHandlerStack::new(),
|
|
use_ir: parent.use_ir,
|
|
recursion_count: parent.recursion_count,
|
|
vars: vec![],
|
|
parent_deletions: vec![],
|
|
out_dest: parent.out_dest.clone(),
|
|
parent_stack: Some(parent),
|
|
}
|
|
}
|
|
|
|
/// Take an [`Arc`] parent, and a child, and apply all the changes from a child back to the parent.
|
|
///
|
|
/// Here it is assumed that `child` was created by a call to [`Stack::with_parent`] with `parent`.
|
|
///
|
|
/// For this to be performant and not clone `parent`, `child` should be the only other
|
|
/// referencer of `parent`.
|
|
pub fn with_changes_from_child(parent: Arc<Stack>, child: Stack) -> Stack {
|
|
// we're going to drop the link to the parent stack on our new stack
|
|
// so that we can unwrap the Arc as a unique reference
|
|
drop(child.parent_stack);
|
|
let mut unique_stack = Arc::unwrap_or_clone(parent);
|
|
|
|
unique_stack
|
|
.vars
|
|
.retain(|(var, _)| !child.parent_deletions.contains(var));
|
|
for (var, value) in child.vars {
|
|
unique_stack.add_var(var, value);
|
|
}
|
|
unique_stack.env_vars = child.env_vars;
|
|
unique_stack.env_hidden = child.env_hidden;
|
|
unique_stack.active_overlays = child.active_overlays;
|
|
unique_stack
|
|
}
|
|
|
|
pub fn with_env(
|
|
&mut self,
|
|
env_vars: &[EnvVars],
|
|
env_hidden: &HashMap<String, HashSet<String>>,
|
|
) {
|
|
// Do not clone the environment if it hasn't changed
|
|
if self.env_vars.iter().any(|scope| !scope.is_empty()) {
|
|
env_vars.clone_into(&mut self.env_vars);
|
|
}
|
|
|
|
if !self.env_hidden.is_empty() {
|
|
self.env_hidden.clone_from(env_hidden);
|
|
}
|
|
}
|
|
|
|
/// Lookup a variable, returning None if it is not present
|
|
fn lookup_var(&self, var_id: VarId) -> Option<Value> {
|
|
for (id, val) in &self.vars {
|
|
if var_id == *id {
|
|
return Some(val.clone());
|
|
}
|
|
}
|
|
|
|
if let Some(stack) = &self.parent_stack {
|
|
if !self.parent_deletions.contains(&var_id) {
|
|
return stack.lookup_var(var_id);
|
|
}
|
|
}
|
|
None
|
|
}
|
|
|
|
/// Lookup a variable, erroring if it is not found
|
|
///
|
|
/// The passed-in span will be used to tag the value
|
|
pub fn get_var(&self, var_id: VarId, span: Span) -> Result<Value, ShellError> {
|
|
match self.lookup_var(var_id) {
|
|
Some(v) => Ok(v.with_span(span)),
|
|
None => Err(ShellError::VariableNotFoundAtRuntime { span }),
|
|
}
|
|
}
|
|
|
|
/// Lookup a variable, erroring if it is not found
|
|
///
|
|
/// While the passed-in span will be used for errors, the returned value
|
|
/// has the span from where it was originally defined
|
|
pub fn get_var_with_origin(&self, var_id: VarId, span: Span) -> Result<Value, ShellError> {
|
|
match self.lookup_var(var_id) {
|
|
Some(v) => Ok(v),
|
|
None => {
|
|
if var_id == NU_VARIABLE_ID || var_id == ENV_VARIABLE_ID {
|
|
return Err(ShellError::GenericError {
|
|
error: "Built-in variables `$env` and `$nu` have no metadata".into(),
|
|
msg: "no metadata available".into(),
|
|
span: Some(span),
|
|
help: None,
|
|
inner: vec![],
|
|
});
|
|
}
|
|
Err(ShellError::VariableNotFoundAtRuntime { span })
|
|
}
|
|
}
|
|
}
|
|
|
|
pub fn add_var(&mut self, var_id: VarId, value: Value) {
|
|
//self.vars.insert(var_id, value);
|
|
for (id, val) in &mut self.vars {
|
|
if *id == var_id {
|
|
*val = value;
|
|
return;
|
|
}
|
|
}
|
|
self.vars.push((var_id, value));
|
|
}
|
|
|
|
pub fn remove_var(&mut self, var_id: VarId) {
|
|
for (idx, (id, _)) in self.vars.iter().enumerate() {
|
|
if *id == var_id {
|
|
self.vars.remove(idx);
|
|
break;
|
|
}
|
|
}
|
|
// even if we did have it in the original layer, we need to make sure to remove it here
|
|
// as well (since the previous update might have simply hid the parent value)
|
|
if self.parent_stack.is_some() {
|
|
self.parent_deletions.push(var_id);
|
|
}
|
|
}
|
|
|
|
pub fn add_env_var(&mut self, var: String, value: Value) {
|
|
if let Some(last_overlay) = self.active_overlays.last() {
|
|
if let Some(env_hidden) = self.env_hidden.get_mut(last_overlay) {
|
|
// if the env var was hidden, let's activate it again
|
|
env_hidden.remove(&var);
|
|
}
|
|
|
|
if let Some(scope) = self.env_vars.last_mut() {
|
|
if let Some(env_vars) = scope.get_mut(last_overlay) {
|
|
env_vars.insert(var, value);
|
|
} else {
|
|
scope.insert(last_overlay.into(), [(var, value)].into_iter().collect());
|
|
}
|
|
} else {
|
|
self.env_vars.push(
|
|
[(last_overlay.into(), [(var, value)].into_iter().collect())]
|
|
.into_iter()
|
|
.collect(),
|
|
);
|
|
}
|
|
} else {
|
|
// TODO: Remove panic
|
|
panic!("internal error: no active overlay");
|
|
}
|
|
}
|
|
|
|
pub fn last_overlay_name(&self) -> Result<String, ShellError> {
|
|
self.active_overlays
|
|
.last()
|
|
.cloned()
|
|
.ok_or_else(|| ShellError::NushellFailed {
|
|
msg: "No active overlay".into(),
|
|
})
|
|
}
|
|
|
|
pub fn captures_to_stack(&self, captures: Vec<(VarId, Value)>) -> Stack {
|
|
self.captures_to_stack_preserve_out_dest(captures).capture()
|
|
}
|
|
|
|
pub fn captures_to_stack_preserve_out_dest(&self, captures: Vec<(VarId, Value)>) -> Stack {
|
|
// FIXME: this is probably slow
|
|
let mut env_vars = self.env_vars.clone();
|
|
env_vars.push(HashMap::new());
|
|
|
|
Stack {
|
|
vars: captures,
|
|
env_vars,
|
|
env_hidden: self.env_hidden.clone(),
|
|
active_overlays: self.active_overlays.clone(),
|
|
arguments: ArgumentStack::new(),
|
|
error_handlers: ErrorHandlerStack::new(),
|
|
use_ir: self.use_ir,
|
|
recursion_count: self.recursion_count,
|
|
parent_stack: None,
|
|
parent_deletions: vec![],
|
|
out_dest: self.out_dest.clone(),
|
|
}
|
|
}
|
|
|
|
pub fn gather_captures(&self, engine_state: &EngineState, captures: &[VarId]) -> Stack {
|
|
let mut vars = vec![];
|
|
|
|
let fake_span = Span::new(0, 0);
|
|
|
|
for capture in captures {
|
|
// Note: this assumes we have calculated captures correctly and that commands
|
|
// that take in a var decl will manually set this into scope when running the blocks
|
|
if let Ok(value) = self.get_var(*capture, fake_span) {
|
|
vars.push((*capture, value));
|
|
} else if let Some(const_val) = &engine_state.get_var(*capture).const_val {
|
|
vars.push((*capture, const_val.clone()));
|
|
}
|
|
}
|
|
|
|
let mut env_vars = self.env_vars.clone();
|
|
env_vars.push(HashMap::new());
|
|
|
|
Stack {
|
|
vars,
|
|
env_vars,
|
|
env_hidden: self.env_hidden.clone(),
|
|
active_overlays: self.active_overlays.clone(),
|
|
arguments: ArgumentStack::new(),
|
|
error_handlers: ErrorHandlerStack::new(),
|
|
use_ir: self.use_ir,
|
|
recursion_count: self.recursion_count,
|
|
parent_stack: None,
|
|
parent_deletions: vec![],
|
|
out_dest: self.out_dest.clone(),
|
|
}
|
|
}
|
|
|
|
/// Flatten the env var scope frames into one frame
|
|
pub fn get_env_vars(&self, engine_state: &EngineState) -> HashMap<String, Value> {
|
|
let mut result = HashMap::new();
|
|
|
|
for active_overlay in self.active_overlays.iter() {
|
|
if let Some(env_vars) = engine_state.env_vars.get(active_overlay) {
|
|
result.extend(
|
|
env_vars
|
|
.iter()
|
|
.filter(|(k, _)| {
|
|
if let Some(env_hidden) = self.env_hidden.get(active_overlay) {
|
|
!env_hidden.contains(*k)
|
|
} else {
|
|
// nothing has been hidden in this overlay
|
|
true
|
|
}
|
|
})
|
|
.map(|(k, v)| (k.clone(), v.clone()))
|
|
.collect::<HashMap<String, Value>>(),
|
|
);
|
|
}
|
|
}
|
|
|
|
result.extend(self.get_stack_env_vars());
|
|
|
|
result
|
|
}
|
|
|
|
/// Get flattened environment variables only from the stack
|
|
pub fn get_stack_env_vars(&self) -> HashMap<String, Value> {
|
|
let mut result = HashMap::new();
|
|
|
|
for scope in &self.env_vars {
|
|
for active_overlay in self.active_overlays.iter() {
|
|
if let Some(env_vars) = scope.get(active_overlay) {
|
|
result.extend(env_vars.clone());
|
|
}
|
|
}
|
|
}
|
|
|
|
result
|
|
}
|
|
|
|
/// Get flattened environment variables only from the stack and one overlay
|
|
pub fn get_stack_overlay_env_vars(&self, overlay_name: &str) -> HashMap<String, Value> {
|
|
let mut result = HashMap::new();
|
|
|
|
for scope in &self.env_vars {
|
|
if let Some(active_overlay) = self.active_overlays.iter().find(|n| n == &overlay_name) {
|
|
if let Some(env_vars) = scope.get(active_overlay) {
|
|
result.extend(env_vars.clone());
|
|
}
|
|
}
|
|
}
|
|
|
|
result
|
|
}
|
|
|
|
/// Same as get_env_vars, but returns only the names as a HashSet
|
|
pub fn get_env_var_names(&self, engine_state: &EngineState) -> HashSet<String> {
|
|
let mut result = HashSet::new();
|
|
|
|
for active_overlay in self.active_overlays.iter() {
|
|
if let Some(env_vars) = engine_state.env_vars.get(active_overlay) {
|
|
result.extend(
|
|
env_vars
|
|
.keys()
|
|
.filter(|k| {
|
|
if let Some(env_hidden) = self.env_hidden.get(active_overlay) {
|
|
!env_hidden.contains(*k)
|
|
} else {
|
|
// nothing has been hidden in this overlay
|
|
true
|
|
}
|
|
})
|
|
.cloned()
|
|
.collect::<HashSet<String>>(),
|
|
);
|
|
}
|
|
}
|
|
|
|
for scope in &self.env_vars {
|
|
for active_overlay in self.active_overlays.iter() {
|
|
if let Some(env_vars) = scope.get(active_overlay) {
|
|
result.extend(env_vars.keys().cloned().collect::<HashSet<String>>());
|
|
}
|
|
}
|
|
}
|
|
|
|
result
|
|
}
|
|
|
|
pub fn get_env_var(&self, engine_state: &EngineState, name: &str) -> Option<Value> {
|
|
for scope in self.env_vars.iter().rev() {
|
|
for active_overlay in self.active_overlays.iter().rev() {
|
|
if let Some(env_vars) = scope.get(active_overlay) {
|
|
if let Some(v) = env_vars.get(name) {
|
|
return Some(v.clone());
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
for active_overlay in self.active_overlays.iter().rev() {
|
|
let is_hidden = if let Some(env_hidden) = self.env_hidden.get(active_overlay) {
|
|
env_hidden.contains(name)
|
|
} else {
|
|
false
|
|
};
|
|
|
|
if !is_hidden {
|
|
if let Some(env_vars) = engine_state.env_vars.get(active_overlay) {
|
|
if let Some(v) = env_vars.get(name) {
|
|
return Some(v.clone());
|
|
}
|
|
}
|
|
}
|
|
}
|
|
None
|
|
}
|
|
|
|
pub fn has_env_var(&self, engine_state: &EngineState, name: &str) -> bool {
|
|
for scope in self.env_vars.iter().rev() {
|
|
for active_overlay in self.active_overlays.iter().rev() {
|
|
if let Some(env_vars) = scope.get(active_overlay) {
|
|
if env_vars.contains_key(name) {
|
|
return true;
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
for active_overlay in self.active_overlays.iter().rev() {
|
|
let is_hidden = if let Some(env_hidden) = self.env_hidden.get(active_overlay) {
|
|
env_hidden.contains(name)
|
|
} else {
|
|
false
|
|
};
|
|
|
|
if !is_hidden {
|
|
if let Some(env_vars) = engine_state.env_vars.get(active_overlay) {
|
|
if env_vars.contains_key(name) {
|
|
return true;
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
false
|
|
}
|
|
|
|
pub fn remove_env_var(&mut self, engine_state: &EngineState, name: &str) -> bool {
|
|
for scope in self.env_vars.iter_mut().rev() {
|
|
for active_overlay in self.active_overlays.iter().rev() {
|
|
if let Some(env_vars) = scope.get_mut(active_overlay) {
|
|
if env_vars.remove(name).is_some() {
|
|
return true;
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
for active_overlay in self.active_overlays.iter().rev() {
|
|
if let Some(env_vars) = engine_state.env_vars.get(active_overlay) {
|
|
if env_vars.get(name).is_some() {
|
|
if let Some(env_hidden) = self.env_hidden.get_mut(active_overlay) {
|
|
env_hidden.insert(name.into());
|
|
} else {
|
|
self.env_hidden
|
|
.insert(active_overlay.into(), [name.into()].into_iter().collect());
|
|
}
|
|
|
|
return true;
|
|
}
|
|
}
|
|
}
|
|
|
|
false
|
|
}
|
|
|
|
pub fn has_env_overlay(&self, name: &str, engine_state: &EngineState) -> bool {
|
|
for scope in self.env_vars.iter().rev() {
|
|
if scope.contains_key(name) {
|
|
return true;
|
|
}
|
|
}
|
|
|
|
engine_state.env_vars.contains_key(name)
|
|
}
|
|
|
|
pub fn is_overlay_active(&self, name: &str) -> bool {
|
|
self.active_overlays.iter().any(|n| n == name)
|
|
}
|
|
|
|
pub fn add_overlay(&mut self, name: String) {
|
|
self.active_overlays.retain(|o| o != &name);
|
|
self.active_overlays.push(name);
|
|
}
|
|
|
|
pub fn remove_overlay(&mut self, name: &str) {
|
|
self.active_overlays.retain(|o| o != name);
|
|
}
|
|
|
|
/// Returns the [`OutDest`] to use for the current command's stdout.
|
|
///
|
|
/// This will be the pipe redirection if one is set,
|
|
/// otherwise it will be the current file redirection,
|
|
/// otherwise it will be the process's stdout indicated by [`OutDest::Inherit`].
|
|
pub fn stdout(&self) -> &OutDest {
|
|
self.out_dest.stdout()
|
|
}
|
|
|
|
/// Returns the [`OutDest`] to use for the current command's stderr.
|
|
///
|
|
/// This will be the pipe redirection if one is set,
|
|
/// otherwise it will be the current file redirection,
|
|
/// otherwise it will be the process's stderr indicated by [`OutDest::Inherit`].
|
|
pub fn stderr(&self) -> &OutDest {
|
|
self.out_dest.stderr()
|
|
}
|
|
|
|
/// Returns the [`OutDest`] of the pipe redirection applied to the current command's stdout.
|
|
pub fn pipe_stdout(&self) -> Option<&OutDest> {
|
|
self.out_dest.pipe_stdout.as_ref()
|
|
}
|
|
|
|
/// Returns the [`OutDest`] of the pipe redirection applied to the current command's stderr.
|
|
pub fn pipe_stderr(&self) -> Option<&OutDest> {
|
|
self.out_dest.pipe_stderr.as_ref()
|
|
}
|
|
|
|
/// Temporarily set the pipe stdout redirection to [`OutDest::Capture`].
|
|
///
|
|
/// This is used before evaluating an expression into a `Value`.
|
|
pub fn start_capture(&mut self) -> StackCaptureGuard {
|
|
StackCaptureGuard::new(self)
|
|
}
|
|
|
|
/// Temporarily use the output redirections in the parent scope.
|
|
///
|
|
/// This is used before evaluating an argument to a call.
|
|
pub fn use_call_arg_out_dest(&mut self) -> StackCallArgGuard {
|
|
StackCallArgGuard::new(self)
|
|
}
|
|
|
|
/// Temporarily apply redirections to stdout and/or stderr.
|
|
pub fn push_redirection(
|
|
&mut self,
|
|
stdout: Option<Redirection>,
|
|
stderr: Option<Redirection>,
|
|
) -> StackIoGuard {
|
|
StackIoGuard::new(self, stdout, stderr)
|
|
}
|
|
|
|
/// Mark stdout for the last command as [`OutDest::Capture`].
|
|
///
|
|
/// This will irreversibly alter the output redirections, and so it only makes sense to use this on an owned `Stack`
|
|
/// (which is why this function does not take `&mut self`).
|
|
///
|
|
/// See [`Stack::start_capture`] which can temporarily set stdout as [`OutDest::Capture`] for a mutable `Stack` reference.
|
|
pub fn capture(mut self) -> Self {
|
|
self.out_dest.pipe_stdout = Some(OutDest::Capture);
|
|
self.out_dest.pipe_stderr = None;
|
|
self
|
|
}
|
|
|
|
/// Clears any pipe and file redirections and resets stdout and stderr to [`OutDest::Inherit`].
|
|
///
|
|
/// This will irreversibly reset the output redirections, and so it only makes sense to use this on an owned `Stack`
|
|
/// (which is why this function does not take `&mut self`).
|
|
pub fn reset_out_dest(mut self) -> Self {
|
|
self.out_dest = StackOutDest::new();
|
|
self
|
|
}
|
|
|
|
/// Clears any pipe redirections, keeping the current stdout and stderr.
|
|
///
|
|
/// This will irreversibly reset some of the output redirections, and so it only makes sense to use this on an owned `Stack`
|
|
/// (which is why this function does not take `&mut self`).
|
|
pub fn reset_pipes(mut self) -> Self {
|
|
self.out_dest.pipe_stdout = None;
|
|
self.out_dest.pipe_stderr = None;
|
|
self
|
|
}
|
|
|
|
/// Replaces the default stdout of the stack with a given file.
|
|
///
|
|
/// This method configures the default stdout to redirect to a specified file.
|
|
/// It is primarily useful for applications using `nu` as a language, where the stdout of
|
|
/// external commands that are not explicitly piped can be redirected to a file.
|
|
///
|
|
/// # Using Pipes
|
|
///
|
|
/// For use in third-party applications pipes might be very useful as they allow using the
|
|
/// stdout of external commands for different uses.
|
|
/// For example the [`os_pipe`](https://docs.rs/os_pipe) crate provides a elegant way to to
|
|
/// access the stdout.
|
|
///
|
|
/// ```
|
|
/// # use std::{fs::File, io::{self, Read}, thread, error};
|
|
/// # use nu_protocol::engine::Stack;
|
|
/// #
|
|
/// let (mut reader, writer) = os_pipe::pipe().unwrap();
|
|
/// // Use a thread to avoid blocking the execution of the called command.
|
|
/// let reader = thread::spawn(move || {
|
|
/// let mut buf: Vec<u8> = Vec::new();
|
|
/// reader.read_to_end(&mut buf)?;
|
|
/// Ok::<_, io::Error>(buf)
|
|
/// });
|
|
///
|
|
/// #[cfg(windows)]
|
|
/// let file = std::os::windows::io::OwnedHandle::from(writer).into();
|
|
/// #[cfg(unix)]
|
|
/// let file = std::os::unix::io::OwnedFd::from(writer).into();
|
|
///
|
|
/// let stack = Stack::new().stdout_file(file);
|
|
///
|
|
/// // Execute some nu code.
|
|
///
|
|
/// drop(stack); // drop the stack so that the writer will be dropped too
|
|
/// let buf = reader.join().unwrap().unwrap();
|
|
/// // Do with your buffer whatever you want.
|
|
/// ```
|
|
pub fn stdout_file(mut self, file: File) -> Self {
|
|
self.out_dest.stdout = OutDest::File(Arc::new(file));
|
|
self
|
|
}
|
|
|
|
/// Replaces the default stderr of the stack with a given file.
|
|
///
|
|
/// For more info, see [`stdout_file`](Self::stdout_file).
|
|
pub fn stderr_file(mut self, file: File) -> Self {
|
|
self.out_dest.stderr = OutDest::File(Arc::new(file));
|
|
self
|
|
}
|
|
|
|
/// Set the PWD environment variable to `path`.
|
|
///
|
|
/// This method accepts `path` with trailing slashes, but they're removed
|
|
/// before writing the value into PWD.
|
|
pub fn set_cwd(&mut self, path: impl AsRef<std::path::Path>) -> Result<(), ShellError> {
|
|
// Helper function to create a simple generic error.
|
|
// Its messages are not especially helpful, but these errors don't occur often, so it's probably fine.
|
|
fn error(msg: &str) -> Result<(), ShellError> {
|
|
Err(ShellError::GenericError {
|
|
error: msg.into(),
|
|
msg: "".into(),
|
|
span: None,
|
|
help: None,
|
|
inner: vec![],
|
|
})
|
|
}
|
|
|
|
let path = path.as_ref();
|
|
|
|
if !path.is_absolute() {
|
|
error("Cannot set $env.PWD to a non-absolute path")
|
|
} else if !path.exists() {
|
|
error("Cannot set $env.PWD to a non-existent directory")
|
|
} else if !path.is_dir() {
|
|
error("Cannot set $env.PWD to a non-directory")
|
|
} else {
|
|
// Strip trailing slashes, if any.
|
|
let path = nu_path::strip_trailing_slash(path);
|
|
let value = Value::string(path.to_string_lossy(), Span::unknown());
|
|
self.add_env_var("PWD".into(), value);
|
|
Ok(())
|
|
}
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod test {
|
|
use std::sync::Arc;
|
|
|
|
use crate::{engine::EngineState, Span, Value};
|
|
|
|
use super::Stack;
|
|
|
|
const ZERO_SPAN: Span = Span { start: 0, end: 0 };
|
|
fn string_value(s: &str) -> Value {
|
|
Value::String {
|
|
val: s.to_string(),
|
|
internal_span: ZERO_SPAN,
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn test_children_see_inner_values() {
|
|
let mut original = Stack::new();
|
|
original.add_var(0, string_value("hello"));
|
|
|
|
let cloned = Stack::with_parent(Arc::new(original));
|
|
assert_eq!(cloned.get_var(0, ZERO_SPAN), Ok(string_value("hello")));
|
|
}
|
|
|
|
#[test]
|
|
fn test_children_dont_see_deleted_values() {
|
|
let mut original = Stack::new();
|
|
original.add_var(0, string_value("hello"));
|
|
|
|
let mut cloned = Stack::with_parent(Arc::new(original));
|
|
cloned.remove_var(0);
|
|
|
|
assert_eq!(
|
|
cloned.get_var(0, ZERO_SPAN),
|
|
Err(crate::ShellError::VariableNotFoundAtRuntime { span: ZERO_SPAN })
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn test_children_changes_override_parent() {
|
|
let mut original = Stack::new();
|
|
original.add_var(0, string_value("hello"));
|
|
|
|
let mut cloned = Stack::with_parent(Arc::new(original));
|
|
cloned.add_var(0, string_value("there"));
|
|
assert_eq!(cloned.get_var(0, ZERO_SPAN), Ok(string_value("there")));
|
|
|
|
cloned.remove_var(0);
|
|
// the underlying value shouldn't magically re-appear
|
|
assert_eq!(
|
|
cloned.get_var(0, ZERO_SPAN),
|
|
Err(crate::ShellError::VariableNotFoundAtRuntime { span: ZERO_SPAN })
|
|
);
|
|
}
|
|
#[test]
|
|
fn test_children_changes_persist_in_offspring() {
|
|
let mut original = Stack::new();
|
|
original.add_var(0, string_value("hello"));
|
|
|
|
let mut cloned = Stack::with_parent(Arc::new(original));
|
|
cloned.add_var(1, string_value("there"));
|
|
|
|
cloned.remove_var(0);
|
|
let cloned = Stack::with_parent(Arc::new(cloned));
|
|
|
|
assert_eq!(
|
|
cloned.get_var(0, ZERO_SPAN),
|
|
Err(crate::ShellError::VariableNotFoundAtRuntime { span: ZERO_SPAN })
|
|
);
|
|
|
|
assert_eq!(cloned.get_var(1, ZERO_SPAN), Ok(string_value("there")));
|
|
}
|
|
|
|
#[test]
|
|
fn test_merging_children_back_to_parent() {
|
|
let mut original = Stack::new();
|
|
let engine_state = EngineState::new();
|
|
original.add_var(0, string_value("hello"));
|
|
|
|
let original_arc = Arc::new(original);
|
|
let mut cloned = Stack::with_parent(original_arc.clone());
|
|
cloned.add_var(1, string_value("there"));
|
|
|
|
cloned.remove_var(0);
|
|
|
|
cloned.add_env_var("ADDED_IN_CHILD".to_string(), string_value("New Env Var"));
|
|
|
|
let original = Stack::with_changes_from_child(original_arc, cloned);
|
|
|
|
assert_eq!(
|
|
original.get_var(0, ZERO_SPAN),
|
|
Err(crate::ShellError::VariableNotFoundAtRuntime { span: ZERO_SPAN })
|
|
);
|
|
|
|
assert_eq!(original.get_var(1, ZERO_SPAN), Ok(string_value("there")));
|
|
|
|
assert_eq!(
|
|
original.get_env_var(&engine_state, "ADDED_IN_CHILD"),
|
|
Some(string_value("New Env Var")),
|
|
);
|
|
}
|
|
}
|