Mirrors/nushell
2
0
Fork 0
mirror of https://github.com/nushell/nushell synced 2024-12-27 05:23:11 +00:00
Code Issues Projects Releases Packages Wiki Activity

IO and redirection overhaul (#11934)

Browse source 
# Description
The PR overhauls how IO redirection is handled, allowing more explicit
and fine-grain control over `stdout` and `stderr` output as well as more
efficient IO and piping.

To summarize the changes in this PR:
- Added a new `IoStream` type to indicate the intended destination for a
pipeline element's `stdout` and `stderr`.
- The `stdout` and `stderr` `IoStream`s are stored in the `Stack` and to
avoid adding 6 additional arguments to every eval function and
`Command::run`. The `stdout` and `stderr` streams can be temporarily
overwritten through functions on `Stack` and these functions will return
a guard that restores the original `stdout` and `stderr` when dropped.
- In the AST, redirections are now directly part of a `PipelineElement`
as a `Option<Redirection>` field instead of having multiple different
`PipelineElement` enum variants for each kind of redirection. This
required changes to the parser, mainly in `lite_parser.rs`.
- `Command`s can also set a `IoStream` override/redirection which will
apply to the previous command in the pipeline. This is used, for
example, in `ignore` to allow the previous external command to have its
stdout redirected to `Stdio::null()` at spawn time. In contrast, the
current implementation has to create an os pipe and manually consume the
output on nushell's side. File and pipe redirections (`o>`, `e>`, `e>|`,
etc.) have precedence over overrides from commands.

This PR improves piping and IO speed, partially addressing #10763. Using
the `throughput` command from that issue, this PR gives the following
speedup on my setup for the commands below:
| Command | Before (MB/s) | After (MB/s) | Bash (MB/s) |
| --------------------------- | -------------:| ------------:|
-----------:|
| `throughput o> /dev/null` | 1169 | 52938 | 54305 |
| `throughput \| ignore` | 840 | 55438 | N/A |
| `throughput \| null` | Error | 53617 | N/A |
| `throughput \| rg 'x'` | 1165 | 3049 | 3736 |
| `(throughput) \| rg 'x'` | 810 | 3085 | 3815 |

(Numbers above are the median samples for throughput)

This PR also paves the way to refactor our `ExternalStream` handling in
the various commands. For example, this PR already fixes the following
code:
```nushell
^sh -c 'echo -n "hello "; sleep 0; echo "world"' | find "hello world"
```
This returns an empty list on 0.90.1 and returns a highlighted "hello
world" on this PR.

Since the `stdout` and `stderr` `IoStream`s are available to commands
when they are run, then this unlocks the potential for more convenient
behavior. E.g., the `find` command can disable its ansi highlighting if
it detects that the output `IoStream` is not the terminal. Knowing the
output streams will also allow background job output to be redirected
more easily and efficiently.

# User-Facing Changes
- External commands returned from closures will be collected (in most
cases):
  ```nushell
  1..2 | each {|_| nu -c "print a" }
  ```
This gives `["a", "a"]` on this PR, whereas this used to print "a\na\n"
and then return an empty list.

  ```nushell
  1..2 | each {|_| nu -c "print -e a" }
  ```
This gives `["", ""]` and prints "a\na\n" to stderr, whereas this used
to return an empty list and print "a\na\n" to stderr.

- Trailing new lines are always trimmed for external commands when
piping into internal commands or collecting it as a value. (Failure to
decode the output as utf-8 will keep the trailing newline for the last
binary value.) In the current nushell version, the following three code
snippets differ only in parenthesis placement, but they all also have
different outputs:

  1. `1..2 | each { ^echo a }`
     ```
     a
     a
     ╭────────────╮
     │ empty list │
     ╰────────────╯
     ```
  2. `1..2 | each { (^echo a) }`
     ```
     ╭───┬───╮
     │ 0 │ a │
     │ 1 │ a │
     ╰───┴───╯
     ```
  3. `1..2 | (each { ^echo a })`
     ```
     ╭───┬───╮
     │ 0 │ a │
     │   │   │
     │ 1 │ a │
     │   │   │
     ╰───┴───╯
     ```

  But in this PR, the above snippets will all have the same output:
  ```
  ╭───┬───╮
  │ 0 │ a │
  │ 1 │ a │
  ╰───┴───╯
  ```

- All existing flags on `run-external` are now deprecated.

- File redirections now apply to all commands inside a code block:
  ```nushell
  (nu -c "print -e a"; nu -c "print -e b") e> test.out
  ```
This gives "a\nb\n" in `test.out` and prints nothing. The same result
would happen when printing to stdout and using a `o>` file redirection.

- External command output will (almost) never be ignored, and ignoring
output must be explicit now:
  ```nushell
  (^echo a; ^echo b)
  ```
This prints "a\nb\n", whereas this used to print only "b\n". This only
applies to external commands; values and internal commands not in return
position will not print anything (e.g., `(echo a; echo b)` still only
prints "b").

- `complete` now always captures stderr (`do` is not necessary).

# After Submitting
The language guide and other documentation will need to be updated.
This commit is contained in:
Ian Manske 2024-03-14 20:51:55 +00:00 committed by GitHub
parent e2907e7e3a
commit b6c7656194
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
113 changed files with 3272 additions and 4022 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

36
crates/nu-cli/src/completions/completer.rs
View file

@ -4,10 +4,9 @@ use crate::completions::{
};
use nu_color_config::{color_record_to_nustyle, lookup_ansi_color_style};
use nu_engine::eval_block;
use nu_parser::{flatten_expression, parse, FlatShape};
use nu_parser::{flatten_pipeline_element, parse, FlatShape};
use nu_protocol::debugger::WithoutDebug;
use nu_protocol::{
ast::PipelineElement,
engine::{EngineState, Stack, StateWorkingSet},
BlockId, PipelineData, Span, Value,
};
@ -25,7 +24,7 @@ impl NuCompleter {
pub fn new(engine_state: Arc<EngineState>, stack: Stack) -> Self {
Self {
engine_state,
stack,
stack: stack.reset_stdio().capture(),
}
}
@ -64,9 +63,10 @@ impl NuCompleter {
offset: usize,
span: Span,
) -> Option<Vec<Suggestion>> {
let stack = self.stack.clone();
let block = self.engine_state.get_block(block_id);
let mut callee_stack = stack.gather_captures(&self.engine_state, &block.captures);
let mut callee_stack = self
.stack
.gather_captures(&self.engine_state, &block.captures);
// Line
if let Some(pos_arg) = block.signature.required_positional.first() {
@ -89,8 +89,6 @@ impl NuCompleter {
&mut callee_stack,
block,
PipelineData::empty(),
true,
true,
);
match result {
@ -128,26 +126,13 @@ impl NuCompleter {
for pipeline in output.pipelines.into_iter() {
for pipeline_element in pipeline.elements {
match pipeline_element {
PipelineElement::Expression(_, expr)
| PipelineElement::ErrPipedExpression(_, expr)
| PipelineElement::OutErrPipedExpression(_, expr)
| PipelineElement::Redirection(_, _, expr, _)
| PipelineElement::And(_, expr)
| PipelineElement::Or(_, expr)
| PipelineElement::SameTargetRedirection { cmd: (_, expr), .. }
| PipelineElement::SeparateRedirection {
out: (_, expr, _), ..
} => {
let flattened: Vec<_> = flatten_expression(&working_set, &expr);
let flattened = flatten_pipeline_element(&working_set, &pipeline_element);
let mut spans: Vec<String> = vec![];
for (flat_idx, flat) in flattened.iter().enumerate() {
let is_passthrough_command = spans
.first()
.filter(|content| {
content.as_str() == "sudo" || content.as_str() == "doas"
})
.filter(|content| content.as_str() == "sudo" || content.as_str() == "doas")
.is_some();
// Read the current spam to string
let current_span = working_set.get_span_contents(flat.0).to_vec();
@ -204,7 +189,7 @@ impl NuCompleter {
// Flags completion
if prefix.starts_with(b"-") {
// Try to complete flag internally
let mut completer = FlagCompletion::new(expr.clone());
let mut completer = FlagCompletion::new(pipeline_element.expr.clone());
let result = self.process_completion(
&mut completer,
&working_set,
@ -234,8 +219,7 @@ impl NuCompleter {
// specially check if it is currently empty - always complete commands
if (is_passthrough_command && flat_idx == 1)
|| (flat_idx == 0
&& working_set.get_span_contents(new_span).is_empty())
|| (flat_idx == 0 && working_set.get_span_contents(new_span).is_empty())
{
let mut completer = CommandCompletion::new(
self.engine_state.clone(),
@ -405,8 +389,6 @@ impl NuCompleter {
}
}
}
}
}
vec![]
}

4
crates/nu-cli/src/completions/custom_completions.rs
View file

@ -25,7 +25,7 @@ impl CustomCompletion {
pub fn new(engine_state: Arc<EngineState>, stack: Stack, decl_id: usize, line: String) -> Self {
Self {
engine_state,
stack,
stack: stack.reset_stdio().capture(),
decl_id,
line,
sort_by: SortBy::None,
@ -67,8 +67,6 @@ impl Completer for CustomCompletion {
custom_completion: None,
}),
],
redirect_stdout: true,
redirect_stderr: true,
parser_info: HashMap::new(),
},
PipelineData::empty(),

10
crates/nu-cli/src/eval_cmds.rs
View file

@ -56,19 +56,13 @@ pub fn evaluate_commands(
}
// Run the block
let exit_code =
match eval_block::<WithoutDebug>(engine_state, stack, &block, input, false, false) {
let exit_code = match eval_block::<WithoutDebug>(engine_state, stack, &block, input) {
Ok(pipeline_data) => {
let mut config = engine_state.get_config().clone();
if let Some(t_mode) = table_mode {
config.table_mode = t_mode.coerce_str()?.parse().unwrap_or_default();
}
crate::eval_file::print_table_or_error(
engine_state,
stack,
pipeline_data,
&mut config,
)
crate::eval_file::print_table_or_error(engine_state, stack, pipeline_data, &mut config)
}
Err(err) => {
let working_set = StateWorkingSet::new(engine_state);

13
crates/nu-cli/src/eval_file.rs
View file

@ -131,14 +131,8 @@ pub fn evaluate_file(
if engine_state.find_decl(b"main", &[]).is_some() {
let args = format!("main {}", args.join(" "));
let pipeline_data = eval_block::<WithoutDebug>(
engine_state,
stack,
&block,
PipelineData::empty(),
false,
false,
);
let pipeline_data =
eval_block::<WithoutDebug>(engine_state, stack, &block, PipelineData::empty());
let pipeline_data = match pipeline_data {
Err(ShellError::Return { .. }) => {
// allows early exists before `main` is run.
@ -214,8 +208,7 @@ pub(crate) fn print_table_or_error(
print_or_exit(pipeline_data, engine_state, config);
} else {
// The final call on table command, it's ok to set redirect_output to false.
let mut call = Call::new(Span::new(0, 0));
call.redirect_stdout = false;
let call = Call::new(Span::new(0, 0));
let table = command.run(engine_state, stack, &call, pipeline_data);
match table {

11
crates/nu-cli/src/menus/menu_completions.rs
View file

@ -28,7 +28,7 @@ impl NuMenuCompleter {
Self {
block_id,
span,
stack,
stack: stack.reset_stdio().capture(),
engine_state,
only_buffer_difference,
}
@ -57,14 +57,7 @@ impl Completer for NuMenuCompleter {
let input = Value::nothing(self.span).into_pipeline_data();
let res = eval_block::<WithoutDebug>(
&self.engine_state,
&mut self.stack,
block,
input,
false,
false,
);
let res = eval_block::<WithoutDebug>(&self.engine_state, &mut self.stack, block, input);
if let Ok(values) = res {
let values = values.into_value(self.span);

12
crates/nu-cli/src/reedline_config.rs
View file

@ -109,17 +109,9 @@ pub(crate) fn add_menus(
(output, working_set.render())
};
let mut temp_stack = Stack::new();
let mut temp_stack = Stack::new().capture();
let input = PipelineData::Empty;
let res = eval_block::<WithoutDebug>(
&engine_state,
&mut temp_stack,
&block,
input,
false,
false,
)?;
let res = eval_block::<WithoutDebug>(&engine_state, &mut temp_stack, &block, input)?;
if let PipelineData::Value(value, None) = res {
for menu in create_menus(&value)? {

36
crates/nu-cli/src/syntax_highlight.rs
View file

@ -3,7 +3,7 @@ use nu_ansi_term::Style;
use nu_color_config::{get_matching_brackets_style, get_shape_color};
use nu_engine::env;
use nu_parser::{flatten_block, parse, FlatShape};
use nu_protocol::ast::{Argument, Block, Expr, Expression, PipelineElement, RecordItem};
use nu_protocol::ast::{Argument, Block, Expr, Expression, PipelineRedirection, RecordItem};
use nu_protocol::engine::{EngineState, Stack, StateWorkingSet};
use nu_protocol::{Config, Span};
use reedline::{Highlighter, StyledText};
@ -262,26 +262,38 @@ fn find_matching_block_end_in_block(
) -> Option<usize> {
for p in &block.pipelines {
for e in &p.elements {
match e {
PipelineElement::Expression(_, e)
| PipelineElement::ErrPipedExpression(_, e)
| PipelineElement::OutErrPipedExpression(_, e)
| PipelineElement::Redirection(_, _, e, _)
| PipelineElement::And(_, e)
| PipelineElement::Or(_, e)
| PipelineElement::SameTargetRedirection { cmd: (_, e), .. }
| PipelineElement::SeparateRedirection { out: (_, e, _), .. } => {
if e.span.contains(global_cursor_offset) {
if e.expr.span.contains(global_cursor_offset) {
if let Some(pos) = find_matching_block_end_in_expr(
line,
working_set,
e,
&e.expr,
global_span_offset,
global_cursor_offset,
) {
return Some(pos);
}
}
if let Some(redirection) = e.redirection.as_ref() {
match redirection {
PipelineRedirection::Single { target, .. }
| PipelineRedirection::Separate { out: target, .. }
| PipelineRedirection::Separate { err: target, .. }
if target.span().contains(global_cursor_offset) =>
{
if let Some(pos) = target.expr().and_then(|expr| {
find_matching_block_end_in_expr(
line,
working_set,
expr,
global_span_offset,
global_cursor_offset,
)
}) {
return Some(pos);
}
}
_ => {}
}
}
}

11
crates/nu-cli/src/util.rs
View file

@ -241,16 +241,9 @@ pub fn eval_source(
}
let b = if allow_return {
eval_block_with_early_return::<WithoutDebug>(
engine_state,
stack,
&block,
input,
false,
false,
)
eval_block_with_early_return::<WithoutDebug>(engine_state, stack, &block, input)
} else {
eval_block::<WithoutDebug>(engine_state, stack, &block, input, false, false)
eval_block::<WithoutDebug>(engine_state, stack, &block, input)
};
match b {

4
crates/nu-cli/tests/support/completions_helpers.rs
View file

@ -200,9 +200,7 @@ pub fn merge_input(
engine_state,
stack,
&block,
PipelineData::Value(Value::nothing(Span::unknown(),), None),
false,
false,
PipelineData::Value(Value::nothing(Span::unknown()), None),
)
.is_ok());

17
crates/nu-cmd-base/src/hook.rs
View file

@ -116,7 +116,7 @@ pub fn eval_hook(
})
.collect();
match eval_block::<WithoutDebug>(engine_state, stack, &block, input, false, false) {
match eval_block::<WithoutDebug>(engine_state, stack, &block, input) {
Ok(pipeline_data) => {
output = pipeline_data;
}
@ -244,14 +244,7 @@ pub fn eval_hook(
})
.collect();
match eval_block::<WithoutDebug>(
engine_state,
stack,
&block,
input,
false,
false,
) {
match eval_block::<WithoutDebug>(engine_state, stack, &block, input) {
Ok(pipeline_data) => {
output = pipeline_data;
}
@ -327,7 +320,9 @@ fn run_hook_block(
let input = optional_input.unwrap_or_else(PipelineData::empty);
let mut callee_stack = stack.gather_captures(engine_state, &block.captures);
let mut callee_stack = stack
.gather_captures(engine_state, &block.captures)
.reset_pipes();
for (idx, PositionalArg { var_id, .. }) in
block.signature.required_positional.iter().enumerate()
@ -349,8 +344,6 @@ fn run_hook_block(
&mut callee_stack,
block,
input,
false,
false,
)?;
if let PipelineData::Value(Value::Error { error, .. }, _) = pipeline_data {

12
crates/nu-cmd-dataframe/src/dataframe/test_dataframe.rs
View file

@ -79,16 +79,10 @@ pub fn test_dataframe_example(engine_state: &mut Box<EngineState>, example: &Exa
.merge_delta(delta)
.expect("Error merging delta");
let mut stack = Stack::new();
let mut stack = Stack::new().capture();
let result = eval_block::<WithoutDebug>(
engine_state,
&mut stack,
&block,
PipelineData::empty(),
true,
true,
)
let result =
eval_block::<WithoutDebug>(engine_state, &mut stack, &block, PipelineData::empty())
.unwrap_or_else(|err| panic!("test eval error in `{}`: {:?}", example.example, err))
.into_value(Span::test_data());

8
crates/nu-cmd-extra/src/extra/filters/each_while.rs
View file

@ -83,8 +83,6 @@ impl Command for EachWhile {
let orig_env_vars = stack.env_vars.clone();
let orig_env_hidden = stack.env_hidden.clone();
let span = call.head;
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let eval_block_with_early_return = get_eval_block_with_early_return(&engine_state);
match input {
@ -111,8 +109,6 @@ impl Command for EachWhile {
&mut stack,
&block,
x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) {
Ok(v) => {
let value = v.into_value(span);
@ -155,8 +151,6 @@ impl Command for EachWhile {
&mut stack,
&block,
x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) {
Ok(v) => {
let value = v.into_value(span);
@ -185,8 +179,6 @@ impl Command for EachWhile {
&mut stack,
&block,
x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
)
}
}

22
crates/nu-cmd-extra/src/extra/filters/update_cells.rs
View file

@ -105,9 +105,6 @@ impl Command for UpdateCells {
let block: Block = engine_state.get_block(block.block_id).clone();
let eval_block_fn = get_eval_block(&engine_state);
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let span = call.head;
stack.with_env(&orig_env_vars, &orig_env_hidden);
@ -130,8 +127,6 @@ impl Command for UpdateCells {
stack,
block,
columns,
redirect_stdout,
redirect_stderr,
span,
eval_block_fn,
}
@ -146,8 +141,6 @@ struct UpdateCellIterator {
engine_state: EngineState,
stack: Stack,
block: Block,
redirect_stdout: bool,
redirect_stderr: bool,
eval_block_fn: EvalBlockFn,
span: Span,
}
@ -177,8 +170,6 @@ impl Iterator for UpdateCellIterator {
&self.engine_state,
&mut self.stack,
&self.block,
self.redirect_stdout,
self.redirect_stderr,
span,
self.eval_block_fn,
),
@ -192,8 +183,6 @@ impl Iterator for UpdateCellIterator {
&self.engine_state,
&mut self.stack,
&self.block,
self.redirect_stdout,
self.redirect_stderr,
self.span,
self.eval_block_fn,
)),
@ -210,8 +199,6 @@ fn process_cell(
engine_state: &EngineState,
stack: &mut Stack,
block: &Block,
redirect_stdout: bool,
redirect_stderr: bool,
span: Span,
eval_block_fn: EvalBlockFn,
) -> Value {
@ -221,14 +208,7 @@ fn process_cell(
}
}
match eval_block_fn(
engine_state,
stack,
block,
val.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) {
match eval_block_fn(engine_state, stack, block, val.into_pipeline_data()) {
Ok(pd) => pd.into_value(span),
Err(e) => Value::error(e, span),
}

2
crates/nu-cmd-extra/src/extra/strings/format/command.rs
View file

@ -295,7 +295,7 @@ fn format_record(
}
}
FormatOperation::ValueNeedEval(_col_name, span) => {
let exp = parse_expression(working_set, &[*span], false);
let exp = parse_expression(working_set, &[*span]);
match working_set.parse_errors.first() {
None => {
let parsed_result = eval_expression(engine_state, stack, &exp);

5
crates/nu-cmd-lang/src/core_commands/collect.rs
View file

@ -45,7 +45,8 @@ impl Command for Collect {
let capture_block: Closure = call.req(engine_state, stack, 0)?;
let block = engine_state.get_block(capture_block.block_id).clone();
let mut stack_captures = stack.captures_to_stack(capture_block.captures.clone());
let mut stack_captures =
stack.captures_to_stack_preserve_stdio(capture_block.captures.clone());
let metadata = input.metadata();
let input: Value = input.into_value(call.head);
@ -65,8 +66,6 @@ impl Command for Collect {
&mut stack_captures,
&block,
input.into_pipeline_data(),
call.redirect_stdout,
call.redirect_stderr,
)
.map(|x| x.set_metadata(metadata));

19
crates/nu-cmd-lang/src/core_commands/do_.rs
View file

@ -5,8 +5,8 @@ use nu_protocol::ast::Call;
use nu_protocol::engine::{Closure, Command, EngineState, Stack};
use nu_protocol::{
Category, Example, IntoSpanned, ListStream, PipelineData, RawStream, ShellError, Signature,
Span, SyntaxShape, Type, Value,
Category, Example, IntoSpanned, IoStream, ListStream, PipelineData, RawStream, ShellError,
Signature, Span, SyntaxShape, Type, Value,
};
#[derive(Clone)]
@ -79,19 +79,12 @@ impl Command for Do {
let capture_errors = call.has_flag(engine_state, caller_stack, "capture-errors")?;
let has_env = call.has_flag(engine_state, caller_stack, "env")?;
let mut callee_stack = caller_stack.captures_to_stack(block.captures);
let mut callee_stack = caller_stack.captures_to_stack_preserve_stdio(block.captures);
let block = engine_state.get_block(block.block_id);
bind_args_to(&mut callee_stack, &block.signature, rest, call.head)?;
let eval_block_with_early_return = get_eval_block_with_early_return(engine_state);
let result = eval_block_with_early_return(
engine_state,
&mut callee_stack,
block,
input,
call.redirect_stdout,
call.redirect_stdout,
);
let result = eval_block_with_early_return(engine_state, &mut callee_stack, block, input);
if has_env {
// Merge the block's environment to the current stack
@ -204,7 +197,9 @@ impl Command for Do {
span,
metadata,
trim_end_newline,
}) if ignore_program_errors && !call.redirect_stdout => {
}) if ignore_program_errors
&& !matches!(caller_stack.stdout(), IoStream::Pipe | IoStream::Capture) =>
{
Ok(PipelineData::ExternalStream {
stdout,
stderr,

50
crates/nu-cmd-lang/src/core_commands/echo.rs
View file

@ -2,8 +2,8 @@ use nu_engine::CallExt;
use nu_protocol::ast::Call;
use nu_protocol::engine::{Command, EngineState, Stack};
use nu_protocol::{
Category, Example, ListStream, PipelineData, ShellError, Signature, Span, SyntaxShape, Type,
Value,
Category, Example, IntoPipelineData, PipelineData, ShellError, Signature, Span, SyntaxShape,
Type, Value,
};
#[derive(Clone)]
@ -38,8 +38,13 @@ little reason to use this over just writing the values as-is."#
call: &Call,
_input: PipelineData,
) -> Result<PipelineData, ShellError> {
let args = call.rest(engine_state, stack, 0);
run(engine_state, args, stack, call)
let mut args = call.rest(engine_state, stack, 0)?;
let value = match args.len() {
0 => Value::string("", call.head),
1 => args.pop().expect("one element"),
_ => Value::list(args, call.head),
};
Ok(value.into_pipeline_data())
}
fn examples(&self) -> Vec<Example> {
@ -62,43 +67,6 @@ little reason to use this over just writing the values as-is."#
}
}
fn run(
engine_state: &EngineState,
args: Result<Vec<Value>, ShellError>,
stack: &mut Stack,
call: &Call,
) -> Result<PipelineData, ShellError> {
let result = args.map(|to_be_echoed| {
let n = to_be_echoed.len();
match n.cmp(&1usize) {
// More than one value is converted in a stream of values
std::cmp::Ordering::Greater => PipelineData::ListStream(
ListStream::from_stream(to_be_echoed.into_iter(), engine_state.ctrlc.clone()),
None,
),
// But a single value can be forwarded as it is
std::cmp::Ordering::Equal => PipelineData::Value(to_be_echoed[0].clone(), None),
// When there are no elements, we echo the empty string
std::cmp::Ordering::Less => PipelineData::Value(Value::string("", call.head), None),
}
});
// If echo is not redirected, then print to the screen (to behave in a similar way to other shells)
if !call.redirect_stdout {
match result {
Ok(pipeline) => {
pipeline.print(engine_state, stack, false, false)?;
Ok(PipelineData::Empty)
}
Err(err) => Err(err),
}
} else {
result
}
}
#[cfg(test)]
mod test {
#[test]

32
crates/nu-cmd-lang/src/core_commands/for_.rs
View file

@ -84,8 +84,8 @@ impl Command for For {
let ctrlc = engine_state.ctrlc.clone();
let engine_state = engine_state.clone();
let block = engine_state.get_block(block.block_id).clone();
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let stack = &mut stack.push_redirection(None, None);
match values {
Value::List { vals, .. } => {
@ -109,14 +109,7 @@ impl Command for For {
},
);
match eval_block(
&engine_state,
stack,
&block,
PipelineData::empty(),
redirect_stdout,
redirect_stderr,
) {
match eval_block(&engine_state, stack, &block, PipelineData::empty()) {
Err(ShellError::Break { .. }) => {
break;
}
@ -154,14 +147,7 @@ impl Command for For {
},
);
match eval_block(
&engine_state,
stack,
&block,
PipelineData::empty(),
redirect_stdout,
redirect_stderr,
) {
match eval_block(&engine_state, stack, &block, PipelineData::empty()) {
Err(ShellError::Break { .. }) => {
break;
}
@ -185,15 +171,7 @@ impl Command for For {
x => {
stack.add_var(var_id, x);
eval_block(
&engine_state,
stack,
&block,
PipelineData::empty(),
redirect_stdout,
redirect_stderr,
)?
.into_value(head);
eval_block(&engine_state, stack, &block, PipelineData::empty())?.into_value(head);
}
}
Ok(PipelineData::empty())

36
crates/nu-cmd-lang/src/core_commands/if_.rs
View file

@ -115,46 +115,18 @@ impl Command for If {
Value::Bool { val, .. } => {
if *val {
let block = engine_state.get_block(then_block.block_id);
eval_block(
engine_state,
stack,
block,
input,
call.redirect_stdout,
call.redirect_stderr,
)
eval_block(engine_state, stack, block, input)
} else if let Some(else_case) = else_case {
if let Some(else_expr) = else_case.as_keyword() {
if let Some(block_id) = else_expr.as_block() {
let block = engine_state.get_block(block_id);
eval_block(
engine_state,
stack,
block,
input,
call.redirect_stdout,
call.redirect_stderr,
)
eval_block(engine_state, stack, block, input)
} else {
eval_expression_with_input(
engine_state,
stack,
else_expr,
input,
call.redirect_stdout,
call.redirect_stderr,
)
eval_expression_with_input(engine_state, stack, else_expr, input)
.map(|res| res.0)
}
} else {
eval_expression_with_input(
engine_state,
stack,
else_case,
input,
call.redirect_stdout,
call.redirect_stderr,
)
eval_expression_with_input(engine_state, stack, else_case, input)
.map(|res| res.0)
}
} else {

8
crates/nu-cmd-lang/src/core_commands/ignore.rs
View file

@ -1,6 +1,8 @@
use nu_protocol::ast::Call;
use nu_protocol::engine::{Command, EngineState, Stack, StateWorkingSet};
use nu_protocol::{Category, Example, PipelineData, ShellError, Signature, Span, Type, Value};
use nu_protocol::{
Category, Example, IoStream, PipelineData, ShellError, Signature, Span, Type, Value,
};
#[derive(Clone)]
pub struct Ignore;
@ -56,6 +58,10 @@ impl Command for Ignore {
result: Some(Value::nothing(Span::test_data())),
}]
}
fn stdio_redirect(&self) -> (Option<IoStream>, Option<IoStream>) {
(Some(IoStream::Null), None)
}
}
#[cfg(test)]

6
crates/nu-cmd-lang/src/core_commands/lazy_make.rs
View file

@ -86,10 +86,12 @@ impl Command for LazyMake {
}
}
let stack = stack.clone().reset_stdio().capture();
Ok(Value::lazy_record(
Box::new(NuLazyRecord {
engine_state: engine_state.clone(),
stack: Arc::new(Mutex::new(stack.clone())),
stack: Arc::new(Mutex::new(stack)),
columns: columns.into_iter().map(|s| s.item).collect(),
get_value,
span,
@ -152,8 +154,6 @@ impl<'a> LazyRecord<'a> for NuLazyRecord {
&mut stack,
block,
PipelineData::Value(column_value, None),
false,
false,
);
pipeline_result.map(|data| match data {

3
crates/nu-cmd-lang/src/core_commands/let_.rs
View file

@ -64,7 +64,8 @@ impl Command for Let {
let block = engine_state.get_block(block_id);
let eval_block = get_eval_block(engine_state);
let pipeline_data = eval_block(engine_state, stack, block, input, true, false)?;
let stack = &mut stack.start_capture();
let pipeline_data = eval_block(engine_state, stack, block, input)?;
let mut value = pipeline_data.into_value(call.head);
// if given variable type is Glob, and our result is string

11
crates/nu-cmd-lang/src/core_commands/loop_.rs
View file

@ -36,6 +36,8 @@ impl Command for Loop {
let block: Block = call.req(engine_state, stack, 0)?;
let eval_block = get_eval_block(engine_state);
let stack = &mut stack.push_redirection(None, None);
loop {
if nu_utils::ctrl_c::was_pressed(&engine_state.ctrlc) {
break;
@ -43,14 +45,7 @@ impl Command for Loop {
let block = engine_state.get_block(block.block_id);
match eval_block(
engine_state,
stack,
block,
PipelineData::empty(),
call.redirect_stdout,
call.redirect_stderr,
) {
match eval_block(engine_state, stack, block, PipelineData::empty()) {
Err(ShellError::Break { .. }) => {
break;
}

18
crates/nu-cmd-lang/src/core_commands/match_.rs
View file

@ -70,23 +70,9 @@ impl Command for Match {
if guard_matches {
return if let Some(block_id) = match_.1.as_block() {
let block = engine_state.get_block(block_id);
eval_block(
engine_state,
stack,
block,
input,
call.redirect_stdout,
call.redirect_stderr,
)
eval_block(engine_state, stack, block, input)
} else {
eval_expression_with_input(
engine_state,
stack,
&match_.1,
input,
call.redirect_stdout,
call.redirect_stderr,
)
eval_expression_with_input(engine_state, stack, &match_.1, input)
.map(|x| x.0)
};
}

11
crates/nu-cmd-lang/src/core_commands/mut_.rs
View file

@ -65,15 +65,8 @@ impl Command for Mut {
let block = engine_state.get_block(block_id);
let eval_block = get_eval_block(engine_state);
let pipeline_data = eval_block(
engine_state,
stack,
block,
input,
call.redirect_stdout,
call.redirect_stderr,
)?;
let stack = &mut stack.start_capture();
let pipeline_data = eval_block(engine_state, stack, block, input)?;
let mut value = pipeline_data.into_value(call.head);
// if given variable type is Glob, and our result is string

14
crates/nu-cmd-lang/src/core_commands/overlay/use_.rs
View file

@ -124,7 +124,9 @@ impl Command for OverlayUse {
)?;
let block = engine_state.get_block(block_id);
let mut callee_stack = caller_stack.gather_captures(engine_state, &block.captures);
let mut callee_stack = caller_stack
.gather_captures(engine_state, &block.captures)
.reset_pipes();
if let Some(path) = &maybe_path {
// Set the currently evaluated directory, if the argument is a valid path
@ -142,15 +144,7 @@ impl Command for OverlayUse {
}
let eval_block = get_eval_block(engine_state);
let _ = eval_block(
engine_state,
&mut callee_stack,
block,
input,
call.redirect_stdout,
call.redirect_stderr,
);
let _ = eval_block(engine_state, &mut callee_stack, block, input);
// The export-env block should see the env vars *before* activating this overlay
caller_stack.add_overlay(overlay_name);

4
crates/nu-cmd-lang/src/core_commands/try_.rs
View file

@ -50,7 +50,7 @@ impl Command for Try {
let try_block = engine_state.get_block(try_block.block_id);
let eval_block = get_eval_block(engine_state);
let result = eval_block(engine_state, stack, try_block, input, false, false);
let result = eval_block(engine_state, stack, try_block, input);
match result {
Err(error) => {
@ -117,8 +117,6 @@ fn handle_catch(
catch_block,
// Make the error accessible with $in, too
err_value.into_pipeline_data(),
false,
false,
)
} else {
Ok(PipelineData::empty())

13
crates/nu-cmd-lang/src/core_commands/use_.rs
View file

@ -105,7 +105,9 @@ This command is a parser keyword. For details, check:
.as_ref()
.and_then(|path| path.parent().map(|p| p.to_path_buf()));
let mut callee_stack = caller_stack.gather_captures(engine_state, &block.captures);
let mut callee_stack = caller_stack
.gather_captures(engine_state, &block.captures)
.reset_pipes();
// If so, set the currently evaluated directory (file-relative PWD)
if let Some(parent) = maybe_parent {
@ -121,14 +123,7 @@ This command is a parser keyword. For details, check:
let eval_block = get_eval_block(engine_state);
// Run the block (discard the result)
let _ = eval_block(
engine_state,
&mut callee_stack,
block,
input,
call.redirect_stdout,
call.redirect_stderr,
)?;
let _ = eval_block(engine_state, &mut callee_stack, block, input)?;
// Merge the block's environment to the current stack
redirect_env(engine_state, caller_stack, &callee_stack);

11
crates/nu-cmd-lang/src/core_commands/while_.rs
View file

@ -48,6 +48,8 @@ impl Command for While {
let eval_expression = get_eval_expression(engine_state);
let eval_block = get_eval_block(engine_state);
let stack = &mut stack.push_redirection(None, None);
loop {
if nu_utils::ctrl_c::was_pressed(&engine_state.ctrlc) {
break;
@ -60,14 +62,7 @@ impl Command for While {
if *val {
let block = engine_state.get_block(block.block_id);
match eval_block(
engine_state,
stack,
block,
PipelineData::empty(),
call.redirect_stdout,
call.redirect_stderr,
) {
match eval_block(engine_state, stack, block, PipelineData::empty()) {
Err(ShellError::Break { .. }) => {
break;
}

7
crates/nu-cmd-lang/src/example_support.rs
View file

@ -112,12 +112,11 @@ pub fn eval_block(
.merge_delta(delta)
.expect("Error merging delta");
let mut stack = Stack::new();
let mut stack = Stack::new().capture();
stack.add_env_var("PWD".to_string(), Value::test_string(cwd.to_string_lossy()));
match nu_engine::eval_block::<WithoutDebug>(engine_state, &mut stack, &block, input, true, true)
{
match nu_engine::eval_block::<WithoutDebug>(engine_state, &mut stack, &block, input) {
Err(err) => panic!("test eval error in `{}`: {:?}", "TODO", err),
Ok(result) => result.into_value(Span::test_data()),
}
@ -128,7 +127,7 @@ pub fn check_example_evaluates_to_expected_output(
cwd: &std::path::Path,
engine_state: &mut Box<EngineState>,
) {
let mut stack = Stack::new();
let mut stack = Stack::new().capture();
// Set up PWD
stack.add_env_var("PWD".to_string(), Value::test_string(cwd.to_string_lossy()));

2
crates/nu-color-config/src/style_computer.rs
View file

@ -78,8 +78,6 @@ impl<'a> StyleComputer<'a> {
&mut stack,
&block,
value.clone().into_pipeline_data(),
false,
false,
) {
Ok(v) => {
let value = v.into_value(span);

5
crates/nu-command/src/debug/explain.rs
View file

@ -73,8 +73,9 @@ pub fn get_pipeline_elements(
let mut i = 0;
while i < pipeline.elements.len() {
let pipeline_element = &pipeline.elements[i];
let pipeline_expression = pipeline_element.expression().clone();
let pipeline_span = pipeline_element.span();
let pipeline_expression = &pipeline_element.expr;
let pipeline_span = pipeline_element.expr.span;
let element_str =
String::from_utf8_lossy(engine_state.get_span_contents(pipeline_span));
let value = Value::string(element_str.to_string(), pipeline_span);

2
crates/nu-command/src/debug/profile.rs
View file

@ -133,8 +133,6 @@ confusing the id/parent_id hierarchy. The --expr flag is helpful for investigati
&mut callee_stack,
block,
input,
call.redirect_stdout,
call.redirect_stdout,
);
// TODO: See eval_source()

18
crates/nu-command/src/debug/timeit.rs
View file

@ -55,24 +55,10 @@ impl Command for TimeIt {
if let Some(block_id) = command_to_run.as_block() {
let eval_block = get_eval_block(engine_state);
let block = engine_state.get_block(block_id);
eval_block(
engine_state,
stack,
block,
input,
call.redirect_stdout,
call.redirect_stderr,
)?
eval_block(engine_state, stack, block, input)?
} else {
let eval_expression_with_input = get_eval_expression_with_input(engine_state);
eval_expression_with_input(
engine_state,
stack,
command_to_run,
input,
call.redirect_stdout,
call.redirect_stderr,
)
eval_expression_with_input(engine_state, stack, command_to_run, input)
.map(|res| res.0)?
}
} else {

8
crates/nu-command/src/env/config/utils.rs vendored
View file

@ -1,7 +1,7 @@
use std::collections::HashMap;
use std::path::PathBuf;
use nu_protocol::{Span, Spanned};
use nu_protocol::{IoStream, Span, Spanned};
use crate::ExternalCommand;
@ -32,10 +32,8 @@ pub(crate) fn gen_command(
name,
args,
arg_keep_raw: vec![false; number_of_args],
redirect_stdout: false,
redirect_stderr: false,
redirect_combine: false,
out: IoStream::Inherit,
err: IoStream::Inherit,
env_vars: env_vars_str,
trim_end_newline: false,
}
}

13
crates/nu-command/src/env/export_env.rs vendored
View file

@ -38,18 +38,13 @@ impl Command for ExportEnv {
) -> Result<PipelineData, ShellError> {
let capture_block: Closure = call.req(engine_state, caller_stack, 0)?;
let block = engine_state.get_block(capture_block.block_id);
let mut callee_stack = caller_stack.captures_to_stack(capture_block.captures);
let mut callee_stack = caller_stack
.captures_to_stack(capture_block.captures)
.reset_pipes();
let eval_block = get_eval_block(engine_state);
let _ = eval_block(
engine_state,
&mut callee_stack,
block,
input,
call.redirect_stdout,
call.redirect_stderr,
);
let _ = eval_block(engine_state, &mut callee_stack, block, input);
redirect_env(engine_state, caller_stack, &callee_stack);

13
crates/nu-command/src/env/source_env.rs vendored
View file

@ -76,18 +76,13 @@ impl Command for SourceEnv {
// Evaluate the block
let block = engine_state.get_block(block_id as usize).clone();
let mut callee_stack = caller_stack.gather_captures(engine_state, &block.captures);
let mut callee_stack = caller_stack
.gather_captures(engine_state, &block.captures)
.reset_pipes();
let eval_block_with_early_return = get_eval_block_with_early_return(engine_state);
let result = eval_block_with_early_return(
engine_state,
&mut callee_stack,
&block,
input,
call.redirect_stdout,
call.redirect_stderr,
);
let result = eval_block_with_early_return(engine_state, &mut callee_stack, &block, input);
// Merge the block's environment to the current stack
redirect_env(engine_state, caller_stack, &callee_stack);

12
crates/nu-command/src/env/with_env.rs vendored
View file

@ -81,12 +81,11 @@ fn with_env(
call: &Call,
input: PipelineData,
) -> Result<PipelineData, ShellError> {
// let external_redirection = args.call_info.args.external_redirection;
let variable: Value = call.req(engine_state, stack, 0)?;
let capture_block: Closure = call.req(engine_state, stack, 1)?;
let block = engine_state.get_block(capture_block.block_id);
let mut stack = stack.captures_to_stack(capture_block.captures);
let mut stack = stack.captures_to_stack_preserve_stdio(capture_block.captures);
let mut env: HashMap<String, Value> = HashMap::new();
@ -145,14 +144,7 @@ fn with_env(
stack.add_env_var(k, v);
}
eval_block::<WithoutDebug>(
engine_state,
&mut stack,
block,
input,
call.redirect_stdout,
call.redirect_stderr,
)
eval_block::<WithoutDebug>(engine_state, &mut stack, block, input)
}
#[cfg(test)]

2
crates/nu-command/src/filesystem/open.rs
View file

@ -194,7 +194,7 @@ impl Command for Open {
let decl = engine_state.get_decl(converter_id);
let command_output = if let Some(block_id) = decl.get_block_id() {
let block = engine_state.get_block(block_id);
eval_block(engine_state, stack, block, file_contents, false, false)
eval_block(engine_state, stack, block, file_contents)
} else {
decl.run(engine_state, stack, &Call::new(call_span), file_contents)
};

65
crates/nu-command/src/filesystem/save.rs
View file

@ -5,7 +5,7 @@ use nu_protocol::ast::{Call, Expr, Expression};
use nu_protocol::engine::{Command, EngineState, Stack};
use nu_protocol::IntoSpanned;
use nu_protocol::{
Category, DataSource, Example, PipelineData, PipelineMetadata, RawStream, ShellError,
Category, DataSource, Example, IoStream, PipelineData, PipelineMetadata, RawStream, ShellError,
Signature, Span, Spanned, SyntaxShape, Type, Value,
};
use std::fs::File;
@ -104,16 +104,7 @@ impl Command for Save {
});
match input {
PipelineData::ExternalStream { stdout: None, .. } => {
// Open files to possibly truncate them
let _ = get_files(&path, stderr_path.as_ref(), append, false, false, force)?;
Ok(PipelineData::empty())
}
PipelineData::ExternalStream {
stdout: Some(stream),
stderr,
..
} => {
PipelineData::ExternalStream { stdout, stderr, .. } => {
let (file, stderr_file) = get_files(
&path,
stderr_path.as_ref(),
@ -123,35 +114,42 @@ impl Command for Save {
force,
)?;
match (stdout, stderr) {
(Some(stdout), stderr) => {
// delegate a thread to redirect stderr to result.
let handler = stderr
.map(|stderr_stream| match stderr_file {
.map(|stderr| match stderr_file {
Some(stderr_file) => thread::Builder::new()
.name("stderr redirector".to_string())
.spawn(move || {
stream_to_file(stderr_stream, stderr_file, span, progress)
stream_to_file(stderr, stderr_file, span, progress)
}),
None => thread::Builder::new()
.name("stderr redirector".to_string())
.spawn(move || {
let _ = stderr_stream.into_bytes();
Ok(PipelineData::empty())
}),
.spawn(move || stderr.drain()),
})
.transpose()
.map_err(|e| e.into_spanned(span))?;
let res = stream_to_file(stream, file, span, progress);
let res = stream_to_file(stdout, file, span, progress);
if let Some(h) = handler {
h.join().map_err(|err| ShellError::ExternalCommand {
label: "Fail to receive external commands stderr message".to_string(),
label: "Fail to receive external commands stderr message"
.to_string(),
help: format!("{err:?}"),
span,
})??;
res
} else {
res
}
res?;
}
(None, Some(stderr)) => match stderr_file {
Some(stderr_file) => stream_to_file(stderr, stderr_file, span, progress)?,
None => stderr.drain()?,
},
(None, None) => {}
};
Ok(PipelineData::Empty)
}
PipelineData::ListStream(ls, pipeline_metadata)
if raw || prepare_path(&path, append, force)?.0.extension().is_none() =>
@ -265,6 +263,10 @@ impl Command for Save {
},
]
}
fn stdio_redirect(&self) -> (Option<IoStream>, Option<IoStream>) {
(Some(IoStream::Capture), Some(IoStream::Capture))
}
}
/// Convert [`PipelineData`] bytes to write in file, possibly converting
@ -430,13 +432,13 @@ fn get_files(
fn stream_to_file(
mut stream: RawStream,
file: File,
mut file: File,
span: Span,
progress: bool,
) -> Result<PipelineData, ShellError> {
) -> Result<(), ShellError> {
// https://github.com/nushell/nushell/pull/9377 contains the reason
// for not using BufWriter<File>
let mut writer = file;
let writer = &mut file;
let mut bytes_processed: u64 = 0;
let bytes_processed_p = &mut bytes_processed;
@ -456,8 +458,7 @@ fn stream_to_file(
(None, None)
};
let result = stream
.try_for_each(move |result| {
stream.try_for_each(move |result| {
let buf = match result {
Ok(v) => match v {
Value::String { val, .. } => val.into_bytes(),
@ -488,15 +489,14 @@ fn stream_to_file(
}
}
if let Err(err) = writer.write(&buf) {
if let Err(err) = writer.write_all(&buf) {
*process_failed_p = true;
return Err(ShellError::IOError {
msg: err.to_string(),
});
}
Ok(())
})
.map(|_| PipelineData::empty());
})?;
// If the `progress` flag is set then
if progress {
@ -508,6 +508,7 @@ fn stream_to_file(
}
}
// And finally return the stream result.
result
file.flush()?;
Ok(())
}

2
crates/nu-command/src/filesystem/watch.rs
View file

@ -217,8 +217,6 @@ impl Command for Watch {
stack,
&block,
Value::nothing(call.span()).into_pipeline_data(),
call.redirect_stdout,
call.redirect_stderr,
);
match eval_result {

10
crates/nu-command/src/filters/each.rs
View file

@ -129,8 +129,6 @@ with 'transpose' first."#
let orig_env_vars = stack.env_vars.clone();
let orig_env_hidden = stack.env_hidden.clone();
let span = call.head;
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
match input {
PipelineData::Empty => Ok(PipelineData::Empty),
@ -157,10 +155,6 @@ with 'transpose' first."#
&mut stack,
&block,
x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
// WithoutDebug,
// &None,
) {
Ok(v) => Some(v.into_value(span)),
Err(ShellError::Continue { span }) => Some(Value::nothing(span)),
@ -205,8 +199,6 @@ with 'transpose' first."#
&mut stack,
&block,
x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) {
Ok(v) => Some(v.into_value(span)),
Err(ShellError::Continue { span }) => Some(Value::nothing(span)),
@ -232,8 +224,6 @@ with 'transpose' first."#
&mut stack,
&block,
x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
)
}
}

8
crates/nu-command/src/filters/filter.rs
View file

@ -62,8 +62,6 @@ a variable. On the other hand, the "row condition" syntax is not supported."#
let orig_env_vars = stack.env_vars.clone();
let orig_env_hidden = stack.env_hidden.clone();
let span = call.head;
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let eval_block = get_eval_block(&engine_state);
match input {
@ -92,8 +90,6 @@ a variable. On the other hand, the "row condition" syntax is not supported."#
&block,
// clone() is used here because x is given to Ok() below.
x.clone().into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) {
Ok(v) => {
if v.into_value(span).is_true() {
@ -136,8 +132,6 @@ a variable. On the other hand, the "row condition" syntax is not supported."#
&block,
// clone() is used here because x is given to Ok() below.
x.clone().into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) {
Ok(v) => {
if v.into_value(span).is_true() {
@ -171,8 +165,6 @@ a variable. On the other hand, the "row condition" syntax is not supported."#
&block,
// clone() is used here because x is given to Ok() below.
x.clone().into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) {
Ok(v) => {
if v.into_value(span).is_true() {

5
crates/nu-command/src/filters/group_by.rs
View file

@ -171,7 +171,7 @@ pub fn group_by(
Value::CellPath { val, .. } => group_cell_path(val, values)?,
Value::Block { .. } | Value::Closure { .. } => {
let block: Option<Closure> = call.opt(engine_state, stack, 0)?;
group_closure(values, span, block, stack, engine_state, call)?
group_closure(values, span, block, stack, engine_state)?
}
_ => {
@ -234,7 +234,6 @@ fn group_closure(
block: Option<Closure>,
stack: &mut Stack,
engine_state: &EngineState,
call: &Call,
) -> Result<IndexMap<String, Vec<Value>>, ShellError> {
let error_key = "error";
let mut groups: IndexMap<String, Vec<Value>> = IndexMap::new();
@ -251,8 +250,6 @@ fn group_closure(
&mut stack,
block,
value.clone().into_pipeline_data(),
call.redirect_stdout,
call.redirect_stderr,
);
let group_key = match pipeline {

28
crates/nu-command/src/filters/insert.rs
View file

@ -130,9 +130,6 @@ fn insert(
let cell_path: CellPath = call.req(engine_state, stack, 0)?;
let replacement: Value = call.req(engine_state, stack, 1)?;
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let ctrlc = engine_state.ctrlc.clone();
let eval_block = get_eval_block(engine_state);
@ -153,8 +150,6 @@ fn insert(
span,
engine_state,
&mut stack,
redirect_stdout,
redirect_stderr,
block,
&cell_path.members,
false,
@ -168,8 +163,6 @@ fn insert(
replacement,
engine_state,
stack,
redirect_stdout,
redirect_stderr,
&cell_path.members,
matches!(first, Some(PathMember::Int { .. })),
eval_block,
@ -225,8 +218,6 @@ fn insert(
&mut stack,
block,
value.clone().into_pipeline_data(),
redirect_stdout,
redirect_stderr,
)?;
pre_elems.push(output.into_value(span));
@ -243,8 +234,6 @@ fn insert(
replacement,
engine_state,
stack,
redirect_stdout,
redirect_stderr,
path,
true,
eval_block,
@ -282,8 +271,6 @@ fn insert(
replacement_span,
&engine_state,
&mut stack,
redirect_stdout,
redirect_stderr,
&block,
&cell_path.members,
false,
@ -330,8 +317,6 @@ fn insert_value_by_closure(
span: Span,
engine_state: &EngineState,
stack: &mut Stack,
redirect_stdout: bool,
redirect_stderr: bool,
block: &Block,
cell_path: &[PathMember],
first_path_member_int: bool,
@ -356,14 +341,7 @@ fn insert_value_by_closure(
.map(IntoPipelineData::into_pipeline_data)
.unwrap_or(PipelineData::Empty);
let output = eval_block_fn(
engine_state,
stack,
block,
input_at_path,
redirect_stdout,
redirect_stderr,
)?;
let output = eval_block_fn(engine_state, stack, block, input_at_path)?;
value.insert_data_at_cell_path(cell_path, output.into_value(span), span)
}
@ -374,8 +352,6 @@ fn insert_single_value_by_closure(
replacement: Value,
engine_state: &EngineState,
stack: &mut Stack,
redirect_stdout: bool,
redirect_stderr: bool,
cell_path: &[PathMember],
first_path_member_int: bool,
eval_block_fn: EvalBlockFn,
@ -390,8 +366,6 @@ fn insert_single_value_by_closure(
span,
engine_state,
&mut stack,
redirect_stdout,
redirect_stderr,
block,
cell_path,
first_path_member_int,

9
crates/nu-command/src/filters/interleave.rs
View file

@ -128,14 +128,7 @@ interleave
// Evaluate the closure on this thread
let block = engine_state.get_block(closure.block_id);
let mut stack = stack.captures_to_stack(closure.captures);
eval_block_with_early_return(
engine_state,
&mut stack,
block,
PipelineData::Empty,
true,
false,
)
eval_block_with_early_return(engine_state, &mut stack, block, PipelineData::Empty)
}))
.try_for_each(|stream| {
stream.and_then(|stream| {

3
crates/nu-command/src/filters/items.rs
View file

@ -54,7 +54,6 @@ impl Command for Items {
let orig_env_vars = stack.env_vars.clone();
let orig_env_hidden = stack.env_hidden.clone();
let span = call.head;
let redirect_stderr = call.redirect_stderr;
let eval_block_with_early_return = get_eval_block_with_early_return(&engine_state);
let input_span = input.span().unwrap_or(call.head);
@ -81,8 +80,6 @@ impl Command for Items {
&mut stack,
&block,
PipelineData::empty(),
true,
redirect_stderr,
) {
Ok(v) => Some(v.into_value(span)),
Err(ShellError::Break { .. }) => None,

12
crates/nu-command/src/filters/par_each.rs
View file

@ -129,8 +129,6 @@ impl Command for ParEach {
let block_id = capture_block.block_id;
let mut stack = stack.captures_to_stack(capture_block.captures);
let span = call.head;
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
// A helper function sorts the output if needed
let apply_order = |mut vec: Vec<(usize, Value)>| {
@ -173,8 +171,6 @@ impl Command for ParEach {
&mut stack,
block,
x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) {
Ok(v) => v.into_value(span),
Err(error) => Value::error(
@ -213,8 +209,6 @@ impl Command for ParEach {
&mut stack,
block,
x.clone().into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) {
Ok(v) => v.into_value(span),
Err(error) => Value::error(
@ -252,8 +246,6 @@ impl Command for ParEach {
&mut stack,
block,
x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) {
Ok(v) => v.into_value(span),
Err(error) => Value::error(
@ -297,8 +289,6 @@ impl Command for ParEach {
&mut stack,
block,
x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) {
Ok(v) => v.into_value(span),
Err(error) => Value::error(error, span),
@ -326,8 +316,6 @@ impl Command for ParEach {
&mut stack,
block,
x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
)
}
}

10
crates/nu-command/src/filters/reduce.rs
View file

@ -107,9 +107,6 @@ impl Command for Reduce {
let orig_env_vars = stack.env_vars.clone();
let orig_env_hidden = stack.env_hidden.clone();
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
// To enumerate over the input (for the index argument),
// it must be converted into an iterator using into_iter().
let mut input_iter = input.into_iter();
@ -130,9 +127,7 @@ impl Command for Reduce {
let mut acc = start_val;
let mut input_iter = input_iter.peekable();
while let Some(x) = input_iter.next() {
for x in input_iter {
// with_env() is used here to ensure that each iteration uses
// a different set of environment variables.
// Hence, a 'cd' in the first loop won't affect the next loop.
@ -157,9 +152,6 @@ impl Command for Reduce {
&mut stack,
block,
PipelineData::empty(),
// redirect stdout until its the last input value
redirect_stdout || input_iter.peek().is_some(),
redirect_stderr,
)?
.into_value(span);

4
crates/nu-command/src/filters/rename.rs
View file

@ -140,8 +140,6 @@ fn rename(
}
None => None,
};
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let block_info =
if let Some(capture_block) = call.get_flag::<Closure>(engine_state, stack, "block")? {
let engine_state = engine_state.clone();
@ -185,8 +183,6 @@ fn rename(
&mut stack,
&block,
Value::string(c.clone(), span).into_pipeline_data(),
redirect_stdout,
redirect_stderr,
);
match eval_result {

12
crates/nu-command/src/filters/skip/skip_until.rs
View file

@ -92,9 +92,6 @@ impl Command for SkipUntil {
let ctrlc = engine_state.ctrlc.clone();
let engine_state = engine_state.clone();
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let eval_block = get_eval_block(&engine_state);
Ok(input
@ -104,14 +101,7 @@ impl Command for SkipUntil {
stack.add_var(var_id, value.clone());
}
!eval_block(
&engine_state,
&mut stack,
&block,
PipelineData::empty(),
redirect_stdout,
redirect_stderr,
)
!eval_block(&engine_state, &mut stack, &block, PipelineData::empty())
.map_or(false, |pipeline_data| {
pipeline_data.into_value(span).is_true()
})

12
crates/nu-command/src/filters/skip/skip_while.rs
View file

@ -97,9 +97,6 @@ impl Command for SkipWhile {
let ctrlc = engine_state.ctrlc.clone();
let engine_state = engine_state.clone();
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let eval_block = get_eval_block(&engine_state);
Ok(input
@ -109,14 +106,7 @@ impl Command for SkipWhile {
stack.add_var(var_id, value.clone());
}
eval_block(
&engine_state,
&mut stack,
&block,
PipelineData::empty(),
redirect_stdout,
redirect_stderr,
)
eval_block(&engine_state, &mut stack, &block, PipelineData::empty())
.map_or(false, |pipeline_data| {
pipeline_data.into_value(span).is_true()
})

12
crates/nu-command/src/filters/take/take_until.rs
View file

@ -89,9 +89,6 @@ impl Command for TakeUntil {
let ctrlc = engine_state.ctrlc.clone();
let engine_state = engine_state.clone();
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let eval_block = get_eval_block(&engine_state);
Ok(input
@ -101,14 +98,7 @@ impl Command for TakeUntil {
stack.add_var(var_id, value.clone());
}
!eval_block(
&engine_state,
&mut stack,
&block,
PipelineData::empty(),
redirect_stdout,
redirect_stderr,
)
!eval_block(&engine_state, &mut stack, &block, PipelineData::empty())
.map_or(false, |pipeline_data| {
pipeline_data.into_value(span).is_true()
})

12
crates/nu-command/src/filters/take/take_while.rs
View file

@ -88,9 +88,6 @@ impl Command for TakeWhile {
let ctrlc = engine_state.ctrlc.clone();
let engine_state = engine_state.clone();
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let eval_block = get_eval_block(&engine_state);
Ok(input
@ -100,14 +97,7 @@ impl Command for TakeWhile {
stack.add_var(var_id, value.clone());
}
eval_block(
&engine_state,
&mut stack,
&block,
PipelineData::empty(),
redirect_stdout,
redirect_stderr,
)
eval_block(&engine_state, &mut stack, &block, PipelineData::empty())
.map_or(false, |pipeline_data| {
pipeline_data.into_value(span).is_true()
})

46
crates/nu-command/src/filters/tee.rs
View file

@ -4,8 +4,8 @@ use nu_engine::{get_eval_block_with_early_return, CallExt};
use nu_protocol::{
ast::Call,
engine::{Closure, Command, EngineState, Stack},
Category, Example, IntoInterruptiblePipelineData, IntoSpanned, PipelineData, RawStream,
ShellError, Signature, Spanned, SyntaxShape, Type, Value,
Category, Example, IntoInterruptiblePipelineData, IntoSpanned, IoStream, PipelineData,
RawStream, ShellError, Signature, Spanned, SyntaxShape, Type, Value,
};
#[derive(Clone)]
@ -49,8 +49,8 @@ use it in your pipeline."#
result: None,
},
Example {
example: "do { nu --commands 'print -e error; print ok' } | \
tee --stderr { save error.log } | complete",
example:
"nu -c 'print -e error; print ok' | tee --stderr { save error.log } | complete",
description: "Save error messages from an external command to a file without \
redirecting them",
result: None,
@ -78,7 +78,9 @@ use it in your pipeline."#
} = call.req(engine_state, stack, 0)?;
let closure_engine_state = engine_state.clone();
let mut closure_stack = stack.captures_to_stack(captures);
let mut closure_stack = stack
.captures_to_stack_preserve_stdio(captures)
.reset_pipes();
let metadata = input.metadata();
let metadata_clone = metadata.clone();
@ -121,46 +123,32 @@ use it in your pipeline."#
&mut closure_stack,
closure_engine_state.get_block(block_id),
input_from_channel,
false,
false,
);
// Make sure to drain any iterator produced to avoid unexpected behavior
result.and_then(|data| data.drain())
};
if use_stderr {
if let Some(stderr) = stderr {
let stderr = stderr
.map(|stderr| {
let iter = tee(stderr.stream, with_stream)
.map_err(|e| e.into_spanned(call.head))?;
let raw_stream = RawStream::new(
Ok::<_, ShellError>(RawStream::new(
Box::new(iter.map(flatten_result)),
stderr.ctrlc,
stderr.span,
stderr.known_size,
);
))
})
.transpose()?;
Ok(PipelineData::ExternalStream {
stdout,
stderr: Some(raw_stream),
stderr,
exit_code,
span,
metadata,
trim_end_newline,
})
} else {
// Throw an error if the stream doesn't have stderr. This is probably the
// user's mistake (e.g., forgetting to use `do`)
Err(ShellError::GenericError {
error: "Stream passed to `tee --stderr` does not have stderr".into(),
msg: "this stream does not contain stderr".into(),
span: Some(span),
help: Some(
"if this is an external command, you probably need to wrap \
it in `do { ... }`"
.into(),
),
inner: vec![],
})
}
} else {
let stdout = stdout
.map(|stdout| {
@ -203,8 +191,6 @@ use it in your pipeline."#
&mut closure_stack,
closure_engine_state.get_block(block_id),
input_from_channel,
false,
false,
);
// Make sure to drain any iterator produced to avoid unexpected behavior
result.and_then(|data| data.drain())
@ -217,6 +203,10 @@ use it in your pipeline."#
}
}
}
fn stdio_redirect(&self) -> (Option<IoStream>, Option<IoStream>) {
(Some(IoStream::Capture), Some(IoStream::Capture))
}
}
fn panic_error() -> ShellError {

19
crates/nu-command/src/filters/update.rs
View file

@ -112,9 +112,6 @@ fn update(
let cell_path: CellPath = call.req(engine_state, stack, 0)?;
let replacement: Value = call.req(engine_state, stack, 1)?;
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let ctrlc = engine_state.ctrlc.clone();
let eval_block = get_eval_block(engine_state);
@ -135,8 +132,6 @@ fn update(
span,
engine_state,
&mut stack,
redirect_stdout,
redirect_stderr,
block,
&cell_path.members,
false,
@ -150,8 +145,6 @@ fn update(
replacement,
engine_state,
stack,
redirect_stdout,
redirect_stderr,
&cell_path.members,
matches!(first, Some(PathMember::Int { .. })),
eval_block,
@ -197,8 +190,6 @@ fn update(
replacement,
engine_state,
stack,
redirect_stdout,
redirect_stderr,
path,
true,
eval_block,
@ -229,8 +220,6 @@ fn update(
replacement_span,
&engine_state,
&mut stack,
redirect_stdout,
redirect_stderr,
&block,
&cell_path.members,
false,
@ -275,8 +264,6 @@ fn update_value_by_closure(
span: Span,
engine_state: &EngineState,
stack: &mut Stack,
redirect_stdout: bool,
redirect_stderr: bool,
block: &Block,
cell_path: &[PathMember],
first_path_member_int: bool,
@ -302,8 +289,6 @@ fn update_value_by_closure(
stack,
block,
input_at_path.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
)?;
value.update_data_at_cell_path(cell_path, output.into_value(span))
@ -315,8 +300,6 @@ fn update_single_value_by_closure(
replacement: Value,
engine_state: &EngineState,
stack: &mut Stack,
redirect_stdout: bool,
redirect_stderr: bool,
cell_path: &[PathMember],
first_path_member_int: bool,
eval_block_fn: EvalBlockFn,
@ -331,8 +314,6 @@ fn update_single_value_by_closure(
span,
engine_state,
&mut stack,
redirect_stdout,
redirect_stderr,
block,
cell_path,
first_path_member_int,

28
crates/nu-command/src/filters/upsert.rs
View file

@ -156,9 +156,6 @@ fn upsert(
let cell_path: CellPath = call.req(engine_state, stack, 0)?;
let replacement: Value = call.req(engine_state, stack, 1)?;
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let eval_block = get_eval_block(engine_state);
let ctrlc = engine_state.ctrlc.clone();
@ -179,8 +176,6 @@ fn upsert(
span,
engine_state,
&mut stack,
redirect_stdout,
redirect_stderr,
block,
&cell_path.members,
false,
@ -194,8 +189,6 @@ fn upsert(
replacement,
engine_state,
stack,
redirect_stdout,
redirect_stderr,
&cell_path.members,
matches!(first, Some(PathMember::Int { .. })),
eval_block,
@ -249,8 +242,6 @@ fn upsert(
&mut stack,
block,
value.clone().into_pipeline_data(),
redirect_stdout,
redirect_stderr,
)?;
pre_elems.push(output.into_value(span));
@ -264,8 +255,6 @@ fn upsert(
replacement,
engine_state,
stack,
redirect_stdout,
redirect_stderr,
path,
true,
eval_block,
@ -303,8 +292,6 @@ fn upsert(
replacement_span,
&engine_state,
&mut stack,
redirect_stdout,
redirect_stderr,
&block,
&cell_path.members,
false,
@ -349,8 +336,6 @@ fn upsert_value_by_closure(
span: Span,
engine_state: &EngineState,
stack: &mut Stack,
redirect_stdout: bool,
redirect_stderr: bool,
block: &Block,
cell_path: &[PathMember],
first_path_member_int: bool,
@ -375,14 +360,7 @@ fn upsert_value_by_closure(
.map(IntoPipelineData::into_pipeline_data)
.unwrap_or(PipelineData::Empty);
let output = eval_block_fn(
engine_state,
stack,
block,
input_at_path,
redirect_stdout,
redirect_stderr,
)?;
let output = eval_block_fn(engine_state, stack, block, input_at_path)?;
value.upsert_data_at_cell_path(cell_path, output.into_value(span))
}
@ -393,8 +371,6 @@ fn upsert_single_value_by_closure(
replacement: Value,
engine_state: &EngineState,
stack: &mut Stack,
redirect_stdout: bool,
redirect_stderr: bool,
cell_path: &[PathMember],
first_path_member_int: bool,
eval_block_fn: EvalBlockFn,
@ -409,8 +385,6 @@ fn upsert_single_value_by_closure(
span,
engine_state,
&mut stack,
redirect_stdout,
redirect_stderr,
block,
cell_path,
first_path_member_int,

9
crates/nu-command/src/filters/utils.rs
View file

@ -53,14 +53,7 @@ pub fn boolean_fold(
stack.add_var(var_id, value.clone());
}
let eval = eval_block(
engine_state,
&mut stack,
block,
value.into_pipeline_data(),
call.redirect_stdout,
call.redirect_stderr,
);
let eval = eval_block(engine_state, &mut stack, block, value.into_pipeline_data());
match eval {
Err(e) => {
return Err(e);

5
crates/nu-command/src/filters/where_.rs
View file

@ -69,9 +69,6 @@ not supported."#
let ctrlc = engine_state.ctrlc.clone();
let engine_state = engine_state.clone();
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let eval_block = get_eval_block(&engine_state);
Ok(input
@ -91,8 +88,6 @@ not supported."#
&block,
// clone() is used here because x is given to Ok() below.
value.clone().into_pipeline_data(),
redirect_stdout,
redirect_stderr,
);
match result {

9
crates/nu-command/src/filters/zip.rs
View file

@ -109,14 +109,7 @@ impl Command for Zip {
Value::Closure { val, .. } => {
let block = engine_state.get_block(val.block_id);
let mut stack = stack.captures_to_stack(val.captures);
eval_block_with_early_return(
engine_state,
&mut stack,
block,
PipelineData::Empty,
true,
false,
)?
eval_block_with_early_return(engine_state, &mut stack, block, PipelineData::Empty)?
}
// If any other value, use it as-is.
val => val.into_pipeline_data(),

23
crates/nu-command/src/formats/from/nuon.rs
View file

@ -1,4 +1,4 @@
use nu_protocol::ast::{Call, Expr, Expression, PipelineElement, RecordItem};
use nu_protocol::ast::{Call, Expr, Expression, RecordItem};
use nu_protocol::engine::{Command, EngineState, Stack, StateWorkingSet};
use nu_protocol::{
record, Category, Example, IntoPipelineData, PipelineData, Range, Record, ShellError,
@ -69,7 +69,7 @@ impl Command for FromNuon {
src: string_input,
error: "error when loading".into(),
msg: "excess values when loading".into(),
span: element.span(),
span: element.expr.span,
}],
});
} else {
@ -109,7 +109,7 @@ impl Command for FromNuon {
src: string_input,
error: "error when loading".into(),
msg: "detected a pipeline in nuon file".into(),
span: expr.span(),
span: expr.expr.span,
}],
});
}
@ -122,22 +122,7 @@ impl Command for FromNuon {
ty: Type::Nothing,
}
} else {
match pipeline.elements.remove(0) {
PipelineElement::Expression(_, expression)
| PipelineElement::ErrPipedExpression(_, expression)
| PipelineElement::OutErrPipedExpression(_, expression)
| PipelineElement::Redirection(_, _, expression, _)
| PipelineElement::And(_, expression)
| PipelineElement::Or(_, expression)
| PipelineElement::SameTargetRedirection {
cmd: (_, expression),
..
}
| PipelineElement::SeparateRedirection {
out: (_, expression, _),
..
} => expression,
}
pipeline.elements.remove(0).expr
}
};

4
crates/nu-command/src/generators/generate.rs
View file

@ -106,8 +106,6 @@ used as the next argument to the closure, otherwise generation stops.
let mut stack = stack.captures_to_stack(capture_block.item.captures);
let orig_env_vars = stack.env_vars.clone();
let orig_env_hidden = stack.env_hidden.clone();
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let eval_block_with_early_return = get_eval_block_with_early_return(&engine_state);
// A type of Option<S> is used to represent state. Invocation
@ -135,8 +133,6 @@ used as the next argument to the closure, otherwise generation stops.
&mut stack,
&block,
arg.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) {
// no data -> output nothing and stop.
Ok(PipelineData::Empty) => (None, None),

9
crates/nu-command/src/misc/source.rs
View file

@ -51,14 +51,7 @@ impl Command for Source {
let eval_block_with_early_return = get_eval_block_with_early_return(engine_state);
eval_block_with_early_return(
engine_state,
stack,
&block,
input,
call.redirect_stdout,
call.redirect_stderr,
)
eval_block_with_early_return(engine_state, stack, &block, input)
}
fn examples(&self) -> Vec<Example> {

22
crates/nu-command/src/system/complete.rs
View file

@ -1,8 +1,8 @@
use nu_protocol::{
ast::Call,
engine::{Command, EngineState, Stack},
Category, Example, IntoPipelineData, IntoSpanned, PipelineData, Record, ShellError, Signature,
Type, Value,
Category, Example, IntoPipelineData, IntoSpanned, IoStream, PipelineData, Record, ShellError,
Signature, Type, Value,
};
use std::thread;
@ -115,19 +115,15 @@ impl Command for Complete {
}
fn examples(&self) -> Vec<Example> {
vec![
Example {
vec![Example {
description:
"Run the external command to completion, capturing stdout and exit_code",
"Run the external command to completion, capturing stdout, stderr, and exit_code",
example: "^external arg1 | complete",
result: None,
},
Example {
description:
"Run external command to completion, capturing, stdout, stderr and exit_code",
example: "do { ^external arg1 } | complete",
result: None,
},
]
}]
}
fn stdio_redirect(&self) -> (Option<IoStream>, Option<IoStream>) {
(Some(IoStream::Capture), Some(IoStream::Capture))
}
}

7
crates/nu-command/src/system/exec.rs
View file

@ -3,7 +3,7 @@ use nu_engine::current_dir;
use nu_protocol::{
ast::Call,
engine::{Command, EngineState, Stack},
Category, Example, PipelineData, ShellError, Signature, Span, SyntaxShape, Type,
Category, Example, IoStream, PipelineData, ShellError, Signature, Span, SyntaxShape, Type,
};
#[derive(Clone)]
@ -62,8 +62,9 @@ fn exec(
stack: &mut Stack,
call: &Call,
) -> Result<PipelineData, ShellError> {
let external_command =
create_external_command(engine_state, stack, call, false, false, false, false)?;
let mut external_command = create_external_command(engine_state, stack, call)?;
external_command.out = IoStream::Inherit;
external_command.err = IoStream::Inherit;
let cwd = current_dir(engine_state, stack)?;
let mut command = external_command.spawn_simple_command(&cwd.to_string_lossy())?;

282
crates/nu-command/src/system/run_external.rs
View file

@ -2,14 +2,12 @@ use nu_cmd_base::hook::eval_hook;
use nu_engine::env_to_strings;
use nu_engine::get_eval_expression;
use nu_engine::CallExt;
use nu_protocol::IntoSpanned;
use nu_protocol::NuGlob;
use nu_protocol::{
ast::{Call, Expr},
did_you_mean,
engine::{Command, EngineState, Stack},
Category, Example, ListStream, PipelineData, RawStream, ShellError, Signature, Span, Spanned,
SyntaxShape, Type, Value,
Category, Example, IntoSpanned, IoStream, ListStream, NuGlob, PipelineData, RawStream,
ShellError, Signature, Span, Spanned, SyntaxShape, Type, Value,
};
use nu_system::ForegroundChild;
use nu_utils::IgnoreCaseExt;
@ -19,14 +17,9 @@ use std::collections::HashMap;
use std::io::{BufRead, BufReader, Read, Write};
use std::path::{Path, PathBuf};
use std::process::{Command as CommandSys, Stdio};
use std::sync::atomic::AtomicBool;
use std::sync::mpsc::{self, SyncSender};
use std::sync::Arc;
use std::sync::mpsc;
use std::thread;
const OUTPUT_BUFFER_SIZE: usize = 1024;
const OUTPUT_BUFFERS_IN_FLIGHT: usize = 3;
#[derive(Clone)]
pub struct External;
@ -76,15 +69,61 @@ impl Command for External {
});
}
let command = create_external_command(
if trim_end_newline {
nu_protocol::report_error_new(
engine_state,
stack,
call,
redirect_stdout,
redirect_stderr,
redirect_combine,
trim_end_newline,
)?;
&ShellError::GenericError {
error: "Deprecated flag".into(),
msg: "`--trim-end-newline` is deprecated".into(),
span: Some(call.arguments_span()),
help: Some(
"trailing new lines are now removed by default when collecting into a value"
.into(),
),
inner: vec![],
},
);
}
if redirect_combine {
nu_protocol::report_error_new(
engine_state,
&ShellError::GenericError {
error: "Deprecated flag".into(),
msg: "`--redirect-combine` is deprecated".into(),
span: Some(call.arguments_span()),
help: Some("use the `o+e>|` pipe redirection instead".into()),
inner: vec![],
},
);
} else if redirect_stdout {
nu_protocol::report_error_new(
engine_state,
&ShellError::GenericError {
error: "Deprecated flag".into(),
msg: "`--redirect-stdout` is deprecated".into(),
span: Some(call.arguments_span()),
help: Some(
"`run-external` will now always redirect stdout if there is a pipe `|` afterwards"
.into(),
),
inner: vec![],
},
);
} else if redirect_stderr {
nu_protocol::report_error_new(
engine_state,
&ShellError::GenericError {
error: "Deprecated flag".into(),
msg: "`--redirect-stderr` is deprecated".into(),
span: Some(call.arguments_span()),
help: Some("use the `e>|` stderr pipe redirection instead".into()),
inner: vec![],
},
);
}
let command = create_external_command(engine_state, stack, call)?;
command.run_with_input(engine_state, stack, input, false)
}
@ -98,7 +137,12 @@ impl Command for External {
},
Example {
description: "Redirect stdout from an external command into the pipeline",
example: r#"run-external --redirect-stdout "echo" "-n" "hello" | split chars"#,
example: r#"run-external "echo" "-n" "hello" | split chars"#,
result: None,
},
Example {
description: "Redirect stderr from an external command into the pipeline",
example: r#"run-external "nu" "-c" "print -e hello" e>| split chars"#,
result: None,
},
]
@ -110,10 +154,6 @@ pub fn create_external_command(
engine_state: &EngineState,
stack: &mut Stack,
call: &Call,
redirect_stdout: bool,
redirect_stderr: bool,
redirect_combine: bool,
trim_end_newline: bool,
) -> Result<ExternalCommand, ShellError> {
let name: Spanned<String> = call.req(engine_state, stack, 0)?;
@ -180,11 +220,9 @@ pub fn create_external_command(
name,
args: spanned_args,
arg_keep_raw,
redirect_stdout,
redirect_stderr,
redirect_combine,
out: stack.stdout().clone(),
err: stack.stderr().clone(),
env_vars: env_vars_str,
trim_end_newline,
})
}
@ -193,11 +231,9 @@ pub struct ExternalCommand {
pub name: Spanned<String>,
pub args: Vec<Spanned<String>>,
pub arg_keep_raw: Vec<bool>,
pub redirect_stdout: bool,
pub redirect_stderr: bool,
pub redirect_combine: bool,
pub out: IoStream,
pub err: IoStream,
pub env_vars: HashMap<String, String>,
pub trim_end_newline: bool,
}
impl ExternalCommand {
@ -364,6 +400,7 @@ impl ExternalCommand {
let mut engine_state = engine_state.clone();
if let Some(hook) = engine_state.config.hooks.command_not_found.clone()
{
let stack = &mut stack.start_capture();
if let Ok(PipelineData::Value(Value::String { val, .. }, ..)) =
eval_hook(
&mut engine_state,
@ -412,6 +449,7 @@ impl ExternalCommand {
thread::Builder::new()
.name("external stdin worker".to_string())
.spawn(move || {
let stack = &mut stack.start_capture();
// Attempt to render the input as a table before piping it to the external.
// This is important for pagers like `less`;
// they need to get Nu data rendered for display to users.
@ -421,7 +459,7 @@ impl ExternalCommand {
let input = crate::Table::run(
&crate::Table,
&engine_state,
&mut stack,
stack,
&Call::new(head),
input,
);
@ -447,42 +485,45 @@ impl ExternalCommand {
#[cfg(unix)]
let commandname = self.name.item.clone();
let redirect_stdout = self.redirect_stdout;
let redirect_stderr = self.redirect_stderr;
let redirect_combine = self.redirect_combine;
let span = self.name.span;
let output_ctrlc = ctrlc.clone();
let stderr_ctrlc = ctrlc.clone();
let (stdout_tx, stdout_rx) = mpsc::sync_channel(OUTPUT_BUFFERS_IN_FLIGHT);
let (exit_code_tx, exit_code_rx) = mpsc::channel();
let stdout = child.as_mut().stdout.take();
let stderr = child.as_mut().stderr.take();
let (stdout, stderr) = if let Some(combined) = reader {
(
Some(RawStream::new(
Box::new(ByteLines::new(combined)),
ctrlc.clone(),
head,
None,
)),
None,
)
} else {
let stdout = child.as_mut().stdout.take().map(|out| {
RawStream::new(Box::new(ByteLines::new(out)), ctrlc.clone(), head, None)
});
// If this external is not the last expression, then its output is piped to a channel
// and we create a ListStream that can be consumed
let stderr = child.as_mut().stderr.take().map(|err| {
RawStream::new(Box::new(ByteLines::new(err)), ctrlc.clone(), head, None)
});
// First create a thread to redirect the external's stdout and wait for an exit code.
thread::Builder::new()
.name("stdout redirector + exit code waiter".to_string())
.spawn(move || {
if redirect_stdout {
let stdout = stdout.ok_or_else(|| {
ShellError::ExternalCommand { label: "Error taking stdout from external".to_string(), help: "Redirects need access to stdout of an external command"
.to_string(), span }
})?;
read_and_redirect_message(stdout, stdout_tx, ctrlc)
} else if redirect_combine {
let stdout = reader.ok_or_else(|| {
ShellError::ExternalCommand { label: "Error taking combined stdout and stderr from external".to_string(), help: "Combined redirects need access to reader pipe of an external command"
.to_string(), span }
})?;
read_and_redirect_message(stdout, stdout_tx, ctrlc)
if matches!(self.err, IoStream::Pipe) {
(stderr, stdout)
} else {
(stdout, stderr)
}
};
// Create a thread to wait for an exit code.
thread::Builder::new()
.name("exit code waiter".into())
.spawn(move || {
match child.as_mut().wait() {
Err(err) => Err(ShellError::ExternalCommand { label: "External command exited with error".into(), help: err.to_string(), span }),
Err(err) => Err(ShellError::ExternalCommand {
label: "External command exited with error".into(),
help: err.to_string(),
span
}),
Ok(x) => {
#[cfg(unix)]
{
@ -531,56 +572,18 @@ impl ExternalCommand {
}
}).map_err(|e| e.into_spanned(head))?;
let (stderr_tx, stderr_rx) = mpsc::sync_channel(OUTPUT_BUFFERS_IN_FLIGHT);
if redirect_stderr {
thread::Builder::new()
.name("stderr redirector".to_string())
.spawn(move || {
let stderr = stderr.ok_or_else(|| ShellError::ExternalCommand {
label: "Error taking stderr from external".to_string(),
help: "Redirects need access to stderr of an external command"
.to_string(),
span,
})?;
read_and_redirect_message(stderr, stderr_tx, stderr_ctrlc);
Ok::<(), ShellError>(())
})
.map_err(|e| e.into_spanned(head))?;
}
let stdout_receiver = ChannelReceiver::new(stdout_rx);
let stderr_receiver = ChannelReceiver::new(stderr_rx);
let exit_code_receiver = ValueReceiver::new(exit_code_rx);
Ok(PipelineData::ExternalStream {
stdout: if redirect_stdout || redirect_combine {
Some(RawStream::new(
Box::new(stdout_receiver),
output_ctrlc.clone(),
head,
None,
))
} else {
None
},
stderr: if redirect_stderr {
Some(RawStream::new(
Box::new(stderr_receiver),
output_ctrlc.clone(),
head,
None,
))
} else {
None
},
stdout,
stderr,
exit_code: Some(ListStream::from_stream(
Box::new(exit_code_receiver),
output_ctrlc,
ctrlc.clone(),
)),
span: head,
metadata: None,
trim_end_newline: self.trim_end_newline,
trim_end_newline: true,
})
}
}
@ -621,20 +624,15 @@ impl ExternalCommand {
// If the external is not the last command, its output will get piped
// either as a string or binary
let reader = if self.redirect_combine {
let reader = if matches!(self.out, IoStream::Pipe) && matches!(self.err, IoStream::Pipe) {
let (reader, writer) = os_pipe::pipe()?;
let writer_clone = writer.try_clone()?;
process.stdout(writer);
process.stderr(writer_clone);
Some(reader)
} else {
if self.redirect_stdout {
process.stdout(Stdio::piped());
}
if self.redirect_stderr {
process.stderr(Stdio::piped());
}
process.stdout(Stdio::try_from(&self.out)?);
process.stderr(Stdio::try_from(&self.err)?);
None
};
@ -824,63 +822,27 @@ fn remove_quotes(input: String) -> String {
}
}
// read message from given `reader`, and send out through `sender`.
//
// `ctrlc` is used to control the process, if ctrl-c is pressed, the read and redirect
// process will be breaked.
fn read_and_redirect_message<R>(
reader: R,
sender: SyncSender<Vec<u8>>,
ctrlc: Option<Arc<AtomicBool>>,
) where
R: Read,
{
// read using the BufferReader. It will do so until there is an
// error or there are no more bytes to read
let mut buf_read = BufReader::with_capacity(OUTPUT_BUFFER_SIZE, reader);
while let Ok(bytes) = buf_read.fill_buf() {
if bytes.is_empty() {
break;
}
struct ByteLines<R: Read>(BufReader<R>);
// The Cow generated from the function represents the conversion
// from bytes to String. If no replacements are required, then the
// borrowed value is a proper UTF-8 string. The Owned option represents
// a string where the values had to be replaced, thus marking it as bytes
let bytes = bytes.to_vec();
let length = bytes.len();
buf_read.consume(length);
if nu_utils::ctrl_c::was_pressed(&ctrlc) {
break;
}
match sender.send(bytes) {
Ok(_) => continue,
Err(_) => break,
}
impl<R: Read> ByteLines<R> {
fn new(read: R) -> Self {
Self(BufReader::new(read))
}
}
// Receiver used for the RawStream
// It implements iterator so it can be used as a RawStream
struct ChannelReceiver {
rx: mpsc::Receiver<Vec<u8>>,
}
impl ChannelReceiver {
pub fn new(rx: mpsc::Receiver<Vec<u8>>) -> Self {
Self { rx }
}
}
impl Iterator for ChannelReceiver {
impl<R: Read> Iterator for ByteLines<R> {
type Item = Result<Vec<u8>, ShellError>;
fn next(&mut self) -> Option<Self::Item> {
match self.rx.recv() {
Ok(v) => Some(Ok(v)),
Err(_) => None,
let mut buf = Vec::new();
// `read_until` will never stop reading unless `\n` or EOF is encountered,
// so let's limit the number of bytes using `take` as the Rust docs suggest.
let capacity = self.0.capacity() as u64;
let mut reader = (&mut self.0).take(capacity);
match reader.read_until(b'\n', &mut buf) {
Ok(0) => None,
Ok(_) => Some(Ok(buf)),
Err(e) => Some(Err(e.into())),
}
}
}

11
crates/nu-command/src/viewers/table.rs
View file

@ -9,10 +9,10 @@ use nu_engine::{env::get_config, env_to_string, CallExt};
use nu_protocol::{
ast::Call,
engine::{Command, EngineState, Stack},
Category, Config, DataSource, Example, IntoPipelineData, ListStream, PipelineData,
PipelineMetadata, RawStream, Record, ShellError, Signature, Span, SyntaxShape, Type, Value,
record, Category, Config, DataSource, Example, IntoPipelineData, IoStream, ListStream,
PipelineData, PipelineMetadata, RawStream, Record, ShellError, Signature, Span, SyntaxShape,
TableMode, Type, Value,
};
use nu_protocol::{record, TableMode};
use nu_table::common::create_nu_table_config;
use nu_table::{
CollapsedTable, ExpandedTable, JustTable, NuTable, NuTableCell, StringResult, TableOpts,
@ -370,7 +370,10 @@ fn handle_table_command(
match input.data {
PipelineData::ExternalStream { .. } => Ok(input.data),
PipelineData::Value(Value::Binary { val, .. }, ..) => {
let stream_list = if input.call.redirect_stdout {
let stream_list = if matches!(
input.stack.stdout(),
IoStream::Pipe | IoStream::Capture | IoStream::Null
) {
vec![Ok(val)]
} else {
let hex = format!("{}\n", nu_pretty_hex::pretty_hex(&val))

69
crates/nu-command/tests/commands/complete.rs
View file

@ -23,7 +23,72 @@ fn basic_exit_code() {
#[test]
fn error() {
let actual = nu!("do { not-found } | complete");
let actual = nu!("not-found | complete");
assert!(actual.err.contains("executable was not found"));
}
#[test]
#[cfg(not(windows))]
fn capture_error_with_too_much_stderr_not_hang_nushell() {
use nu_test_support::fs::Stub::FileWithContent;
use nu_test_support::playground::Playground;
Playground::setup("external with many stderr message", |dirs, sandbox| {
let bytes: usize = 81920;
let mut large_file_body = String::with_capacity(bytes);
for _ in 0..bytes {
large_file_body.push('a');
}
sandbox.with_files(vec![FileWithContent("a_large_file.txt", &large_file_body)]);
let actual =
nu!(cwd: dirs.test(), "sh -c 'cat a_large_file.txt 1>&2' | complete | get stderr");
assert_eq!(actual.out, large_file_body);
})
}
#[test]
#[cfg(not(windows))]
fn capture_error_with_too_much_stdout_not_hang_nushell() {
use nu_test_support::fs::Stub::FileWithContent;
use nu_test_support::playground::Playground;
Playground::setup("external with many stdout message", |dirs, sandbox| {
let bytes: usize = 81920;
let mut large_file_body = String::with_capacity(bytes);
for _ in 0..bytes {
large_file_body.push('a');
}
sandbox.with_files(vec![FileWithContent("a_large_file.txt", &large_file_body)]);
let actual = nu!(cwd: dirs.test(), "sh -c 'cat a_large_file.txt' | complete | get stdout");
assert_eq!(actual.out, large_file_body);
})
}
#[test]
#[cfg(not(windows))]
fn capture_error_with_both_stdout_stderr_messages_not_hang_nushell() {
use nu_test_support::fs::Stub::FileWithContent;
use nu_test_support::playground::Playground;
Playground::setup(
"external with many stdout and stderr messages",
|dirs, sandbox| {
let script_body = r#"
x=$(printf '=%.0s' $(seq 40960))
echo $x
echo $x 1>&2
"#;
let expect_body = "=".repeat(40960);
sandbox.with_files(vec![FileWithContent("test.sh", script_body)]);
// check for stdout
let actual = nu!(cwd: dirs.test(), "sh test.sh | complete | get stdout | str trim");
assert_eq!(actual.out, expect_body);
// check for stderr
let actual = nu!(cwd: dirs.test(), "sh test.sh | complete | get stderr | str trim");
assert_eq!(actual.out, expect_body);
},
)
}

85
crates/nu-command/tests/commands/do_.rs
View file

@ -1,6 +1,4 @@
use nu_test_support::nu;
#[cfg(not(windows))]
use nu_test_support::pipeline;
#[test]
fn capture_errors_works() {
@ -63,89 +61,6 @@ fn ignore_error_should_work_for_external_command() {
assert_eq!(actual.out, "post");
}
#[test]
#[cfg(not(windows))]
fn capture_error_with_too_much_stderr_not_hang_nushell() {
use nu_test_support::fs::Stub::FileWithContent;
use nu_test_support::pipeline;
use nu_test_support::playground::Playground;
Playground::setup("external with many stderr message", |dirs, sandbox| {
let bytes: usize = 81920;
let mut large_file_body = String::with_capacity(bytes);
for _ in 0..bytes {
large_file_body.push('a');
}
sandbox.with_files(vec![FileWithContent("a_large_file.txt", &large_file_body)]);
let actual = nu!(
cwd: dirs.test(), pipeline(
r#"
do -c {sh -c "cat a_large_file.txt 1>&2"} | complete | get stderr
"#,
));
assert_eq!(actual.out, large_file_body);
})
}
#[test]
#[cfg(not(windows))]
fn capture_error_with_too_much_stdout_not_hang_nushell() {
use nu_test_support::fs::Stub::FileWithContent;
use nu_test_support::pipeline;
use nu_test_support::playground::Playground;
Playground::setup("external with many stdout message", |dirs, sandbox| {
let bytes: usize = 81920;
let mut large_file_body = String::with_capacity(bytes);
for _ in 0..bytes {
large_file_body.push('a');
}
sandbox.with_files(vec![FileWithContent("a_large_file.txt", &large_file_body)]);
let actual = nu!(
cwd: dirs.test(), pipeline(
r#"
do -c {sh -c "cat a_large_file.txt"} | complete | get stdout
"#,
));
assert_eq!(actual.out, large_file_body);
})
}
#[test]
#[cfg(not(windows))]
fn capture_error_with_both_stdout_stderr_messages_not_hang_nushell() {
use nu_test_support::fs::Stub::FileWithContent;
use nu_test_support::playground::Playground;
Playground::setup(
"external with many stdout and stderr messages",
|dirs, sandbox| {
let script_body = r#"
x=$(printf '=%.0s' $(seq 40960))
echo $x
echo $x 1>&2
"#;
let expect_body = "=".repeat(40960);
sandbox.with_files(vec![FileWithContent("test.sh", script_body)]);
// check for stdout
let actual = nu!(
cwd: dirs.test(), pipeline(
"do -c {sh test.sh} | complete | get stdout | str trim",
));
assert_eq!(actual.out, expect_body);
// check for stderr
let actual = nu!(
cwd: dirs.test(), pipeline(
"do -c {sh test.sh} | complete | get stderr | str trim",
));
assert_eq!(actual.out, expect_body);
},
)
}
#[test]
fn ignore_error_works_with_list_stream() {
let actual = nu!(r#"do -i { ["a", null, "b"] | ansi strip }"#);

8
crates/nu-command/tests/commands/let_.rs
View file

@ -65,9 +65,7 @@ fn let_err_pipeline_redirects_externals() {
let actual = nu!(
r#"let x = with-env [FOO "foo"] {nu --testbin echo_env_stderr FOO e>| str length}; $x"#
);
// have an extra \n, so length is 4.
assert_eq!(actual.out, "4");
assert_eq!(actual.out, "3");
}
#[test]
@ -75,9 +73,7 @@ fn let_outerr_pipeline_redirects_externals() {
let actual = nu!(
r#"let x = with-env [FOO "foo"] {nu --testbin echo_env_stderr FOO o+e>| str length}; $x"#
);
// have an extra \n, so length is 4.
assert_eq!(actual.out, "4");
assert_eq!(actual.out, "3");
}
#[ignore]

14
crates/nu-command/tests/commands/redirection.rs
View file

@ -164,7 +164,7 @@ fn redirection_keep_exit_codes() {
Playground::setup("redirection preserves exit code", |dirs, _| {
let out = nu!(
cwd: dirs.test(),
"do -i { nu --testbin fail e> a.txt } | complete | get exit_code"
"nu --testbin fail e> a.txt | complete | get exit_code"
);
// needs to use contains "1", because it complete will output `Some(RawStream)`.
assert!(out.out.contains('1'));
@ -358,7 +358,7 @@ fn redirection_with_out_pipe() {
r#"$env.BAZ = "message"; nu --testbin echo_env_mixed out-err BAZ BAZ err> tmp_file | str length"#,
);
assert_eq!(actual.out, "8");
assert_eq!(actual.out, "7");
// check for stderr redirection file.
let expected_out_file = dirs.test().join("tmp_file");
let actual_len = file_contents(expected_out_file).len();
@ -376,7 +376,7 @@ fn redirection_with_err_pipe() {
r#"$env.BAZ = "message"; nu --testbin echo_env_mixed out-err BAZ BAZ out> tmp_file e>| str length"#,
);
assert_eq!(actual.out, "8");
assert_eq!(actual.out, "7");
// check for stdout redirection file.
let expected_out_file = dirs.test().join("tmp_file");
let actual_len = file_contents(expected_out_file).len();
@ -392,7 +392,7 @@ fn no_redirection_with_outerr_pipe() {
cwd: dirs.test(),
&format!("echo 3 {redirect_type} a.txt o+e>| str length")
);
assert!(actual.err.contains("not allowed to use with redirection"));
assert!(actual.err.contains("Multiple redirections provided"));
assert!(
!dirs.test().join("a.txt").exists(),
"No file should be created on error"
@ -404,7 +404,7 @@ fn no_redirection_with_outerr_pipe() {
cwd: dirs.test(),
"echo 3 o> a.txt e> b.txt o+e>| str length"
);
assert!(actual.err.contains("not allowed to use with redirection"));
assert!(actual.err.contains("Multiple redirections provided"));
assert!(
!dirs.test().join("a.txt").exists(),
"No file should be created on error"
@ -423,7 +423,7 @@ fn no_duplicate_redirection() {
cwd: dirs.test(),
"echo 3 o> a.txt o> a.txt"
);
assert!(actual.err.contains("Redirection can be set only once"));
assert!(actual.err.contains("Multiple redirections provided"));
assert!(
!dirs.test().join("a.txt").exists(),
"No file should be created on error"
@ -432,7 +432,7 @@ fn no_duplicate_redirection() {
cwd: dirs.test(),
"echo 3 e> a.txt e> a.txt"
);
assert!(actual.err.contains("Redirection can be set only once"));
assert!(actual.err.contains("Multiple redirections provided"));
assert!(
!dirs.test().join("a.txt").exists(),
"No file should be created on error"

2
crates/nu-command/tests/commands/run_external.rs
View file

@ -325,7 +325,7 @@ fn redirect_combine() {
let actual = nu!(
cwd: dirs.test(), pipeline(
r#"
run-external --redirect-combine sh ...[-c 'echo Foo; echo >&2 Bar']
run-external sh ...[-c 'echo Foo; echo >&2 Bar'] o+e>| print
"#
));

6
crates/nu-engine/src/call_ext.rs
View file

@ -70,6 +70,7 @@ impl CallExt for Call {
if flag_name == name.0.item {
return if let Some(expr) = &name.2 {
// Check --flag=false
let stack = &mut stack.use_call_arg_stdio();
let result = eval_expression::<WithoutDebug>(engine_state, stack, expr)?;
match result {
Value::Bool { val, .. } => Ok(val),
@ -96,6 +97,7 @@ impl CallExt for Call {
name: &str,
) -> Result<Option<T>, ShellError> {
if let Some(expr) = self.get_flag_expr(name) {
let stack = &mut stack.use_call_arg_stdio();
let result = eval_expression::<WithoutDebug>(engine_state, stack, expr)?;
FromValue::from_value(result).map(Some)
} else {
@ -109,6 +111,7 @@ impl CallExt for Call {
stack: &mut Stack,
starting_pos: usize,
) -> Result<Vec<T>, ShellError> {
let stack = &mut stack.use_call_arg_stdio();
let mut output = vec![];
for result in self.rest_iter_flattened(starting_pos, |expr| {
@ -127,6 +130,7 @@ impl CallExt for Call {
pos: usize,
) -> Result<Option<T>, ShellError> {
if let Some(expr) = self.positional_nth(pos) {
let stack = &mut stack.use_call_arg_stdio();
let result = eval_expression::<WithoutDebug>(engine_state, stack, expr)?;
FromValue::from_value(result).map(Some)
} else {
@ -154,6 +158,7 @@ impl CallExt for Call {
pos: usize,
) -> Result<T, ShellError> {
if let Some(expr) = self.positional_nth(pos) {
let stack = &mut stack.use_call_arg_stdio();
let result = eval_expression::<WithoutDebug>(engine_state, stack, expr)?;
FromValue::from_value(result)
} else if self.positional_len() == 0 {
@ -173,6 +178,7 @@ impl CallExt for Call {
name: &str,
) -> Result<T, ShellError> {
if let Some(expr) = self.get_parser_info(name) {
let stack = &mut stack.use_call_arg_stdio();
let result = eval_expression::<WithoutDebug>(engine_state, stack, expr)?;
FromValue::from_value(result)
} else if self.parser_info.is_empty() {

15
crates/nu-engine/src/documentation.rs
View file

@ -23,6 +23,9 @@ pub fn get_full_help(
no_color: !config.use_ansi_coloring,
brief: false,
};
let stack = &mut stack.start_capture();
get_documentation(
sig,
examples,
@ -235,16 +238,14 @@ fn get_documentation(
));
}
let mut caller_stack = Stack::new();
let caller_stack = &mut Stack::new().capture();
if let Ok(result) = eval_call::<WithoutDebug>(
engine_state,
&mut caller_stack,
caller_stack,
&Call {
decl_id,
head: span,
arguments: vec![],
redirect_stdout: true,
redirect_stderr: true,
parser_info: HashMap::new(),
},
PipelineData::Value(Value::list(vals, span), None),
@ -340,7 +341,7 @@ fn get_ansi_color_for_component_or_default(
default: &str,
) -> String {
if let Some(color) = &engine_state.get_config().color_config.get(theme_component) {
let mut caller_stack = Stack::new();
let caller_stack = &mut Stack::new().capture();
let span = Span::unknown();
let argument_opt = get_argument_for_color_value(engine_state, color, span);
@ -350,13 +351,11 @@ fn get_ansi_color_for_component_or_default(
if let Some(decl_id) = engine_state.find_decl(b"ansi", &[]) {
if let Ok(result) = eval_call::<WithoutDebug>(
engine_state,
&mut caller_stack,
caller_stack,
&Call {
decl_id,
head: span,
arguments: vec![argument],
redirect_stdout: true,
redirect_stderr: true,
parser_info: HashMap::new(),
},
PipelineData::Empty,

4
crates/nu-engine/src/env.rs
View file

@ -379,7 +379,7 @@ fn get_converted_value(
let block = engine_state.get_block(val.block_id);
if let Some(var) = block.signature.get_positional(0) {
let mut stack = stack.gather_captures(engine_state, &block.captures);
let mut stack = stack.captures_to_stack(val.captures.clone());
if let Some(var_id) = &var.var_id {
stack.add_var(*var_id, orig_val.clone());
}
@ -391,8 +391,6 @@ fn get_converted_value(
&mut stack,
block,
PipelineData::new_with_metadata(None, val_span),
true,
true,
);
match result {

969
crates/nu-engine/src/eval.rs
View file

File diff suppressed because it is too large Load diff

22
crates/nu-engine/src/eval_helpers.rs
View file

@ -8,24 +8,12 @@ use nu_protocol::engine::{EngineState, Stack};
use nu_protocol::{PipelineData, ShellError, Value};
/// Type of eval_block() function
pub type EvalBlockFn = fn(
&EngineState,
&mut Stack,
&Block,
PipelineData,
bool,
bool,
) -> Result<PipelineData, ShellError>;
pub type EvalBlockFn =
fn(&EngineState, &mut Stack, &Block, PipelineData) -> Result<PipelineData, ShellError>;
/// Type of eval_block_with_early_return() function
pub type EvalBlockWithEarlyReturnFn = fn(
&EngineState,
&mut Stack,
&Block,
PipelineData,
bool,
bool,
) -> Result<PipelineData, ShellError>;
pub type EvalBlockWithEarlyReturnFn =
fn(&EngineState, &mut Stack, &Block, PipelineData) -> Result<PipelineData, ShellError>;
/// Type of eval_expression() function
pub type EvalExpressionFn = fn(&EngineState, &mut Stack, &Expression) -> Result<Value, ShellError>;
@ -36,8 +24,6 @@ pub type EvalExpressionWithInputFn = fn(
&mut Stack,
&Expression,
PipelineData,
bool,
bool,
) -> Result<(PipelineData, bool), ShellError>;
/// Type of eval_subexpression() function

10
crates/nu-explore/src/nu_common/command.rs
View file

@ -2,8 +2,8 @@ use nu_engine::eval_block;
use nu_parser::parse;
use nu_protocol::debugger::WithoutDebug;
use nu_protocol::{
engine::{EngineState, Stack, StateWorkingSet},
PipelineData, ShellError, Value,
engine::{EngineState, Redirection, Stack, StateWorkingSet},
IoStream, PipelineData, ShellError, Value,
};
pub fn run_command_with_value(
@ -91,5 +91,9 @@ fn eval_source2(
block.pipelines.drain(..block.pipelines.len() - 1);
}
eval_block::<WithoutDebug>(engine_state, stack, &block, input, true, true)
let stack = &mut stack.push_redirection(
Some(Redirection::Pipe(IoStream::Capture)),
Some(Redirection::Pipe(IoStream::Capture)),
);
eval_block::<WithoutDebug>(engine_state, stack, &block, input)
}

85
crates/nu-parser/src/flatten.rs
View file

@ -1,6 +1,6 @@
use nu_protocol::ast::{
Argument, Block, Expr, Expression, ExternalArgument, ImportPatternMember, MatchPattern,
PathMember, Pattern, Pipeline, PipelineElement, RecordItem,
PathMember, Pattern, Pipeline, PipelineElement, PipelineRedirection, RecordItem,
};
use nu_protocol::{engine::StateWorkingSet, Span};
use nu_protocol::{DeclId, VarId};
@ -223,7 +223,7 @@ pub fn flatten_expression(
output.extend(args);
output
}
Expr::ExternalCall(head, args, _) => {
Expr::ExternalCall(head, args) => {
let mut output = vec![];
match **head {
@ -559,60 +559,43 @@ pub fn flatten_pipeline_element(
working_set: &StateWorkingSet,
pipeline_element: &PipelineElement,
) -> Vec<(Span, FlatShape)> {
match pipeline_element {
PipelineElement::Expression(span, expr)
| PipelineElement::ErrPipedExpression(span, expr)
| PipelineElement::OutErrPipedExpression(span, expr) => {
if let Some(span) = span {
let mut output = vec![(*span, FlatShape::Pipe)];
output.append(&mut flatten_expression(working_set, expr));
let mut output = if let Some(span) = pipeline_element.pipe {
let mut output = vec![(span, FlatShape::Pipe)];
output.extend(flatten_expression(working_set, &pipeline_element.expr));
output
} else {
flatten_expression(working_set, expr)
}
}
PipelineElement::Redirection(span, _, expr, _) => {
let mut output = vec![(*span, FlatShape::Redirection)];
output.append(&mut flatten_expression(working_set, expr));
output
}
PipelineElement::SeparateRedirection {
out: (out_span, out_expr, _),
err: (err_span, err_expr, _),
} => {
let mut output = vec![(*out_span, FlatShape::Redirection)];
output.append(&mut flatten_expression(working_set, out_expr));
output.push((*err_span, FlatShape::Redirection));
output.append(&mut flatten_expression(working_set, err_expr));
output
}
PipelineElement::SameTargetRedirection {
cmd: (cmd_span, cmd_expr),
redirection: (redirect_span, redirect_expr, _),
} => {
let mut output = if let Some(span) = cmd_span {
let mut output = vec![(*span, FlatShape::Pipe)];
output.append(&mut flatten_expression(working_set, cmd_expr));
output
} else {
flatten_expression(working_set, cmd_expr)
flatten_expression(working_set, &pipeline_element.expr)
};
output.push((*redirect_span, FlatShape::Redirection));
output.append(&mut flatten_expression(working_set, redirect_expr));
if let Some(redirection) = pipeline_element.redirection.as_ref() {
match redirection {
PipelineRedirection::Single { target, .. } => {
output.push((target.span(), FlatShape::Redirection));
if let Some(expr) = target.expr() {
output.extend(flatten_expression(working_set, expr));
}
}
PipelineRedirection::Separate { out, err } => {
let (out, err) = if out.span() <= err.span() {
(out, err)
} else {
(err, out)
};
output.push((out.span(), FlatShape::Redirection));
if let Some(expr) = out.expr() {
output.extend(flatten_expression(working_set, expr));
}
output.push((err.span(), FlatShape::Redirection));
if let Some(expr) = err.expr() {
output.extend(flatten_expression(working_set, expr));
}
}
}
}
output
}
PipelineElement::And(span, expr) => {
let mut output = vec![(*span, FlatShape::And)];
output.append(&mut flatten_expression(working_set, expr));
output
}
PipelineElement::Or(span, expr) => {
let mut output = vec![(*span, FlatShape::Or)];
output.append(&mut flatten_expression(working_set, expr));
output
}
}
}
pub fn flatten_pipeline(
working_set: &StateWorkingSet,

25
crates/nu-parser/src/known_external.rs
View file

@ -4,7 +4,7 @@ use nu_protocol::{
engine::Command,
ShellError, Signature,
};
use nu_protocol::{PipelineData, Spanned, Type};
use nu_protocol::{PipelineData, Type};
#[derive(Clone)]
pub struct KnownExternal {
@ -42,7 +42,6 @@ impl Command for KnownExternal {
call: &Call,
input: PipelineData,
) -> Result<PipelineData, ShellError> {
let call_span = call.span();
let head_span = call.head;
let decl_id = engine_state
.find_decl("run-external".as_bytes(), &[])
@ -110,28 +109,6 @@ impl Command for KnownExternal {
}
}
if call.redirect_stdout {
extern_call.add_named((
Spanned {
item: "redirect-stdout".into(),
span: call_span,
},
None,
None,
))
}
if call.redirect_stderr {
extern_call.add_named((
Spanned {
item: "redirect-stderr".into(),
span: call_span,
},
None,
None,
))
}
command.run(engine_state, stack, &extern_call, input)
}
}

2
crates/nu-parser/src/lib.rs
View file

@ -17,7 +17,7 @@ pub use flatten::{
};
pub use known_external::KnownExternal;
pub use lex::{lex, lex_signature, Token, TokenContents};
pub use lite_parser::{lite_parse, LiteBlock, LiteElement};
pub use lite_parser::{lite_parse, LiteBlock, LiteCommand};
pub use parse_keywords::*;
pub use parser_path::*;

630
crates/nu-parser/src/lite_parser.rs
View file

@ -1,212 +1,135 @@
/// Lite parsing converts a flat stream of tokens from the lexer to a syntax element structure that
/// can be parsed.
//! Lite parsing converts a flat stream of tokens from the lexer to a syntax element structure that
//! can be parsed.
use std::mem;
use crate::{Token, TokenContents};
use nu_protocol::{ast::Redirection, ParseError, Span};
use nu_protocol::{ast::RedirectionSource, ParseError, Span};
#[derive(Debug)]
#[derive(Debug, Clone, Copy)]
pub enum LiteRedirectionTarget {
File {
connector: Span,
file: Span,
append: bool,
},
Pipe {
connector: Span,
},
}
impl LiteRedirectionTarget {
pub fn connector(&self) -> Span {
match self {
LiteRedirectionTarget::File { connector, .. }
| LiteRedirectionTarget::Pipe { connector } => *connector,
}
}
}
#[derive(Debug, Clone)]
pub enum LiteRedirection {
Single {
source: RedirectionSource,
target: LiteRedirectionTarget,
},
Separate {
out: LiteRedirectionTarget,
err: LiteRedirectionTarget,
},
}
#[derive(Debug, Clone, Default)]
pub struct LiteCommand {
pub pipe: Option<Span>,
pub comments: Vec<Span>,
pub parts: Vec<Span>,
}
impl Default for LiteCommand {
fn default() -> Self {
Self::new()
}
pub redirection: Option<LiteRedirection>,
}
impl LiteCommand {
pub fn new() -> Self {
Self {
comments: vec![],
parts: vec![],
}
}
pub fn push(&mut self, span: Span) {
fn push(&mut self, span: Span) {
self.parts.push(span);
}
pub fn is_empty(&self) -> bool {
self.parts.is_empty()
fn try_add_redirection(
&mut self,
source: RedirectionSource,
target: LiteRedirectionTarget,
) -> Result<(), ParseError> {
let redirection = match (self.redirection.take(), source) {
(None, source) => Ok(LiteRedirection::Single { source, target }),
(
Some(LiteRedirection::Single {
source: RedirectionSource::Stdout,
target: out,
}),
RedirectionSource::Stderr,
) => Ok(LiteRedirection::Separate { out, err: target }),
(
Some(LiteRedirection::Single {
source: RedirectionSource::Stderr,
target: err,
}),
RedirectionSource::Stdout,
) => Ok(LiteRedirection::Separate { out: target, err }),
(
Some(LiteRedirection::Single {
source,
target: first,
}),
_,
) => Err(ParseError::MultipleRedirections(
source,
first.connector(),
target.connector(),
)),
(
Some(LiteRedirection::Separate { out, .. }),
RedirectionSource::Stdout | RedirectionSource::StdoutAndStderr,
) => Err(ParseError::MultipleRedirections(
RedirectionSource::Stdout,
out.connector(),
target.connector(),
)),
(Some(LiteRedirection::Separate { err, .. }), RedirectionSource::Stderr) => {
Err(ParseError::MultipleRedirections(
RedirectionSource::Stderr,
err.connector(),
target.connector(),
))
}
}?;
self.redirection = Some(redirection);
Ok(())
}
}
// Note: the Span is the span of the connector not the whole element
#[derive(Debug)]
pub enum LiteElement {
Command(Option<Span>, LiteCommand),
// Similar to LiteElement::Command, except the previous command's output is stderr piped.
// e.g: `e>| cmd`
ErrPipedCommand(Option<Span>, LiteCommand),
// Similar to LiteElement::Command, except the previous command's output is stderr + stdout piped.
// e.g: `o+e>| cmd`
OutErrPipedCommand(Option<Span>, LiteCommand),
// final field indicates if it's in append mode
Redirection(Span, Redirection, LiteCommand, bool),
// SeparateRedirection variant can only be generated by two different Redirection variant
// final bool field indicates if it's in append mode
SeparateRedirection {
out: (Span, LiteCommand, bool),
err: (Span, LiteCommand, bool),
},
// SameTargetRedirection variant can only be generated by Command with Redirection::OutAndErr
// redirection's final bool field indicates if it's in append mode
SameTargetRedirection {
cmd: (Option<Span>, LiteCommand),
redirection: (Span, LiteCommand, bool),
},
}
#[derive(Debug, Default)]
#[derive(Debug, Clone, Default)]
pub struct LitePipeline {
pub commands: Vec<LiteElement>,
pub commands: Vec<LiteCommand>,
}
impl LitePipeline {
pub fn new() -> Self {
Self { commands: vec![] }
fn push(&mut self, element: &mut LiteCommand) {
if !element.parts.is_empty() || element.redirection.is_some() {
self.commands.push(mem::take(element));
}
pub fn push(&mut self, element: LiteElement) {
self.commands.push(element);
}
pub fn insert(&mut self, index: usize, element: LiteElement) {
self.commands.insert(index, element);
}
pub fn is_empty(&self) -> bool {
self.commands.is_empty()
}
pub fn exists(&self, new_target: &Redirection) -> bool {
for cmd in &self.commands {
if let LiteElement::Redirection(_, exists_target, _, _) = cmd {
if exists_target == new_target {
return true;
}
}
}
false
}
}
#[derive(Debug)]
#[derive(Debug, Clone, Default)]
pub struct LiteBlock {
pub block: Vec<LitePipeline>,
}
impl Default for LiteBlock {
fn default() -> Self {
Self::new()
}
}
impl LiteBlock {
pub fn new() -> Self {
Self { block: vec![] }
}
pub fn push(&mut self, mut pipeline: LitePipeline) {
// once we push `pipeline` to our block
// the block takes ownership of `pipeline`, which means that
// our `pipeline` is complete on collecting commands.
self.merge_redirections(&mut pipeline);
self.merge_cmd_with_outerr_redirection(&mut pipeline);
self.block.push(pipeline);
}
pub fn is_empty(&self) -> bool {
self.block.is_empty()
}
fn merge_cmd_with_outerr_redirection(&self, pipeline: &mut LitePipeline) {
let mut cmd_index = None;
let mut outerr_index = None;
for (index, cmd) in pipeline.commands.iter().enumerate() {
if let LiteElement::Command(..) = cmd {
cmd_index = Some(index);
}
if let LiteElement::Redirection(
_span,
Redirection::StdoutAndStderr,
_target_cmd,
_is_append_mode,
) = cmd
{
outerr_index = Some(index);
break;
}
}
if let (Some(cmd_index), Some(outerr_index)) = (cmd_index, outerr_index) {
// we can make sure that cmd_index is less than outerr_index.
let outerr_redirect = pipeline.commands.remove(outerr_index);
let cmd = pipeline.commands.remove(cmd_index);
// `outerr_redirect` and `cmd` should always be `LiteElement::Command` and `LiteElement::Redirection`
if let (
LiteElement::Command(cmd_span, lite_cmd),
LiteElement::Redirection(span, _, outerr_cmd, is_append_mode),
) = (cmd, outerr_redirect)
{
pipeline.insert(
cmd_index,
LiteElement::SameTargetRedirection {
cmd: (cmd_span, lite_cmd),
redirection: (span, outerr_cmd, is_append_mode),
},
)
}
}
}
fn merge_redirections(&self, pipeline: &mut LitePipeline) {
// In case our command may contains both stdout and stderr redirection.
// We pick them out and Combine them into one LiteElement::SeparateRedirection variant.
let mut stdout_index = None;
let mut stderr_index = None;
for (index, cmd) in pipeline.commands.iter().enumerate() {
if let LiteElement::Redirection(_span, redirection, _target_cmd, _is_append_mode) = cmd
{
match *redirection {
Redirection::Stderr => stderr_index = Some(index),
Redirection::Stdout => stdout_index = Some(index),
Redirection::StdoutAndStderr => {}
}
}
}
if let (Some(out_indx), Some(err_indx)) = (stdout_index, stderr_index) {
let (out_redirect, err_redirect, new_indx) = {
// to avoid panic, we need to remove commands which have larger index first.
if out_indx > err_indx {
let out_redirect = pipeline.commands.remove(out_indx);
let err_redirect = pipeline.commands.remove(err_indx);
(out_redirect, err_redirect, err_indx)
} else {
let err_redirect = pipeline.commands.remove(err_indx);
let out_redirect = pipeline.commands.remove(out_indx);
(out_redirect, err_redirect, out_indx)
}
};
// `out_redirect` and `err_redirect` should always be `LiteElement::Redirection`
if let (
LiteElement::Redirection(out_span, _, out_command, out_append_mode),
LiteElement::Redirection(err_span, _, err_command, err_append_mode),
) = (out_redirect, err_redirect)
{
// using insert with specific index to keep original
// pipeline commands order.
pipeline.insert(
new_indx,
LiteElement::SeparateRedirection {
out: (out_span, out_command, out_append_mode),
err: (err_span, err_command, err_append_mode),
},
)
}
fn push(&mut self, pipeline: &mut LitePipeline) {
if !pipeline.commands.is_empty() {
self.block.push(mem::take(pipeline));
}
}
}
@ -226,37 +149,68 @@ fn last_non_comment_token(tokens: &[Token], cur_idx: usize) -> Option<TokenConte
}
pub fn lite_parse(tokens: &[Token]) -> (LiteBlock, Option<ParseError>) {
let mut block = LiteBlock::new();
let mut curr_pipeline = LitePipeline::new();
let mut curr_command = LiteCommand::new();
let mut last_token = TokenContents::Eol;
let mut last_connector = TokenContents::Pipe;
let mut last_connector_span: Option<Span> = None;
if tokens.is_empty() {
return (LiteBlock::new(), None);
return (LiteBlock::default(), None);
}
let mut curr_comment: Option<Vec<Span>> = None;
let mut block = LiteBlock::default();
let mut pipeline = LitePipeline::default();
let mut command = LiteCommand::default();
let mut last_token = TokenContents::Eol;
let mut file_redirection = None;
let mut curr_comment: Option<Vec<Span>> = None;
let mut error = None;
for (idx, token) in tokens.iter().enumerate() {
if let Some((source, append, span)) = file_redirection.take() {
if command.parts.is_empty() {
error = error.or(Some(ParseError::LabeledError(
"Redirection without command or expression".into(),
"there is nothing to redirect".into(),
span,
)));
command.push(span);
match token.contents {
TokenContents::Comment => {
command.comments.push(token.span);
curr_comment = None;
}
TokenContents::Pipe
| TokenContents::ErrGreaterPipe
| TokenContents::OutErrGreaterPipe => {
pipeline.push(&mut command);
command.pipe = Some(token.span);
}
TokenContents::Semicolon => {
pipeline.push(&mut command);
block.push(&mut pipeline);
}
TokenContents::Eol => {
pipeline.push(&mut command);
}
_ => command.push(token.span),
}
} else {
match &token.contents {
TokenContents::PipePipe => {
error = error.or(Some(ParseError::ShellOrOr(token.span)));
curr_command.push(token.span);
last_token = TokenContents::Item;
command.push(span);
command.push(token.span);
}
TokenContents::Item => {
// If we have a comment, go ahead and attach it
if let Some(curr_comment) = curr_comment.take() {
curr_command.comments = curr_comment;
let target = LiteRedirectionTarget::File {
connector: span,
file: token.span,
append,
};
if let Err(err) = command.try_add_redirection(source, target) {
error = error.or(Some(err));
command.push(span);
command.push(token.span)
}
curr_command.push(token.span);
last_token = TokenContents::Item;
}
TokenContents::OutGreaterThan
| TokenContents::OutGreaterGreaterThan
@ -264,36 +218,112 @@ pub fn lite_parse(tokens: &[Token]) -> (LiteBlock, Option<ParseError>) {
| TokenContents::ErrGreaterGreaterThan
| TokenContents::OutErrGreaterThan
| TokenContents::OutErrGreaterGreaterThan => {
if let Some(err) = push_command_to(
&mut curr_pipeline,
curr_command,
last_connector,
last_connector_span,
) {
error = Some(err);
error =
error.or(Some(ParseError::Expected("redirection target", token.span)));
command.push(span);
command.push(token.span);
}
curr_command = LiteCommand::new();
last_token = token.contents;
last_connector = token.contents;
last_connector_span = Some(token.span);
}
pipe_token @ (TokenContents::Pipe
TokenContents::Pipe
| TokenContents::ErrGreaterPipe
| TokenContents::OutErrGreaterPipe) => {
if let Some(err) = push_command_to(
&mut curr_pipeline,
curr_command,
last_connector,
last_connector_span,
) {
error = Some(err);
| TokenContents::OutErrGreaterPipe => {
error =
error.or(Some(ParseError::Expected("redirection target", token.span)));
command.push(span);
pipeline.push(&mut command);
command.pipe = Some(token.span);
}
TokenContents::Eol => {
error =
error.or(Some(ParseError::Expected("redirection target", token.span)));
command.push(span);
pipeline.push(&mut command);
}
TokenContents::Semicolon => {
error =
error.or(Some(ParseError::Expected("redirection target", token.span)));
command.push(span);
pipeline.push(&mut command);
block.push(&mut pipeline);
}
TokenContents::Comment => {
error = error.or(Some(ParseError::Expected("redirection target", span)));
command.push(span);
command.comments.push(token.span);
curr_comment = None;
}
}
}
} else {
match &token.contents {
TokenContents::PipePipe => {
error = error.or(Some(ParseError::ShellOrOr(token.span)));
command.push(token.span);
}
TokenContents::Item => {
// This is commented out to preserve old parser behavior,
// but we should probably error here.
//
// if element.redirection.is_some() {
// error = error.or(Some(ParseError::LabeledError(
// "Unexpected positional".into(),
// "cannot add positional arguments after output redirection".into(),
// token.span,
// )));
// }
//
// For example, this is currently allowed: ^echo thing o> out.txt extra_arg
curr_command = LiteCommand::new();
last_token = *pipe_token;
last_connector = *pipe_token;
last_connector_span = Some(token.span);
// If we have a comment, go ahead and attach it
if let Some(curr_comment) = curr_comment.take() {
command.comments = curr_comment;
}
command.push(token.span);
}
TokenContents::OutGreaterThan => {
file_redirection = Some((RedirectionSource::Stdout, false, token.span));
}
TokenContents::OutGreaterGreaterThan => {
file_redirection = Some((RedirectionSource::Stdout, true, token.span));
}
TokenContents::ErrGreaterThan => {
file_redirection = Some((RedirectionSource::Stderr, false, token.span));
}
TokenContents::ErrGreaterGreaterThan => {
file_redirection = Some((RedirectionSource::Stderr, true, token.span));
}
TokenContents::OutErrGreaterThan => {
file_redirection =
Some((RedirectionSource::StdoutAndStderr, false, token.span));
}
TokenContents::OutErrGreaterGreaterThan => {
file_redirection = Some((RedirectionSource::StdoutAndStderr, true, token.span));
}
TokenContents::ErrGreaterPipe => {
let target = LiteRedirectionTarget::Pipe {
connector: token.span,
};
if let Err(err) = command.try_add_redirection(RedirectionSource::Stderr, target)
{
error = error.or(Some(err));
}
pipeline.push(&mut command);
command.pipe = Some(token.span);
}
TokenContents::OutErrGreaterPipe => {
let target = LiteRedirectionTarget::Pipe {
connector: token.span,
};
if let Err(err) =
command.try_add_redirection(RedirectionSource::StdoutAndStderr, target)
{
error = error.or(Some(err));
}
pipeline.push(&mut command);
command.pipe = Some(token.span);
}
TokenContents::Pipe => {
pipeline.push(&mut command);
command.pipe = Some(token.span);
}
TokenContents::Eol => {
// Handle `[Command] [Pipe] ([Comment] | [Eol])+ [Command]`
@ -301,60 +331,24 @@ pub fn lite_parse(tokens: &[Token]) -> (LiteBlock, Option<ParseError>) {
// `[Eol]` branch checks if previous token is `[Pipe]` to construct pipeline
// and so `[Comment] | [Eol]` should be ignore to make it work
let actual_token = last_non_comment_token(tokens, idx);
if actual_token != Some(TokenContents::Pipe)
&& actual_token != Some(TokenContents::OutGreaterThan)
{
if let Some(err) = push_command_to(
&mut curr_pipeline,
curr_command,
last_connector,
last_connector_span,
) {
error = Some(err);
}
curr_command = LiteCommand::new();
if !curr_pipeline.is_empty() {
block.push(curr_pipeline);
curr_pipeline = LitePipeline::new();
last_connector = TokenContents::Pipe;
last_connector_span = None;
}
if actual_token != Some(TokenContents::Pipe) {
pipeline.push(&mut command);
block.push(&mut pipeline);
}
if last_token == TokenContents::Eol {
// Clear out the comment as we're entering a new comment
curr_comment = None;
}
last_token = TokenContents::Eol;
}
TokenContents::Semicolon => {
if let Some(err) = push_command_to(
&mut curr_pipeline,
curr_command,
last_connector,
last_connector_span,
) {
error = Some(err);
}
curr_command = LiteCommand::new();
if !curr_pipeline.is_empty() {
block.push(curr_pipeline);
curr_pipeline = LitePipeline::new();
last_connector = TokenContents::Pipe;
last_connector_span = None;
}
last_token = TokenContents::Semicolon;
pipeline.push(&mut command);
block.push(&mut pipeline);
}
TokenContents::Comment => {
// Comment is beside something
if last_token != TokenContents::Eol {
curr_command.comments.push(token.span);
command.comments.push(token.span);
curr_comment = None;
} else {
// Comment precedes something
@ -364,24 +358,21 @@ pub fn lite_parse(tokens: &[Token]) -> (LiteBlock, Option<ParseError>) {
curr_comment = Some(vec![token.span]);
}
}
last_token = TokenContents::Comment;
}
}
}
if let Some(err) = push_command_to(
&mut curr_pipeline,
curr_command,
last_connector,
last_connector_span,
) {
error = Some(err);
last_token = token.contents;
}
if !curr_pipeline.is_empty() {
block.push(curr_pipeline);
if let Some((_, _, span)) = file_redirection {
command.push(span);
error = error.or(Some(ParseError::Expected("redirection target", span)));
}
pipeline.push(&mut command);
block.push(&mut pipeline);
if last_non_comment_token(tokens, tokens.len()) == Some(TokenContents::Pipe) {
(
block,
@ -394,86 +385,3 @@ pub fn lite_parse(tokens: &[Token]) -> (LiteBlock, Option<ParseError>) {
(block, error)
}
}
fn get_redirection(connector: TokenContents) -> Option<(Redirection, bool)> {
match connector {
TokenContents::OutGreaterThan => Some((Redirection::Stdout, false)),
TokenContents::OutGreaterGreaterThan => Some((Redirection::Stdout, true)),
TokenContents::ErrGreaterThan => Some((Redirection::Stderr, false)),
TokenContents::ErrGreaterGreaterThan => Some((Redirection::Stderr, true)),
TokenContents::OutErrGreaterThan => Some((Redirection::StdoutAndStderr, false)),
TokenContents::OutErrGreaterGreaterThan => Some((Redirection::StdoutAndStderr, true)),
_ => None,
}
}
/// push a `command` to `pipeline`
///
/// It will return Some(err) if `command` is empty and we want to push a
/// redirection command, or we have meet the same redirection in `pipeline`.
fn push_command_to(
pipeline: &mut LitePipeline,
command: LiteCommand,
last_connector: TokenContents,
last_connector_span: Option<Span>,
) -> Option<ParseError> {
if !command.is_empty() {
match get_redirection(last_connector) {
Some((redirect, is_append_mode)) => {
let span = last_connector_span
.expect("internal error: redirection missing span information");
if pipeline.exists(&redirect) {
return Some(ParseError::LabeledError(
"Redirection can be set only once".into(),
"try to remove one".into(),
span,
));
}
pipeline.push(LiteElement::Redirection(
last_connector_span
.expect("internal error: redirection missing span information"),
redirect,
command,
is_append_mode,
))
}
None => {
if last_connector == TokenContents::ErrGreaterPipe {
pipeline.push(LiteElement::ErrPipedCommand(last_connector_span, command))
} else if last_connector == TokenContents::OutErrGreaterPipe {
// Don't allow o+e>| along with redirection.
for cmd in &pipeline.commands {
if matches!(
cmd,
LiteElement::Redirection { .. }
| LiteElement::SameTargetRedirection { .. }
| LiteElement::SeparateRedirection { .. }
) {
return Some(ParseError::LabeledError(
"`o+e>|` pipe is not allowed to use with redirection".into(),
"try to use different type of pipe, or remove redirection".into(),
last_connector_span
.expect("internal error: outerr pipe missing span information"),
));
}
}
pipeline.push(LiteElement::OutErrPipedCommand(
last_connector_span,
command,
))
} else {
pipeline.push(LiteElement::Command(last_connector_span, command))
}
}
}
None
} else if get_redirection(last_connector).is_some() {
Some(ParseError::Expected(
"redirection target",
last_connector_span.expect("internal error: redirection missing span information"),
))
} else {
None
}
}

314
crates/nu-parser/src/parse_keywords.rs
View file

@ -1,6 +1,7 @@
use crate::{
exportable::Exportable,
parse_block,
parser::{parse_redirection, redirecting_builtin_error},
parser_path::ParserPath,
type_check::{check_block_input_output, type_compatible},
};
@ -28,7 +29,7 @@ use crate::{
is_math_expression_like,
known_external::KnownExternal,
lex,
lite_parser::{lite_parse, LiteCommand, LiteElement},
lite_parser::{lite_parse, LiteCommand},
parser::{
check_call, check_name, garbage, garbage_pipeline, parse, parse_call, parse_expression,
parse_full_signature, parse_import_pattern, parse_internal_call, parse_multispan_value,
@ -88,17 +89,8 @@ pub fn is_unaliasable_parser_keyword(working_set: &StateWorkingSet, spans: &[Spa
/// This is a new more compact method of calling parse_xxx() functions without repeating the
/// parse_call() in each function. Remaining keywords can be moved here.
pub fn parse_keyword(
working_set: &mut StateWorkingSet,
lite_command: &LiteCommand,
is_subexpression: bool,
) -> Pipeline {
let call_expr = parse_call(
working_set,
&lite_command.parts,
lite_command.parts[0],
is_subexpression,
);
pub fn parse_keyword(working_set: &mut StateWorkingSet, lite_command: &LiteCommand) -> Pipeline {
let call_expr = parse_call(working_set, &lite_command.parts, lite_command.parts[0]);
// if err.is_some() {
// return (Pipeline::from_vec(vec![call_expr]), err);
@ -246,7 +238,8 @@ pub fn parse_def_predecl(working_set: &mut StateWorkingSet, spans: &[Span]) {
}
}
pub fn parse_for(working_set: &mut StateWorkingSet, spans: &[Span]) -> Expression {
pub fn parse_for(working_set: &mut StateWorkingSet, lite_command: &LiteCommand) -> Expression {
let spans = &lite_command.parts;
// Checking that the function is used with the correct name
// Maybe this is not necessary but it is a sanity check
if working_set.get_span_contents(spans[0]) != b"for" {
@ -256,6 +249,10 @@ pub fn parse_for(working_set: &mut StateWorkingSet, spans: &[Span]) -> Expressio
));
return garbage(spans[0]);
}
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("for", redirection));
return garbage(spans[0]);
}
// Parsing the spans and checking that they match the register signature
// Using a parsed call makes more sense than checking for how many spans are in the call
@ -393,6 +390,10 @@ pub fn parse_def(
));
return (garbage_pipeline(spans), None);
}
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("def", redirection));
return (garbage_pipeline(spans), None);
}
// Parsing the spans and checking that they match the register signature
// Using a parsed call makes more sense than checking for how many spans are in the call
@ -667,6 +668,10 @@ pub fn parse_extern(
));
return garbage_pipeline(spans);
}
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("extern", redirection));
return garbage_pipeline(spans);
}
// Parsing the spans and checking that they match the register signature
// Using a parsed call makes more sense than checking for how many spans are in the call
@ -818,6 +823,10 @@ pub fn parse_alias(
));
return garbage_pipeline(spans);
}
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("alias", redirection));
return garbage_pipeline(spans);
}
if let Some(span) = check_name(working_set, spans) {
return Pipeline::from_vec(vec![garbage(*span)]);
@ -907,7 +916,7 @@ pub fn parse_alias(
{
// TODO: Maybe we need to implement a Display trait for Expression?
let starting_error_count = working_set.parse_errors.len();
let expr = parse_expression(working_set, replacement_spans, false);
let expr = parse_expression(working_set, replacement_spans);
working_set.parse_errors.truncate(starting_error_count);
let msg = format!("{:?}", expr.expr);
@ -923,12 +932,7 @@ pub fn parse_alias(
let starting_error_count = working_set.parse_errors.len();
working_set.search_predecls = false;
let expr = parse_call(
working_set,
replacement_spans,
replacement_spans[0],
false, // TODO: Should this be set properly???
);
let expr = parse_call(working_set, replacement_spans, replacement_spans[0]);
working_set.search_predecls = true;
@ -1063,24 +1067,32 @@ pub fn parse_export_in_block(
let full_name = if lite_command.parts.len() > 1 {
let sub = working_set.get_span_contents(lite_command.parts[1]);
match sub {
b"alias" | b"def" | b"extern" | b"use" | b"module" | b"const" => {
[b"export ", sub].concat()
}
_ => b"export".to_vec(),
b"alias" => "export alias",
b"def" => "export def",
b"extern" => "export extern",
b"use" => "export use",
b"module" => "export module",
b"const" => "export const",
_ => "export",
}
} else {
b"export".to_vec()
"export"
};
if let Some(decl_id) = working_set.find_decl(&full_name) {
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error(full_name, redirection));
return garbage_pipeline(&lite_command.parts);
}
if let Some(decl_id) = working_set.find_decl(full_name.as_bytes()) {
let ParsedInternalCall { call, output, .. } = parse_internal_call(
working_set,
if full_name == b"export" {
if full_name == "export" {
lite_command.parts[0]
} else {
span(&lite_command.parts[0..2])
},
if full_name == b"export" {
if full_name == "export" {
&lite_command.parts[1..]
} else {
&lite_command.parts[2..]
@ -1107,16 +1119,13 @@ pub fn parse_export_in_block(
}
} else {
working_set.error(ParseError::UnknownState(
format!(
"internal error: '{}' declaration not found",
String::from_utf8_lossy(&full_name)
),
format!("internal error: '{full_name}' declaration not found",),
span(&lite_command.parts),
));
return garbage_pipeline(&lite_command.parts);
};
if &full_name == b"export" {
if full_name == "export" {
// export by itself is meaningless
working_set.error(ParseError::UnexpectedKeyword(
"export".into(),
@ -1125,19 +1134,16 @@ pub fn parse_export_in_block(
return garbage_pipeline(&lite_command.parts);
}
match full_name.as_slice() {
b"export alias" => parse_alias(working_set, lite_command, None),
b"export def" => parse_def(working_set, lite_command, None).0,
b"export const" => parse_const(working_set, &lite_command.parts[1..]),
b"export use" => {
let (pipeline, _) = parse_use(working_set, &lite_command.parts);
pipeline
}
b"export module" => parse_module(working_set, lite_command, None).0,
b"export extern" => parse_extern(working_set, lite_command, None),
match full_name {
"export alias" => parse_alias(working_set, lite_command, None),
"export def" => parse_def(working_set, lite_command, None).0,
"export const" => parse_const(working_set, &lite_command.parts[1..]),
"export use" => parse_use(working_set, lite_command).0,
"export module" => parse_module(working_set, lite_command, None).0,
"export extern" => parse_extern(working_set, lite_command, None),
_ => {
working_set.error(ParseError::UnexpectedKeyword(
String::from_utf8_lossy(&full_name).to_string(),
full_name.into(),
lite_command.parts[0],
));
@ -1186,8 +1192,6 @@ pub fn parse_export_in_module(
head: spans[0],
decl_id: export_decl_id,
arguments: vec![],
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(),
});
@ -1198,6 +1202,8 @@ pub fn parse_export_in_module(
let lite_command = LiteCommand {
comments: lite_command.comments.clone(),
parts: spans[1..].to_vec(),
pipe: lite_command.pipe,
redirection: lite_command.redirection.clone(),
};
let (pipeline, cmd_result) =
parse_def(working_set, &lite_command, Some(module_name));
@ -1222,16 +1228,9 @@ pub fn parse_export_in_module(
};
// Trying to warp the 'def' call into the 'export def' in a very clumsy way
if let Some(PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(ref def_call),
..
},
)) = pipeline.elements.first()
if let Some(Expr::Call(def_call)) = pipeline.elements.first().map(|e| &e.expr.expr)
{
call = def_call.clone();
call.head = span(&spans[0..=1]);
call.decl_id = export_def_decl_id;
} else {
@ -1247,6 +1246,8 @@ pub fn parse_export_in_module(
let lite_command = LiteCommand {
comments: lite_command.comments.clone(),
parts: spans[1..].to_vec(),
pipe: lite_command.pipe,
redirection: lite_command.redirection.clone(),
};
let extern_name = [b"export ", kw_name].concat();
@ -1263,16 +1264,9 @@ pub fn parse_export_in_module(
};
// Trying to warp the 'def' call into the 'export def' in a very clumsy way
if let Some(PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(ref def_call),
..
},
)) = pipeline.elements.first()
if let Some(Expr::Call(def_call)) = pipeline.elements.first().map(|e| &e.expr.expr)
{
call = def_call.clone();
call.head = span(&spans[0..=1]);
call.decl_id = export_def_decl_id;
} else {
@ -1308,6 +1302,8 @@ pub fn parse_export_in_module(
let lite_command = LiteCommand {
comments: lite_command.comments.clone(),
parts: spans[1..].to_vec(),
pipe: lite_command.pipe,
redirection: lite_command.redirection.clone(),
};
let pipeline = parse_alias(working_set, &lite_command, Some(module_name));
@ -1323,13 +1319,8 @@ pub fn parse_export_in_module(
};
// Trying to warp the 'alias' call into the 'export alias' in a very clumsy way
if let Some(PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(ref alias_call),
..
},
)) = pipeline.elements.first()
if let Some(Expr::Call(alias_call)) =
pipeline.elements.first().map(|e| &e.expr.expr)
{
call = alias_call.clone();
@ -1368,8 +1359,10 @@ pub fn parse_export_in_module(
let lite_command = LiteCommand {
comments: lite_command.comments.clone(),
parts: spans[1..].to_vec(),
pipe: lite_command.pipe,
redirection: lite_command.redirection.clone(),
};
let (pipeline, exportables) = parse_use(working_set, &lite_command.parts);
let (pipeline, exportables) = parse_use(working_set, &lite_command);
let export_use_decl_id = if let Some(id) = working_set.find_decl(b"export use") {
id
@ -1382,13 +1375,7 @@ pub fn parse_export_in_module(
};
// Trying to warp the 'use' call into the 'export use' in a very clumsy way
if let Some(PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(ref use_call),
..
},
)) = pipeline.elements.first()
if let Some(Expr::Call(use_call)) = pipeline.elements.first().map(|e| &e.expr.expr)
{
call = use_call.clone();
@ -1419,13 +1406,8 @@ pub fn parse_export_in_module(
};
// Trying to warp the 'module' call into the 'export module' in a very clumsy way
if let Some(PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(ref module_call),
..
},
)) = pipeline.elements.first()
if let Some(Expr::Call(module_call)) =
pipeline.elements.first().map(|e| &e.expr.expr)
{
call = module_call.clone();
@ -1476,13 +1458,7 @@ pub fn parse_export_in_module(
};
// Trying to warp the 'const' call into the 'export const' in a very clumsy way
if let Some(PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(ref def_call),
..
},
)) = pipeline.elements.first()
if let Some(Expr::Call(def_call)) = pipeline.elements.first().map(|e| &e.expr.expr)
{
call = def_call.clone();
@ -1690,9 +1666,7 @@ pub fn parse_module_block(
for pipeline in &output.block {
if pipeline.commands.len() == 1 {
if let LiteElement::Command(_, command) = &pipeline.commands[0] {
parse_def_predecl(working_set, &command.parts);
}
parse_def_predecl(working_set, &pipeline.commands[0].parts);
}
}
@ -1702,10 +1676,8 @@ pub fn parse_module_block(
for pipeline in output.block.iter() {
if pipeline.commands.len() == 1 {
match &pipeline.commands[0] {
LiteElement::Command(_, command)
| LiteElement::ErrPipedCommand(_, command)
| LiteElement::OutErrPipedCommand(_, command) => {
let command = &pipeline.commands[0];
let name = working_set.get_span_contents(command.parts[0]);
match name {
@ -1733,7 +1705,7 @@ pub fn parse_module_block(
))
}
b"use" => {
let (pipeline, _) = parse_use(working_set, &command.parts);
let (pipeline, _) = parse_use(working_set, command);
block.pipelines.push(pipeline)
}
@ -1756,24 +1728,16 @@ pub fn parse_module_block(
if &name == b"main" {
if module.main.is_some() {
let err_span = if !pipe.elements.is_empty() {
if let PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) = &pipe.elements[0]
{
if let Expr::Call(call) = &pipe.elements[0].expr.expr {
call.head
} else {
pipe.elements[0].span()
pipe.elements[0].expr.span
}
} else {
span
};
working_set.error(ParseError::ModuleDoubleMain(
String::from_utf8_lossy(module_name)
.to_string(),
String::from_utf8_lossy(module_name).to_string(),
err_span,
));
} else {
@ -1785,17 +1749,9 @@ pub fn parse_module_block(
}
Exportable::Module { name, id } => {
if &name == b"mod" {
let (
submodule_main,
submodule_decls,
submodule_submodules,
) = {
let (submodule_main, submodule_decls, submodule_submodules) = {
let submodule = working_set.get_module(id);
(
submodule.main,
submodule.decls(),
submodule.submodules(),
)
(submodule.main, submodule.decls(), submodule.submodules())
};
// Add submodule's decls to the parent module
@ -1807,37 +1763,27 @@ pub fn parse_module_block(
if let Some(main_decl_id) = submodule_main {
if module.main.is_some() {
let err_span = if !pipe.elements.is_empty() {
if let PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) = &pipe.elements[0]
if let Expr::Call(call) =
&pipe.elements[0].expr.expr
{
call.head
} else {
pipe.elements[0].span()
pipe.elements[0].expr.span
}
} else {
span
};
working_set.error(
ParseError::ModuleDoubleMain(
String::from_utf8_lossy(module_name)
.to_string(),
working_set.error(ParseError::ModuleDoubleMain(
String::from_utf8_lossy(module_name).to_string(),
err_span,
),
);
));
} else {
module.main = Some(main_decl_id);
}
}
// Add submodule's submodules to the parent module
for (submodule_name, submodule_id) in
submodule_submodules
{
for (submodule_name, submodule_id) in submodule_submodules {
module.add_submodule(submodule_name, submodule_id);
}
} else {
@ -1853,8 +1799,7 @@ pub fn parse_module_block(
block.pipelines.push(pipe)
}
b"export-env" => {
let (pipe, maybe_env_block) =
parse_export_env(working_set, &command.parts);
let (pipe, maybe_env_block) = parse_export_env(working_set, &command.parts);
if let Some(block_id) = maybe_env_block {
module.add_env_block(block_id);
@ -1864,25 +1809,14 @@ pub fn parse_module_block(
}
_ => {
working_set.error(ParseError::ExpectedKeyword(
"def, const, extern, alias, use, module, export or export-env keyword".into(),
"def, const, extern, alias, use, module, export or export-env keyword"
.into(),
command.parts[0],
));
block.pipelines.push(garbage_pipeline(&command.parts))
}
}
}
LiteElement::Redirection(_, _, command, _) => {
block.pipelines.push(garbage_pipeline(&command.parts))
}
LiteElement::SeparateRedirection {
out: (_, command, _),
..
} => block.pipelines.push(garbage_pipeline(&command.parts)),
LiteElement::SameTargetRedirection {
cmd: (_, command), ..
} => block.pipelines.push(garbage_pipeline(&command.parts)),
}
} else {
working_set.error(ParseError::Expected("not a pipeline", span));
block.pipelines.push(garbage_pipeline(&[span]))
@ -2069,6 +2003,12 @@ pub fn parse_module(
// visible and usable in this module's scope). We want to disable that for files.
let spans = &lite_command.parts;
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("module", redirection));
return (garbage_pipeline(spans), None);
}
let mut module_comments = lite_command.comments.clone();
let split_id = if spans.len() > 1 && working_set.get_span_contents(spans[0]) == b"export" {
@ -2236,8 +2176,6 @@ pub fn parse_module(
Argument::Positional(module_name_or_path_expr),
Argument::Positional(block_expr),
],
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(),
});
@ -2252,7 +2190,12 @@ pub fn parse_module(
)
}
pub fn parse_use(working_set: &mut StateWorkingSet, spans: &[Span]) -> (Pipeline, Vec<Exportable>) {
pub fn parse_use(
working_set: &mut StateWorkingSet,
lite_command: &LiteCommand,
) -> (Pipeline, Vec<Exportable>) {
let spans = &lite_command.parts;
let (name_span, split_id) =
if spans.len() > 1 && working_set.get_span_contents(spans[0]) == b"export" {
(spans[1], 2)
@ -2277,6 +2220,11 @@ pub fn parse_use(working_set: &mut StateWorkingSet, spans: &[Span]) -> (Pipeline
return (garbage_pipeline(spans), vec![]);
}
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("use", redirection));
return (garbage_pipeline(spans), vec![]);
}
let (call, call_span, args_spans) = match working_set.find_decl(b"use") {
Some(decl_id) => {
let (command_spans, rest_spans) = spans.split_at(split_id);
@ -2460,7 +2408,9 @@ pub fn parse_use(working_set: &mut StateWorkingSet, spans: &[Span]) -> (Pipeline
)
}
pub fn parse_hide(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline {
pub fn parse_hide(working_set: &mut StateWorkingSet, lite_command: &LiteCommand) -> Pipeline {
let spans = &lite_command.parts;
if working_set.get_span_contents(spans[0]) != b"hide" {
working_set.error(ParseError::UnknownState(
"internal error: Wrong call name for 'hide' command".into(),
@ -2468,6 +2418,10 @@ pub fn parse_hide(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline
));
return garbage_pipeline(spans);
}
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("hide", redirection));
return garbage_pipeline(spans);
}
let (call, args_spans) = match working_set.find_decl(b"hide") {
Some(decl_id) => {
@ -3054,8 +3008,6 @@ pub fn parse_let(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline
decl_id,
head: spans[0],
arguments: vec![Argument::Positional(lvalue), Argument::Positional(rvalue)],
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(),
});
@ -3198,8 +3150,6 @@ pub fn parse_const(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipelin
decl_id,
head: spans[0],
arguments: vec![Argument::Positional(lvalue), Argument::Positional(rvalue)],
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(),
});
@ -3315,8 +3265,6 @@ pub fn parse_mut(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline
decl_id,
head: spans[0],
arguments: vec![Argument::Positional(lvalue), Argument::Positional(rvalue)],
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(),
});
@ -3353,10 +3301,21 @@ pub fn parse_mut(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline
garbage_pipeline(spans)
}
pub fn parse_source(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline {
pub fn parse_source(working_set: &mut StateWorkingSet, lite_command: &LiteCommand) -> Pipeline {
let spans = &lite_command.parts;
let name = working_set.get_span_contents(spans[0]);
if name == b"source" || name == b"source-env" {
if let Some(redirection) = lite_command.redirection.as_ref() {
let name = if name == b"source" {
"source"
} else {
"source-env"
};
working_set.error(redirecting_builtin_error(name, redirection));
return garbage_pipeline(spans);
}
let scoped = name == b"source-env";
if let Some(decl_id) = working_set.find_decl(name) {
@ -3537,13 +3496,26 @@ pub fn parse_where_expr(working_set: &mut StateWorkingSet, spans: &[Span]) -> Ex
}
}
pub fn parse_where(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline {
let expression = parse_where_expr(working_set, spans);
Pipeline::from_vec(vec![expression])
pub fn parse_where(working_set: &mut StateWorkingSet, lite_command: &LiteCommand) -> Pipeline {
let expr = parse_where_expr(working_set, &lite_command.parts);
let redirection = lite_command
.redirection
.as_ref()
.map(|r| parse_redirection(working_set, r));
let element = PipelineElement {
pipe: None,
expr,
redirection,
};
Pipeline {
elements: vec![element],
}
}
#[cfg(feature = "plugin")]
pub fn parse_register(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline {
pub fn parse_register(working_set: &mut StateWorkingSet, lite_command: &LiteCommand) -> Pipeline {
use std::sync::Arc;
use nu_plugin::{get_signature, PersistentPlugin, PluginDeclaration};
@ -3551,6 +3523,8 @@ pub fn parse_register(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipe
engine::Stack, IntoSpanned, PluginIdentity, PluginSignature, RegisteredPlugin,
};
let spans = &lite_command.parts;
let cwd = working_set.get_cwd();
// Checking that the function is used with the correct name
@ -3562,6 +3536,10 @@ pub fn parse_register(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipe
));
return garbage_pipeline(spans);
}
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("register", redirection));
return garbage_pipeline(spans);
}
// Parsing the spans and checking that they match the register signature
// Using a parsed call makes more sense than checking for how many spans are in the call
@ -3677,7 +3655,7 @@ pub fn parse_register(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipe
// We need the current environment variables for `python` based plugins
// Or we'll likely have a problem when a plugin is implemented in a virtual Python environment.
let get_envs = || {
let stack = Stack::new();
let stack = Stack::new().capture();
nu_engine::env::env_to_strings(working_set.permanent_state, &stack)
};

5
crates/nu-parser/src/parse_patterns.rs
View file

@ -7,7 +7,6 @@ use nu_protocol::{
use crate::{
lex, lite_parse,
parser::{is_variable, parse_value},
LiteElement,
};
pub fn garbage(span: Span) -> MatchPattern {
@ -108,10 +107,9 @@ pub fn parse_list_pattern(working_set: &mut StateWorkingSet, span: Span) -> Matc
let mut args = vec![];
if !output.block.is_empty() {
for arg in &output.block[0].commands {
for command in &output.block[0].commands {
let mut spans_idx = 0;
if let LiteElement::Command(_, command) = arg {
while spans_idx < command.parts.len() {
let contents = working_set.get_span_contents(command.parts[spans_idx]);
if contents == b".." {
@ -152,7 +150,6 @@ pub fn parse_list_pattern(working_set: &mut StateWorkingSet, span: Span) -> Matc
}
}
}
}
MatchPattern {
pattern: Pattern::List(args),

528
crates/nu-parser/src/parser.rs
View file

@ -1,6 +1,6 @@
use crate::{
lex::{lex, lex_signature},
lite_parser::{lite_parse, LiteCommand, LiteElement, LitePipeline},
lite_parser::{lite_parse, LiteCommand, LitePipeline, LiteRedirection, LiteRedirectionTarget},
parse_mut,
parse_patterns::parse_pattern,
parse_shape_specs::{parse_shape_name, parse_type, ShapeDescriptorUse},
@ -14,7 +14,7 @@ use nu_protocol::{
Argument, Assignment, Bits, Block, Boolean, Call, CellPath, Comparison, Expr, Expression,
ExternalArgument, FullCellPath, ImportPattern, ImportPatternHead, ImportPatternMember,
MatchPattern, Math, Operator, PathMember, Pattern, Pipeline, PipelineElement,
RangeInclusion, RangeOperator, RecordItem,
PipelineRedirection, RangeInclusion, RangeOperator, RecordItem, RedirectionTarget,
},
engine::StateWorkingSet,
eval_const::eval_constant,
@ -303,11 +303,7 @@ fn parse_external_arg(working_set: &mut StateWorkingSet, span: Span) -> External
}
}
pub fn parse_external_call(
working_set: &mut StateWorkingSet,
spans: &[Span],
is_subexpression: bool,
) -> Expression {
pub fn parse_external_call(working_set: &mut StateWorkingSet, spans: &[Span]) -> Expression {
trace!("parse external");
let mut args = vec![];
@ -324,7 +320,7 @@ pub fn parse_external_call(
let head = if head_contents.starts_with(b"$") || head_contents.starts_with(b"(") {
// the expression is inside external_call, so it's a subexpression
let arg = parse_expression(working_set, &[head_span], true);
let arg = parse_expression(working_set, &[head_span]);
Box::new(arg)
} else {
let (contents, err) = unescape_unquote_string(&head_contents, head_span);
@ -346,7 +342,7 @@ pub fn parse_external_call(
}
Expression {
expr: Expr::ExternalCall(head, args, is_subexpression),
expr: Expr::ExternalCall(head, args),
span: span(spans),
ty: Type::Any,
custom_completion: None,
@ -708,7 +704,7 @@ pub fn parse_multispan_value(
// is it subexpression?
// Not sure, but let's make it not, so the behavior is the same as previous version of nushell.
let arg = parse_expression(working_set, &spans[*spans_idx..], false);
let arg = parse_expression(working_set, &spans[*spans_idx..]);
*spans_idx = spans.len() - 1;
arg
@ -1095,12 +1091,7 @@ pub fn parse_internal_call(
}
}
pub fn parse_call(
working_set: &mut StateWorkingSet,
spans: &[Span],
head: Span,
is_subexpression: bool,
) -> Expression {
pub fn parse_call(working_set: &mut StateWorkingSet, spans: &[Span], head: Span) -> Expression {
trace!("parsing: call");
if spans.is_empty() {
@ -1179,7 +1170,7 @@ pub fn parse_call(
let parsed_call = if let Some(alias) = decl.as_alias() {
if let Expression {
expr: Expr::ExternalCall(head, args, is_subexpression),
expr: Expr::ExternalCall(head, args),
span: _,
ty,
custom_completion,
@ -1198,7 +1189,7 @@ pub fn parse_call(
head.span = spans[0]; // replacing the spans preserves syntax highlighting
return Expression {
expr: Expr::ExternalCall(head, final_args, *is_subexpression),
expr: Expr::ExternalCall(head, final_args),
span: span(spans),
ty: ty.clone(),
custom_completion: *custom_completion,
@ -1246,7 +1237,7 @@ pub fn parse_call(
trace!("parsing: external call");
// Otherwise, try external command
parse_external_call(working_set, spans, is_subexpression)
parse_external_call(working_set, spans)
}
}
@ -3139,9 +3130,11 @@ pub fn parse_row_condition(working_set: &mut StateWorkingSet, spans: &[Span]) ->
// We have an expression, so let's convert this into a block.
let mut block = Block::new();
let mut pipeline = Pipeline::new();
pipeline
.elements
.push(PipelineElement::Expression(None, expression));
pipeline.elements.push(PipelineElement {
pipe: None,
expr: expression,
redirection: None,
});
block.pipelines.push(pipeline);
@ -3846,10 +3839,9 @@ pub fn parse_list_expression(
let mut contained_type: Option<Type> = None;
if !output.block.is_empty() {
for arg in output.block.remove(0).commands {
for mut command in output.block.remove(0).commands {
let mut spans_idx = 0;
if let LiteElement::Command(_, mut command) = arg {
while spans_idx < command.parts.len() {
let curr_span = command.parts[spans_idx];
let curr_tok = working_set.get_span_contents(curr_span);
@ -3903,7 +3895,6 @@ pub fn parse_list_expression(
}
}
}
}
Expression {
expr: Expr::List(args),
@ -4861,7 +4852,7 @@ pub fn parse_math_expression(
if first_span == b"if" || first_span == b"match" {
// If expression
if spans.len() > 1 {
return parse_call(working_set, spans, spans[0], false);
return parse_call(working_set, spans, spans[0]);
} else {
working_set.error(ParseError::Expected(
"expression",
@ -4936,7 +4927,7 @@ pub fn parse_math_expression(
// allow `if` to be a special value for assignment.
if content == b"if" || content == b"match" {
let rhs = parse_call(working_set, &spans[idx..], spans[0], false);
let rhs = parse_call(working_set, &spans[idx..], spans[0]);
expr_stack.push(op);
expr_stack.push(rhs);
break;
@ -5055,11 +5046,7 @@ pub fn parse_math_expression(
.expect("internal error: expression stack empty")
}
pub fn parse_expression(
working_set: &mut StateWorkingSet,
spans: &[Span],
is_subexpression: bool,
) -> Expression {
pub fn parse_expression(working_set: &mut StateWorkingSet, spans: &[Span]) -> Expression {
trace!("parsing: expression");
let mut pos = 0;
@ -5134,7 +5121,7 @@ pub fn parse_expression(
spans[0],
));
parse_call(working_set, &spans[pos..], spans[0], is_subexpression)
parse_call(working_set, &spans[pos..], spans[0])
}
b"let" | b"const" | b"mut" => {
working_set.error(ParseError::AssignInPipeline(
@ -5152,19 +5139,19 @@ pub fn parse_expression(
.to_string(),
spans[0],
));
parse_call(working_set, &spans[pos..], spans[0], is_subexpression)
parse_call(working_set, &spans[pos..], spans[0])
}
b"overlay" => {
if spans.len() > 1 && working_set.get_span_contents(spans[1]) == b"list" {
// whitelist 'overlay list'
parse_call(working_set, &spans[pos..], spans[0], is_subexpression)
parse_call(working_set, &spans[pos..], spans[0])
} else {
working_set.error(ParseError::BuiltinCommandInPipeline(
"overlay".into(),
spans[0],
));
parse_call(working_set, &spans[pos..], spans[0], is_subexpression)
parse_call(working_set, &spans[pos..], spans[0])
}
}
b"where" => parse_where_expr(working_set, &spans[pos..]),
@ -5175,10 +5162,10 @@ pub fn parse_expression(
spans[0],
));
parse_call(working_set, &spans[pos..], spans[0], is_subexpression)
parse_call(working_set, &spans[pos..], spans[0])
}
_ => parse_call(working_set, &spans[pos..], spans[0], is_subexpression),
_ => parse_call(working_set, &spans[pos..], spans[0]),
}
};
@ -5217,8 +5204,6 @@ pub fn parse_expression(
head: Span::unknown(),
decl_id,
arguments,
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(),
}));
@ -5251,7 +5236,6 @@ pub fn parse_variable(working_set: &mut StateWorkingSet, span: Span) -> Option<V
pub fn parse_builtin_commands(
working_set: &mut StateWorkingSet,
lite_command: &LiteCommand,
is_subexpression: bool,
) -> Pipeline {
trace!("parsing: builtin commands");
if !is_math_expression_like(working_set, lite_command.parts[0])
@ -5264,12 +5248,7 @@ pub fn parse_builtin_commands(
if cmd.is_alias() {
// Parse keywords that can be aliased. Note that we check for "unaliasable" keywords
// because alias can have any name, therefore, we can't check for "aliasable" keywords.
let call_expr = parse_call(
working_set,
&lite_command.parts,
lite_command.parts[0],
is_subexpression,
);
let call_expr = parse_call(working_set, &lite_command.parts, lite_command.parts[0]);
if let Expression {
expr: Expr::Call(call),
@ -5299,26 +5278,31 @@ pub fn parse_builtin_commands(
b"const" => parse_const(working_set, &lite_command.parts),
b"mut" => parse_mut(working_set, &lite_command.parts),
b"for" => {
let expr = parse_for(working_set, &lite_command.parts);
let expr = parse_for(working_set, lite_command);
Pipeline::from_vec(vec![expr])
}
b"alias" => parse_alias(working_set, lite_command, None),
b"module" => parse_module(working_set, lite_command, None).0,
b"use" => {
let (pipeline, _) = parse_use(working_set, &lite_command.parts);
pipeline
b"use" => parse_use(working_set, lite_command).0,
b"overlay" => {
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("overlay", redirection));
return garbage_pipeline(&lite_command.parts);
}
b"overlay" => parse_keyword(working_set, lite_command, is_subexpression),
b"source" | b"source-env" => parse_source(working_set, &lite_command.parts),
parse_keyword(working_set, lite_command)
}
b"source" | b"source-env" => parse_source(working_set, lite_command),
b"export" => parse_export_in_block(working_set, lite_command),
b"hide" => parse_hide(working_set, &lite_command.parts),
b"where" => parse_where(working_set, &lite_command.parts),
b"hide" => parse_hide(working_set, lite_command),
b"where" => parse_where(working_set, lite_command),
#[cfg(feature = "plugin")]
b"register" => parse_register(working_set, &lite_command.parts),
b"register" => parse_register(working_set, lite_command),
_ => {
let expr = parse_expression(working_set, &lite_command.parts, is_subexpression);
let element = parse_pipeline_element(working_set, lite_command);
Pipeline::from_vec(vec![expr])
Pipeline {
elements: vec![element],
}
}
}
}
@ -5459,6 +5443,76 @@ pub fn parse_record(working_set: &mut StateWorkingSet, span: Span) -> Expression
}
}
fn parse_redirection_target(
working_set: &mut StateWorkingSet,
target: &LiteRedirectionTarget,
) -> RedirectionTarget {
match target {
LiteRedirectionTarget::File {
connector,
file,
append,
} => RedirectionTarget::File {
expr: parse_value(working_set, *file, &SyntaxShape::Any),
append: *append,
span: *connector,
},
LiteRedirectionTarget::Pipe { connector } => RedirectionTarget::Pipe { span: *connector },
}
}
pub(crate) fn parse_redirection(
working_set: &mut StateWorkingSet,
target: &LiteRedirection,
) -> PipelineRedirection {
match target {
LiteRedirection::Single { source, target } => PipelineRedirection::Single {
source: *source,
target: parse_redirection_target(working_set, target),
},
LiteRedirection::Separate { out, err } => PipelineRedirection::Separate {
out: parse_redirection_target(working_set, out),
err: parse_redirection_target(working_set, err),
},
}
}
fn parse_pipeline_element(
working_set: &mut StateWorkingSet,
command: &LiteCommand,
) -> PipelineElement {
trace!("parsing: pipeline element");
let expr = parse_expression(working_set, &command.parts);
let redirection = command
.redirection
.as_ref()
.map(|r| parse_redirection(working_set, r));
PipelineElement {
pipe: command.pipe,
expr,
redirection,
}
}
pub(crate) fn redirecting_builtin_error(
name: &'static str,
redirection: &LiteRedirection,
) -> ParseError {
match redirection {
LiteRedirection::Single { target, .. } => {
ParseError::RedirectingBuiltinCommand(name, target.connector(), None)
}
LiteRedirection::Separate { out, err } => ParseError::RedirectingBuiltinCommand(
name,
out.connector().min(err.connector()),
Some(out.connector().max(err.connector())),
),
}
}
pub fn parse_pipeline(
working_set: &mut StateWorkingSet,
pipeline: &LitePipeline,
@ -5467,38 +5521,28 @@ pub fn parse_pipeline(
) -> Pipeline {
if pipeline.commands.len() > 1 {
// Special case: allow `let` and `mut` to consume the whole pipeline, eg) `let abc = "foo" | str length`
match &pipeline.commands[0] {
LiteElement::Command(_, command) if !command.parts.is_empty() => {
if working_set.get_span_contents(command.parts[0]) == b"let"
|| working_set.get_span_contents(command.parts[0]) == b"mut"
{
if let Some(&first) = pipeline.commands[0].parts.first() {
let first = working_set.get_span_contents(first);
if first == b"let" || first == b"mut" {
let name = if first == b"let" { "let" } else { "mut" };
let mut new_command = LiteCommand {
comments: vec![],
parts: command.parts.clone(),
parts: pipeline.commands[0].parts.clone(),
pipe: None,
redirection: None,
};
for command in &pipeline.commands[1..] {
match command {
LiteElement::Command(Some(pipe_span), command)
| LiteElement::ErrPipedCommand(Some(pipe_span), command)
| LiteElement::OutErrPipedCommand(Some(pipe_span), command) => {
new_command.parts.push(*pipe_span);
if let Some(redirection) = pipeline.commands[0].redirection.as_ref() {
working_set.error(redirecting_builtin_error(name, redirection));
}
new_command.comments.extend_from_slice(&command.comments);
new_command.parts.extend_from_slice(&command.parts);
}
LiteElement::Redirection(span, ..) => {
working_set.error(ParseError::RedirectionInLetMut(*span, None))
}
LiteElement::SeparateRedirection { out, err } => {
working_set.error(ParseError::RedirectionInLetMut(
out.0.min(err.0),
Some(out.0.max(err.0)),
))
}
LiteElement::SameTargetRedirection { redirection, .. } => working_set
.error(ParseError::RedirectionInLetMut(redirection.0, None)),
_ => panic!("unsupported"),
for element in &pipeline.commands[1..] {
if let Some(redirection) = pipeline.commands[0].redirection.as_ref() {
working_set.error(redirecting_builtin_error(name, redirection));
} else {
new_command.parts.push(element.pipe.expect("pipe span"));
new_command.comments.extend_from_slice(&element.comments);
new_command.parts.extend_from_slice(&element.parts);
}
}
@ -5509,21 +5553,13 @@ pub fn parse_pipeline(
new_command.parts.truncate(3);
new_command.parts.push(rhs_span);
let mut pipeline =
parse_builtin_commands(working_set, &new_command, is_subexpression);
let mut pipeline = parse_builtin_commands(working_set, &new_command);
if pipeline_index == 0 {
let let_decl_id = working_set.find_decl(b"let");
let mut_decl_id = working_set.find_decl(b"mut");
for element in pipeline.elements.iter_mut() {
if let PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) = element
{
if let Expr::Call(call) = &element.expr.expr {
if Some(call.decl_id) == let_decl_id
|| Some(call.decl_id) == mut_decl_id
{
@ -5531,39 +5567,32 @@ pub fn parse_pipeline(
if let Some(Expression {
expr: Expr::Block(block_id),
..
}) = call.positional_iter_mut().nth(1)
}) = call.positional_iter().nth(1)
{
let block = working_set.get_block(*block_id);
if let Some(PipelineElement::Expression(
prepend,
expr,
)) = block
if let Some(element) = block
.pipelines
.first()
.and_then(|p| p.elements.first())
.cloned()
{
if expr.has_in_variable(working_set) {
let new_expr = PipelineElement::Expression(
prepend,
wrap_expr_with_collect(working_set, &expr),
if element.has_in_variable(working_set) {
let element = wrap_element_with_collect(
working_set,
&element,
);
let block =
working_set.get_block_mut(*block_id);
block.pipelines[0].elements[0] = new_expr;
let block = working_set.get_block_mut(*block_id);
block.pipelines[0].elements[0] = element;
}
}
}
continue;
} else if element.has_in_variable(working_set)
&& !is_subexpression
} else if element.has_in_variable(working_set) && !is_subexpression
{
*element = wrap_element_with_collect(working_set, element);
}
} else if element.has_in_variable(working_set) && !is_subexpression
{
} else if element.has_in_variable(working_set) && !is_subexpression {
*element = wrap_element_with_collect(working_set, element);
}
}
@ -5573,133 +5602,66 @@ pub fn parse_pipeline(
}
}
}
_ => {}
};
let mut output = pipeline
let mut elements = pipeline
.commands
.iter()
.map(|command| match command {
LiteElement::Command(span, command) => {
trace!("parsing: pipeline element: command");
let expr = parse_expression(working_set, &command.parts, is_subexpression);
PipelineElement::Expression(*span, expr)
}
LiteElement::ErrPipedCommand(span, command) => {
trace!("parsing: pipeline element: err piped command");
let expr = parse_expression(working_set, &command.parts, is_subexpression);
PipelineElement::ErrPipedExpression(*span, expr)
}
LiteElement::OutErrPipedCommand(span, command) => {
trace!("parsing: pipeline element: err piped command");
let expr = parse_expression(working_set, &command.parts, is_subexpression);
PipelineElement::OutErrPipedExpression(*span, expr)
}
LiteElement::Redirection(span, redirection, command, is_append_mode) => {
let expr = parse_value(working_set, command.parts[0], &SyntaxShape::Any);
PipelineElement::Redirection(*span, redirection.clone(), expr, *is_append_mode)
}
LiteElement::SeparateRedirection {
out: (out_span, out_command, out_append_mode),
err: (err_span, err_command, err_append_mode),
} => {
trace!("parsing: pipeline element: separate redirection");
let out_expr =
parse_value(working_set, out_command.parts[0], &SyntaxShape::Any);
let err_expr =
parse_value(working_set, err_command.parts[0], &SyntaxShape::Any);
PipelineElement::SeparateRedirection {
out: (*out_span, out_expr, *out_append_mode),
err: (*err_span, err_expr, *err_append_mode),
}
}
LiteElement::SameTargetRedirection {
cmd: (cmd_span, command),
redirection: (redirect_span, redirect_command, is_append_mode),
} => {
trace!("parsing: pipeline element: same target redirection");
let expr = parse_expression(working_set, &command.parts, is_subexpression);
let redirect_expr =
parse_value(working_set, redirect_command.parts[0], &SyntaxShape::Any);
PipelineElement::SameTargetRedirection {
cmd: (*cmd_span, expr),
redirection: (*redirect_span, redirect_expr, *is_append_mode),
}
}
})
.collect::<Vec<PipelineElement>>();
.map(|element| parse_pipeline_element(working_set, element))
.collect::<Vec<_>>();
if is_subexpression {
for element in output.iter_mut().skip(1) {
for element in elements.iter_mut().skip(1) {
if element.has_in_variable(working_set) {
*element = wrap_element_with_collect(working_set, element);
}
}
} else {
for element in output.iter_mut() {
for element in elements.iter_mut() {
if element.has_in_variable(working_set) {
*element = wrap_element_with_collect(working_set, element);
}
}
}
Pipeline { elements: output }
Pipeline { elements }
} else {
match &pipeline.commands[0] {
LiteElement::Command(_, command)
| LiteElement::ErrPipedCommand(_, command)
| LiteElement::OutErrPipedCommand(_, command)
| LiteElement::Redirection(_, _, command, _)
| LiteElement::SeparateRedirection {
out: (_, command, _),
..
} => {
let mut pipeline = parse_builtin_commands(working_set, command, is_subexpression);
if let Some(&first) = pipeline.commands[0].parts.first() {
let first = working_set.get_span_contents(first);
if first == b"let" || first == b"mut" {
if let Some(redirection) = pipeline.commands[0].redirection.as_ref() {
let name = if first == b"let" { "let" } else { "mut" };
working_set.error(redirecting_builtin_error(name, redirection));
}
}
}
let mut pipeline = parse_builtin_commands(working_set, &pipeline.commands[0]);
let let_decl_id = working_set.find_decl(b"let");
let mut_decl_id = working_set.find_decl(b"mut");
if pipeline_index == 0 {
for element in pipeline.elements.iter_mut() {
if let PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) = element
{
if Some(call.decl_id) == let_decl_id
|| Some(call.decl_id) == mut_decl_id
{
if let Expr::Call(call) = &element.expr.expr {
if Some(call.decl_id) == let_decl_id || Some(call.decl_id) == mut_decl_id {
// Do an expansion
if let Some(Expression {
expr: Expr::Block(block_id),
..
}) = call.positional_iter_mut().nth(1)
}) = call.positional_iter().nth(1)
{
let block = working_set.get_block(*block_id);
if let Some(PipelineElement::Expression(prepend, expr)) = block
if let Some(element) = block
.pipelines
.first()
.and_then(|p| p.elements.first())
.cloned()
{
if expr.has_in_variable(working_set) {
let new_expr = PipelineElement::Expression(
prepend,
wrap_expr_with_collect(working_set, &expr),
);
if element.has_in_variable(working_set) {
let element = wrap_element_with_collect(working_set, &element);
let block = working_set.get_block_mut(*block_id);
block.pipelines[0].elements[0] = new_expr;
block.pipelines[0].elements[0] = element;
}
}
}
@ -5712,27 +5674,9 @@ pub fn parse_pipeline(
}
}
}
pipeline
}
LiteElement::SameTargetRedirection {
cmd: (span, command),
redirection: (redirect_span, redirect_cmd, is_append_mode),
} => {
trace!("parsing: pipeline element: same target redirection");
let expr = parse_expression(working_set, &command.parts, is_subexpression);
let redirect_expr =
parse_value(working_set, redirect_cmd.parts[0], &SyntaxShape::Any);
Pipeline {
elements: vec![PipelineElement::SameTargetRedirection {
cmd: (*span, expr),
redirection: (*redirect_span, redirect_expr, *is_append_mode),
}],
}
}
}
}
}
pub fn parse_block(
@ -5757,19 +5701,7 @@ pub fn parse_block(
// that share the same block can see each other
for pipeline in &lite_block.block {
if pipeline.commands.len() == 1 {
match &pipeline.commands[0] {
LiteElement::Command(_, command)
| LiteElement::ErrPipedCommand(_, command)
| LiteElement::OutErrPipedCommand(_, command)
| LiteElement::Redirection(_, _, command, _)
| LiteElement::SeparateRedirection {
out: (_, command, _),
..
}
| LiteElement::SameTargetRedirection {
cmd: (_, command), ..
} => parse_def_predecl(working_set, &command.parts),
}
parse_def_predecl(working_set, &pipeline.commands[0].parts)
}
}
@ -5852,33 +5784,28 @@ pub fn discover_captures_in_pipeline_element(
seen_blocks: &mut HashMap<BlockId, Vec<(VarId, Span)>>,
output: &mut Vec<(VarId, Span)>,
) -> Result<(), ParseError> {
match element {
PipelineElement::Expression(_, expression)
| PipelineElement::ErrPipedExpression(_, expression)
| PipelineElement::OutErrPipedExpression(_, expression)
| PipelineElement::Redirection(_, _, expression, _)
| PipelineElement::And(_, expression)
| PipelineElement::Or(_, expression) => {
discover_captures_in_expr(working_set, expression, seen, seen_blocks, output)
discover_captures_in_expr(working_set, &element.expr, seen, seen_blocks, output)?;
if let Some(redirection) = element.redirection.as_ref() {
match redirection {
PipelineRedirection::Single { target, .. } => {
if let Some(expr) = target.expr() {
discover_captures_in_expr(working_set, expr, seen, seen_blocks, output)?;
}
PipelineElement::SeparateRedirection {
out: (_, out_expr, _),
err: (_, err_expr, _),
} => {
discover_captures_in_expr(working_set, out_expr, seen, seen_blocks, output)?;
discover_captures_in_expr(working_set, err_expr, seen, seen_blocks, output)?;
}
PipelineRedirection::Separate { out, err } => {
if let Some(expr) = out.expr() {
discover_captures_in_expr(working_set, expr, seen, seen_blocks, output)?;
}
if let Some(expr) = err.expr() {
discover_captures_in_expr(working_set, expr, seen, seen_blocks, output)?;
}
}
}
}
Ok(())
}
PipelineElement::SameTargetRedirection {
cmd: (_, cmd_expr),
redirection: (_, redirect_expr, _),
} => {
discover_captures_in_expr(working_set, cmd_expr, seen, seen_blocks, output)?;
discover_captures_in_expr(working_set, redirect_expr, seen, seen_blocks, output)?;
Ok(())
}
}
}
pub fn discover_captures_in_pattern(pattern: &MatchPattern, seen: &mut Vec<VarId>) {
match &pattern.pattern {
@ -6043,7 +5970,7 @@ pub fn discover_captures_in_expr(
}
Expr::CellPath(_) => {}
Expr::DateTime(_) => {}
Expr::ExternalCall(head, args, _) => {
Expr::ExternalCall(head, args) => {
discover_captures_in_expr(working_set, head, seen, seen_blocks, output)?;
for ExternalArgument::Regular(expr) | ExternalArgument::Spread(expr) in args {
@ -6193,66 +6120,37 @@ pub fn discover_captures_in_expr(
Ok(())
}
fn wrap_redirection_with_collect(
working_set: &mut StateWorkingSet,
target: &RedirectionTarget,
) -> RedirectionTarget {
match target {
RedirectionTarget::File { expr, append, span } => RedirectionTarget::File {
expr: wrap_expr_with_collect(working_set, expr),
span: *span,
append: *append,
},
RedirectionTarget::Pipe { span } => RedirectionTarget::Pipe { span: *span },
}
}
fn wrap_element_with_collect(
working_set: &mut StateWorkingSet,
element: &PipelineElement,
) -> PipelineElement {
match element {
PipelineElement::Expression(span, expression) => {
PipelineElement::Expression(*span, wrap_expr_with_collect(working_set, expression))
}
PipelineElement::ErrPipedExpression(span, expression) => {
PipelineElement::ErrPipedExpression(
*span,
wrap_expr_with_collect(working_set, expression),
)
}
PipelineElement::OutErrPipedExpression(span, expression) => {
PipelineElement::OutErrPipedExpression(
*span,
wrap_expr_with_collect(working_set, expression),
)
}
PipelineElement::Redirection(span, redirection, expression, is_append_mode) => {
PipelineElement::Redirection(
*span,
redirection.clone(),
wrap_expr_with_collect(working_set, expression),
*is_append_mode,
)
}
PipelineElement::SeparateRedirection {
out: (out_span, out_exp, out_append_mode),
err: (err_span, err_exp, err_append_mode),
} => PipelineElement::SeparateRedirection {
out: (
*out_span,
wrap_expr_with_collect(working_set, out_exp),
*out_append_mode,
),
err: (
*err_span,
wrap_expr_with_collect(working_set, err_exp),
*err_append_mode,
),
PipelineElement {
pipe: element.pipe,
expr: wrap_expr_with_collect(working_set, &element.expr),
redirection: element.redirection.as_ref().map(|r| match r {
PipelineRedirection::Single { source, target } => PipelineRedirection::Single {
source: *source,
target: wrap_redirection_with_collect(working_set, target),
},
PipelineElement::SameTargetRedirection {
cmd: (cmd_span, cmd_exp),
redirection: (redirect_span, redirect_exp, is_append_mode),
} => PipelineElement::SameTargetRedirection {
cmd: (*cmd_span, wrap_expr_with_collect(working_set, cmd_exp)),
redirection: (
*redirect_span,
wrap_expr_with_collect(working_set, redirect_exp),
*is_append_mode,
),
PipelineRedirection::Separate { out, err } => PipelineRedirection::Separate {
out: wrap_redirection_with_collect(working_set, out),
err: wrap_redirection_with_collect(working_set, err),
},
PipelineElement::And(span, expression) => {
PipelineElement::And(*span, wrap_expr_with_collect(working_set, expression))
}
PipelineElement::Or(span, expression) => {
PipelineElement::Or(*span, wrap_expr_with_collect(working_set, expression))
}
}),
}
}
@ -6304,8 +6202,6 @@ fn wrap_expr_with_collect(working_set: &mut StateWorkingSet, expr: &Expression)
head: Span::new(0, 0),
arguments: output,
decl_id,
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(),
})),
span,

27
crates/nu-parser/src/type_check.rs
View file

@ -1,7 +1,6 @@
use nu_protocol::{
ast::{
Assignment, Bits, Block, Boolean, Comparison, Expr, Expression, Math, Operator, Pipeline,
PipelineElement,
},
engine::StateWorkingSet,
ParseError, Type,
@ -917,14 +916,9 @@ pub fn check_pipeline_type(
let mut output_errors: Option<Vec<ParseError>> = None;
'elem: for elem in &pipeline.elements {
match elem {
PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) => {
if elem.redirection.is_some() {
current_type = Type::Any;
} else if let Expr::Call(call) = &elem.expr.expr {
let decl = working_set.get_decl(call.decl_id);
if current_type == Type::Any {
@ -961,18 +955,12 @@ pub fn check_pipeline_type(
if let Some(output_errors) = &mut output_errors {
output_errors.push(ParseError::InputMismatch(current_type, call.head))
} else {
output_errors =
Some(vec![ParseError::InputMismatch(current_type, call.head)]);
output_errors = Some(vec![ParseError::InputMismatch(current_type, call.head)]);
}
}
current_type = Type::Any;
}
PipelineElement::Expression(_, Expression { ty, .. }) => {
current_type = ty.clone();
}
_ => {
current_type = Type::Any;
}
} else {
current_type = elem.expr.ty.clone();
}
}
@ -1015,7 +1003,8 @@ pub fn check_block_input_output(working_set: &StateWorkingSet, block: &Block) ->
.elements
.last()
.expect("internal error: we should have elements")
.span()
.expr
.span
};
output_errors.push(ParseError::OutputMismatch(output_type.clone(), span))

546
crates/nu-parser/tests/test_parser.rs
View file

@ -1,10 +1,8 @@
use nu_parser::*;
use nu_protocol::ast::{Argument, Call, PathMember};
use nu_protocol::Span;
use nu_protocol::{
ast::{Expr, Expression, PipelineElement},
ast::{Argument, Call, Expr, PathMember},
engine::{Command, EngineState, Stack, StateWorkingSet},
ParseError, PipelineData, ShellError, Signature, SyntaxShape,
ParseError, PipelineData, ShellError, Signature, Span, SyntaxShape,
};
use rstest::rstest;
@ -73,21 +71,15 @@ fn test_int(
} else {
assert!(err.is_none(), "{test_tag}: unexpected error {err:#?}");
assert_eq!(block.len(), 1, "{test_tag}: result block length > 1");
let expressions = &block.pipelines[0];
let pipeline = &block.pipelines[0];
assert_eq!(
expressions.len(),
pipeline.len(),
1,
"{test_tag}: got multiple result expressions, expected 1"
);
if let PipelineElement::Expression(
_,
Expression {
expr: observed_val, ..
},
) = &expressions.elements[0]
{
compare_rhs_binary_op(test_tag, &expected_val, observed_val);
}
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
compare_rhs_binary_op(test_tag, &expected_val, &element.expr.expr);
}
}
@ -112,7 +104,7 @@ fn compare_rhs_binary_op(
"{test_tag}: Expected: {expected:#?}, observed: {observed:#?}"
)
}
Expr::ExternalCall(e, _, _) => {
Expr::ExternalCall(e, _) => {
let observed_expr = &e.expr;
assert_eq!(
expected, observed_expr,
@ -259,6 +251,7 @@ pub fn multi_test_parse_number() {
test_int(test.0, test.1, test.2, test.3);
}
}
#[ignore]
#[test]
fn test_parse_any() {
@ -277,6 +270,7 @@ fn test_parse_any() {
}
}
}
#[test]
pub fn parse_int() {
let engine_state = EngineState::new();
@ -286,18 +280,11 @@ pub fn parse_int() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
assert!(matches!(
expressions.elements[0],
PipelineElement::Expression(
_,
Expression {
expr: Expr::Int(3),
..
}
)
))
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::Int(3));
}
#[test]
@ -309,18 +296,11 @@ pub fn parse_int_with_underscores() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
assert!(matches!(
expressions.elements[0],
PipelineElement::Expression(
_,
Expression {
expr: Expr::Int(420692023),
..
}
)
))
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::Int(420692023));
}
#[test]
@ -339,19 +319,13 @@ pub fn parse_cell_path() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
// hoo boy this pattern matching is a pain
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
if let Expr::FullCellPath(b) = &expr.expr {
assert!(matches!(
b.head,
Expression {
expr: Expr::Var(_),
..
}
));
if let Expr::FullCellPath(b) = &element.expr.expr {
assert!(matches!(b.head.expr, Expr::Var(_)));
if let [a, b] = &b.tail[..] {
if let PathMember::String { val, optional, .. } = a {
assert_eq!(val, "bar");
@ -372,9 +346,6 @@ pub fn parse_cell_path() {
} else {
panic!("Not a cell path");
}
} else {
panic!("Not an expression")
}
}
#[test]
@ -394,19 +365,13 @@ pub fn parse_cell_path_optional() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
// hoo boy this pattern matching is a pain
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
if let Expr::FullCellPath(b) = &expr.expr {
assert!(matches!(
b.head,
Expression {
expr: Expr::Var(_),
..
}
));
if let Expr::FullCellPath(b) = &element.expr.expr {
assert!(matches!(b.head.expr, Expr::Var(_)));
if let [a, b] = &b.tail[..] {
if let PathMember::String { val, optional, .. } = a {
assert_eq!(val, "bar");
@ -427,9 +392,6 @@ pub fn parse_cell_path_optional() {
} else {
panic!("Not a cell path");
}
} else {
panic!("Not an expression")
}
}
#[test]
@ -441,13 +403,11 @@ pub fn parse_binary_with_hex_format() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert_eq!(expr.expr, Expr::Binary(vec![0x13]))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::Binary(vec![0x13]));
}
#[test]
@ -459,13 +419,11 @@ pub fn parse_binary_with_incomplete_hex_format() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert_eq!(expr.expr, Expr::Binary(vec![0x03]))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::Binary(vec![0x03]));
}
#[test]
@ -477,13 +435,11 @@ pub fn parse_binary_with_binary_format() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert_eq!(expr.expr, Expr::Binary(vec![0b10101000]))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::Binary(vec![0b10101000]));
}
#[test]
@ -495,13 +451,11 @@ pub fn parse_binary_with_incomplete_binary_format() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert_eq!(expr.expr, Expr::Binary(vec![0b00000010]))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::Binary(vec![0b00000010]));
}
#[test]
@ -513,13 +467,11 @@ pub fn parse_binary_with_octal_format() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert_eq!(expr.expr, Expr::Binary(vec![0o250]))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::Binary(vec![0o250]));
}
#[test]
@ -531,13 +483,11 @@ pub fn parse_binary_with_incomplete_octal_format() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert_eq!(expr.expr, Expr::Binary(vec![0o2]))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::Binary(vec![0o2]));
}
#[test]
@ -549,13 +499,11 @@ pub fn parse_binary_with_invalid_octal_format() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert!(!matches!(&expr.expr, Expr::Binary(_)))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert!(!matches!(element.expr.expr, Expr::Binary(_)));
}
#[test]
@ -569,13 +517,11 @@ pub fn parse_binary_with_multi_byte_char() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert!(!matches!(&expr.expr, Expr::Binary(_)))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert!(!matches!(element.expr.expr, Expr::Binary(_)))
}
#[test]
@ -591,17 +537,12 @@ pub fn parse_call() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
if let PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) = &expressions.elements[0]
{
if let Expr::Call(call) = &element.expr.expr {
assert_eq!(call.decl_id, 0);
}
}
@ -650,17 +591,12 @@ pub fn parse_call_short_flag_batch_arg_allowed() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
if let PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) = &expressions.elements[0]
{
if let Expr::Call(call) = &element.expr.expr {
assert_eq!(call.decl_id, 0);
assert_eq!(call.arguments.len(), 2);
matches!(call.arguments[0], Argument::Named((_, None, None)));
@ -767,42 +703,28 @@ fn test_nothing_comparison_eq() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
assert!(matches!(
&expressions.elements[0],
PipelineElement::Expression(
_,
Expression {
expr: Expr::BinaryOp(..),
..
}
)
))
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert!(matches!(&element.expr.expr, Expr::BinaryOp(..)));
}
#[rstest]
#[case(b"let a = 1 err> /dev/null", "RedirectionInLetMut")]
#[case(b"let a = 1 out> /dev/null", "RedirectionInLetMut")]
#[case(b"mut a = 1 err> /dev/null", "RedirectionInLetMut")]
#[case(b"mut a = 1 out> /dev/null", "RedirectionInLetMut")]
// This two cases cause AssignInPipeline instead of RedirectionInLetMut
#[case(b"let a = 1 out+err> /dev/null", "AssignInPipeline")]
#[case(b"mut a = 1 out+err> /dev/null", "AssignInPipeline")]
fn test_redirection_with_letmut(#[case] phase: &[u8], #[case] expected: &str) {
#[case(b"let a = 1 err> /dev/null")]
#[case(b"let a = 1 out> /dev/null")]
#[case(b"mut a = 1 err> /dev/null")]
#[case(b"mut a = 1 out> /dev/null")]
#[case(b"let a = 1 out+err> /dev/null")]
#[case(b"mut a = 1 out+err> /dev/null")]
fn test_redirection_with_letmut(#[case] phase: &[u8]) {
let engine_state = EngineState::new();
let mut working_set = StateWorkingSet::new(&engine_state);
let _block = parse(&mut working_set, None, phase, true);
match expected {
"RedirectionInLetMut" => assert!(matches!(
assert!(matches!(
working_set.parse_errors.first(),
Some(ParseError::RedirectionInLetMut(_, _))
)),
"AssignInPipeline" => assert!(matches!(
working_set.parse_errors.first(),
Some(ParseError::AssignInPipeline(_, _, _, _))
)),
_ => panic!("unexpected pattern"),
}
Some(ParseError::RedirectingBuiltinCommand(_, _, _))
));
}
#[test]
@ -814,18 +736,11 @@ fn test_nothing_comparison_neq() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
assert!(matches!(
&expressions.elements[0],
PipelineElement::Expression(
_,
Expression {
expr: Expr::BinaryOp(..),
..
}
)
))
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert!(matches!(&element.expr.expr, Expr::BinaryOp(..)));
}
mod string {
@ -840,13 +755,11 @@ mod string {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert_eq!(expr.expr, Expr::String("hello nushell".to_string()))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::String("hello nushell".to_string()))
}
mod interpolation {
@ -864,15 +777,15 @@ mod string {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
let subexprs: Vec<&Expr> = match expr {
Expression {
expr: Expr::StringInterpolation(expressions),
..
} => expressions.iter().map(|e| &e.expr).collect(),
let subexprs: Vec<&Expr> = match &element.expr.expr {
Expr::StringInterpolation(expressions) => {
expressions.iter().map(|e| &e.expr).collect()
}
_ => panic!("Expected an `Expr::StringInterpolation`"),
};
@ -881,9 +794,6 @@ mod string {
assert_eq!(subexprs[0], &Expr::String("hello ".to_string()));
assert!(matches!(subexprs[1], &Expr::FullCellPath(..)));
} else {
panic!("Not an expression")
}
}
#[test]
@ -896,25 +806,21 @@ mod string {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
let subexprs: Vec<&Expr> = match expr {
Expression {
expr: Expr::StringInterpolation(expressions),
..
} => expressions.iter().map(|e| &e.expr).collect(),
let subexprs: Vec<&Expr> = match &element.expr.expr {
Expr::StringInterpolation(expressions) => {
expressions.iter().map(|e| &e.expr).collect()
}
_ => panic!("Expected an `Expr::StringInterpolation`"),
};
assert_eq!(subexprs.len(), 1);
assert_eq!(subexprs[0], &Expr::String("hello (39 + 3)".to_string()));
} else {
panic!("Not an expression")
}
}
#[test]
@ -927,16 +833,15 @@ mod string {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
let subexprs: Vec<&Expr> = match expr {
Expression {
expr: Expr::StringInterpolation(expressions),
..
} => expressions.iter().map(|e| &e.expr).collect(),
let subexprs: Vec<&Expr> = match &element.expr.expr {
Expr::StringInterpolation(expressions) => {
expressions.iter().map(|e| &e.expr).collect()
}
_ => panic!("Expected an `Expr::StringInterpolation`"),
};
@ -945,9 +850,6 @@ mod string {
assert_eq!(subexprs[0], &Expr::String("hello \\".to_string()));
assert!(matches!(subexprs[1], &Expr::FullCellPath(..)));
} else {
panic!("Not an expression")
}
}
#[test]
@ -960,24 +862,20 @@ mod string {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
let subexprs: Vec<&Expr> = match expr {
Expression {
expr: Expr::StringInterpolation(expressions),
..
} => expressions.iter().map(|e| &e.expr).collect(),
let subexprs: Vec<&Expr> = match &element.expr.expr {
Expr::StringInterpolation(expressions) => {
expressions.iter().map(|e| &e.expr).collect()
}
_ => panic!("Expected an `Expr::StringInterpolation`"),
};
assert_eq!(subexprs.len(), 1);
assert_eq!(subexprs[0], &Expr::String("(1 + 3)(7 - 5)".to_string()));
} else {
panic!("Not an expression")
}
}
#[test]
@ -1084,13 +982,11 @@ mod range {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1, "{tag}: block length");
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1, "{tag}: expression length");
if let PipelineElement::Expression(
_,
Expression {
expr:
Expr::Range(
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1, "{tag}: expression length");
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
if let Expr::Range(
Some(_),
None,
Some(_),
@ -1098,10 +994,7 @@ mod range {
inclusion: the_inclusion,
..
},
),
..
},
) = expressions.elements[0]
) = element.expr.expr
{
assert_eq!(
the_inclusion, inclusion,
@ -1143,13 +1036,11 @@ mod range {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 2, "{tag} block len 2");
let expressions = &block.pipelines[1];
assert_eq!(expressions.len(), 1, "{tag}: expression length 1");
if let PipelineElement::Expression(
_,
Expression {
expr:
Expr::Range(
let pipeline = &block.pipelines[1];
assert_eq!(pipeline.len(), 1, "{tag}: expression length 1");
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
if let Expr::Range(
Some(_),
None,
Some(_),
@ -1157,10 +1048,7 @@ mod range {
inclusion: the_inclusion,
..
},
),
..
},
) = expressions.elements[0]
) = element.expr.expr
{
assert_eq!(
the_inclusion, inclusion,
@ -1189,13 +1077,11 @@ mod range {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1, "{tag}: block len 1");
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1, "{tag}: expression length 1");
if let PipelineElement::Expression(
_,
Expression {
expr:
Expr::Range(
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1, "{tag}: expression length");
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
if let Expr::Range(
Some(_),
None,
None,
@ -1203,10 +1089,7 @@ mod range {
inclusion: the_inclusion,
..
},
),
..
},
) = expressions.elements[0]
) = element.expr.expr
{
assert_eq!(
the_inclusion, inclusion,
@ -1235,13 +1118,11 @@ mod range {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1, "{tag}: block len 1");
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1, "{tag}: expression length 1");
if let PipelineElement::Expression(
_,
Expression {
expr:
Expr::Range(
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1, "{tag}: expression length");
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
if let Expr::Range(
None,
None,
Some(_),
@ -1249,10 +1130,7 @@ mod range {
inclusion: the_inclusion,
..
},
),
..
},
) = expressions.elements[0]
) = element.expr.expr
{
assert_eq!(
the_inclusion, inclusion,
@ -1281,13 +1159,11 @@ mod range {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1, "{tag}: block length 1");
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1, "{tag}: expression length 1");
if let PipelineElement::Expression(
_,
Expression {
expr:
Expr::Range(
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1, "{tag}: expression length");
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
if let Expr::Range(
Some(_),
Some(_),
Some(_),
@ -1295,10 +1171,7 @@ mod range {
inclusion: the_inclusion,
..
},
),
..
},
) = expressions.elements[0]
) = element.expr.expr
{
assert_eq!(
the_inclusion, inclusion,
@ -1671,31 +1544,21 @@ mod input_types {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 2);
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 2);
assert!(pipeline.elements[0].redirection.is_none());
assert!(pipeline.elements[1].redirection.is_none());
match &expressions.elements[0] {
PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) => {
match &pipeline.elements[0].expr.expr {
Expr::Call(call) => {
let expected_id = working_set.find_decl(b"ls").unwrap();
assert_eq!(call.decl_id, expected_id)
}
_ => panic!("Expected expression Call not found"),
}
match &expressions.elements[1] {
PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) => {
match &pipeline.elements[1].expr.expr {
Expr::Call(call) => {
let expected_id = working_set.find_decl(b"group-by").unwrap();
assert_eq!(call.decl_id, expected_id)
}
@ -1718,15 +1581,10 @@ mod input_types {
engine_state.merge_delta(delta).unwrap();
let expressions = &block.pipelines[0];
match &expressions.elements[3] {
PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) => {
let pipeline = &block.pipelines[0];
assert!(pipeline.elements[3].redirection.is_none());
match &pipeline.elements[3].expr.expr {
Expr::Call(call) => {
let arg = &call.arguments[0];
match arg {
Argument::Positional(a) => match &a.expr {
@ -1734,17 +1592,12 @@ mod input_types {
Expr::Subexpression(id) => {
let block = engine_state.get_block(*id);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 2);
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 2);
assert!(pipeline.elements[1].redirection.is_none());
match &expressions.elements[1] {
PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) => {
match &pipeline.elements[1].expr.expr {
Expr::Call(call) => {
let working_set = StateWorkingSet::new(&engine_state);
let expected_id = working_set.find_decl(b"min").unwrap();
assert_eq!(call.decl_id, expected_id)
@ -1776,29 +1629,20 @@ mod input_types {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
match &expressions.elements[2] {
PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) => {
let pipeline = &block.pipelines[0];
assert!(pipeline.elements[2].redirection.is_none());
assert!(pipeline.elements[3].redirection.is_none());
match &pipeline.elements[2].expr.expr {
Expr::Call(call) => {
let expected_id = working_set.find_decl(b"with-column").unwrap();
assert_eq!(call.decl_id, expected_id)
}
_ => panic!("Expected expression Call not found"),
}
match &expressions.elements[3] {
PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) => {
match &pipeline.elements[3].expr.expr {
Expr::Call(call) => {
let expected_id = working_set.find_decl(b"collect").unwrap();
assert_eq!(call.decl_id, expected_id)
}

18
crates/nu-parser/tests/test_parser_unicode_escapes.rs
View file

@ -1,13 +1,9 @@
#![cfg(test)]
//use nu_parser::ParseError;
use nu_parser::*;
use nu_protocol::{
//ast::{Expr, Expression, PipelineElement},
ast::{Expr, PipelineElement},
//engine::{Command, EngineState, Stack, StateWorkingSet},
ast::Expr,
engine::{EngineState, StateWorkingSet},
//Signature, SyntaxShape,
};
pub fn do_test(test: &[u8], expected: &str, error_contains: Option<&str>) {
@ -19,13 +15,11 @@ pub fn do_test(test: &[u8], expected: &str, error_contains: Option<&str>) {
match working_set.parse_errors.first() {
None => {
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert_eq!(expr.expr, Expr::String(expected.to_string()))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::String(expected.to_string()));
}
Some(pev) => match error_contains {
None => {

34
crates/nu-plugin/src/plugin/context.rs
View file

@ -6,8 +6,8 @@ use std::{
use nu_engine::get_eval_block_with_early_return;
use nu_protocol::{
ast::Call,
engine::{Closure, EngineState, Stack},
Config, IntoSpanned, PipelineData, PluginIdentity, ShellError, Span, Spanned, Value,
engine::{Closure, EngineState, Redirection, Stack},
Config, IntoSpanned, IoStream, PipelineData, PluginIdentity, ShellError, Span, Spanned, Value,
};
/// Object safe trait for abstracting operations required of the plugin context.
@ -107,8 +107,6 @@ impl PluginExecutionContext for PluginExecutionCommandContext {
&mut stack,
&block,
input,
false,
false,
) {
Ok(v) => v.into_value(span),
Err(e) => Value::error(e, self.call.head),
@ -155,7 +153,24 @@ impl PluginExecutionContext for PluginExecutionCommandContext {
inner: vec![],
})?;
let mut stack = self.stack.captures_to_stack(closure.item.captures);
let mut stack = self
.stack
.captures_to_stack(closure.item.captures)
.reset_pipes();
let stdout = if redirect_stdout {
Some(Redirection::Pipe(IoStream::Capture))
} else {
None
};
let stderr = if redirect_stderr {
Some(Redirection::Pipe(IoStream::Capture))
} else {
None
};
let stack = &mut stack.push_redirection(stdout, stderr);
// Set up the positional arguments
for (idx, value) in positional.into_iter().enumerate() {
@ -174,14 +189,7 @@ impl PluginExecutionContext for PluginExecutionCommandContext {
let eval_block_with_early_return = get_eval_block_with_early_return(&self.engine_state);
eval_block_with_early_return(
&self.engine_state,
&mut stack,
block,
input,
redirect_stdout,
redirect_stderr,
)
eval_block_with_early_return(&self.engine_state, stack, block, input)
}
}

1
crates/nu-plugin/src/plugin/declaration.rs
View file

@ -92,6 +92,7 @@ impl Command for PluginDeclaration {
// We need the current environment variables for `python` based plugins. Or
// we'll likely have a problem when a plugin is implemented in a virtual Python
// environment.
let stack = &mut stack.start_capture();
nu_engine::env::env_to_strings(engine_state, stack)
})
})

6
crates/nu-protocol/src/alias.rs
View file

@ -1,9 +1,7 @@
use crate::engine::{EngineState, Stack};
use crate::PipelineData;
use crate::{
ast::{Call, Expression},
engine::Command,
ShellError, Signature,
engine::{Command, EngineState, Stack},
PipelineData, ShellError, Signature,
};
#[derive(Clone)]

26
crates/nu-protocol/src/ast/block.rs
View file

@ -1,5 +1,5 @@
use super::Pipeline;
use crate::{ast::PipelineElement, Signature, Span, Type, VarId};
use crate::{engine::EngineState, IoStream, Signature, Span, Type, VarId};
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Serialize, Deserialize)]
@ -19,6 +19,17 @@ impl Block {
pub fn is_empty(&self) -> bool {
self.pipelines.is_empty()
}
pub fn stdio_redirect(
&self,
engine_state: &EngineState,
) -> (Option<IoStream>, Option<IoStream>) {
if let Some(first) = self.pipelines.first() {
first.stdio_redirect(engine_state)
} else {
(None, None)
}
}
}
impl Default for Block {
@ -51,15 +62,10 @@ impl Block {
pub fn output_type(&self) -> Type {
if let Some(last) = self.pipelines.last() {
if let Some(last) = last.elements.last() {
match last {
PipelineElement::Expression(_, expr) => expr.ty.clone(),
PipelineElement::ErrPipedExpression(_, expr) => expr.ty.clone(),
PipelineElement::OutErrPipedExpression(_, expr) => expr.ty.clone(),
PipelineElement::Redirection(_, _, _, _) => Type::Any,
PipelineElement::SeparateRedirection { .. } => Type::Any,
PipelineElement::SameTargetRedirection { .. } => Type::Any,
PipelineElement::And(_, expr) => expr.ty.clone(),
PipelineElement::Or(_, expr) => expr.ty.clone(),
if last.redirection.is_some() {
Type::Any
} else {
last.expr.ty.clone()
}
} else {
Type::Nothing

9
crates/nu-protocol/src/ast/call.rs
View file

@ -2,10 +2,9 @@ use std::collections::HashMap;
use serde::{Deserialize, Serialize};
use super::Expression;
use crate::{
engine::StateWorkingSet, eval_const::eval_constant, DeclId, FromValue, ShellError, Span,
Spanned, Value,
ast::Expression, engine::StateWorkingSet, eval_const::eval_constant, DeclId, FromValue,
ShellError, Span, Spanned, Value,
};
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
@ -51,8 +50,6 @@ pub struct Call {
pub decl_id: DeclId,
pub head: Span,
pub arguments: Vec<Argument>,
pub redirect_stdout: bool,
pub redirect_stderr: bool,
/// this field is used by the parser to pass additional command-specific information
pub parser_info: HashMap<String, Expression>,
}
@ -63,8 +60,6 @@ impl Call {
decl_id: 0,
head,
arguments: vec![],
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(),
}
}

74
crates/nu-protocol/src/ast/expr.rs
View file

@ -5,7 +5,10 @@ use super::{
Call, CellPath, Expression, ExternalArgument, FullCellPath, MatchPattern, Operator,
RangeOperator,
};
use crate::{ast::ImportPattern, ast::Unit, BlockId, Signature, Span, Spanned, VarId};
use crate::{
ast::ImportPattern, ast::Unit, engine::EngineState, BlockId, IoStream, Signature, Span,
Spanned, VarId,
};
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub enum Expr {
@ -22,7 +25,7 @@ pub enum Expr {
Var(VarId),
VarDecl(VarId),
Call(Box<Call>),
ExternalCall(Box<Expression>, Vec<ExternalArgument>, bool), // head, args, is_subexpression
ExternalCall(Box<Expression>, Vec<ExternalArgument>), // head, args
Operator(Operator),
RowCondition(BlockId),
UnaryNot(Box<Expression>),
@ -52,6 +55,73 @@ pub enum Expr {
Garbage,
}
impl Expr {
pub fn stdio_redirect(
&self,
engine_state: &EngineState,
) -> (Option<IoStream>, Option<IoStream>) {
// Usages of `$in` will be wrapped by a `collect` call by the parser,
// so we do not have to worry about that when considering
// which of the expressions below may consume pipeline output.
match self {
Expr::Call(call) => engine_state.get_decl(call.decl_id).stdio_redirect(),
Expr::Subexpression(block_id) | Expr::Block(block_id) => engine_state
.get_block(*block_id)
.stdio_redirect(engine_state),
Expr::FullCellPath(cell_path) => cell_path.head.expr.stdio_redirect(engine_state),
Expr::Bool(_)
| Expr::Int(_)
| Expr::Float(_)
| Expr::Binary(_)
| Expr::Range(_, _, _, _)
| Expr::Var(_)
| Expr::UnaryNot(_)
| Expr::BinaryOp(_, _, _)
| Expr::Closure(_) // piping into a closure value, not into a closure call
| Expr::List(_)
| Expr::Table(_, _)
| Expr::Record(_)
| Expr::ValueWithUnit(_, _)
| Expr::DateTime(_)
| Expr::String(_)
| Expr::CellPath(_)
| Expr::StringInterpolation(_)
| Expr::Nothing => {
// These expressions do not use the output of the pipeline in any meaningful way,
// so we can discard the previous output by redirecting it to `Null`.
(Some(IoStream::Null), None)
}
Expr::VarDecl(_)
| Expr::Operator(_)
| Expr::Filepath(_, _)
| Expr::Directory(_, _)
| Expr::GlobPattern(_, _)
| Expr::ImportPattern(_)
| Expr::Overlay(_)
| Expr::Signature(_)
| Expr::Spread(_)
| Expr::Garbage => {
// These should be impossible to pipe to,
// but even it is, the pipeline output is not used in any way.
(Some(IoStream::Null), None)
}
Expr::RowCondition(_) | Expr::MatchBlock(_) => {
// These should be impossible to pipe to,
// but if they are, then the pipeline output could be used.
(None, None)
}
Expr::ExternalCall(_, _) => {
// No override necessary, pipes will always be created in eval
(None, None)
}
Expr::Keyword(_, _, _) => {
// Not sure about this; let's return no redirection override for now.
(None, None)
}
}
}
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub enum RecordItem {
/// A key: val mapping

4
crates/nu-protocol/src/ast/expression.rs
View file

@ -184,7 +184,7 @@ impl Expression {
}
Expr::CellPath(_) => false,
Expr::DateTime(_) => false,
Expr::ExternalCall(head, args, _) => {
Expr::ExternalCall(head, args) => {
if head.has_in_variable(working_set) {
return true;
}
@ -369,7 +369,7 @@ impl Expression {
}
Expr::CellPath(_) => {}
Expr::DateTime(_) => {}
Expr::ExternalCall(head, args, _) => {
Expr::ExternalCall(head, args) => {
head.replace_span(working_set, replaced, new_span);
for ExternalArgument::Regular(expr) | ExternalArgument::Spread(expr) in args {
expr.replace_span(working_set, replaced, new_span);

209
crates/nu-protocol/src/ast/pipeline.rs
View file

@ -1,102 +1,58 @@
use crate::{ast::Expression, engine::StateWorkingSet, Span};
use crate::{
ast::Expression,
engine::{EngineState, StateWorkingSet},
IoStream, Span,
};
use serde::{Deserialize, Serialize};
use std::fmt::Display;
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq)]
pub enum Redirection {
#[derive(Debug, Clone, Copy, Serialize, Deserialize, Eq, PartialEq)]
pub enum RedirectionSource {
Stdout,
Stderr,
StdoutAndStderr,
}
// Note: Span in the below is for the span of the connector not the whole element
#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum PipelineElement {
Expression(Option<Span>, Expression),
ErrPipedExpression(Option<Span>, Expression),
OutErrPipedExpression(Option<Span>, Expression),
// final field indicates if it's in append mode
Redirection(Span, Redirection, Expression, bool),
// final bool field indicates if it's in append mode
SeparateRedirection {
out: (Span, Expression, bool),
err: (Span, Expression, bool),
},
// redirection's final bool field indicates if it's in append mode
SameTargetRedirection {
cmd: (Option<Span>, Expression),
redirection: (Span, Expression, bool),
},
And(Span, Expression),
Or(Span, Expression),
}
impl PipelineElement {
pub fn expression(&self) -> &Expression {
match self {
PipelineElement::Expression(_, expression)
| PipelineElement::ErrPipedExpression(_, expression)
| PipelineElement::OutErrPipedExpression(_, expression) => expression,
PipelineElement::Redirection(_, _, expression, _) => expression,
PipelineElement::SeparateRedirection {
out: (_, expression, _),
..
} => expression,
PipelineElement::SameTargetRedirection {
cmd: (_, expression),
..
} => expression,
PipelineElement::And(_, expression) => expression,
PipelineElement::Or(_, expression) => expression,
impl Display for RedirectionSource {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(match self {
RedirectionSource::Stdout => "stdout",
RedirectionSource::Stderr => "stderr",
RedirectionSource::StdoutAndStderr => "stdout and stderr",
})
}
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
pub enum RedirectionTarget {
File {
expr: Expression,
append: bool,
span: Span,
},
Pipe {
span: Span,
},
}
impl RedirectionTarget {
pub fn span(&self) -> Span {
match self {
PipelineElement::Expression(None, expression)
| PipelineElement::ErrPipedExpression(None, expression)
| PipelineElement::OutErrPipedExpression(None, expression)
| PipelineElement::SameTargetRedirection {
cmd: (None, expression),
..
} => expression.span,
PipelineElement::Expression(Some(span), expression)
| PipelineElement::ErrPipedExpression(Some(span), expression)
| PipelineElement::OutErrPipedExpression(Some(span), expression)
| PipelineElement::Redirection(span, _, expression, _)
| PipelineElement::SeparateRedirection {
out: (span, expression, _),
..
}
| PipelineElement::And(span, expression)
| PipelineElement::Or(span, expression)
| PipelineElement::SameTargetRedirection {
cmd: (Some(span), expression),
..
} => Span {
start: span.start,
end: expression.span.end,
},
RedirectionTarget::File { span, .. } | RedirectionTarget::Pipe { span } => *span,
}
}
pub fn has_in_variable(&self, working_set: &StateWorkingSet) -> bool {
pub fn expr(&self) -> Option<&Expression> {
match self {
PipelineElement::Expression(_, expression)
| PipelineElement::ErrPipedExpression(_, expression)
| PipelineElement::OutErrPipedExpression(_, expression)
| PipelineElement::Redirection(_, _, expression, _)
| PipelineElement::And(_, expression)
| PipelineElement::Or(_, expression)
| PipelineElement::SameTargetRedirection {
cmd: (_, expression),
..
} => expression.has_in_variable(working_set),
PipelineElement::SeparateRedirection {
out: (_, out_expr, _),
err: (_, err_expr, _),
} => out_expr.has_in_variable(working_set) || err_expr.has_in_variable(working_set),
RedirectionTarget::File { expr, .. } => Some(expr),
RedirectionTarget::Pipe { .. } => None,
}
}
pub fn has_in_variable(&self, working_set: &StateWorkingSet) -> bool {
self.expr().is_some_and(|e| e.has_in_variable(working_set))
}
pub fn replace_span(
&mut self,
working_set: &mut StateWorkingSet,
@ -104,24 +60,72 @@ impl PipelineElement {
new_span: Span,
) {
match self {
PipelineElement::Expression(_, expression)
| PipelineElement::ErrPipedExpression(_, expression)
| PipelineElement::OutErrPipedExpression(_, expression)
| PipelineElement::Redirection(_, _, expression, _)
| PipelineElement::And(_, expression)
| PipelineElement::Or(_, expression)
| PipelineElement::SameTargetRedirection {
cmd: (_, expression),
..
RedirectionTarget::File { expr, .. } => {
expr.replace_span(working_set, replaced, new_span)
}
| PipelineElement::SeparateRedirection {
out: (_, expression, _),
..
} => expression.replace_span(working_set, replaced, new_span),
RedirectionTarget::Pipe { .. } => {}
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
pub enum PipelineRedirection {
Single {
source: RedirectionSource,
target: RedirectionTarget,
},
Separate {
out: RedirectionTarget,
err: RedirectionTarget,
},
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct PipelineElement {
pub pipe: Option<Span>,
pub expr: Expression,
pub redirection: Option<PipelineRedirection>,
}
impl PipelineElement {
pub fn has_in_variable(&self, working_set: &StateWorkingSet) -> bool {
self.expr.has_in_variable(working_set)
|| self.redirection.as_ref().is_some_and(|r| match r {
PipelineRedirection::Single { target, .. } => target.has_in_variable(working_set),
PipelineRedirection::Separate { out, err } => {
out.has_in_variable(working_set) || err.has_in_variable(working_set)
}
})
}
pub fn replace_span(
&mut self,
working_set: &mut StateWorkingSet,
replaced: Span,
new_span: Span,
) {
self.expr.replace_span(working_set, replaced, new_span);
if let Some(expr) = self.redirection.as_mut() {
match expr {
PipelineRedirection::Single { target, .. } => {
target.replace_span(working_set, replaced, new_span)
}
PipelineRedirection::Separate { out, err } => {
out.replace_span(working_set, replaced, new_span);
err.replace_span(working_set, replaced, new_span);
}
}
}
}
pub fn stdio_redirect(
&self,
engine_state: &EngineState,
) -> (Option<IoStream>, Option<IoStream>) {
self.expr.expr.stdio_redirect(engine_state)
}
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Pipeline {
pub elements: Vec<PipelineElement>,
@ -143,8 +147,10 @@ impl Pipeline {
elements: expressions
.into_iter()
.enumerate()
.map(|(idx, x)| {
PipelineElement::Expression(if idx == 0 { None } else { Some(x.span) }, x)
.map(|(idx, expr)| PipelineElement {
pipe: if idx == 0 { None } else { Some(expr.span) },
expr,
redirection: None,
})
.collect(),
}
@ -157,4 +163,15 @@ impl Pipeline {
pub fn is_empty(&self) -> bool {
self.elements.is_empty()
}
pub fn stdio_redirect(
&self,
engine_state: &EngineState,
) -> (Option<IoStream>, Option<IoStream>) {
if let Some(first) = self.elements.first() {
first.stdio_redirect(engine_state)
} else {
(None, None)
}
}
}

13
crates/nu-protocol/src/debugger/profiler.rs
View file

@ -142,19 +142,14 @@ impl Debugger for Profiler {
};
let expr_opt = if self.collect_exprs {
Some(match element {
PipelineElement::Expression(_, expression) => {
expr_to_string(engine_state, &expression.expr)
}
_ => "other".to_string(),
})
Some(expr_to_string(engine_state, &element.expr.expr))
} else {
None
};
let new_id = ElementId(self.elements.len());
let mut new_element = ElementInfo::new(self.depth, element.span());
let mut new_element = ElementInfo::new(self.depth, element.expr.span);
new_element.expr = expr_opt;
self.elements.push(new_element);
@ -178,7 +173,7 @@ impl Debugger for Profiler {
return;
}
let element_span = element.span();
let element_span = element.expr.span;
let out_opt = if self.collect_values {
Some(match result {
@ -250,7 +245,7 @@ fn expr_to_string(engine_state: &EngineState, expr: &Expr) -> String {
Expr::Closure(_) => "closure".to_string(),
Expr::DateTime(_) => "datetime".to_string(),
Expr::Directory(_, _) => "directory".to_string(),
Expr::ExternalCall(_, _, _) => "external call".to_string(),
Expr::ExternalCall(_, _) => "external call".to_string(),
Expr::Filepath(_, _) => "filepath".to_string(),
Expr::Float(_) => "float".to_string(),
Expr::FullCellPath(full_cell_path) => {

6
crates/nu-protocol/src/engine/command.rs
View file

@ -1,4 +1,4 @@
use crate::{ast::Call, Alias, BlockId, Example, PipelineData, ShellError, Signature};
use crate::{ast::Call, Alias, BlockId, Example, IoStream, PipelineData, ShellError, Signature};
use super::{EngineState, Stack, StateWorkingSet};
@ -133,6 +133,10 @@ pub trait Command: Send + Sync + CommandClone {
_ => CommandType::Other,
}
}
fn stdio_redirect(&self) -> (Option<IoStream>, Option<IoStream>) {
(None, None)
}
}
pub trait CommandClone {

2
crates/nu-protocol/src/engine/mod.rs
View file

@ -7,6 +7,7 @@ mod pattern_match;
mod stack;
mod state_delta;
mod state_working_set;
mod stdio;
mod usage;
mod variable;
@ -19,4 +20,5 @@ pub use pattern_match::*;
pub use stack::*;
pub use state_delta::*;
pub use state_working_set::*;
pub use stdio::*;
pub use variable::*;

125
crates/nu-protocol/src/engine/stack.rs
View file

@ -1,10 +1,12 @@
use std::collections::{HashMap, HashSet};
use std::sync::Arc;
use crate::engine::EngineState;
use crate::engine::DEFAULT_OVERLAY_NAME;
use crate::{ShellError, Span, Value, VarId};
use crate::{ENV_VARIABLE_ID, NU_VARIABLE_ID};
use crate::{
engine::{EngineState, DEFAULT_OVERLAY_NAME},
IoStream, ShellError, Span, Value, VarId, ENV_VARIABLE_ID, NU_VARIABLE_ID,
};
use super::{Redirection, StackCallArgGuard, StackCaptureGuard, StackIoGuard, StackStdio};
/// Environment variables per overlay
pub type EnvVars = HashMap<String, HashMap<String, Value>>;
@ -37,22 +39,36 @@ pub struct Stack {
/// List of active overlays
pub active_overlays: Vec<String>,
pub recursion_count: u64,
pub parent_stack: Option<Arc<Stack>>,
/// Variables that have been deleted (this is used to hide values from parent stack lookups)
pub parent_deletions: Vec<VarId>,
pub(crate) stdio: StackStdio,
}
impl Default for Stack {
fn default() -> Self {
Self::new()
}
}
impl Stack {
pub fn new() -> Stack {
Stack {
vars: vec![],
env_vars: vec![],
/// Create a new stack.
///
/// Stdio will be set to [`IoStream::Inherit`]. So, if the last command is an external command,
/// then its output will be forwarded to the terminal/stdio streams.
///
/// Use [`Stack::capture`] afterwards if you need to evaluate an expression to a [`Value`](crate::Value)
/// (as opposed to a [`PipelineData`](crate::PipelineData)).
pub fn new() -> Self {
Self {
vars: Vec::new(),
env_vars: Vec::new(),
env_hidden: HashMap::new(),
active_overlays: vec![DEFAULT_OVERLAY_NAME.to_string()],
recursion_count: 0,
parent_stack: None,
parent_deletions: vec![],
stdio: StackStdio::new(),
}
}
@ -82,9 +98,10 @@ impl Stack {
env_hidden: parent.env_hidden.clone(),
active_overlays: parent.active_overlays.clone(),
recursion_count: parent.recursion_count,
parent_stack: Some(parent),
vars: vec![],
parent_deletions: vec![],
stdio: parent.stdio.clone(),
parent_stack: Some(parent),
}
}
@ -235,6 +252,10 @@ impl Stack {
}
pub fn captures_to_stack(&self, captures: Vec<(VarId, Value)>) -> Stack {
self.captures_to_stack_preserve_stdio(captures).capture()
}
pub fn captures_to_stack_preserve_stdio(&self, captures: Vec<(VarId, Value)>) -> Stack {
// FIXME: this is probably slow
let mut env_vars = self.env_vars.clone();
env_vars.push(HashMap::new());
@ -247,6 +268,7 @@ impl Stack {
recursion_count: self.recursion_count,
parent_stack: None,
parent_deletions: vec![],
stdio: self.stdio.clone(),
}
}
@ -276,6 +298,7 @@ impl Stack {
recursion_count: self.recursion_count,
parent_stack: None,
parent_deletions: vec![],
stdio: self.stdio.clone(),
}
}
@ -481,11 +504,87 @@ impl Stack {
pub fn remove_overlay(&mut self, name: &str) {
self.active_overlays.retain(|o| o != name);
}
/// Returns the [`IoStream`] to use for the current command's stdout.
///
/// This will be the pipe redirection if one is set,
/// otherwise it will be the current file redirection,
/// otherwise it will be the process's stdout indicated by [`IoStream::Inherit`].
pub fn stdout(&self) -> &IoStream {
self.stdio.stdout()
}
impl Default for Stack {
fn default() -> Self {
Self::new()
/// Returns the [`IoStream`] to use for the current command's stderr.
///
/// This will be the pipe redirection if one is set,
/// otherwise it will be the current file redirection,
/// otherwise it will be the process's stderr indicated by [`IoStream::Inherit`].
pub fn stderr(&self) -> &IoStream {
self.stdio.stderr()
}
/// Returns the [`IoStream`] to use for the last command's stdout.
pub fn pipe_stdout(&self) -> Option<&IoStream> {
self.stdio.pipe_stdout.as_ref()
}
/// Returns the [`IoStream`] to use for the last command's stderr.
pub fn pipe_stderr(&self) -> Option<&IoStream> {
self.stdio.pipe_stderr.as_ref()
}
/// Temporarily set the pipe stdout redirection to [`IoStream::Capture`].
///
/// This is used before evaluating an expression into a `Value`.
pub fn start_capture(&mut self) -> StackCaptureGuard {
StackCaptureGuard::new(self)
}
/// Temporarily use the stdio redirections in the parent scope.
///
/// This is used before evaluating an argument to a call.
pub fn use_call_arg_stdio(&mut self) -> StackCallArgGuard {
StackCallArgGuard::new(self)
}
/// Temporarily apply redirections to stdout and/or stderr.
pub fn push_redirection(
&mut self,
stdout: Option<Redirection>,
stderr: Option<Redirection>,
) -> StackIoGuard {
StackIoGuard::new(self, stdout, stderr)
}
/// Mark stdout for the last command as [`IoStream::Capture`].
///
/// This will irreversibly alter the stdio redirections, and so it only makes sense to use this on an owned `Stack`
/// (which is why this function does not take `&mut self`).
///
/// See [`Stack::start_capture`] which can temporarily set stdout as [`IoStream::Capture`] for a mutable `Stack` reference.
pub fn capture(mut self) -> Self {
self.stdio.pipe_stdout = Some(IoStream::Capture);
self.stdio.pipe_stderr = None;
self
}
/// Clears any pipe and file redirections and resets stdout and stderr to [`IoStream::Inherit`].
///
/// This will irreversibly reset the stdio redirections, and so it only makes sense to use this on an owned `Stack`
/// (which is why this function does not take `&mut self`).
pub fn reset_stdio(mut self) -> Self {
self.stdio = StackStdio::new();
self
}
/// Clears any pipe redirections, keeping the current stdout and stderr.
///
/// This will irreversibly reset some of the stdio redirections, and so it only makes sense to use this on an owned `Stack`
/// (which is why this function does not take `&mut self`).
pub fn reset_pipes(mut self) -> Self {
self.stdio.pipe_stdout = None;
self.stdio.pipe_stderr = None;
self
}
}

Some files were not shown because too many files have changed in this diff Show more