Mirrors/nushell
2
0
Fork 0
mirror of https://github.com/nushell/nushell synced 2025-01-09 11:49:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
nushell/crates/nu-errors/src/lib.rs

913 lines
31 KiB
Rust
Raw Normal View History

Produce ArgumentError for signature mismatch ArgumentError also automatically produces diagnostics
2019-06-30 06:14:40 +00:00
use ansi_term::Color;
Extract core stuff into own crates This commit extracts five new crates: - nu-source, which contains the core source-code handling logic in Nu, including Text, Span, and also the pretty.rs-based debug logic - nu-parser, which is the parser and expander logic - nu-protocol, which is the bulk of the types and basic conveniences used by plugins - nu-errors, which contains ShellError, ParseError and error handling conveniences - nu-textview, which is the textview plugin extracted into a crate One of the major consequences of this refactor is that it's no longer possible to `impl X for Spanned<Y>` outside of the `nu-source` crate, so a lot of types became more concrete (Value became a concrete type instead of Spanned<Value>, for example). This also turned a number of inherent methods in the main nu crate into plain functions (impl Value {} became a bunch of functions in the `value` namespace in `crate::data::value`).
2019-11-26 02:30:48 +00:00
use bigdecimal::BigDecimal;
objectshell initial commit
2019-05-10 16:59:12 +00:00
use derive_new::new;
Add pretty errors to commands
2019-06-07 22:35:07 +00:00
use language_reporting::{Diagnostic, Label, Severity};
Extract core stuff into own crates This commit extracts five new crates: - nu-source, which contains the core source-code handling logic in Nu, including Text, Span, and also the pretty.rs-based debug logic - nu-parser, which is the parser and expander logic - nu-protocol, which is the bulk of the types and basic conveniences used by plugins - nu-errors, which contains ShellError, ParseError and error handling conveniences - nu-textview, which is the textview plugin extracted into a crate One of the major consequences of this refactor is that it's no longer possible to `impl X for Spanned<Y>` outside of the `nu-source` crate, so a lot of types became more concrete (Value became a concrete type instead of Spanned<Value>, for example). This also turned a number of inherent methods in the main nu crate into plain functions (impl Value {} became a bunch of functions in the `value` namespace in `crate::data::value`).
2019-11-26 02:30:48 +00:00
use nu_source::{b, DebugDocBuilder, PrettyDebug, Span, Spanned, SpannedItem, TracableContext};
use num_bigint::BigInt;
use num_traits::ToPrimitive;
"Add plugin arg errors. Bring remaining errors to parity"
2019-07-13 16:59:59 +00:00
use serde::{Deserialize, Serialize};
Add support for ~ expansion This ended up being a bit of a yak shave. The basic idea in this commit is to expand `~` in paths, but only in paths. The way this is accomplished is by doing the expansion inside of the code that parses literal syntax for `SyntaxType::Path`. As a quick refresher: every command is entitled to expand its arguments in a custom way. While this could in theory be used for general-purpose macros, today the expansion facility is limited to syntactic hints. For example, the syntax `where cpu > 0` expands under the hood to `where { $it.cpu > 0 }`. This happens because the first argument to `where` is defined as a `SyntaxType::Block`, and the parser coerces binary expressions whose left-hand-side looks like a member into a block when the command is expecting one. This is mildly more magical than what most programming languages would do, but we believe that it makes sense to allow commands to fine-tune the syntax because of the domain nushell is in (command-line shells). The syntactic expansions supported by this facility are relatively limited. For example, we don't allow `$it` to become a bare word, simply because the command asks for a string in the relevant position. That would quickly become more confusing than it's worth. This PR adds a new `SyntaxType` rule: `SyntaxType::Path`. When a command declares a parameter as a `SyntaxType::Path`, string literals and bare words passed as an argument to that parameter are processed using the path expansion rules. Right now, that only means that `~` is expanded into the home directory, but additional rules are possible in the future. By restricting this expansion to a syntactic expansion when passed as an argument to a command expecting a path, we avoid making `~` a generally reserved character. This will also allow us to give good tab completion for paths with `~` characters in them when a command is expecting a path. In order to accomplish the above, this commit changes the parsing functions to take a `Context` instead of just a `CommandRegistry`. From the perspective of macro expansion, you can think of the `CommandRegistry` as a dictionary of in-scope macros, and the `Context` as the compile-time state used in expansion. This could gain additional functionality over time as we find more uses for the expansion system.
2019-08-26 19:21:03 +00:00
use std::fmt;
--no-edit
2019-11-04 15:47:03 +00:00
use std::ops::Range;
objectshell initial commit
2019-05-10 16:59:12 +00:00
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
/// A structured reason for a ParseError. Note that parsing in nu is more like macro expansion in
/// other languages, so the kinds of errors that can occur during parsing are more contextual than
/// you might expect.
Modernize external parse and improve trace The original purpose of this PR was to modernize the external parser to use the new Shape system. This commit does include some of that change, but a more important aspect of this change is an improvement to the expansion trace. Previous commit 6a7c00ea adding trace infrastructure to the syntax coloring feature. This commit adds tracing to the expander. The bulk of that work, in addition to the tree builder logic, was an overhaul of the formatter traits to make them more general purpose, and more structured. Some highlights: - `ToDebug` was split into two traits (`ToDebug` and `DebugFormat`) because implementations needed to become objects, but a convenience method on `ToDebug` didn't qualify - `DebugFormat`'s `fmt_debug` method now takes a `DebugFormatter` rather than a standard formatter, and `DebugFormatter` has a new (but still limited) facility for structured formatting. - Implementations of `ExpandSyntax` need to produce output that implements `DebugFormat`. Unlike the highlighter changes, these changes are fairly focused in the trace output, so these changes aren't behind a flag.
2019-10-28 14:46:50 +00:00
#[derive(Debug, Clone)]
pub enum ParseErrorReason {
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
/// The parser encountered an EOF rather than what it was expecting
Eof { expected: &'static str, span: Span },
Add Range and start Signature support This commit contains two improvements: - Support for a Range syntax (and a corresponding Range value) - Work towards a signature syntax Implementing the Range syntax resulted in cleaning up how operators in the core syntax works. There are now two kinds of infix operators - tight operators (`.` and `..`) - loose operators Tight operators may not be interspersed (`$it.left..$it.right` is a syntax error). Loose operators require whitespace on both sides of the operator, and can be arbitrarily interspersed. Precedence is left to right in the core syntax. Note that delimited syntax (like `( ... )` or `[ ... ]`) is a single token node in the core syntax. A single token node can be parsed from beginning to end in a context-free manner. The rule for `.` is `<token node>.<member>`. The rule for `..` is `<token node>..<token node>`. Loose operators all have the same syntactic rule: `<token node><space><loose op><space><token node>`. The second aspect of this pull request is the beginning of support for a signature syntax. Before implementing signatures, a necessary prerequisite is for the core syntax to support multi-line programs. That work establishes a few things: - `;` and newlines are handled in the core grammar, and both count as "separators" - line comments begin with `#` and continue until the end of the line In this commit, multi-token productions in the core grammar can use separators interchangably with spaces. However, I think we will ultimately want a different rule preventing separators from occurring before an infix operator, so that the end of a line is always unambiguous. This would avoid gratuitous differences between modules and repl usage. We already effectively have this rule, because otherwise `x<newline> | y` would be a single pipeline, but of course that wouldn't work.
2019-12-04 21:14:52 +00:00
/// The parser expected to see the end of a token stream (possibly the token
/// stream from inside a delimited token node), but found something else.
ExtraTokens { actual: Spanned<String> },
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
/// The parser encountered something other than what it was expecting
Modernize external parse and improve trace The original purpose of this PR was to modernize the external parser to use the new Shape system. This commit does include some of that change, but a more important aspect of this change is an improvement to the expansion trace. Previous commit 6a7c00ea adding trace infrastructure to the syntax coloring feature. This commit adds tracing to the expander. The bulk of that work, in addition to the tree builder logic, was an overhaul of the formatter traits to make them more general purpose, and more structured. Some highlights: - `ToDebug` was split into two traits (`ToDebug` and `DebugFormat`) because implementations needed to become objects, but a convenience method on `ToDebug` didn't qualify - `DebugFormat`'s `fmt_debug` method now takes a `DebugFormatter` rather than a standard formatter, and `DebugFormatter` has a new (but still limited) facility for structured formatting. - Implementations of `ExpandSyntax` need to produce output that implements `DebugFormat`. Unlike the highlighter changes, these changes are fairly focused in the trace output, so these changes aren't behind a flag.
2019-10-28 14:46:50 +00:00
Mismatch {
expected: &'static str,
--no-edit
2019-11-04 15:47:03 +00:00
actual: Spanned<String>,
Modernize external parse and improve trace The original purpose of this PR was to modernize the external parser to use the new Shape system. This commit does include some of that change, but a more important aspect of this change is an improvement to the expansion trace. Previous commit 6a7c00ea adding trace infrastructure to the syntax coloring feature. This commit adds tracing to the expander. The bulk of that work, in addition to the tree builder logic, was an overhaul of the formatter traits to make them more general purpose, and more structured. Some highlights: - `ToDebug` was split into two traits (`ToDebug` and `DebugFormat`) because implementations needed to become objects, but a convenience method on `ToDebug` didn't qualify - `DebugFormat`'s `fmt_debug` method now takes a `DebugFormatter` rather than a standard formatter, and `DebugFormatter` has a new (but still limited) facility for structured formatting. - Implementations of `ExpandSyntax` need to produce output that implements `DebugFormat`. Unlike the highlighter changes, these changes are fairly focused in the trace output, so these changes aren't behind a flag.
2019-10-28 14:46:50 +00:00
},
Another batch of un-unwrapping (#1148) Another batch of un-unwrappings
2020-01-02 04:02:46 +00:00
/// An unexpected internal error has occurred
InternalError { message: Spanned<String> },
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
/// The parser tried to parse an argument for a command, but it failed for
/// some reason
Modernize external parse and improve trace The original purpose of this PR was to modernize the external parser to use the new Shape system. This commit does include some of that change, but a more important aspect of this change is an improvement to the expansion trace. Previous commit 6a7c00ea adding trace infrastructure to the syntax coloring feature. This commit adds tracing to the expander. The bulk of that work, in addition to the tree builder logic, was an overhaul of the formatter traits to make them more general purpose, and more structured. Some highlights: - `ToDebug` was split into two traits (`ToDebug` and `DebugFormat`) because implementations needed to become objects, but a convenience method on `ToDebug` didn't qualify - `DebugFormat`'s `fmt_debug` method now takes a `DebugFormatter` rather than a standard formatter, and `DebugFormatter` has a new (but still limited) facility for structured formatting. - Implementations of `ExpandSyntax` need to produce output that implements `DebugFormat`. Unlike the highlighter changes, these changes are fairly focused in the trace output, so these changes aren't behind a flag.
2019-10-28 14:46:50 +00:00
ArgumentError {
--no-edit
2019-11-04 15:47:03 +00:00
command: Spanned<String>,
Modernize external parse and improve trace The original purpose of this PR was to modernize the external parser to use the new Shape system. This commit does include some of that change, but a more important aspect of this change is an improvement to the expansion trace. Previous commit 6a7c00ea adding trace infrastructure to the syntax coloring feature. This commit adds tracing to the expander. The bulk of that work, in addition to the tree builder logic, was an overhaul of the formatter traits to make them more general purpose, and more structured. Some highlights: - `ToDebug` was split into two traits (`ToDebug` and `DebugFormat`) because implementations needed to become objects, but a convenience method on `ToDebug` didn't qualify - `DebugFormat`'s `fmt_debug` method now takes a `DebugFormatter` rather than a standard formatter, and `DebugFormatter` has a new (but still limited) facility for structured formatting. - Implementations of `ExpandSyntax` need to produce output that implements `DebugFormat`. Unlike the highlighter changes, these changes are fairly focused in the trace output, so these changes aren't behind a flag.
2019-10-28 14:46:50 +00:00
error: ArgumentError,
},
}
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
/// A newtype for `ParseErrorReason`
Modernize external parse and improve trace The original purpose of this PR was to modernize the external parser to use the new Shape system. This commit does include some of that change, but a more important aspect of this change is an improvement to the expansion trace. Previous commit 6a7c00ea adding trace infrastructure to the syntax coloring feature. This commit adds tracing to the expander. The bulk of that work, in addition to the tree builder logic, was an overhaul of the formatter traits to make them more general purpose, and more structured. Some highlights: - `ToDebug` was split into two traits (`ToDebug` and `DebugFormat`) because implementations needed to become objects, but a convenience method on `ToDebug` didn't qualify - `DebugFormat`'s `fmt_debug` method now takes a `DebugFormatter` rather than a standard formatter, and `DebugFormatter` has a new (but still limited) facility for structured formatting. - Implementations of `ExpandSyntax` need to produce output that implements `DebugFormat`. Unlike the highlighter changes, these changes are fairly focused in the trace output, so these changes aren't behind a flag.
2019-10-28 14:46:50 +00:00
#[derive(Debug, Clone)]
pub struct ParseError {
reason: ParseErrorReason,
}
impl ParseError {
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
/// Construct a [ParseErrorReason::Eof](ParseErrorReason::Eof)
Modernize external parse and improve trace The original purpose of this PR was to modernize the external parser to use the new Shape system. This commit does include some of that change, but a more important aspect of this change is an improvement to the expansion trace. Previous commit 6a7c00ea adding trace infrastructure to the syntax coloring feature. This commit adds tracing to the expander. The bulk of that work, in addition to the tree builder logic, was an overhaul of the formatter traits to make them more general purpose, and more structured. Some highlights: - `ToDebug` was split into two traits (`ToDebug` and `DebugFormat`) because implementations needed to become objects, but a convenience method on `ToDebug` didn't qualify - `DebugFormat`'s `fmt_debug` method now takes a `DebugFormatter` rather than a standard formatter, and `DebugFormatter` has a new (but still limited) facility for structured formatting. - Implementations of `ExpandSyntax` need to produce output that implements `DebugFormat`. Unlike the highlighter changes, these changes are fairly focused in the trace output, so these changes aren't behind a flag.
2019-10-28 14:46:50 +00:00
pub fn unexpected_eof(expected: &'static str, span: Span) -> ParseError {
ParseError {
--no-edit
2019-11-04 15:47:03 +00:00
reason: ParseErrorReason::Eof { expected, span },
Modernize external parse and improve trace The original purpose of this PR was to modernize the external parser to use the new Shape system. This commit does include some of that change, but a more important aspect of this change is an improvement to the expansion trace. Previous commit 6a7c00ea adding trace infrastructure to the syntax coloring feature. This commit adds tracing to the expander. The bulk of that work, in addition to the tree builder logic, was an overhaul of the formatter traits to make them more general purpose, and more structured. Some highlights: - `ToDebug` was split into two traits (`ToDebug` and `DebugFormat`) because implementations needed to become objects, but a convenience method on `ToDebug` didn't qualify - `DebugFormat`'s `fmt_debug` method now takes a `DebugFormatter` rather than a standard formatter, and `DebugFormatter` has a new (but still limited) facility for structured formatting. - Implementations of `ExpandSyntax` need to produce output that implements `DebugFormat`. Unlike the highlighter changes, these changes are fairly focused in the trace output, so these changes aren't behind a flag.
2019-10-28 14:46:50 +00:00
}
}
Add Range and start Signature support This commit contains two improvements: - Support for a Range syntax (and a corresponding Range value) - Work towards a signature syntax Implementing the Range syntax resulted in cleaning up how operators in the core syntax works. There are now two kinds of infix operators - tight operators (`.` and `..`) - loose operators Tight operators may not be interspersed (`$it.left..$it.right` is a syntax error). Loose operators require whitespace on both sides of the operator, and can be arbitrarily interspersed. Precedence is left to right in the core syntax. Note that delimited syntax (like `( ... )` or `[ ... ]`) is a single token node in the core syntax. A single token node can be parsed from beginning to end in a context-free manner. The rule for `.` is `<token node>.<member>`. The rule for `..` is `<token node>..<token node>`. Loose operators all have the same syntactic rule: `<token node><space><loose op><space><token node>`. The second aspect of this pull request is the beginning of support for a signature syntax. Before implementing signatures, a necessary prerequisite is for the core syntax to support multi-line programs. That work establishes a few things: - `;` and newlines are handled in the core grammar, and both count as "separators" - line comments begin with `#` and continue until the end of the line In this commit, multi-token productions in the core grammar can use separators interchangably with spaces. However, I think we will ultimately want a different rule preventing separators from occurring before an infix operator, so that the end of a line is always unambiguous. This would avoid gratuitous differences between modules and repl usage. We already effectively have this rule, because otherwise `x<newline> | y` would be a single pipeline, but of course that wouldn't work.
2019-12-04 21:14:52 +00:00
/// Construct a [ParseErrorReason::ExtraTokens](ParseErrorReason::ExtraTokens)
pub fn extra_tokens(actual: Spanned<impl Into<String>>) -> ParseError {
let Spanned { span, item } = actual;
ParseError {
reason: ParseErrorReason::ExtraTokens {
actual: item.into().spanned(span),
},
}
}
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
/// Construct a [ParseErrorReason::Mismatch](ParseErrorReason::Mismatch)
--no-edit
2019-11-04 15:47:03 +00:00
pub fn mismatch(expected: &'static str, actual: Spanned<impl Into<String>>) -> ParseError {
let Spanned { span, item } = actual;
Modernize external parse and improve trace The original purpose of this PR was to modernize the external parser to use the new Shape system. This commit does include some of that change, but a more important aspect of this change is an improvement to the expansion trace. Previous commit 6a7c00ea adding trace infrastructure to the syntax coloring feature. This commit adds tracing to the expander. The bulk of that work, in addition to the tree builder logic, was an overhaul of the formatter traits to make them more general purpose, and more structured. Some highlights: - `ToDebug` was split into two traits (`ToDebug` and `DebugFormat`) because implementations needed to become objects, but a convenience method on `ToDebug` didn't qualify - `DebugFormat`'s `fmt_debug` method now takes a `DebugFormatter` rather than a standard formatter, and `DebugFormatter` has a new (but still limited) facility for structured formatting. - Implementations of `ExpandSyntax` need to produce output that implements `DebugFormat`. Unlike the highlighter changes, these changes are fairly focused in the trace output, so these changes aren't behind a flag.
2019-10-28 14:46:50 +00:00
ParseError {
reason: ParseErrorReason::Mismatch {
expected,
--no-edit
2019-11-04 15:47:03 +00:00
actual: item.into().spanned(span),
Modernize external parse and improve trace The original purpose of this PR was to modernize the external parser to use the new Shape system. This commit does include some of that change, but a more important aspect of this change is an improvement to the expansion trace. Previous commit 6a7c00ea adding trace infrastructure to the syntax coloring feature. This commit adds tracing to the expander. The bulk of that work, in addition to the tree builder logic, was an overhaul of the formatter traits to make them more general purpose, and more structured. Some highlights: - `ToDebug` was split into two traits (`ToDebug` and `DebugFormat`) because implementations needed to become objects, but a convenience method on `ToDebug` didn't qualify - `DebugFormat`'s `fmt_debug` method now takes a `DebugFormatter` rather than a standard formatter, and `DebugFormatter` has a new (but still limited) facility for structured formatting. - Implementations of `ExpandSyntax` need to produce output that implements `DebugFormat`. Unlike the highlighter changes, these changes are fairly focused in the trace output, so these changes aren't behind a flag.
2019-10-28 14:46:50 +00:00
},
}
}
Another batch of un-unwrapping (#1148) Another batch of un-unwrappings
2020-01-02 04:02:46 +00:00
/// Construct a [ParseErrorReason::InternalError](ParseErrorReason::InternalError)
pub fn internal_error(message: Spanned<impl Into<String>>) -> ParseError {
ParseError {
reason: ParseErrorReason::InternalError {
message: message.item.into().spanned(message.span),
},
}
}
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
/// Construct a [ParseErrorReason::ArgumentError](ParseErrorReason::ArgumentError)
--no-edit
2019-11-04 15:47:03 +00:00
pub fn argument_error(command: Spanned<impl Into<String>>, kind: ArgumentError) -> ParseError {
Modernize external parse and improve trace The original purpose of this PR was to modernize the external parser to use the new Shape system. This commit does include some of that change, but a more important aspect of this change is an improvement to the expansion trace. Previous commit 6a7c00ea adding trace infrastructure to the syntax coloring feature. This commit adds tracing to the expander. The bulk of that work, in addition to the tree builder logic, was an overhaul of the formatter traits to make them more general purpose, and more structured. Some highlights: - `ToDebug` was split into two traits (`ToDebug` and `DebugFormat`) because implementations needed to become objects, but a convenience method on `ToDebug` didn't qualify - `DebugFormat`'s `fmt_debug` method now takes a `DebugFormatter` rather than a standard formatter, and `DebugFormatter` has a new (but still limited) facility for structured formatting. - Implementations of `ExpandSyntax` need to produce output that implements `DebugFormat`. Unlike the highlighter changes, these changes are fairly focused in the trace output, so these changes aren't behind a flag.
2019-10-28 14:46:50 +00:00
ParseError {
reason: ParseErrorReason::ArgumentError {
--no-edit
2019-11-04 15:47:03 +00:00
command: command.item.into().spanned(command.span),
Modernize external parse and improve trace The original purpose of this PR was to modernize the external parser to use the new Shape system. This commit does include some of that change, but a more important aspect of this change is an improvement to the expansion trace. Previous commit 6a7c00ea adding trace infrastructure to the syntax coloring feature. This commit adds tracing to the expander. The bulk of that work, in addition to the tree builder logic, was an overhaul of the formatter traits to make them more general purpose, and more structured. Some highlights: - `ToDebug` was split into two traits (`ToDebug` and `DebugFormat`) because implementations needed to become objects, but a convenience method on `ToDebug` didn't qualify - `DebugFormat`'s `fmt_debug` method now takes a `DebugFormatter` rather than a standard formatter, and `DebugFormatter` has a new (but still limited) facility for structured formatting. - Implementations of `ExpandSyntax` need to produce output that implements `DebugFormat`. Unlike the highlighter changes, these changes are fairly focused in the trace output, so these changes aren't behind a flag.
2019-10-28 14:46:50 +00:00
error: kind,
},
}
}
}
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
/// Convert a [ParseError](ParseError) into a [ShellError](ShellError)
Modernize external parse and improve trace The original purpose of this PR was to modernize the external parser to use the new Shape system. This commit does include some of that change, but a more important aspect of this change is an improvement to the expansion trace. Previous commit 6a7c00ea adding trace infrastructure to the syntax coloring feature. This commit adds tracing to the expander. The bulk of that work, in addition to the tree builder logic, was an overhaul of the formatter traits to make them more general purpose, and more structured. Some highlights: - `ToDebug` was split into two traits (`ToDebug` and `DebugFormat`) because implementations needed to become objects, but a convenience method on `ToDebug` didn't qualify - `DebugFormat`'s `fmt_debug` method now takes a `DebugFormatter` rather than a standard formatter, and `DebugFormatter` has a new (but still limited) facility for structured formatting. - Implementations of `ExpandSyntax` need to produce output that implements `DebugFormat`. Unlike the highlighter changes, these changes are fairly focused in the trace output, so these changes aren't behind a flag.
2019-10-28 14:46:50 +00:00
impl From<ParseError> for ShellError {
fn from(error: ParseError) -> ShellError {
match error.reason {
--no-edit
2019-11-04 15:47:03 +00:00
ParseErrorReason::Eof { expected, span } => ShellError::unexpected_eof(expected, span),
Move to using clippy (#1142) * Clippy fixes * Finish converting to use clippy * fix warnings in new master * fix windows * fix windows Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
2019-12-31 07:36:08 +00:00
ParseErrorReason::ExtraTokens { actual } => ShellError::type_error("nothing", actual),
Modernize external parse and improve trace The original purpose of this PR was to modernize the external parser to use the new Shape system. This commit does include some of that change, but a more important aspect of this change is an improvement to the expansion trace. Previous commit 6a7c00ea adding trace infrastructure to the syntax coloring feature. This commit adds tracing to the expander. The bulk of that work, in addition to the tree builder logic, was an overhaul of the formatter traits to make them more general purpose, and more structured. Some highlights: - `ToDebug` was split into two traits (`ToDebug` and `DebugFormat`) because implementations needed to become objects, but a convenience method on `ToDebug` didn't qualify - `DebugFormat`'s `fmt_debug` method now takes a `DebugFormatter` rather than a standard formatter, and `DebugFormatter` has a new (but still limited) facility for structured formatting. - Implementations of `ExpandSyntax` need to produce output that implements `DebugFormat`. Unlike the highlighter changes, these changes are fairly focused in the trace output, so these changes aren't behind a flag.
2019-10-28 14:46:50 +00:00
ParseErrorReason::Mismatch { actual, expected } => {
Move to using clippy (#1142) * Clippy fixes * Finish converting to use clippy * fix warnings in new master * fix windows * fix windows Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
2019-12-31 07:36:08 +00:00
ShellError::type_error(expected, actual)
Modernize external parse and improve trace The original purpose of this PR was to modernize the external parser to use the new Shape system. This commit does include some of that change, but a more important aspect of this change is an improvement to the expansion trace. Previous commit 6a7c00ea adding trace infrastructure to the syntax coloring feature. This commit adds tracing to the expander. The bulk of that work, in addition to the tree builder logic, was an overhaul of the formatter traits to make them more general purpose, and more structured. Some highlights: - `ToDebug` was split into two traits (`ToDebug` and `DebugFormat`) because implementations needed to become objects, but a convenience method on `ToDebug` didn't qualify - `DebugFormat`'s `fmt_debug` method now takes a `DebugFormatter` rather than a standard formatter, and `DebugFormatter` has a new (but still limited) facility for structured formatting. - Implementations of `ExpandSyntax` need to produce output that implements `DebugFormat`. Unlike the highlighter changes, these changes are fairly focused in the trace output, so these changes aren't behind a flag.
2019-10-28 14:46:50 +00:00
}
Another batch of un-unwrapping (#1148) Another batch of un-unwrappings
2020-01-02 04:02:46 +00:00
ParseErrorReason::InternalError { message } => ShellError::labeled_error(
format!("Internal error: {}", message.item),
&message.item,
&message.span,
),
--no-edit
2019-11-04 15:47:03 +00:00
ParseErrorReason::ArgumentError { command, error } => {
ShellError::argument_error(command, error)
}
Modernize external parse and improve trace The original purpose of this PR was to modernize the external parser to use the new Shape system. This commit does include some of that change, but a more important aspect of this change is an improvement to the expansion trace. Previous commit 6a7c00ea adding trace infrastructure to the syntax coloring feature. This commit adds tracing to the expander. The bulk of that work, in addition to the tree builder logic, was an overhaul of the formatter traits to make them more general purpose, and more structured. Some highlights: - `ToDebug` was split into two traits (`ToDebug` and `DebugFormat`) because implementations needed to become objects, but a convenience method on `ToDebug` didn't qualify - `DebugFormat`'s `fmt_debug` method now takes a `DebugFormatter` rather than a standard formatter, and `DebugFormatter` has a new (but still limited) facility for structured formatting. - Implementations of `ExpandSyntax` need to produce output that implements `DebugFormat`. Unlike the highlighter changes, these changes are fairly focused in the trace output, so these changes aren't behind a flag.
2019-10-28 14:46:50 +00:00
}
}
}
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
/// ArgumentError describes various ways that the parser could fail because of unexpected arguments.
/// Nu commands are like a combination of functions and macros, and these errors correspond to
/// problems that could be identified during expansion based on the syntactic signature of a
/// command.
--no-edit
2019-11-04 15:47:03 +00:00
#[derive(Debug, Eq, PartialEq, Clone, Ord, Hash, PartialOrd, Serialize, Deserialize)]
Produce ArgumentError for signature mismatch ArgumentError also automatically produces diagnostics
2019-06-30 06:14:40 +00:00
pub enum ArgumentError {
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
/// The command specified a mandatory flag, but it was missing.
Produce ArgumentError for signature mismatch ArgumentError also automatically produces diagnostics
2019-06-30 06:14:40 +00:00
MissingMandatoryFlag(String),
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
/// The command specified a mandatory positional argument, but it was missing.
Produce ArgumentError for signature mismatch ArgumentError also automatically produces diagnostics
2019-06-30 06:14:40 +00:00
MissingMandatoryPositional(String),
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
/// A flag was found, and it should have been followed by a value, but no value was found
Produce ArgumentError for signature mismatch ArgumentError also automatically produces diagnostics
2019-06-30 06:14:40 +00:00
MissingValueForName(String),
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
/// A sequence of characters was found that was not syntactically valid (but would have
/// been valid if the command was an external command)
Close a bunch of holes in external command args Previously, there was a single parsing rule for "bare words" that applied to both internal and external commands. This meant that, because `cargo +nightly` needed to work, we needed to add `+` as a valid character in bare words. The number of characters continued to grow, and the situation was becoming untenable. The current strategy would eventually eat up all syntax and make it impossible to add syntax like `@foo` to internal commands. This patch significantly restricts bare words and introduces a new token type (`ExternalWord`). An `ExternalWord` expands to an error in the internal syntax, but expands to a bare word in the external syntax. `ExternalWords` are highlighted in grey in the shell.
2019-09-09 17:43:10 +00:00
InvalidExternalWord,
Produce ArgumentError for signature mismatch ArgumentError also automatically produces diagnostics
2019-06-30 06:14:40 +00:00
}
Remove unused Display implementations After the previous commit, nushell uses PrettyDebug and PrettyDebugWithSource for our pretty-printed display output. PrettyDebug produces a structured `pretty.rs` document rather than writing directly into a fmt::Formatter, and types that implement `PrettyDebug` have a convenience `display` method that produces a string (to be used in situations where `Display` is needed for compatibility with other traits, or where simple rendering is appropriate).
2019-11-25 18:07:20 +00:00
impl PrettyDebug for ArgumentError {
fn pretty(&self) -> DebugDocBuilder {
match self {
ArgumentError::MissingMandatoryFlag(flag) => {
b::description("missing `")
+ b::description(flag)
+ b::description("` as mandatory flag")
}
ArgumentError::MissingMandatoryPositional(pos) => {
b::description("missing `")
+ b::description(pos)
+ b::description("` as mandatory positional argument")
}
ArgumentError::MissingValueForName(name) => {
b::description("missing value for flag `")
+ b::description(name)
+ b::description("`")
}
ArgumentError::InvalidExternalWord => b::description("invalid word"),
}
}
}
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
/// A `ShellError` is a proximate error and a possible cause, which could have its own cause,
/// creating a cause chain.
--no-edit
2019-11-04 15:47:03 +00:00
#[derive(Debug, Eq, PartialEq, Ord, PartialOrd, Clone, Serialize, Deserialize, Hash)]
Tests pass
2019-07-09 04:31:26 +00:00
pub struct ShellError {
error: ProximateShellError,
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
cause: Option<Box<ShellError>>,
A bunch of rework I'm gonna use a VecDeque now instead of trying to get async streams working to make progress, but the intent is that we should be able to use async streams in and out to interleave the work better.
2019-05-13 17:30:51 +00:00
}
Move to using clippy (#1142) * Clippy fixes * Finish converting to use clippy * fix warnings in new master * fix windows * fix windows Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
2019-12-31 07:36:08 +00:00
/// `PrettyDebug` is for internal debugging. For user-facing debugging, [into_diagnostic](ShellError::into_diagnostic)
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
/// is used, which prints an error, highlighting spans.
Remove unused Display implementations After the previous commit, nushell uses PrettyDebug and PrettyDebugWithSource for our pretty-printed display output. PrettyDebug produces a structured `pretty.rs` document rather than writing directly into a fmt::Formatter, and types that implement `PrettyDebug` have a convenience `display` method that produces a string (to be used in situations where `Display` is needed for compatibility with other traits, or where simple rendering is appropriate).
2019-11-25 18:07:20 +00:00
impl PrettyDebug for ShellError {
fn pretty(&self) -> DebugDocBuilder {
match &self.error {
ProximateShellError::SyntaxError { problem } => {
b::error("Syntax Error")
+ b::space()
+ b::delimit("(", b::description(&problem.item), ")")
}
ProximateShellError::UnexpectedEof { .. } => b::error("Unexpected end"),
ProximateShellError::TypeError { expected, actual } => {
b::error("Type Error")
+ b::space()
+ b::delimit(
"(",
b::description("expected:")
+ b::space()
+ b::description(expected)
+ b::description(",")
+ b::space()
+ b::description("actual:")
+ b::space()
Move to using clippy (#1142) * Clippy fixes * Finish converting to use clippy * fix warnings in new master * fix windows * fix windows Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
2019-12-31 07:36:08 +00:00
+ b::option(actual.item.as_ref().map(b::description)),
Remove unused Display implementations After the previous commit, nushell uses PrettyDebug and PrettyDebugWithSource for our pretty-printed display output. PrettyDebug produces a structured `pretty.rs` document rather than writing directly into a fmt::Formatter, and types that implement `PrettyDebug` have a convenience `display` method that produces a string (to be used in situations where `Display` is needed for compatibility with other traits, or where simple rendering is appropriate).
2019-11-25 18:07:20 +00:00
")",
)
}
ProximateShellError::MissingProperty { subpath, expr } => {
b::error("Missing Property")
+ b::space()
+ b::delimit(
"(",
b::description("expr:")
+ b::space()
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
+ b::description(&expr.item)
Remove unused Display implementations After the previous commit, nushell uses PrettyDebug and PrettyDebugWithSource for our pretty-printed display output. PrettyDebug produces a structured `pretty.rs` document rather than writing directly into a fmt::Formatter, and types that implement `PrettyDebug` have a convenience `display` method that produces a string (to be used in situations where `Display` is needed for compatibility with other traits, or where simple rendering is appropriate).
2019-11-25 18:07:20 +00:00
+ b::description(",")
+ b::space()
+ b::description("subpath:")
+ b::space()
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
+ b::description(&subpath.item),
Remove unused Display implementations After the previous commit, nushell uses PrettyDebug and PrettyDebugWithSource for our pretty-printed display output. PrettyDebug produces a structured `pretty.rs` document rather than writing directly into a fmt::Formatter, and types that implement `PrettyDebug` have a convenience `display` method that produces a string (to be used in situations where `Display` is needed for compatibility with other traits, or where simple rendering is appropriate).
2019-11-25 18:07:20 +00:00
")",
)
}
ProximateShellError::InvalidIntegerIndex { subpath, .. } => {
b::error("Invalid integer index")
+ b::space()
+ b::delimit(
"(",
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
b::description("subpath:") + b::space() + b::description(&subpath.item),
Remove unused Display implementations After the previous commit, nushell uses PrettyDebug and PrettyDebugWithSource for our pretty-printed display output. PrettyDebug produces a structured `pretty.rs` document rather than writing directly into a fmt::Formatter, and types that implement `PrettyDebug` have a convenience `display` method that produces a string (to be used in situations where `Display` is needed for compatibility with other traits, or where simple rendering is appropriate).
2019-11-25 18:07:20 +00:00
")",
)
}
ProximateShellError::MissingValue { reason, .. } => {
b::error("Missing Value")
+ b::space()
+ b::delimit(
"(",
b::description("reason:") + b::space() + b::description(reason),
")",
)
}
ProximateShellError::ArgumentError { command, error } => {
b::error("Argument Error")
+ b::space()
+ b::delimit(
"(",
b::description("command:")
+ b::space()
+ b::description(&command.item)
+ b::description(",")
+ b::space()
+ b::description("error:")
+ b::space()
+ error.pretty(),
")",
)
}
ProximateShellError::RangeError {
kind,
actual_kind,
operation,
} => {
b::error("Range Error")
+ b::space()
+ b::delimit(
"(",
b::description("expected:")
+ b::space()
+ kind.pretty()
+ b::description(",")
+ b::space()
+ b::description("actual:")
+ b::space()
+ b::description(&actual_kind.item)
+ b::description(",")
+ b::space()
+ b::description("operation:")
+ b::space()
+ b::description(operation),
")",
)
}
ProximateShellError::Diagnostic(_) => b::error("diagnostic"),
ProximateShellError::CoerceError { left, right } => {
b::error("Coercion Error")
+ b::space()
+ b::delimit(
"(",
b::description("left:")
+ b::space()
+ b::description(&left.item)
+ b::description(",")
+ b::space()
+ b::description("right:")
+ b::space()
+ b::description(&right.item),
")",
)
}
ProximateShellError::UntaggedRuntimeError { reason } => {
b::error("Unknown Error") + b::delimit("(", b::description(reason), ")")
}
}
}
}
impl std::fmt::Display for ShellError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", self.pretty().display())
}
}
Restructuring
2019-08-02 19:15:07 +00:00
impl serde::de::Error for ShellError {
fn custom<T>(msg: T) -> Self
where
T: std::fmt::Display,
{
Overhaul the coloring system This commit replaces the previous naive coloring system with a coloring system that is more aligned with the parser. The main benefit of this change is that it allows us to use parsing rules to decide how to color tokens. For example, consider the following syntax: ``` $ ps | where cpu > 10 ``` Ideally, we could color `cpu` like a column name and not a string, because `cpu > 10` is a shorthand block syntax that expands to `{ $it.cpu > 10 }`. The way that we know that it's a shorthand block is that the `where` command declares that its first parameter is a `SyntaxShape::Block`, which allows the shorthand block form. In order to accomplish this, we need to color the tokens in a way that corresponds to their expanded semantics, which means that high-fidelity coloring requires expansion. This commit adds a `ColorSyntax` trait that corresponds to the `ExpandExpression` trait. The semantics are fairly similar, with a few differences. First `ExpandExpression` consumes N tokens and returns a single `hir::Expression`. `ColorSyntax` consumes N tokens and writes M `FlatShape` tokens to the output. Concretely, for syntax like `[1 2 3]` - `ExpandExpression` takes a single token node and produces a single `hir::Expression` - `ColorSyntax` takes the same token node and emits 7 `FlatShape`s (open delimiter, int, whitespace, int, whitespace, int, close delimiter) Second, `ColorSyntax` is more willing to plow through failures than `ExpandExpression`. In particular, consider syntax like ``` $ ps | where cpu > ``` In this case - `ExpandExpression` will see that the `where` command is expecting a block, see that it's not a literal block and try to parse it as a shorthand block. It will successfully find a member followed by an infix operator, but not a following expression. That means that the entire pipeline part fails to parse and is a syntax error. - `ColorSyntax` will also try to parse it as a shorthand block and ultimately fail, but it will fall back to "backoff coloring mode", which parsing any unidentified tokens in an unfallible, simple way. In this case, `cpu` will color as a string and `>` will color as an operator. Finally, it's very important that coloring a pipeline infallibly colors the entire string, doesn't fail, and doesn't get stuck in an infinite loop. In order to accomplish this, this PR separates `ColorSyntax`, which is infallible from `FallibleColorSyntax`, which might fail. This allows the type system to let us know if our coloring rules bottom out at at an infallible rule. It's not perfect: it's still possible for the coloring process to get stuck or consume tokens non-atomically. I intend to reduce the opportunity for those problems in a future commit. In the meantime, the current system catches a number of mistakes (like trying to use a fallible coloring rule in a loop without thinking about the possibility that it will never terminate).
2019-10-06 20:22:50 +00:00
ShellError::untagged_runtime_error(msg.to_string())
Restructuring
2019-08-02 19:15:07 +00:00
}
}
A bunch of rework I'm gonna use a VecDeque now instead of trying to get async streams working to make progress, but the intent is that we should be able to use async streams in and out to interleave the work better.
2019-05-13 17:30:51 +00:00
impl ShellError {
Overhaul the coloring system This commit replaces the previous naive coloring system with a coloring system that is more aligned with the parser. The main benefit of this change is that it allows us to use parsing rules to decide how to color tokens. For example, consider the following syntax: ``` $ ps | where cpu > 10 ``` Ideally, we could color `cpu` like a column name and not a string, because `cpu > 10` is a shorthand block syntax that expands to `{ $it.cpu > 10 }`. The way that we know that it's a shorthand block is that the `where` command declares that its first parameter is a `SyntaxShape::Block`, which allows the shorthand block form. In order to accomplish this, we need to color the tokens in a way that corresponds to their expanded semantics, which means that high-fidelity coloring requires expansion. This commit adds a `ColorSyntax` trait that corresponds to the `ExpandExpression` trait. The semantics are fairly similar, with a few differences. First `ExpandExpression` consumes N tokens and returns a single `hir::Expression`. `ColorSyntax` consumes N tokens and writes M `FlatShape` tokens to the output. Concretely, for syntax like `[1 2 3]` - `ExpandExpression` takes a single token node and produces a single `hir::Expression` - `ColorSyntax` takes the same token node and emits 7 `FlatShape`s (open delimiter, int, whitespace, int, whitespace, int, close delimiter) Second, `ColorSyntax` is more willing to plow through failures than `ExpandExpression`. In particular, consider syntax like ``` $ ps | where cpu > ``` In this case - `ExpandExpression` will see that the `where` command is expecting a block, see that it's not a literal block and try to parse it as a shorthand block. It will successfully find a member followed by an infix operator, but not a following expression. That means that the entire pipeline part fails to parse and is a syntax error. - `ColorSyntax` will also try to parse it as a shorthand block and ultimately fail, but it will fall back to "backoff coloring mode", which parsing any unidentified tokens in an unfallible, simple way. In this case, `cpu` will color as a string and `>` will color as an operator. Finally, it's very important that coloring a pipeline infallibly colors the entire string, doesn't fail, and doesn't get stuck in an infinite loop. In order to accomplish this, this PR separates `ColorSyntax`, which is infallible from `FallibleColorSyntax`, which might fail. This allows the type system to let us know if our coloring rules bottom out at at an infallible rule. It's not perfect: it's still possible for the coloring process to get stuck or consume tokens non-atomically. I intend to reduce the opportunity for those problems in a future commit. In the meantime, the current system catches a number of mistakes (like trying to use a fallible coloring rule in a loop without thinking about the possibility that it will never terminate).
2019-10-06 20:22:50 +00:00
pub fn type_error(
Support evaluating most expressions Blocks, paths, and others Plus a bunch of other infra improvements
2019-06-29 08:55:42 +00:00
expected: impl Into<String>,
--no-edit
2019-11-04 15:47:03 +00:00
actual: Spanned<impl Into<String>>,
Support evaluating most expressions Blocks, paths, and others Plus a bunch of other infra improvements
2019-06-29 08:55:42 +00:00
) -> ShellError {
Tests pass
2019-07-09 04:31:26 +00:00
ProximateShellError::TypeError {
Support evaluating most expressions Blocks, paths, and others Plus a bunch of other infra improvements
2019-06-29 08:55:42 +00:00
expected: expected.into(),
actual: actual.map(|i| Some(i.into())),
}
Tests pass
2019-07-09 04:31:26 +00:00
.start()
}
--no-edit
2019-11-04 15:47:03 +00:00
pub fn missing_property(
subpath: Spanned<impl Into<String>>,
expr: Spanned<impl Into<String>>,
) -> ShellError {
ProximateShellError::MissingProperty {
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
subpath: subpath.map(|s| s.into()),
expr: expr.map(|e| e.into()),
--no-edit
2019-11-04 15:47:03 +00:00
}
.start()
}
pub fn invalid_integer_index(
subpath: Spanned<impl Into<String>>,
integer: impl Into<Span>,
) -> ShellError {
ProximateShellError::InvalidIntegerIndex {
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
subpath: subpath.map(|s| s.into()),
--no-edit
2019-11-04 15:47:03 +00:00
integer: integer.into(),
}
.start()
}
Overhaul the coloring system This commit replaces the previous naive coloring system with a coloring system that is more aligned with the parser. The main benefit of this change is that it allows us to use parsing rules to decide how to color tokens. For example, consider the following syntax: ``` $ ps | where cpu > 10 ``` Ideally, we could color `cpu` like a column name and not a string, because `cpu > 10` is a shorthand block syntax that expands to `{ $it.cpu > 10 }`. The way that we know that it's a shorthand block is that the `where` command declares that its first parameter is a `SyntaxShape::Block`, which allows the shorthand block form. In order to accomplish this, we need to color the tokens in a way that corresponds to their expanded semantics, which means that high-fidelity coloring requires expansion. This commit adds a `ColorSyntax` trait that corresponds to the `ExpandExpression` trait. The semantics are fairly similar, with a few differences. First `ExpandExpression` consumes N tokens and returns a single `hir::Expression`. `ColorSyntax` consumes N tokens and writes M `FlatShape` tokens to the output. Concretely, for syntax like `[1 2 3]` - `ExpandExpression` takes a single token node and produces a single `hir::Expression` - `ColorSyntax` takes the same token node and emits 7 `FlatShape`s (open delimiter, int, whitespace, int, whitespace, int, close delimiter) Second, `ColorSyntax` is more willing to plow through failures than `ExpandExpression`. In particular, consider syntax like ``` $ ps | where cpu > ``` In this case - `ExpandExpression` will see that the `where` command is expecting a block, see that it's not a literal block and try to parse it as a shorthand block. It will successfully find a member followed by an infix operator, but not a following expression. That means that the entire pipeline part fails to parse and is a syntax error. - `ColorSyntax` will also try to parse it as a shorthand block and ultimately fail, but it will fall back to "backoff coloring mode", which parsing any unidentified tokens in an unfallible, simple way. In this case, `cpu` will color as a string and `>` will color as an operator. Finally, it's very important that coloring a pipeline infallibly colors the entire string, doesn't fail, and doesn't get stuck in an infinite loop. In order to accomplish this, this PR separates `ColorSyntax`, which is infallible from `FallibleColorSyntax`, which might fail. This allows the type system to let us know if our coloring rules bottom out at at an infallible rule. It's not perfect: it's still possible for the coloring process to get stuck or consume tokens non-atomically. I intend to reduce the opportunity for those problems in a future commit. In the meantime, the current system catches a number of mistakes (like trying to use a fallible coloring rule in a loop without thinking about the possibility that it will never terminate).
2019-10-06 20:22:50 +00:00
pub fn untagged_runtime_error(error: impl Into<String>) -> ShellError {
ProximateShellError::UntaggedRuntimeError {
reason: error.into(),
}
.start()
}
Extract core stuff into own crates This commit extracts five new crates: - nu-source, which contains the core source-code handling logic in Nu, including Text, Span, and also the pretty.rs-based debug logic - nu-parser, which is the parser and expander logic - nu-protocol, which is the bulk of the types and basic conveniences used by plugins - nu-errors, which contains ShellError, ParseError and error handling conveniences - nu-textview, which is the textview plugin extracted into a crate One of the major consequences of this refactor is that it's no longer possible to `impl X for Spanned<Y>` outside of the `nu-source` crate, so a lot of types became more concrete (Value became a concrete type instead of Spanned<Value>, for example). This also turned a number of inherent methods in the main nu crate into plain functions (impl Value {} became a bunch of functions in the `value` namespace in `crate::data::value`).
2019-11-26 02:30:48 +00:00
pub fn unexpected_eof(expected: impl Into<String>, span: impl Into<Span>) -> ShellError {
Overhaul the expansion system The main thrust of this (very large) commit is an overhaul of the expansion system. The parsing pipeline is: - Lightly parse the source file for atoms, basic delimiters and pipeline structure into a token tree - Expand the token tree into a HIR (high-level intermediate representation) based upon the baseline syntax rules for expressions and the syntactic shape of commands. Somewhat non-traditionally, nu doesn't have an AST at all. It goes directly from the token tree, which doesn't represent many important distinctions (like the difference between `hello` and `5KB`) directly into a high-level representation that doesn't have a direct correspondence to the source code. At a high level, nu commands work like macros, in the sense that the syntactic shape of the invocation of a command depends on the definition of a command. However, commands do not have the ability to perform unrestricted expansions of the token tree. Instead, they describe their arguments in terms of syntactic shapes, and the expander expands the token tree into HIR based upon that definition. For example, the `where` command says that it takes a block as its first required argument, and the description of the block syntactic shape expands the syntax `cpu > 10` into HIR that represents `{ $it.cpu > 10 }`. This commit overhauls that system so that the syntactic shapes are described in terms of a few new traits (`ExpandSyntax` and `ExpandExpression` are the primary ones) that are more composable than the previous system. The first big win of this new system is the addition of the `ColumnPath` shape, which looks like `cpu."max ghz"` or `package.version`. Previously, while a variable path could look like `$it.cpu."max ghz"`, the tail of a variable path could not be easily reused in other contexts. Now, that tail is its own syntactic shape, and it can be used as part of a command's signature. This cleans up commands like `inc`, `add` and `edit` as well as shorthand blocks, which can now look like `| where cpu."max ghz" > 10`
2019-09-17 22:26:27 +00:00
ProximateShellError::UnexpectedEof {
expected: expected.into(),
--no-edit
2019-11-04 15:47:03 +00:00
span: span.into(),
Overhaul the expansion system The main thrust of this (very large) commit is an overhaul of the expansion system. The parsing pipeline is: - Lightly parse the source file for atoms, basic delimiters and pipeline structure into a token tree - Expand the token tree into a HIR (high-level intermediate representation) based upon the baseline syntax rules for expressions and the syntactic shape of commands. Somewhat non-traditionally, nu doesn't have an AST at all. It goes directly from the token tree, which doesn't represent many important distinctions (like the difference between `hello` and `5KB`) directly into a high-level representation that doesn't have a direct correspondence to the source code. At a high level, nu commands work like macros, in the sense that the syntactic shape of the invocation of a command depends on the definition of a command. However, commands do not have the ability to perform unrestricted expansions of the token tree. Instead, they describe their arguments in terms of syntactic shapes, and the expander expands the token tree into HIR based upon that definition. For example, the `where` command says that it takes a block as its first required argument, and the description of the block syntactic shape expands the syntax `cpu > 10` into HIR that represents `{ $it.cpu > 10 }`. This commit overhauls that system so that the syntactic shapes are described in terms of a few new traits (`ExpandSyntax` and `ExpandExpression` are the primary ones) that are more composable than the previous system. The first big win of this new system is the addition of the `ColumnPath` shape, which looks like `cpu."max ghz"` or `package.version`. Previously, while a variable path could look like `$it.cpu."max ghz"`, the tail of a variable path could not be easily reused in other contexts. Now, that tail is its own syntactic shape, and it can be used as part of a command's signature. This cleans up commands like `inc`, `add` and `edit` as well as shorthand blocks, which can now look like `| where cpu."max ghz" > 10`
2019-09-17 22:26:27 +00:00
}
.start()
}
Extract core stuff into own crates This commit extracts five new crates: - nu-source, which contains the core source-code handling logic in Nu, including Text, Span, and also the pretty.rs-based debug logic - nu-parser, which is the parser and expander logic - nu-protocol, which is the bulk of the types and basic conveniences used by plugins - nu-errors, which contains ShellError, ParseError and error handling conveniences - nu-textview, which is the textview plugin extracted into a crate One of the major consequences of this refactor is that it's no longer possible to `impl X for Spanned<Y>` outside of the `nu-source` crate, so a lot of types became more concrete (Value became a concrete type instead of Spanned<Value>, for example). This also turned a number of inherent methods in the main nu crate into plain functions (impl Value {} became a bunch of functions in the `value` namespace in `crate::data::value`).
2019-11-26 02:30:48 +00:00
pub fn range_error(
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
expected: impl Into<ExpectedRange>,
Extract nu_source into a crate This commit extracts Tag, Span, Text, as well as source-related debug facilities into a new crate called nu_source. This change is much bigger than one might have expected because the previous code relied heavily on implementing inherent methods on `Tagged<T>` and `Spanned<T>`, which is no longer possible. As a result, this change creates more concrete types instead of using `Tagged<T>`. One notable example: Tagged<Value> became Value, and Value became UntaggedValue. This change clarifies the intent of the code in many places, but it does make it a big change.
2019-11-21 14:33:14 +00:00
actual: &Spanned<impl fmt::Debug>,
--no-edit
2019-11-04 15:47:03 +00:00
operation: impl Into<String>,
Basic support for decimal numbers This commit is more substantial than it looks: there was basically no real support for decimals before, and that impacted values all the way through. I also made Size contain a decimal instead of an integer (`1.6kb` is a reasonable thing to type), which impacted a bunch of code. The biggest impact of this commit is that it creates many more possible ways for valid nu types to fail to serialize as toml, json, etc. which typically can't support the full range of Decimal (or Bigint, which I also think we should support). This commit makes to-toml fallible, and a similar effort is necessary for the rest of the serializations. We also need to figure out how to clearly communicate to users what has happened, but failing to serialize to toml seems clearly superior to me than weird errors in basic math operations.
2019-08-30 17:29:04 +00:00
) -> ShellError {
ProximateShellError::RangeError {
kind: expected.into(),
Extract nu_source into a crate This commit extracts Tag, Span, Text, as well as source-related debug facilities into a new crate called nu_source. This change is much bigger than one might have expected because the previous code relied heavily on implementing inherent methods on `Tagged<T>` and `Spanned<T>`, which is no longer possible. As a result, this change creates more concrete types instead of using `Tagged<T>`. One notable example: Tagged<Value> became Value, and Value became UntaggedValue. This change clarifies the intent of the code in many places, but it does make it a big change.
2019-11-21 14:33:14 +00:00
actual_kind: format!("{:?}", actual.item).spanned(actual.span),
--no-edit
2019-11-04 15:47:03 +00:00
operation: operation.into(),
Basic support for decimal numbers This commit is more substantial than it looks: there was basically no real support for decimals before, and that impacted values all the way through. I also made Size contain a decimal instead of an integer (`1.6kb` is a reasonable thing to type), which impacted a bunch of code. The biggest impact of this commit is that it creates many more possible ways for valid nu types to fail to serialize as toml, json, etc. which typically can't support the full range of Decimal (or Bigint, which I also think we should support). This commit makes to-toml fallible, and a similar effort is necessary for the rest of the serializations. We also need to figure out how to clearly communicate to users what has happened, but failing to serialize to toml seems clearly superior to me than weird errors in basic math operations.
2019-08-30 17:29:04 +00:00
}
.start()
}
Extract core stuff into own crates This commit extracts five new crates: - nu-source, which contains the core source-code handling logic in Nu, including Text, Span, and also the pretty.rs-based debug logic - nu-parser, which is the parser and expander logic - nu-protocol, which is the bulk of the types and basic conveniences used by plugins - nu-errors, which contains ShellError, ParseError and error handling conveniences - nu-textview, which is the textview plugin extracted into a crate One of the major consequences of this refactor is that it's no longer possible to `impl X for Spanned<Y>` outside of the `nu-source` crate, so a lot of types became more concrete (Value became a concrete type instead of Spanned<Value>, for example). This also turned a number of inherent methods in the main nu crate into plain functions (impl Value {} became a bunch of functions in the `value` namespace in `crate::data::value`).
2019-11-26 02:30:48 +00:00
pub fn syntax_error(problem: Spanned<impl Into<String>>) -> ShellError {
Add support for external escape valve (`^dir`) This commit makes it possible to force nu to treat a command as an external command by prefixing it with `^`. For example `^dir` will force `dir` to run an external command, even if `dir` is also a registered nu command. This ensures that users don't need to leave nu just because we happened to use a command they need. This commit adds a new token type for external commands, which, among other things, makes it pretty straight forward to syntax highlight external commands uniquely, and generally to treat them as special.
2019-08-15 22:18:18 +00:00
ProximateShellError::SyntaxError {
problem: problem.map(|p| p.into()),
}
.start()
}
Extract core stuff into own crates This commit extracts five new crates: - nu-source, which contains the core source-code handling logic in Nu, including Text, Span, and also the pretty.rs-based debug logic - nu-parser, which is the parser and expander logic - nu-protocol, which is the bulk of the types and basic conveniences used by plugins - nu-errors, which contains ShellError, ParseError and error handling conveniences - nu-textview, which is the textview plugin extracted into a crate One of the major consequences of this refactor is that it's no longer possible to `impl X for Spanned<Y>` outside of the `nu-source` crate, so a lot of types became more concrete (Value became a concrete type instead of Spanned<Value>, for example). This also turned a number of inherent methods in the main nu crate into plain functions (impl Value {} became a bunch of functions in the `value` namespace in `crate::data::value`).
2019-11-26 02:30:48 +00:00
pub fn coerce_error(
--no-edit
2019-11-04 15:47:03 +00:00
left: Spanned<impl Into<String>>,
right: Spanned<impl Into<String>>,
Tests pass
2019-07-09 04:31:26 +00:00
) -> ShellError {
ProximateShellError::CoerceError {
left: left.map(|l| l.into()),
right: right.map(|r| r.into()),
}
.start()
}
Extract core stuff into own crates This commit extracts five new crates: - nu-source, which contains the core source-code handling logic in Nu, including Text, Span, and also the pretty.rs-based debug logic - nu-parser, which is the parser and expander logic - nu-protocol, which is the bulk of the types and basic conveniences used by plugins - nu-errors, which contains ShellError, ParseError and error handling conveniences - nu-textview, which is the textview plugin extracted into a crate One of the major consequences of this refactor is that it's no longer possible to `impl X for Spanned<Y>` outside of the `nu-source` crate, so a lot of types became more concrete (Value became a concrete type instead of Spanned<Value>, for example). This also turned a number of inherent methods in the main nu crate into plain functions (impl Value {} became a bunch of functions in the `value` namespace in `crate::data::value`).
2019-11-26 02:30:48 +00:00
pub fn argument_error(command: Spanned<impl Into<String>>, kind: ArgumentError) -> ShellError {
Tests pass
2019-07-09 04:31:26 +00:00
ProximateShellError::ArgumentError {
--no-edit
2019-11-04 15:47:03 +00:00
command: command.map(|c| c.into()),
Tests pass
2019-07-09 04:31:26 +00:00
error: kind,
}
.start()
Support evaluating most expressions Blocks, paths, and others Plus a bunch of other infra improvements
2019-06-29 08:55:42 +00:00
}
Extract core stuff into own crates This commit extracts five new crates: - nu-source, which contains the core source-code handling logic in Nu, including Text, Span, and also the pretty.rs-based debug logic - nu-parser, which is the parser and expander logic - nu-protocol, which is the bulk of the types and basic conveniences used by plugins - nu-errors, which contains ShellError, ParseError and error handling conveniences - nu-textview, which is the textview plugin extracted into a crate One of the major consequences of this refactor is that it's no longer possible to `impl X for Spanned<Y>` outside of the `nu-source` crate, so a lot of types became more concrete (Value became a concrete type instead of Spanned<Value>, for example). This also turned a number of inherent methods in the main nu crate into plain functions (impl Value {} became a bunch of functions in the `value` namespace in `crate::data::value`).
2019-11-26 02:30:48 +00:00
pub fn parse_error(
Revert "Revert "Migrate most uses of the Span concept to Tag"" This reverts commit bee7c5639cc78d65e6656518627a86009a522500.
2019-09-14 16:30:24 +00:00
error: nom::Err<(
Overhaul the expansion system The main thrust of this (very large) commit is an overhaul of the expansion system. The parsing pipeline is: - Lightly parse the source file for atoms, basic delimiters and pipeline structure into a token tree - Expand the token tree into a HIR (high-level intermediate representation) based upon the baseline syntax rules for expressions and the syntactic shape of commands. Somewhat non-traditionally, nu doesn't have an AST at all. It goes directly from the token tree, which doesn't represent many important distinctions (like the difference between `hello` and `5KB`) directly into a high-level representation that doesn't have a direct correspondence to the source code. At a high level, nu commands work like macros, in the sense that the syntactic shape of the invocation of a command depends on the definition of a command. However, commands do not have the ability to perform unrestricted expansions of the token tree. Instead, they describe their arguments in terms of syntactic shapes, and the expander expands the token tree into HIR based upon that definition. For example, the `where` command says that it takes a block as its first required argument, and the description of the block syntactic shape expands the syntax `cpu > 10` into HIR that represents `{ $it.cpu > 10 }`. This commit overhauls that system so that the syntactic shapes are described in terms of a few new traits (`ExpandSyntax` and `ExpandExpression` are the primary ones) that are more composable than the previous system. The first big win of this new system is the addition of the `ColumnPath` shape, which looks like `cpu."max ghz"` or `package.version`. Previously, while a variable path could look like `$it.cpu."max ghz"`, the tail of a variable path could not be easily reused in other contexts. Now, that tail is its own syntactic shape, and it can be used as part of a command's signature. This cleans up commands like `inc`, `add` and `edit` as well as shorthand blocks, which can now look like `| where cpu."max ghz" > 10`
2019-09-17 22:26:27 +00:00
nom_locate::LocatedSpanEx<&str, TracableContext>,
Revert "Revert "Migrate most uses of the Span concept to Tag"" This reverts commit bee7c5639cc78d65e6656518627a86009a522500.
2019-09-14 16:30:24 +00:00
nom::error::ErrorKind,
)>,
Better errors and more fleshed out semantics
2019-05-30 04:19:46 +00:00
) -> ShellError {
use language_reporting::*;
match error {
Remove unwraps from the parser I intend to add regression tests for these cases to the parser as a follow-up PR. Fixes #490 Fixes #494
2019-08-27 21:20:18 +00:00
nom::Err::Incomplete(_) => {
// TODO: Get span of EOF
let diagnostic = Diagnostic::new(
Severity::Error,
Move to using clippy (#1142) * Clippy fixes * Finish converting to use clippy * fix warnings in new master * fix windows * fix windows Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
2019-12-31 07:36:08 +00:00
"Parse Error: Unexpected end of line".to_string(),
Remove unwraps from the parser I intend to add regression tests for these cases to the parser as a follow-up PR. Fixes #490 Fixes #494
2019-08-27 21:20:18 +00:00
);
ShellError::diagnostic(diagnostic)
}
WIP
2019-06-22 03:43:37 +00:00
nom::Err::Failure(span) | nom::Err::Error(span) => {
Move to using clippy (#1142) * Clippy fixes * Finish converting to use clippy * fix warnings in new master * fix windows * fix windows Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
2019-12-31 07:36:08 +00:00
let diagnostic = Diagnostic::new(Severity::Error, "Parse Error".to_string())
Stream support (#812) * Moves off of draining between filters. Instead, the sink will pull on the stream, and will drain element-wise. This moves the whole stream to being lazy. * Adds ctrl-c support and connects it into some of the key points where we pull on the stream. If a ctrl-c is detect, we immediately halt pulling on the stream and return to the prompt. * Moves away from having a SourceMap where anchor locations are stored. Now AnchorLocation is kept directly in the Tag. * To make this possible, split tag and span. Span is largely used in the parser and is copyable. Tag is now no longer copyable.
2019-10-13 04:12:43 +00:00
.with_label(Label::new_primary(Span::from(span.0)));
Better errors and more fleshed out semantics
2019-05-30 04:19:46 +00:00
Add pretty errors to commands
2019-06-07 22:35:07 +00:00
ShellError::diagnostic(diagnostic)
Tests pass
2019-07-09 04:31:26 +00:00
}
Better errors and more fleshed out semantics
2019-05-30 04:19:46 +00:00
}
}
Extract core stuff into own crates This commit extracts five new crates: - nu-source, which contains the core source-code handling logic in Nu, including Text, Span, and also the pretty.rs-based debug logic - nu-parser, which is the parser and expander logic - nu-protocol, which is the bulk of the types and basic conveniences used by plugins - nu-errors, which contains ShellError, ParseError and error handling conveniences - nu-textview, which is the textview plugin extracted into a crate One of the major consequences of this refactor is that it's no longer possible to `impl X for Spanned<Y>` outside of the `nu-source` crate, so a lot of types became more concrete (Value became a concrete type instead of Spanned<Value>, for example). This also turned a number of inherent methods in the main nu crate into plain functions (impl Value {} became a bunch of functions in the `value` namespace in `crate::data::value`).
2019-11-26 02:30:48 +00:00
pub fn diagnostic(diagnostic: Diagnostic<Span>) -> ShellError {
Tests pass
2019-07-09 04:31:26 +00:00
ProximateShellError::Diagnostic(ShellDiagnostic { diagnostic }).start()
Add pretty errors to commands
2019-06-07 22:35:07 +00:00
}
Move to using clippy (#1142) * Clippy fixes * Finish converting to use clippy * fix warnings in new master * fix windows * fix windows Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
2019-12-31 07:36:08 +00:00
pub fn into_diagnostic(self) -> Diagnostic<Span> {
Tests pass
2019-07-09 04:31:26 +00:00
match self.error {
--no-edit
2019-11-04 15:47:03 +00:00
ProximateShellError::MissingValue { span, reason } => {
Reduce unwraps Remove a number of unwraps. In some cases, a `?` just worked as is. I also made it possible to use `?` to go from Result<OutputStream, ShellError> to OutputStream. Finally, started updating PerItemCommand to be able to use the signature deserialization logic, which substantially reduces unwraps. This is still in-progress work, but tests pass and it should be clear to merge and keep iterating on master.
2019-08-17 03:53:39 +00:00
let mut d = Diagnostic::new(
Severity::Bug,
format!("Internal Error (missing value) :: {}", reason),
);
--no-edit
2019-11-04 15:47:03 +00:00
if let Some(span) = span {
d = d.with_label(Label::new_primary(span));
Reduce unwraps Remove a number of unwraps. In some cases, a `?` just worked as is. I also made it possible to use `?` to go from Result<OutputStream, ShellError> to OutputStream. Finally, started updating PerItemCommand to be able to use the signature deserialization logic, which substantially reduces unwraps. This is still in-progress work, but tests pass and it should be clear to merge and keep iterating on master.
2019-08-17 03:53:39 +00:00
}
d
}
Tests pass
2019-07-09 04:31:26 +00:00
ProximateShellError::ArgumentError {
WIP improve error infrastructure Also simplify commands and reduce papercuts
2019-07-03 20:31:15 +00:00
command,
error,
} => match error {
Close a bunch of holes in external command args Previously, there was a single parsing rule for "bare words" that applied to both internal and external commands. This meant that, because `cargo +nightly` needed to work, we needed to add `+` as a valid character in bare words. The number of characters continued to grow, and the situation was becoming untenable. The current strategy would eventually eat up all syntax and make it impossible to add syntax like `@foo` to internal commands. This patch significantly restricts bare words and introduces a new token type (`ExternalWord`). An `ExternalWord` expands to an error in the internal syntax, but expands to a bare word in the external syntax. `ExternalWords` are highlighted in grey in the shell.
2019-09-09 17:43:10 +00:00
ArgumentError::InvalidExternalWord => Diagnostic::new(
Severity::Error,
Move to using clippy (#1142) * Clippy fixes * Finish converting to use clippy * fix warnings in new master * fix windows * fix windows Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
2019-12-31 07:36:08 +00:00
"Invalid bare word for Nu command (did you intend to invoke an external command?)".to_string())
--no-edit
2019-11-04 15:47:03 +00:00
.with_label(Label::new_primary(command.span)),
Produce ArgumentError for signature mismatch ArgumentError also automatically produces diagnostics
2019-06-30 06:14:40 +00:00
ArgumentError::MissingMandatoryFlag(name) => Diagnostic::new(
Severity::Error,
format!(
WIP improve error infrastructure Also simplify commands and reduce papercuts
2019-07-03 20:31:15 +00:00
"{} requires {}{}",
--no-edit
2019-11-04 15:47:03 +00:00
Color::Cyan.paint(&command.item),
WIP improve error infrastructure Also simplify commands and reduce papercuts
2019-07-03 20:31:15 +00:00
Color::Black.bold().paint("--"),
Color::Black.bold().paint(name)
Produce ArgumentError for signature mismatch ArgumentError also automatically produces diagnostics
2019-06-30 06:14:40 +00:00
),
)
--no-edit
2019-11-04 15:47:03 +00:00
.with_label(Label::new_primary(command.span)),
Produce ArgumentError for signature mismatch ArgumentError also automatically produces diagnostics
2019-06-30 06:14:40 +00:00
ArgumentError::MissingMandatoryPositional(name) => Diagnostic::new(
Severity::Error,
WIP improve error infrastructure Also simplify commands and reduce papercuts
2019-07-03 20:31:15 +00:00
format!(
Finish updating the last cases
2019-08-20 06:11:11 +00:00
"{} requires {} parameter",
--no-edit
2019-11-04 15:47:03 +00:00
Color::Cyan.paint(&command.item),
Finish updating the last cases
2019-08-20 06:11:11 +00:00
Color::Green.bold().paint(name.clone())
WIP improve error infrastructure Also simplify commands and reduce papercuts
2019-07-03 20:31:15 +00:00
),
Produce ArgumentError for signature mismatch ArgumentError also automatically produces diagnostics
2019-06-30 06:14:40 +00:00
)
Finish updating the last cases
2019-08-20 06:11:11 +00:00
.with_label(
--no-edit
2019-11-04 15:47:03 +00:00
Label::new_primary(command.span).with_message(format!("requires {} parameter", name)),
Finish updating the last cases
2019-08-20 06:11:11 +00:00
),
Produce ArgumentError for signature mismatch ArgumentError also automatically produces diagnostics
2019-06-30 06:14:40 +00:00
ArgumentError::MissingValueForName(name) => Diagnostic::new(
Severity::Error,
format!(
WIP improve error infrastructure Also simplify commands and reduce papercuts
2019-07-03 20:31:15 +00:00
"{} is missing value for flag {}{}",
--no-edit
2019-11-04 15:47:03 +00:00
Color::Cyan.paint(&command.item),
WIP improve error infrastructure Also simplify commands and reduce papercuts
2019-07-03 20:31:15 +00:00
Color::Black.bold().paint("--"),
Color::Black.bold().paint(name)
Produce ArgumentError for signature mismatch ArgumentError also automatically produces diagnostics
2019-06-30 06:14:40 +00:00
),
)
--no-edit
2019-11-04 15:47:03 +00:00
.with_label(Label::new_primary(command.span)),
Produce ArgumentError for signature mismatch ArgumentError also automatically produces diagnostics
2019-06-30 06:14:40 +00:00
},
Tests pass
2019-07-09 04:31:26 +00:00
ProximateShellError::TypeError {
Support evaluating most expressions Blocks, paths, and others Plus a bunch of other infra improvements
2019-06-29 08:55:42 +00:00
expected,
actual:
--no-edit
2019-11-04 15:47:03 +00:00
Spanned {
Support evaluating most expressions Blocks, paths, and others Plus a bunch of other infra improvements
2019-06-29 08:55:42 +00:00
item: Some(actual),
--no-edit
2019-11-04 15:47:03 +00:00
span,
Support evaluating most expressions Blocks, paths, and others Plus a bunch of other infra improvements
2019-06-29 08:55:42 +00:00
},
} => Diagnostic::new(Severity::Error, "Type Error").with_label(
--no-edit
2019-11-04 15:47:03 +00:00
Label::new_primary(span)
Support evaluating most expressions Blocks, paths, and others Plus a bunch of other infra improvements
2019-06-29 08:55:42 +00:00
.with_message(format!("Expected {}, found {}", expected, actual)),
),
Tests pass
2019-07-09 04:31:26 +00:00
ProximateShellError::TypeError {
Support evaluating most expressions Blocks, paths, and others Plus a bunch of other infra improvements
2019-06-29 08:55:42 +00:00
expected,
initial change to Tagged<Value>
2019-08-01 01:58:42 +00:00
actual:
--no-edit
2019-11-04 15:47:03 +00:00
Spanned {
initial change to Tagged<Value>
2019-08-01 01:58:42 +00:00
item: None,
--no-edit
2019-11-04 15:47:03 +00:00
span
initial change to Tagged<Value>
2019-08-01 01:58:42 +00:00
},
Support evaluating most expressions Blocks, paths, and others Plus a bunch of other infra improvements
2019-06-29 08:55:42 +00:00
} => Diagnostic::new(Severity::Error, "Type Error")
--no-edit
2019-11-04 15:47:03 +00:00
.with_label(Label::new_primary(span).with_message(expected)),
Fix a bunch of bugs
2019-06-24 00:55:31 +00:00
Overhaul the expansion system The main thrust of this (very large) commit is an overhaul of the expansion system. The parsing pipeline is: - Lightly parse the source file for atoms, basic delimiters and pipeline structure into a token tree - Expand the token tree into a HIR (high-level intermediate representation) based upon the baseline syntax rules for expressions and the syntactic shape of commands. Somewhat non-traditionally, nu doesn't have an AST at all. It goes directly from the token tree, which doesn't represent many important distinctions (like the difference between `hello` and `5KB`) directly into a high-level representation that doesn't have a direct correspondence to the source code. At a high level, nu commands work like macros, in the sense that the syntactic shape of the invocation of a command depends on the definition of a command. However, commands do not have the ability to perform unrestricted expansions of the token tree. Instead, they describe their arguments in terms of syntactic shapes, and the expander expands the token tree into HIR based upon that definition. For example, the `where` command says that it takes a block as its first required argument, and the description of the block syntactic shape expands the syntax `cpu > 10` into HIR that represents `{ $it.cpu > 10 }`. This commit overhauls that system so that the syntactic shapes are described in terms of a few new traits (`ExpandSyntax` and `ExpandExpression` are the primary ones) that are more composable than the previous system. The first big win of this new system is the addition of the `ColumnPath` shape, which looks like `cpu."max ghz"` or `package.version`. Previously, while a variable path could look like `$it.cpu."max ghz"`, the tail of a variable path could not be easily reused in other contexts. Now, that tail is its own syntactic shape, and it can be used as part of a command's signature. This cleans up commands like `inc`, `add` and `edit` as well as shorthand blocks, which can now look like `| where cpu."max ghz" > 10`
2019-09-17 22:26:27 +00:00
ProximateShellError::UnexpectedEof {
--no-edit
2019-11-04 15:47:03 +00:00
expected, span
Move to using clippy (#1142) * Clippy fixes * Finish converting to use clippy * fix warnings in new master * fix windows * fix windows Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
2019-12-31 07:36:08 +00:00
} => Diagnostic::new(Severity::Error, "Unexpected end of input".to_string())
--no-edit
2019-11-04 15:47:03 +00:00
.with_label(Label::new_primary(span).with_message(format!("Expected {}", expected))),
Overhaul the expansion system The main thrust of this (very large) commit is an overhaul of the expansion system. The parsing pipeline is: - Lightly parse the source file for atoms, basic delimiters and pipeline structure into a token tree - Expand the token tree into a HIR (high-level intermediate representation) based upon the baseline syntax rules for expressions and the syntactic shape of commands. Somewhat non-traditionally, nu doesn't have an AST at all. It goes directly from the token tree, which doesn't represent many important distinctions (like the difference between `hello` and `5KB`) directly into a high-level representation that doesn't have a direct correspondence to the source code. At a high level, nu commands work like macros, in the sense that the syntactic shape of the invocation of a command depends on the definition of a command. However, commands do not have the ability to perform unrestricted expansions of the token tree. Instead, they describe their arguments in terms of syntactic shapes, and the expander expands the token tree into HIR based upon that definition. For example, the `where` command says that it takes a block as its first required argument, and the description of the block syntactic shape expands the syntax `cpu > 10` into HIR that represents `{ $it.cpu > 10 }`. This commit overhauls that system so that the syntactic shapes are described in terms of a few new traits (`ExpandSyntax` and `ExpandExpression` are the primary ones) that are more composable than the previous system. The first big win of this new system is the addition of the `ColumnPath` shape, which looks like `cpu."max ghz"` or `package.version`. Previously, while a variable path could look like `$it.cpu."max ghz"`, the tail of a variable path could not be easily reused in other contexts. Now, that tail is its own syntactic shape, and it can be used as part of a command's signature. This cleans up commands like `inc`, `add` and `edit` as well as shorthand blocks, which can now look like `| where cpu."max ghz" > 10`
2019-09-17 22:26:27 +00:00
Basic support for decimal numbers This commit is more substantial than it looks: there was basically no real support for decimals before, and that impacted values all the way through. I also made Size contain a decimal instead of an integer (`1.6kb` is a reasonable thing to type), which impacted a bunch of code. The biggest impact of this commit is that it creates many more possible ways for valid nu types to fail to serialize as toml, json, etc. which typically can't support the full range of Decimal (or Bigint, which I also think we should support). This commit makes to-toml fallible, and a similar effort is necessary for the rest of the serializations. We also need to figure out how to clearly communicate to users what has happened, but failing to serialize to toml seems clearly superior to me than weird errors in basic math operations.
2019-08-30 17:29:04 +00:00
ProximateShellError::RangeError {
kind,
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
operation,
Basic support for decimal numbers This commit is more substantial than it looks: there was basically no real support for decimals before, and that impacted values all the way through. I also made Size contain a decimal instead of an integer (`1.6kb` is a reasonable thing to type), which impacted a bunch of code. The biggest impact of this commit is that it creates many more possible ways for valid nu types to fail to serialize as toml, json, etc. which typically can't support the full range of Decimal (or Bigint, which I also think we should support). This commit makes to-toml fallible, and a similar effort is necessary for the rest of the serializations. We also need to figure out how to clearly communicate to users what has happened, but failing to serialize to toml seems clearly superior to me than weird errors in basic math operations.
2019-08-30 17:29:04 +00:00
actual_kind:
--no-edit
2019-11-04 15:47:03 +00:00
Spanned {
Basic support for decimal numbers This commit is more substantial than it looks: there was basically no real support for decimals before, and that impacted values all the way through. I also made Size contain a decimal instead of an integer (`1.6kb` is a reasonable thing to type), which impacted a bunch of code. The biggest impact of this commit is that it creates many more possible ways for valid nu types to fail to serialize as toml, json, etc. which typically can't support the full range of Decimal (or Bigint, which I also think we should support). This commit makes to-toml fallible, and a similar effort is necessary for the rest of the serializations. We also need to figure out how to clearly communicate to users what has happened, but failing to serialize to toml seems clearly superior to me than weird errors in basic math operations.
2019-08-30 17:29:04 +00:00
item,
--no-edit
2019-11-04 15:47:03 +00:00
span
Basic support for decimal numbers This commit is more substantial than it looks: there was basically no real support for decimals before, and that impacted values all the way through. I also made Size contain a decimal instead of an integer (`1.6kb` is a reasonable thing to type), which impacted a bunch of code. The biggest impact of this commit is that it creates many more possible ways for valid nu types to fail to serialize as toml, json, etc. which typically can't support the full range of Decimal (or Bigint, which I also think we should support). This commit makes to-toml fallible, and a similar effort is necessary for the rest of the serializations. We also need to figure out how to clearly communicate to users what has happened, but failing to serialize to toml seems clearly superior to me than weird errors in basic math operations.
2019-08-30 17:29:04 +00:00
},
} => Diagnostic::new(Severity::Error, "Range Error").with_label(
--no-edit
2019-11-04 15:47:03 +00:00
Label::new_primary(span).with_message(format!(
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
"Expected to convert {} to {} while {}, but it was out of range",
item,
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
kind.display(),
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
operation
Basic support for decimal numbers This commit is more substantial than it looks: there was basically no real support for decimals before, and that impacted values all the way through. I also made Size contain a decimal instead of an integer (`1.6kb` is a reasonable thing to type), which impacted a bunch of code. The biggest impact of this commit is that it creates many more possible ways for valid nu types to fail to serialize as toml, json, etc. which typically can't support the full range of Decimal (or Bigint, which I also think we should support). This commit makes to-toml fallible, and a similar effort is necessary for the rest of the serializations. We also need to figure out how to clearly communicate to users what has happened, but failing to serialize to toml seems clearly superior to me than weird errors in basic math operations.
2019-08-30 17:29:04 +00:00
)),
),
Add support for external escape valve (`^dir`) This commit makes it possible to force nu to treat a command as an external command by prefixing it with `^`. For example `^dir` will force `dir` to run an external command, even if `dir` is also a registered nu command. This ensures that users don't need to leave nu just because we happened to use a command they need. This commit adds a new token type for external commands, which, among other things, makes it pretty straight forward to syntax highlight external commands uniquely, and generally to treat them as special.
2019-08-15 22:18:18 +00:00
ProximateShellError::SyntaxError {
problem:
--no-edit
2019-11-04 15:47:03 +00:00
Spanned {
span,
Overhaul the expansion system The main thrust of this (very large) commit is an overhaul of the expansion system. The parsing pipeline is: - Lightly parse the source file for atoms, basic delimiters and pipeline structure into a token tree - Expand the token tree into a HIR (high-level intermediate representation) based upon the baseline syntax rules for expressions and the syntactic shape of commands. Somewhat non-traditionally, nu doesn't have an AST at all. It goes directly from the token tree, which doesn't represent many important distinctions (like the difference between `hello` and `5KB`) directly into a high-level representation that doesn't have a direct correspondence to the source code. At a high level, nu commands work like macros, in the sense that the syntactic shape of the invocation of a command depends on the definition of a command. However, commands do not have the ability to perform unrestricted expansions of the token tree. Instead, they describe their arguments in terms of syntactic shapes, and the expander expands the token tree into HIR based upon that definition. For example, the `where` command says that it takes a block as its first required argument, and the description of the block syntactic shape expands the syntax `cpu > 10` into HIR that represents `{ $it.cpu > 10 }`. This commit overhauls that system so that the syntactic shapes are described in terms of a few new traits (`ExpandSyntax` and `ExpandExpression` are the primary ones) that are more composable than the previous system. The first big win of this new system is the addition of the `ColumnPath` shape, which looks like `cpu."max ghz"` or `package.version`. Previously, while a variable path could look like `$it.cpu."max ghz"`, the tail of a variable path could not be easily reused in other contexts. Now, that tail is its own syntactic shape, and it can be used as part of a command's signature. This cleans up commands like `inc`, `add` and `edit` as well as shorthand blocks, which can now look like `| where cpu."max ghz" > 10`
2019-09-17 22:26:27 +00:00
item
Add support for external escape valve (`^dir`) This commit makes it possible to force nu to treat a command as an external command by prefixing it with `^`. For example `^dir` will force `dir` to run an external command, even if `dir` is also a registered nu command. This ensures that users don't need to leave nu just because we happened to use a command they need. This commit adds a new token type for external commands, which, among other things, makes it pretty straight forward to syntax highlight external commands uniquely, and generally to treat them as special.
2019-08-15 22:18:18 +00:00
},
} => Diagnostic::new(Severity::Error, "Syntax Error")
--no-edit
2019-11-04 15:47:03 +00:00
.with_label(Label::new_primary(span).with_message(item)),
Add support for external escape valve (`^dir`) This commit makes it possible to force nu to treat a command as an external command by prefixing it with `^`. For example `^dir` will force `dir` to run an external command, even if `dir` is also a registered nu command. This ensures that users don't need to leave nu just because we happened to use a command they need. This commit adds a new token type for external commands, which, among other things, makes it pretty straight forward to syntax highlight external commands uniquely, and generally to treat them as special.
2019-08-15 22:18:18 +00:00
Overhaul the coloring system This commit replaces the previous naive coloring system with a coloring system that is more aligned with the parser. The main benefit of this change is that it allows us to use parsing rules to decide how to color tokens. For example, consider the following syntax: ``` $ ps | where cpu > 10 ``` Ideally, we could color `cpu` like a column name and not a string, because `cpu > 10` is a shorthand block syntax that expands to `{ $it.cpu > 10 }`. The way that we know that it's a shorthand block is that the `where` command declares that its first parameter is a `SyntaxShape::Block`, which allows the shorthand block form. In order to accomplish this, we need to color the tokens in a way that corresponds to their expanded semantics, which means that high-fidelity coloring requires expansion. This commit adds a `ColorSyntax` trait that corresponds to the `ExpandExpression` trait. The semantics are fairly similar, with a few differences. First `ExpandExpression` consumes N tokens and returns a single `hir::Expression`. `ColorSyntax` consumes N tokens and writes M `FlatShape` tokens to the output. Concretely, for syntax like `[1 2 3]` - `ExpandExpression` takes a single token node and produces a single `hir::Expression` - `ColorSyntax` takes the same token node and emits 7 `FlatShape`s (open delimiter, int, whitespace, int, whitespace, int, close delimiter) Second, `ColorSyntax` is more willing to plow through failures than `ExpandExpression`. In particular, consider syntax like ``` $ ps | where cpu > ``` In this case - `ExpandExpression` will see that the `where` command is expecting a block, see that it's not a literal block and try to parse it as a shorthand block. It will successfully find a member followed by an infix operator, but not a following expression. That means that the entire pipeline part fails to parse and is a syntax error. - `ColorSyntax` will also try to parse it as a shorthand block and ultimately fail, but it will fall back to "backoff coloring mode", which parsing any unidentified tokens in an unfallible, simple way. In this case, `cpu` will color as a string and `>` will color as an operator. Finally, it's very important that coloring a pipeline infallibly colors the entire string, doesn't fail, and doesn't get stuck in an infinite loop. In order to accomplish this, this PR separates `ColorSyntax`, which is infallible from `FallibleColorSyntax`, which might fail. This allows the type system to let us know if our coloring rules bottom out at at an infallible rule. It's not perfect: it's still possible for the coloring process to get stuck or consume tokens non-atomically. I intend to reduce the opportunity for those problems in a future commit. In the meantime, the current system catches a number of mistakes (like trying to use a fallible coloring rule in a loop without thinking about the possibility that it will never terminate).
2019-10-06 20:22:50 +00:00
ProximateShellError::MissingProperty { subpath, expr, .. } => {
Fix a bunch of bugs
2019-06-24 00:55:31 +00:00
let mut diag = Diagnostic::new(Severity::Error, "Missing property");
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
if subpath.span == Span::unknown() {
diag.message = format!("Missing property (for {})", subpath.item);
} else {
let subpath = Label::new_primary(subpath.span).with_message(subpath.item);
diag = diag.with_label(subpath);
if expr.span != Span::unknown() {
let expr = Label::new_primary(expr.span).with_message(expr.item);
diag = diag.with_label(expr)
}
Fix a bunch of bugs
2019-06-24 00:55:31 +00:00
}
diag
}
--no-edit
2019-11-04 15:47:03 +00:00
ProximateShellError::InvalidIntegerIndex { subpath,integer } => {
let mut diag = Diagnostic::new(Severity::Error, "Invalid integer property");
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
if subpath.span == Span::unknown() {
diag.message = format!("Invalid integer property (for {})", subpath.item)
} else {
let label = Label::new_primary(subpath.span).with_message(subpath.item);
diag = diag.with_label(label)
--no-edit
2019-11-04 15:47:03 +00:00
}
diag = diag.with_label(Label::new_secondary(integer).with_message("integer"));
diag
}
Tests pass
2019-07-09 04:31:26 +00:00
ProximateShellError::Diagnostic(diag) => diag.diagnostic,
ProximateShellError::CoerceError { left, right } => {
Fix a bunch of bugs
2019-06-24 00:55:31 +00:00
Diagnostic::new(Severity::Error, "Coercion error")
--no-edit
2019-11-04 15:47:03 +00:00
.with_label(Label::new_primary(left.span).with_message(left.item))
.with_label(Label::new_secondary(right.span).with_message(right.item))
Fix a bunch of bugs
2019-06-24 00:55:31 +00:00
}
Overhaul the coloring system This commit replaces the previous naive coloring system with a coloring system that is more aligned with the parser. The main benefit of this change is that it allows us to use parsing rules to decide how to color tokens. For example, consider the following syntax: ``` $ ps | where cpu > 10 ``` Ideally, we could color `cpu` like a column name and not a string, because `cpu > 10` is a shorthand block syntax that expands to `{ $it.cpu > 10 }`. The way that we know that it's a shorthand block is that the `where` command declares that its first parameter is a `SyntaxShape::Block`, which allows the shorthand block form. In order to accomplish this, we need to color the tokens in a way that corresponds to their expanded semantics, which means that high-fidelity coloring requires expansion. This commit adds a `ColorSyntax` trait that corresponds to the `ExpandExpression` trait. The semantics are fairly similar, with a few differences. First `ExpandExpression` consumes N tokens and returns a single `hir::Expression`. `ColorSyntax` consumes N tokens and writes M `FlatShape` tokens to the output. Concretely, for syntax like `[1 2 3]` - `ExpandExpression` takes a single token node and produces a single `hir::Expression` - `ColorSyntax` takes the same token node and emits 7 `FlatShape`s (open delimiter, int, whitespace, int, whitespace, int, close delimiter) Second, `ColorSyntax` is more willing to plow through failures than `ExpandExpression`. In particular, consider syntax like ``` $ ps | where cpu > ``` In this case - `ExpandExpression` will see that the `where` command is expecting a block, see that it's not a literal block and try to parse it as a shorthand block. It will successfully find a member followed by an infix operator, but not a following expression. That means that the entire pipeline part fails to parse and is a syntax error. - `ColorSyntax` will also try to parse it as a shorthand block and ultimately fail, but it will fall back to "backoff coloring mode", which parsing any unidentified tokens in an unfallible, simple way. In this case, `cpu` will color as a string and `>` will color as an operator. Finally, it's very important that coloring a pipeline infallibly colors the entire string, doesn't fail, and doesn't get stuck in an infinite loop. In order to accomplish this, this PR separates `ColorSyntax`, which is infallible from `FallibleColorSyntax`, which might fail. This allows the type system to let us know if our coloring rules bottom out at at an infallible rule. It's not perfect: it's still possible for the coloring process to get stuck or consume tokens non-atomically. I intend to reduce the opportunity for those problems in a future commit. In the meantime, the current system catches a number of mistakes (like trying to use a fallible coloring rule in a loop without thinking about the possibility that it will never terminate).
2019-10-06 20:22:50 +00:00
ProximateShellError::UntaggedRuntimeError { reason } => Diagnostic::new(Severity::Error, format!("Error: {}", reason))
Fix a bunch of bugs
2019-06-24 00:55:31 +00:00
}
}
"Add plugin arg errors. Bring remaining errors to parity"
2019-07-13 16:59:59 +00:00
pub fn labeled_error(
Add pretty errors to commands
2019-06-07 22:35:07 +00:00
msg: impl Into<String>,
label: impl Into<String>,
Extract nu_source into a crate This commit extracts Tag, Span, Text, as well as source-related debug facilities into a new crate called nu_source. This change is much bigger than one might have expected because the previous code relied heavily on implementing inherent methods on `Tagged<T>` and `Spanned<T>`, which is no longer possible. As a result, this change creates more concrete types instead of using `Tagged<T>`. One notable example: Tagged<Value> became Value, and Value became UntaggedValue. This change clarifies the intent of the code in many places, but it does make it a big change.
2019-11-21 14:33:14 +00:00
span: impl Into<Span>,
Add pretty errors to commands
2019-06-07 22:35:07 +00:00
) -> ShellError {
ShellError::diagnostic(
Diagnostic::new(Severity::Error, msg.into())
Extract nu_source into a crate This commit extracts Tag, Span, Text, as well as source-related debug facilities into a new crate called nu_source. This change is much bigger than one might have expected because the previous code relied heavily on implementing inherent methods on `Tagged<T>` and `Spanned<T>`, which is no longer possible. As a result, this change creates more concrete types instead of using `Tagged<T>`. One notable example: Tagged<Value> became Value, and Value became UntaggedValue. This change clarifies the intent of the code in many places, but it does make it a big change.
2019-11-21 14:33:14 +00:00
.with_label(Label::new_primary(span.into()).with_message(label.into())),
Add pretty errors to commands
2019-06-07 22:35:07 +00:00
)
Better errors and more fleshed out semantics
2019-05-30 04:19:46 +00:00
}
Move more parts to tags and away from spans
2019-08-05 08:54:29 +00:00
pub fn labeled_error_with_secondary(
Update errors and improve ctrl-c
2019-06-15 18:36:17 +00:00
msg: impl Into<String>,
Move more parts to tags and away from spans
2019-08-05 08:54:29 +00:00
primary_label: impl Into<String>,
Extract nu_source into a crate This commit extracts Tag, Span, Text, as well as source-related debug facilities into a new crate called nu_source. This change is much bigger than one might have expected because the previous code relied heavily on implementing inherent methods on `Tagged<T>` and `Spanned<T>`, which is no longer possible. As a result, this change creates more concrete types instead of using `Tagged<T>`. One notable example: Tagged<Value> became Value, and Value became UntaggedValue. This change clarifies the intent of the code in many places, but it does make it a big change.
2019-11-21 14:33:14 +00:00
primary_span: impl Into<Span>,
Move more parts to tags and away from spans
2019-08-05 08:54:29 +00:00
secondary_label: impl Into<String>,
Extract nu_source into a crate This commit extracts Tag, Span, Text, as well as source-related debug facilities into a new crate called nu_source. This change is much bigger than one might have expected because the previous code relied heavily on implementing inherent methods on `Tagged<T>` and `Spanned<T>`, which is no longer possible. As a result, this change creates more concrete types instead of using `Tagged<T>`. One notable example: Tagged<Value> became Value, and Value became UntaggedValue. This change clarifies the intent of the code in many places, but it does make it a big change.
2019-11-21 14:33:14 +00:00
secondary_span: impl Into<Span>,
Update errors and improve ctrl-c
2019-06-15 18:36:17 +00:00
) -> ShellError {
Move more parts to tags and away from spans
2019-08-05 08:54:29 +00:00
ShellError::diagnostic(
Diagnostic::new_error(msg.into())
.with_label(
Extract nu_source into a crate This commit extracts Tag, Span, Text, as well as source-related debug facilities into a new crate called nu_source. This change is much bigger than one might have expected because the previous code relied heavily on implementing inherent methods on `Tagged<T>` and `Spanned<T>`, which is no longer possible. As a result, this change creates more concrete types instead of using `Tagged<T>`. One notable example: Tagged<Value> became Value, and Value became UntaggedValue. This change clarifies the intent of the code in many places, but it does make it a big change.
2019-11-21 14:33:14 +00:00
Label::new_primary(primary_span.into()).with_message(primary_label.into()),
Revert "Revert "Migrate most uses of the Span concept to Tag"" This reverts commit bee7c5639cc78d65e6656518627a86009a522500.
2019-09-14 16:30:24 +00:00
)
.with_label(
Extract nu_source into a crate This commit extracts Tag, Span, Text, as well as source-related debug facilities into a new crate called nu_source. This change is much bigger than one might have expected because the previous code relied heavily on implementing inherent methods on `Tagged<T>` and `Spanned<T>`, which is no longer possible. As a result, this change creates more concrete types instead of using `Tagged<T>`. One notable example: Tagged<Value> became Value, and Value became UntaggedValue. This change clarifies the intent of the code in many places, but it does make it a big change.
2019-11-21 14:33:14 +00:00
Label::new_secondary(secondary_span.into())
Revert "Revert "Migrate most uses of the Span concept to Tag"" This reverts commit bee7c5639cc78d65e6656518627a86009a522500.
2019-09-14 16:30:24 +00:00
.with_message(secondary_label.into()),
Move more parts to tags and away from spans
2019-08-05 08:54:29 +00:00
),
)
Update errors and improve ctrl-c
2019-06-15 18:36:17 +00:00
}
Extract core stuff into own crates This commit extracts five new crates: - nu-source, which contains the core source-code handling logic in Nu, including Text, Span, and also the pretty.rs-based debug logic - nu-parser, which is the parser and expander logic - nu-protocol, which is the bulk of the types and basic conveniences used by plugins - nu-errors, which contains ShellError, ParseError and error handling conveniences - nu-textview, which is the textview plugin extracted into a crate One of the major consequences of this refactor is that it's no longer possible to `impl X for Spanned<Y>` outside of the `nu-source` crate, so a lot of types became more concrete (Value became a concrete type instead of Spanned<Value>, for example). This also turned a number of inherent methods in the main nu crate into plain functions (impl Value {} became a bunch of functions in the `value` namespace in `crate::data::value`).
2019-11-26 02:30:48 +00:00
pub fn unimplemented(title: impl Into<String>) -> ShellError {
Overhaul the coloring system This commit replaces the previous naive coloring system with a coloring system that is more aligned with the parser. The main benefit of this change is that it allows us to use parsing rules to decide how to color tokens. For example, consider the following syntax: ``` $ ps | where cpu > 10 ``` Ideally, we could color `cpu` like a column name and not a string, because `cpu > 10` is a shorthand block syntax that expands to `{ $it.cpu > 10 }`. The way that we know that it's a shorthand block is that the `where` command declares that its first parameter is a `SyntaxShape::Block`, which allows the shorthand block form. In order to accomplish this, we need to color the tokens in a way that corresponds to their expanded semantics, which means that high-fidelity coloring requires expansion. This commit adds a `ColorSyntax` trait that corresponds to the `ExpandExpression` trait. The semantics are fairly similar, with a few differences. First `ExpandExpression` consumes N tokens and returns a single `hir::Expression`. `ColorSyntax` consumes N tokens and writes M `FlatShape` tokens to the output. Concretely, for syntax like `[1 2 3]` - `ExpandExpression` takes a single token node and produces a single `hir::Expression` - `ColorSyntax` takes the same token node and emits 7 `FlatShape`s (open delimiter, int, whitespace, int, whitespace, int, close delimiter) Second, `ColorSyntax` is more willing to plow through failures than `ExpandExpression`. In particular, consider syntax like ``` $ ps | where cpu > ``` In this case - `ExpandExpression` will see that the `where` command is expecting a block, see that it's not a literal block and try to parse it as a shorthand block. It will successfully find a member followed by an infix operator, but not a following expression. That means that the entire pipeline part fails to parse and is a syntax error. - `ColorSyntax` will also try to parse it as a shorthand block and ultimately fail, but it will fall back to "backoff coloring mode", which parsing any unidentified tokens in an unfallible, simple way. In this case, `cpu` will color as a string and `>` will color as an operator. Finally, it's very important that coloring a pipeline infallibly colors the entire string, doesn't fail, and doesn't get stuck in an infinite loop. In order to accomplish this, this PR separates `ColorSyntax`, which is infallible from `FallibleColorSyntax`, which might fail. This allows the type system to let us know if our coloring rules bottom out at at an infallible rule. It's not perfect: it's still possible for the coloring process to get stuck or consume tokens non-atomically. I intend to reduce the opportunity for those problems in a future commit. In the meantime, the current system catches a number of mistakes (like trying to use a fallible coloring rule in a loop without thinking about the possibility that it will never terminate).
2019-10-06 20:22:50 +00:00
ShellError::untagged_runtime_error(&format!("Unimplemented: {}", title.into()))
Improve parser
2019-06-04 21:42:31 +00:00
}
Extract core stuff into own crates This commit extracts five new crates: - nu-source, which contains the core source-code handling logic in Nu, including Text, Span, and also the pretty.rs-based debug logic - nu-parser, which is the parser and expander logic - nu-protocol, which is the bulk of the types and basic conveniences used by plugins - nu-errors, which contains ShellError, ParseError and error handling conveniences - nu-textview, which is the textview plugin extracted into a crate One of the major consequences of this refactor is that it's no longer possible to `impl X for Spanned<Y>` outside of the `nu-source` crate, so a lot of types became more concrete (Value became a concrete type instead of Spanned<Value>, for example). This also turned a number of inherent methods in the main nu crate into plain functions (impl Value {} became a bunch of functions in the `value` namespace in `crate::data::value`).
2019-11-26 02:30:48 +00:00
pub fn unexpected(title: impl Into<String>) -> ShellError {
Overhaul the coloring system This commit replaces the previous naive coloring system with a coloring system that is more aligned with the parser. The main benefit of this change is that it allows us to use parsing rules to decide how to color tokens. For example, consider the following syntax: ``` $ ps | where cpu > 10 ``` Ideally, we could color `cpu` like a column name and not a string, because `cpu > 10` is a shorthand block syntax that expands to `{ $it.cpu > 10 }`. The way that we know that it's a shorthand block is that the `where` command declares that its first parameter is a `SyntaxShape::Block`, which allows the shorthand block form. In order to accomplish this, we need to color the tokens in a way that corresponds to their expanded semantics, which means that high-fidelity coloring requires expansion. This commit adds a `ColorSyntax` trait that corresponds to the `ExpandExpression` trait. The semantics are fairly similar, with a few differences. First `ExpandExpression` consumes N tokens and returns a single `hir::Expression`. `ColorSyntax` consumes N tokens and writes M `FlatShape` tokens to the output. Concretely, for syntax like `[1 2 3]` - `ExpandExpression` takes a single token node and produces a single `hir::Expression` - `ColorSyntax` takes the same token node and emits 7 `FlatShape`s (open delimiter, int, whitespace, int, whitespace, int, close delimiter) Second, `ColorSyntax` is more willing to plow through failures than `ExpandExpression`. In particular, consider syntax like ``` $ ps | where cpu > ``` In this case - `ExpandExpression` will see that the `where` command is expecting a block, see that it's not a literal block and try to parse it as a shorthand block. It will successfully find a member followed by an infix operator, but not a following expression. That means that the entire pipeline part fails to parse and is a syntax error. - `ColorSyntax` will also try to parse it as a shorthand block and ultimately fail, but it will fall back to "backoff coloring mode", which parsing any unidentified tokens in an unfallible, simple way. In this case, `cpu` will color as a string and `>` will color as an operator. Finally, it's very important that coloring a pipeline infallibly colors the entire string, doesn't fail, and doesn't get stuck in an infinite loop. In order to accomplish this, this PR separates `ColorSyntax`, which is infallible from `FallibleColorSyntax`, which might fail. This allows the type system to let us know if our coloring rules bottom out at at an infallible rule. It's not perfect: it's still possible for the coloring process to get stuck or consume tokens non-atomically. I intend to reduce the opportunity for those problems in a future commit. In the meantime, the current system catches a number of mistakes (like trying to use a fallible coloring rule in a loop without thinking about the possibility that it will never terminate).
2019-10-06 20:22:50 +00:00
ShellError::untagged_runtime_error(&format!("Unexpected: {}", title.into()))
WIP
2019-06-22 03:43:37 +00:00
}
Better errors and more fleshed out semantics
2019-05-30 04:19:46 +00:00
}
Refactor the repl loop
2019-05-16 21:43:36 +00:00
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
/// `ExpectedRange` describes a range of values that was expected by a command. In addition
/// to typical ranges, this enum allows an error to specify that the range of allowed values
/// corresponds to a particular numeric type (which is a dominant use-case for the
/// [RangeError](ProximateShellError::RangeError) error type).
--no-edit
2019-11-04 15:47:03 +00:00
#[derive(Debug, Eq, PartialEq, Ord, PartialOrd, Hash, Clone, Serialize, Deserialize)]
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
pub enum ExpectedRange {
I8,
I16,
I32,
I64,
I128,
U8,
U16,
U32,
U64,
U128,
F32,
F64,
--no-edit
2019-11-04 15:47:03 +00:00
Usize,
Size,
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
BigInt,
BigDecimal,
--no-edit
2019-11-04 15:47:03 +00:00
Range { start: usize, end: usize },
}
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
/// Convert a Rust range into an [ExpectedRange](ExpectedRange).
--no-edit
2019-11-04 15:47:03 +00:00
impl From<Range<usize>> for ExpectedRange {
fn from(range: Range<usize>) -> Self {
ExpectedRange::Range {
start: range.start,
end: range.end,
}
}
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
}
Remove unused Display implementations After the previous commit, nushell uses PrettyDebug and PrettyDebugWithSource for our pretty-printed display output. PrettyDebug produces a structured `pretty.rs` document rather than writing directly into a fmt::Formatter, and types that implement `PrettyDebug` have a convenience `display` method that produces a string (to be used in situations where `Display` is needed for compatibility with other traits, or where simple rendering is appropriate).
2019-11-25 18:07:20 +00:00
impl PrettyDebug for ExpectedRange {
fn pretty(&self) -> DebugDocBuilder {
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
b::description(match self {
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
ExpectedRange::I8 => "an 8-bit signed integer",
ExpectedRange::I16 => "a 16-bit signed integer",
ExpectedRange::I32 => "a 32-bit signed integer",
ExpectedRange::I64 => "a 64-bit signed integer",
ExpectedRange::I128 => "a 128-bit signed integer",
ExpectedRange::U8 => "an 8-bit unsigned integer",
ExpectedRange::U16 => "a 16-bit unsigned integer",
ExpectedRange::U32 => "a 32-bit unsigned integer",
ExpectedRange::U64 => "a 64-bit unsigned integer",
ExpectedRange::U128 => "a 128-bit unsigned integer",
ExpectedRange::F32 => "a 32-bit float",
ExpectedRange::F64 => "a 64-bit float",
--no-edit
2019-11-04 15:47:03 +00:00
ExpectedRange::Usize => "an list index",
ExpectedRange::Size => "a list offset",
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
ExpectedRange::BigDecimal => "a decimal",
ExpectedRange::BigInt => "an integer",
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
ExpectedRange::Range { start, end } => {
return b::description(format!("{} to {}", start, end))
}
})
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
}
}
--no-edit
2019-11-04 15:47:03 +00:00
#[derive(Debug, Eq, PartialEq, Clone, Ord, PartialOrd, Serialize, Deserialize, Hash)]
Tests pass
2019-07-09 04:31:26 +00:00
pub enum ProximateShellError {
Add support for external escape valve (`^dir`) This commit makes it possible to force nu to treat a command as an external command by prefixing it with `^`. For example `^dir` will force `dir` to run an external command, even if `dir` is also a registered nu command. This ensures that users don't need to leave nu just because we happened to use a command they need. This commit adds a new token type for external commands, which, among other things, makes it pretty straight forward to syntax highlight external commands uniquely, and generally to treat them as special.
2019-08-15 22:18:18 +00:00
SyntaxError {
--no-edit
2019-11-04 15:47:03 +00:00
problem: Spanned<String>,
Add support for external escape valve (`^dir`) This commit makes it possible to force nu to treat a command as an external command by prefixing it with `^`. For example `^dir` will force `dir` to run an external command, even if `dir` is also a registered nu command. This ensures that users don't need to leave nu just because we happened to use a command they need. This commit adds a new token type for external commands, which, among other things, makes it pretty straight forward to syntax highlight external commands uniquely, and generally to treat them as special.
2019-08-15 22:18:18 +00:00
},
Overhaul the expansion system The main thrust of this (very large) commit is an overhaul of the expansion system. The parsing pipeline is: - Lightly parse the source file for atoms, basic delimiters and pipeline structure into a token tree - Expand the token tree into a HIR (high-level intermediate representation) based upon the baseline syntax rules for expressions and the syntactic shape of commands. Somewhat non-traditionally, nu doesn't have an AST at all. It goes directly from the token tree, which doesn't represent many important distinctions (like the difference between `hello` and `5KB`) directly into a high-level representation that doesn't have a direct correspondence to the source code. At a high level, nu commands work like macros, in the sense that the syntactic shape of the invocation of a command depends on the definition of a command. However, commands do not have the ability to perform unrestricted expansions of the token tree. Instead, they describe their arguments in terms of syntactic shapes, and the expander expands the token tree into HIR based upon that definition. For example, the `where` command says that it takes a block as its first required argument, and the description of the block syntactic shape expands the syntax `cpu > 10` into HIR that represents `{ $it.cpu > 10 }`. This commit overhauls that system so that the syntactic shapes are described in terms of a few new traits (`ExpandSyntax` and `ExpandExpression` are the primary ones) that are more composable than the previous system. The first big win of this new system is the addition of the `ColumnPath` shape, which looks like `cpu."max ghz"` or `package.version`. Previously, while a variable path could look like `$it.cpu."max ghz"`, the tail of a variable path could not be easily reused in other contexts. Now, that tail is its own syntactic shape, and it can be used as part of a command's signature. This cleans up commands like `inc`, `add` and `edit` as well as shorthand blocks, which can now look like `| where cpu."max ghz" > 10`
2019-09-17 22:26:27 +00:00
UnexpectedEof {
expected: String,
--no-edit
2019-11-04 15:47:03 +00:00
span: Span,
Add support for external escape valve (`^dir`) This commit makes it possible to force nu to treat a command as an external command by prefixing it with `^`. For example `^dir` will force `dir` to run an external command, even if `dir` is also a registered nu command. This ensures that users don't need to leave nu just because we happened to use a command they need. This commit adds a new token type for external commands, which, among other things, makes it pretty straight forward to syntax highlight external commands uniquely, and generally to treat them as special.
2019-08-15 22:18:18 +00:00
},
Tests pass
2019-07-09 04:31:26 +00:00
TypeError {
expected: String,
--no-edit
2019-11-04 15:47:03 +00:00
actual: Spanned<Option<String>>,
Tests pass
2019-07-09 04:31:26 +00:00
},
MissingProperty {
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
subpath: Spanned<String>,
expr: Spanned<String>,
--no-edit
2019-11-04 15:47:03 +00:00
},
InvalidIntegerIndex {
Try to re-apply #1039
2019-12-02 16:14:05 +00:00
subpath: Spanned<String>,
--no-edit
2019-11-04 15:47:03 +00:00
integer: Span,
Tests pass
2019-07-09 04:31:26 +00:00
},
Reduce unwraps Remove a number of unwraps. In some cases, a `?` just worked as is. I also made it possible to use `?` to go from Result<OutputStream, ShellError> to OutputStream. Finally, started updating PerItemCommand to be able to use the signature deserialization logic, which substantially reduces unwraps. This is still in-progress work, but tests pass and it should be clear to merge and keep iterating on master.
2019-08-17 03:53:39 +00:00
MissingValue {
--no-edit
2019-11-04 15:47:03 +00:00
span: Option<Span>,
Reduce unwraps Remove a number of unwraps. In some cases, a `?` just worked as is. I also made it possible to use `?` to go from Result<OutputStream, ShellError> to OutputStream. Finally, started updating PerItemCommand to be able to use the signature deserialization logic, which substantially reduces unwraps. This is still in-progress work, but tests pass and it should be clear to merge and keep iterating on master.
2019-08-17 03:53:39 +00:00
reason: String,
},
Tests pass
2019-07-09 04:31:26 +00:00
ArgumentError {
--no-edit
2019-11-04 15:47:03 +00:00
command: Spanned<String>,
Tests pass
2019-07-09 04:31:26 +00:00
error: ArgumentError,
},
Basic support for decimal numbers This commit is more substantial than it looks: there was basically no real support for decimals before, and that impacted values all the way through. I also made Size contain a decimal instead of an integer (`1.6kb` is a reasonable thing to type), which impacted a bunch of code. The biggest impact of this commit is that it creates many more possible ways for valid nu types to fail to serialize as toml, json, etc. which typically can't support the full range of Decimal (or Bigint, which I also think we should support). This commit makes to-toml fallible, and a similar effort is necessary for the rest of the serializations. We also need to figure out how to clearly communicate to users what has happened, but failing to serialize to toml seems clearly superior to me than weird errors in basic math operations.
2019-08-30 17:29:04 +00:00
RangeError {
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
kind: ExpectedRange,
--no-edit
2019-11-04 15:47:03 +00:00
actual_kind: Spanned<String>,
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
operation: String,
Basic support for decimal numbers This commit is more substantial than it looks: there was basically no real support for decimals before, and that impacted values all the way through. I also made Size contain a decimal instead of an integer (`1.6kb` is a reasonable thing to type), which impacted a bunch of code. The biggest impact of this commit is that it creates many more possible ways for valid nu types to fail to serialize as toml, json, etc. which typically can't support the full range of Decimal (or Bigint, which I also think we should support). This commit makes to-toml fallible, and a similar effort is necessary for the rest of the serializations. We also need to figure out how to clearly communicate to users what has happened, but failing to serialize to toml seems clearly superior to me than weird errors in basic math operations.
2019-08-30 17:29:04 +00:00
},
Tests pass
2019-07-09 04:31:26 +00:00
Diagnostic(ShellDiagnostic),
CoerceError {
--no-edit
2019-11-04 15:47:03 +00:00
left: Spanned<String>,
right: Spanned<String>,
Tests pass
2019-07-09 04:31:26 +00:00
},
Overhaul the coloring system This commit replaces the previous naive coloring system with a coloring system that is more aligned with the parser. The main benefit of this change is that it allows us to use parsing rules to decide how to color tokens. For example, consider the following syntax: ``` $ ps | where cpu > 10 ``` Ideally, we could color `cpu` like a column name and not a string, because `cpu > 10` is a shorthand block syntax that expands to `{ $it.cpu > 10 }`. The way that we know that it's a shorthand block is that the `where` command declares that its first parameter is a `SyntaxShape::Block`, which allows the shorthand block form. In order to accomplish this, we need to color the tokens in a way that corresponds to their expanded semantics, which means that high-fidelity coloring requires expansion. This commit adds a `ColorSyntax` trait that corresponds to the `ExpandExpression` trait. The semantics are fairly similar, with a few differences. First `ExpandExpression` consumes N tokens and returns a single `hir::Expression`. `ColorSyntax` consumes N tokens and writes M `FlatShape` tokens to the output. Concretely, for syntax like `[1 2 3]` - `ExpandExpression` takes a single token node and produces a single `hir::Expression` - `ColorSyntax` takes the same token node and emits 7 `FlatShape`s (open delimiter, int, whitespace, int, whitespace, int, close delimiter) Second, `ColorSyntax` is more willing to plow through failures than `ExpandExpression`. In particular, consider syntax like ``` $ ps | where cpu > ``` In this case - `ExpandExpression` will see that the `where` command is expecting a block, see that it's not a literal block and try to parse it as a shorthand block. It will successfully find a member followed by an infix operator, but not a following expression. That means that the entire pipeline part fails to parse and is a syntax error. - `ColorSyntax` will also try to parse it as a shorthand block and ultimately fail, but it will fall back to "backoff coloring mode", which parsing any unidentified tokens in an unfallible, simple way. In this case, `cpu` will color as a string and `>` will color as an operator. Finally, it's very important that coloring a pipeline infallibly colors the entire string, doesn't fail, and doesn't get stuck in an infinite loop. In order to accomplish this, this PR separates `ColorSyntax`, which is infallible from `FallibleColorSyntax`, which might fail. This allows the type system to let us know if our coloring rules bottom out at at an infallible rule. It's not perfect: it's still possible for the coloring process to get stuck or consume tokens non-atomically. I intend to reduce the opportunity for those problems in a future commit. In the meantime, the current system catches a number of mistakes (like trying to use a fallible coloring rule in a loop without thinking about the possibility that it will never terminate).
2019-10-06 20:22:50 +00:00
UntaggedRuntimeError {
reason: String,
},
Tests pass
2019-07-09 04:31:26 +00:00
}
Add support for ~ expansion This ended up being a bit of a yak shave. The basic idea in this commit is to expand `~` in paths, but only in paths. The way this is accomplished is by doing the expansion inside of the code that parses literal syntax for `SyntaxType::Path`. As a quick refresher: every command is entitled to expand its arguments in a custom way. While this could in theory be used for general-purpose macros, today the expansion facility is limited to syntactic hints. For example, the syntax `where cpu > 0` expands under the hood to `where { $it.cpu > 0 }`. This happens because the first argument to `where` is defined as a `SyntaxType::Block`, and the parser coerces binary expressions whose left-hand-side looks like a member into a block when the command is expecting one. This is mildly more magical than what most programming languages would do, but we believe that it makes sense to allow commands to fine-tune the syntax because of the domain nushell is in (command-line shells). The syntactic expansions supported by this facility are relatively limited. For example, we don't allow `$it` to become a bare word, simply because the command asks for a string in the relevant position. That would quickly become more confusing than it's worth. This PR adds a new `SyntaxType` rule: `SyntaxType::Path`. When a command declares a parameter as a `SyntaxType::Path`, string literals and bare words passed as an argument to that parameter are processed using the path expansion rules. Right now, that only means that `~` is expanded into the home directory, but additional rules are possible in the future. By restricting this expansion to a syntactic expansion when passed as an argument to a command expecting a path, we avoid making `~` a generally reserved character. This will also allow us to give good tab completion for paths with `~` characters in them when a command is expecting a path. In order to accomplish the above, this commit changes the parsing functions to take a `Context` instead of just a `CommandRegistry`. From the perspective of macro expansion, you can think of the `CommandRegistry` as a dictionary of in-scope macros, and the `Context` as the compile-time state used in expansion. This could gain additional functionality over time as we find more uses for the expansion system.
2019-08-26 19:21:03 +00:00
Tests pass
2019-07-09 04:31:26 +00:00
impl ProximateShellError {
fn start(self) -> ShellError {
ShellError {
cause: None,
error: self,
}
}
}
"Add plugin arg errors. Bring remaining errors to parity"
2019-07-13 16:59:59 +00:00
#[derive(Debug, Clone, Serialize, Deserialize)]
Better errors and more fleshed out semantics
2019-05-30 04:19:46 +00:00
pub struct ShellDiagnostic {
Stream support (#812) * Moves off of draining between filters. Instead, the sink will pull on the stream, and will drain element-wise. This moves the whole stream to being lazy. * Adds ctrl-c support and connects it into some of the key points where we pull on the stream. If a ctrl-c is detect, we immediately halt pulling on the stream and return to the prompt. * Moves away from having a SourceMap where anchor locations are stored. Now AnchorLocation is kept directly in the Tag. * To make this possible, split tag and span. Span is largely used in the parser and is copyable. Tag is now no longer copyable.
2019-10-13 04:12:43 +00:00
pub(crate) diagnostic: Diagnostic<Span>,
Better errors and more fleshed out semantics
2019-05-30 04:19:46 +00:00
}
--no-edit
2019-11-04 15:47:03 +00:00
impl std::hash::Hash for ShellDiagnostic {
fn hash<H: std::hash::Hasher>(&self, state: &mut H) {
self.diagnostic.severity.hash(state);
self.diagnostic.code.hash(state);
self.diagnostic.message.hash(state);
for label in &self.diagnostic.labels {
label.span.hash(state);
label.message.hash(state);
match label.style {
language_reporting::LabelStyle::Primary => 0.hash(state),
language_reporting::LabelStyle::Secondary => 1.hash(state),
}
}
}
}
Better errors and more fleshed out semantics
2019-05-30 04:19:46 +00:00
impl PartialEq for ShellDiagnostic {
fn eq(&self, _other: &ShellDiagnostic) -> bool {
false
Refactor the repl loop
2019-05-16 21:43:36 +00:00
}
objectshell initial commit
2019-05-10 16:59:12 +00:00
}
Better errors and more fleshed out semantics
2019-05-30 04:19:46 +00:00
impl Eq for ShellDiagnostic {}
impl std::cmp::PartialOrd for ShellDiagnostic {
fn partial_cmp(&self, _other: &Self) -> Option<std::cmp::Ordering> {
Some(std::cmp::Ordering::Less)
}
}
impl std::cmp::Ord for ShellDiagnostic {
fn cmp(&self, _other: &Self) -> std::cmp::Ordering {
std::cmp::Ordering::Less
}
}
Improve signature infrastructure The `config` command uses different kinds of named arguments, which illustrates how it works.
2019-06-01 05:50:16 +00:00
#[derive(Debug, Ord, PartialOrd, Eq, PartialEq, new, Clone, Serialize, Deserialize)]
Better errors and more fleshed out semantics
2019-05-30 04:19:46 +00:00
pub struct StringError {
title: String,
Stream support (#812) * Moves off of draining between filters. Instead, the sink will pull on the stream, and will drain element-wise. This moves the whole stream to being lazy. * Adds ctrl-c support and connects it into some of the key points where we pull on the stream. If a ctrl-c is detect, we immediately halt pulling on the stream and return to the prompt. * Moves away from having a SourceMap where anchor locations are stored. Now AnchorLocation is kept directly in the Tag. * To make this possible, split tag and span. Span is largely used in the parser and is copyable. Tag is now no longer copyable.
2019-10-13 04:12:43 +00:00
error: String,
Better errors and more fleshed out semantics
2019-05-30 04:19:46 +00:00
}
objectshell initial commit
2019-05-10 16:59:12 +00:00
impl std::error::Error for ShellError {}
Unwrap cleanup mitigation.
2019-08-21 12:08:23 +00:00
impl std::convert::From<Box<dyn std::error::Error>> for ShellError {
fn from(input: Box<dyn std::error::Error>) -> ShellError {
Overhaul the coloring system This commit replaces the previous naive coloring system with a coloring system that is more aligned with the parser. The main benefit of this change is that it allows us to use parsing rules to decide how to color tokens. For example, consider the following syntax: ``` $ ps | where cpu > 10 ``` Ideally, we could color `cpu` like a column name and not a string, because `cpu > 10` is a shorthand block syntax that expands to `{ $it.cpu > 10 }`. The way that we know that it's a shorthand block is that the `where` command declares that its first parameter is a `SyntaxShape::Block`, which allows the shorthand block form. In order to accomplish this, we need to color the tokens in a way that corresponds to their expanded semantics, which means that high-fidelity coloring requires expansion. This commit adds a `ColorSyntax` trait that corresponds to the `ExpandExpression` trait. The semantics are fairly similar, with a few differences. First `ExpandExpression` consumes N tokens and returns a single `hir::Expression`. `ColorSyntax` consumes N tokens and writes M `FlatShape` tokens to the output. Concretely, for syntax like `[1 2 3]` - `ExpandExpression` takes a single token node and produces a single `hir::Expression` - `ColorSyntax` takes the same token node and emits 7 `FlatShape`s (open delimiter, int, whitespace, int, whitespace, int, close delimiter) Second, `ColorSyntax` is more willing to plow through failures than `ExpandExpression`. In particular, consider syntax like ``` $ ps | where cpu > ``` In this case - `ExpandExpression` will see that the `where` command is expecting a block, see that it's not a literal block and try to parse it as a shorthand block. It will successfully find a member followed by an infix operator, but not a following expression. That means that the entire pipeline part fails to parse and is a syntax error. - `ColorSyntax` will also try to parse it as a shorthand block and ultimately fail, but it will fall back to "backoff coloring mode", which parsing any unidentified tokens in an unfallible, simple way. In this case, `cpu` will color as a string and `>` will color as an operator. Finally, it's very important that coloring a pipeline infallibly colors the entire string, doesn't fail, and doesn't get stuck in an infinite loop. In order to accomplish this, this PR separates `ColorSyntax`, which is infallible from `FallibleColorSyntax`, which might fail. This allows the type system to let us know if our coloring rules bottom out at at an infallible rule. It's not perfect: it's still possible for the coloring process to get stuck or consume tokens non-atomically. I intend to reduce the opportunity for those problems in a future commit. In the meantime, the current system catches a number of mistakes (like trying to use a fallible coloring rule in a loop without thinking about the possibility that it will never terminate).
2019-10-06 20:22:50 +00:00
ShellError::untagged_runtime_error(format!("{}", input))
Unwrap cleanup mitigation.
2019-08-21 12:08:23 +00:00
}
}
objectshell initial commit
2019-05-10 16:59:12 +00:00
impl std::convert::From<std::io::Error> for ShellError {
fn from(input: std::io::Error) -> ShellError {
Overhaul the coloring system This commit replaces the previous naive coloring system with a coloring system that is more aligned with the parser. The main benefit of this change is that it allows us to use parsing rules to decide how to color tokens. For example, consider the following syntax: ``` $ ps | where cpu > 10 ``` Ideally, we could color `cpu` like a column name and not a string, because `cpu > 10` is a shorthand block syntax that expands to `{ $it.cpu > 10 }`. The way that we know that it's a shorthand block is that the `where` command declares that its first parameter is a `SyntaxShape::Block`, which allows the shorthand block form. In order to accomplish this, we need to color the tokens in a way that corresponds to their expanded semantics, which means that high-fidelity coloring requires expansion. This commit adds a `ColorSyntax` trait that corresponds to the `ExpandExpression` trait. The semantics are fairly similar, with a few differences. First `ExpandExpression` consumes N tokens and returns a single `hir::Expression`. `ColorSyntax` consumes N tokens and writes M `FlatShape` tokens to the output. Concretely, for syntax like `[1 2 3]` - `ExpandExpression` takes a single token node and produces a single `hir::Expression` - `ColorSyntax` takes the same token node and emits 7 `FlatShape`s (open delimiter, int, whitespace, int, whitespace, int, close delimiter) Second, `ColorSyntax` is more willing to plow through failures than `ExpandExpression`. In particular, consider syntax like ``` $ ps | where cpu > ``` In this case - `ExpandExpression` will see that the `where` command is expecting a block, see that it's not a literal block and try to parse it as a shorthand block. It will successfully find a member followed by an infix operator, but not a following expression. That means that the entire pipeline part fails to parse and is a syntax error. - `ColorSyntax` will also try to parse it as a shorthand block and ultimately fail, but it will fall back to "backoff coloring mode", which parsing any unidentified tokens in an unfallible, simple way. In this case, `cpu` will color as a string and `>` will color as an operator. Finally, it's very important that coloring a pipeline infallibly colors the entire string, doesn't fail, and doesn't get stuck in an infinite loop. In order to accomplish this, this PR separates `ColorSyntax`, which is infallible from `FallibleColorSyntax`, which might fail. This allows the type system to let us know if our coloring rules bottom out at at an infallible rule. It's not perfect: it's still possible for the coloring process to get stuck or consume tokens non-atomically. I intend to reduce the opportunity for those problems in a future commit. In the meantime, the current system catches a number of mistakes (like trying to use a fallible coloring rule in a loop without thinking about the possibility that it will never terminate).
2019-10-06 20:22:50 +00:00
ShellError::untagged_runtime_error(format!("{}", input))
objectshell initial commit
2019-05-10 16:59:12 +00:00
}
}
Improved streams
2019-05-24 04:34:43 +00:00
Streams are wired up pairwise
2019-05-24 07:29:16 +00:00
impl std::convert::From<subprocess::PopenError> for ShellError {
fn from(input: subprocess::PopenError) -> ShellError {
Overhaul the coloring system This commit replaces the previous naive coloring system with a coloring system that is more aligned with the parser. The main benefit of this change is that it allows us to use parsing rules to decide how to color tokens. For example, consider the following syntax: ``` $ ps | where cpu > 10 ``` Ideally, we could color `cpu` like a column name and not a string, because `cpu > 10` is a shorthand block syntax that expands to `{ $it.cpu > 10 }`. The way that we know that it's a shorthand block is that the `where` command declares that its first parameter is a `SyntaxShape::Block`, which allows the shorthand block form. In order to accomplish this, we need to color the tokens in a way that corresponds to their expanded semantics, which means that high-fidelity coloring requires expansion. This commit adds a `ColorSyntax` trait that corresponds to the `ExpandExpression` trait. The semantics are fairly similar, with a few differences. First `ExpandExpression` consumes N tokens and returns a single `hir::Expression`. `ColorSyntax` consumes N tokens and writes M `FlatShape` tokens to the output. Concretely, for syntax like `[1 2 3]` - `ExpandExpression` takes a single token node and produces a single `hir::Expression` - `ColorSyntax` takes the same token node and emits 7 `FlatShape`s (open delimiter, int, whitespace, int, whitespace, int, close delimiter) Second, `ColorSyntax` is more willing to plow through failures than `ExpandExpression`. In particular, consider syntax like ``` $ ps | where cpu > ``` In this case - `ExpandExpression` will see that the `where` command is expecting a block, see that it's not a literal block and try to parse it as a shorthand block. It will successfully find a member followed by an infix operator, but not a following expression. That means that the entire pipeline part fails to parse and is a syntax error. - `ColorSyntax` will also try to parse it as a shorthand block and ultimately fail, but it will fall back to "backoff coloring mode", which parsing any unidentified tokens in an unfallible, simple way. In this case, `cpu` will color as a string and `>` will color as an operator. Finally, it's very important that coloring a pipeline infallibly colors the entire string, doesn't fail, and doesn't get stuck in an infinite loop. In order to accomplish this, this PR separates `ColorSyntax`, which is infallible from `FallibleColorSyntax`, which might fail. This allows the type system to let us know if our coloring rules bottom out at at an infallible rule. It's not perfect: it's still possible for the coloring process to get stuck or consume tokens non-atomically. I intend to reduce the opportunity for those problems in a future commit. In the meantime, the current system catches a number of mistakes (like trying to use a fallible coloring rule in a loop without thinking about the possibility that it will never terminate).
2019-10-06 20:22:50 +00:00
ShellError::untagged_runtime_error(format!("{}", input))
Streams are wired up pairwise
2019-05-24 07:29:16 +00:00
}
}
A real parser (lalrpop)
2019-05-26 06:54:41 +00:00
Unwrap cleanup mitigation.
2019-08-21 12:08:23 +00:00
impl std::convert::From<serde_yaml::Error> for ShellError {
fn from(input: serde_yaml::Error) -> ShellError {
Overhaul the coloring system This commit replaces the previous naive coloring system with a coloring system that is more aligned with the parser. The main benefit of this change is that it allows us to use parsing rules to decide how to color tokens. For example, consider the following syntax: ``` $ ps | where cpu > 10 ``` Ideally, we could color `cpu` like a column name and not a string, because `cpu > 10` is a shorthand block syntax that expands to `{ $it.cpu > 10 }`. The way that we know that it's a shorthand block is that the `where` command declares that its first parameter is a `SyntaxShape::Block`, which allows the shorthand block form. In order to accomplish this, we need to color the tokens in a way that corresponds to their expanded semantics, which means that high-fidelity coloring requires expansion. This commit adds a `ColorSyntax` trait that corresponds to the `ExpandExpression` trait. The semantics are fairly similar, with a few differences. First `ExpandExpression` consumes N tokens and returns a single `hir::Expression`. `ColorSyntax` consumes N tokens and writes M `FlatShape` tokens to the output. Concretely, for syntax like `[1 2 3]` - `ExpandExpression` takes a single token node and produces a single `hir::Expression` - `ColorSyntax` takes the same token node and emits 7 `FlatShape`s (open delimiter, int, whitespace, int, whitespace, int, close delimiter) Second, `ColorSyntax` is more willing to plow through failures than `ExpandExpression`. In particular, consider syntax like ``` $ ps | where cpu > ``` In this case - `ExpandExpression` will see that the `where` command is expecting a block, see that it's not a literal block and try to parse it as a shorthand block. It will successfully find a member followed by an infix operator, but not a following expression. That means that the entire pipeline part fails to parse and is a syntax error. - `ColorSyntax` will also try to parse it as a shorthand block and ultimately fail, but it will fall back to "backoff coloring mode", which parsing any unidentified tokens in an unfallible, simple way. In this case, `cpu` will color as a string and `>` will color as an operator. Finally, it's very important that coloring a pipeline infallibly colors the entire string, doesn't fail, and doesn't get stuck in an infinite loop. In order to accomplish this, this PR separates `ColorSyntax`, which is infallible from `FallibleColorSyntax`, which might fail. This allows the type system to let us know if our coloring rules bottom out at at an infallible rule. It's not perfect: it's still possible for the coloring process to get stuck or consume tokens non-atomically. I intend to reduce the opportunity for those problems in a future commit. In the meantime, the current system catches a number of mistakes (like trying to use a fallible coloring rule in a loop without thinking about the possibility that it will never terminate).
2019-10-06 20:22:50 +00:00
ShellError::untagged_runtime_error(format!("{:?}", input))
Unwrap cleanup mitigation.
2019-08-21 12:08:23 +00:00
}
}
Improve signature infrastructure The `config` command uses different kinds of named arguments, which illustrates how it works.
2019-06-01 05:50:16 +00:00
impl std::convert::From<toml::ser::Error> for ShellError {
fn from(input: toml::ser::Error) -> ShellError {
Overhaul the coloring system This commit replaces the previous naive coloring system with a coloring system that is more aligned with the parser. The main benefit of this change is that it allows us to use parsing rules to decide how to color tokens. For example, consider the following syntax: ``` $ ps | where cpu > 10 ``` Ideally, we could color `cpu` like a column name and not a string, because `cpu > 10` is a shorthand block syntax that expands to `{ $it.cpu > 10 }`. The way that we know that it's a shorthand block is that the `where` command declares that its first parameter is a `SyntaxShape::Block`, which allows the shorthand block form. In order to accomplish this, we need to color the tokens in a way that corresponds to their expanded semantics, which means that high-fidelity coloring requires expansion. This commit adds a `ColorSyntax` trait that corresponds to the `ExpandExpression` trait. The semantics are fairly similar, with a few differences. First `ExpandExpression` consumes N tokens and returns a single `hir::Expression`. `ColorSyntax` consumes N tokens and writes M `FlatShape` tokens to the output. Concretely, for syntax like `[1 2 3]` - `ExpandExpression` takes a single token node and produces a single `hir::Expression` - `ColorSyntax` takes the same token node and emits 7 `FlatShape`s (open delimiter, int, whitespace, int, whitespace, int, close delimiter) Second, `ColorSyntax` is more willing to plow through failures than `ExpandExpression`. In particular, consider syntax like ``` $ ps | where cpu > ``` In this case - `ExpandExpression` will see that the `where` command is expecting a block, see that it's not a literal block and try to parse it as a shorthand block. It will successfully find a member followed by an infix operator, but not a following expression. That means that the entire pipeline part fails to parse and is a syntax error. - `ColorSyntax` will also try to parse it as a shorthand block and ultimately fail, but it will fall back to "backoff coloring mode", which parsing any unidentified tokens in an unfallible, simple way. In this case, `cpu` will color as a string and `>` will color as an operator. Finally, it's very important that coloring a pipeline infallibly colors the entire string, doesn't fail, and doesn't get stuck in an infinite loop. In order to accomplish this, this PR separates `ColorSyntax`, which is infallible from `FallibleColorSyntax`, which might fail. This allows the type system to let us know if our coloring rules bottom out at at an infallible rule. It's not perfect: it's still possible for the coloring process to get stuck or consume tokens non-atomically. I intend to reduce the opportunity for those problems in a future commit. In the meantime, the current system catches a number of mistakes (like trying to use a fallible coloring rule in a loop without thinking about the possibility that it will never terminate).
2019-10-06 20:22:50 +00:00
ShellError::untagged_runtime_error(format!("{:?}", input))
Improve signature infrastructure The `config` command uses different kinds of named arguments, which illustrates how it works.
2019-06-01 05:50:16 +00:00
}
}
Reduce unwraps Remove a number of unwraps. In some cases, a `?` just worked as is. I also made it possible to use `?` to go from Result<OutputStream, ShellError> to OutputStream. Finally, started updating PerItemCommand to be able to use the signature deserialization logic, which substantially reduces unwraps. This is still in-progress work, but tests pass and it should be clear to merge and keep iterating on master.
2019-08-17 03:53:39 +00:00
impl std::convert::From<serde_json::Error> for ShellError {
fn from(input: serde_json::Error) -> ShellError {
Overhaul the coloring system This commit replaces the previous naive coloring system with a coloring system that is more aligned with the parser. The main benefit of this change is that it allows us to use parsing rules to decide how to color tokens. For example, consider the following syntax: ``` $ ps | where cpu > 10 ``` Ideally, we could color `cpu` like a column name and not a string, because `cpu > 10` is a shorthand block syntax that expands to `{ $it.cpu > 10 }`. The way that we know that it's a shorthand block is that the `where` command declares that its first parameter is a `SyntaxShape::Block`, which allows the shorthand block form. In order to accomplish this, we need to color the tokens in a way that corresponds to their expanded semantics, which means that high-fidelity coloring requires expansion. This commit adds a `ColorSyntax` trait that corresponds to the `ExpandExpression` trait. The semantics are fairly similar, with a few differences. First `ExpandExpression` consumes N tokens and returns a single `hir::Expression`. `ColorSyntax` consumes N tokens and writes M `FlatShape` tokens to the output. Concretely, for syntax like `[1 2 3]` - `ExpandExpression` takes a single token node and produces a single `hir::Expression` - `ColorSyntax` takes the same token node and emits 7 `FlatShape`s (open delimiter, int, whitespace, int, whitespace, int, close delimiter) Second, `ColorSyntax` is more willing to plow through failures than `ExpandExpression`. In particular, consider syntax like ``` $ ps | where cpu > ``` In this case - `ExpandExpression` will see that the `where` command is expecting a block, see that it's not a literal block and try to parse it as a shorthand block. It will successfully find a member followed by an infix operator, but not a following expression. That means that the entire pipeline part fails to parse and is a syntax error. - `ColorSyntax` will also try to parse it as a shorthand block and ultimately fail, but it will fall back to "backoff coloring mode", which parsing any unidentified tokens in an unfallible, simple way. In this case, `cpu` will color as a string and `>` will color as an operator. Finally, it's very important that coloring a pipeline infallibly colors the entire string, doesn't fail, and doesn't get stuck in an infinite loop. In order to accomplish this, this PR separates `ColorSyntax`, which is infallible from `FallibleColorSyntax`, which might fail. This allows the type system to let us know if our coloring rules bottom out at at an infallible rule. It's not perfect: it's still possible for the coloring process to get stuck or consume tokens non-atomically. I intend to reduce the opportunity for those problems in a future commit. In the meantime, the current system catches a number of mistakes (like trying to use a fallible coloring rule in a loop without thinking about the possibility that it will never terminate).
2019-10-06 20:22:50 +00:00
ShellError::untagged_runtime_error(format!("{:?}", input))
Reduce unwraps Remove a number of unwraps. In some cases, a `?` just worked as is. I also made it possible to use `?` to go from Result<OutputStream, ShellError> to OutputStream. Finally, started updating PerItemCommand to be able to use the signature deserialization logic, which substantially reduces unwraps. This is still in-progress work, but tests pass and it should be clear to merge and keep iterating on master.
2019-08-17 03:53:39 +00:00
}
}
switch from reqwest to surf
2019-08-24 19:36:19 +00:00
impl std::convert::From<Box<dyn std::error::Error + Send + Sync>> for ShellError {
fn from(input: Box<dyn std::error::Error + Send + Sync>) -> ShellError {
Overhaul the coloring system This commit replaces the previous naive coloring system with a coloring system that is more aligned with the parser. The main benefit of this change is that it allows us to use parsing rules to decide how to color tokens. For example, consider the following syntax: ``` $ ps | where cpu > 10 ``` Ideally, we could color `cpu` like a column name and not a string, because `cpu > 10` is a shorthand block syntax that expands to `{ $it.cpu > 10 }`. The way that we know that it's a shorthand block is that the `where` command declares that its first parameter is a `SyntaxShape::Block`, which allows the shorthand block form. In order to accomplish this, we need to color the tokens in a way that corresponds to their expanded semantics, which means that high-fidelity coloring requires expansion. This commit adds a `ColorSyntax` trait that corresponds to the `ExpandExpression` trait. The semantics are fairly similar, with a few differences. First `ExpandExpression` consumes N tokens and returns a single `hir::Expression`. `ColorSyntax` consumes N tokens and writes M `FlatShape` tokens to the output. Concretely, for syntax like `[1 2 3]` - `ExpandExpression` takes a single token node and produces a single `hir::Expression` - `ColorSyntax` takes the same token node and emits 7 `FlatShape`s (open delimiter, int, whitespace, int, whitespace, int, close delimiter) Second, `ColorSyntax` is more willing to plow through failures than `ExpandExpression`. In particular, consider syntax like ``` $ ps | where cpu > ``` In this case - `ExpandExpression` will see that the `where` command is expecting a block, see that it's not a literal block and try to parse it as a shorthand block. It will successfully find a member followed by an infix operator, but not a following expression. That means that the entire pipeline part fails to parse and is a syntax error. - `ColorSyntax` will also try to parse it as a shorthand block and ultimately fail, but it will fall back to "backoff coloring mode", which parsing any unidentified tokens in an unfallible, simple way. In this case, `cpu` will color as a string and `>` will color as an operator. Finally, it's very important that coloring a pipeline infallibly colors the entire string, doesn't fail, and doesn't get stuck in an infinite loop. In order to accomplish this, this PR separates `ColorSyntax`, which is infallible from `FallibleColorSyntax`, which might fail. This allows the type system to let us know if our coloring rules bottom out at at an infallible rule. It's not perfect: it's still possible for the coloring process to get stuck or consume tokens non-atomically. I intend to reduce the opportunity for those problems in a future commit. In the meantime, the current system catches a number of mistakes (like trying to use a fallible coloring rule in a loop without thinking about the possibility that it will never terminate).
2019-10-06 20:22:50 +00:00
ShellError::untagged_runtime_error(format!("{:?}", input))
switch from reqwest to surf
2019-08-24 19:36:19 +00:00
}
}
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
pub trait CoerceInto<U> {
fn coerce_into(self, operation: impl Into<String>) -> Result<U, ShellError>;
Basic support for decimal numbers This commit is more substantial than it looks: there was basically no real support for decimals before, and that impacted values all the way through. I also made Size contain a decimal instead of an integer (`1.6kb` is a reasonable thing to type), which impacted a bunch of code. The biggest impact of this commit is that it creates many more possible ways for valid nu types to fail to serialize as toml, json, etc. which typically can't support the full range of Decimal (or Bigint, which I also think we should support). This commit makes to-toml fallible, and a similar effort is necessary for the rest of the serializations. We also need to figure out how to clearly communicate to users what has happened, but failing to serialize to toml seems clearly superior to me than weird errors in basic math operations.
2019-08-30 17:29:04 +00:00
}
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
trait ToExpectedRange {
fn to_expected_range() -> ExpectedRange;
}
macro_rules! ranged_int {
($ty:tt -> $op:tt -> $variant:tt) => {
impl ToExpectedRange for $ty {
fn to_expected_range() -> ExpectedRange {
ExpectedRange::$variant
}
}
Extract nu_source into a crate This commit extracts Tag, Span, Text, as well as source-related debug facilities into a new crate called nu_source. This change is much bigger than one might have expected because the previous code relied heavily on implementing inherent methods on `Tagged<T>` and `Spanned<T>`, which is no longer possible. As a result, this change creates more concrete types instead of using `Tagged<T>`. One notable example: Tagged<Value> became Value, and Value became UntaggedValue. This change clarifies the intent of the code in many places, but it does make it a big change.
2019-11-21 14:33:14 +00:00
impl CoerceInto<$ty> for nu_source::Tagged<BigInt> {
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
fn coerce_into(self, operation: impl Into<String>) -> Result<$ty, ShellError> {
match self.$op() {
Some(v) => Ok(v),
None => Err(ShellError::range_error(
$ty::to_expected_range(),
Extract nu_source into a crate This commit extracts Tag, Span, Text, as well as source-related debug facilities into a new crate called nu_source. This change is much bigger than one might have expected because the previous code relied heavily on implementing inherent methods on `Tagged<T>` and `Spanned<T>`, which is no longer possible. As a result, this change creates more concrete types instead of using `Tagged<T>`. One notable example: Tagged<Value> became Value, and Value became UntaggedValue. This change clarifies the intent of the code in many places, but it does make it a big change.
2019-11-21 14:33:14 +00:00
&self.item.spanned(self.tag.span),
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
operation.into(),
)),
}
}
}
Extract nu_source into a crate This commit extracts Tag, Span, Text, as well as source-related debug facilities into a new crate called nu_source. This change is much bigger than one might have expected because the previous code relied heavily on implementing inherent methods on `Tagged<T>` and `Spanned<T>`, which is no longer possible. As a result, this change creates more concrete types instead of using `Tagged<T>`. One notable example: Tagged<Value> became Value, and Value became UntaggedValue. This change clarifies the intent of the code in many places, but it does make it a big change.
2019-11-21 14:33:14 +00:00
impl CoerceInto<$ty> for nu_source::Tagged<&BigInt> {
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
fn coerce_into(self, operation: impl Into<String>) -> Result<$ty, ShellError> {
match self.$op() {
Some(v) => Ok(v),
None => Err(ShellError::range_error(
$ty::to_expected_range(),
Extract nu_source into a crate This commit extracts Tag, Span, Text, as well as source-related debug facilities into a new crate called nu_source. This change is much bigger than one might have expected because the previous code relied heavily on implementing inherent methods on `Tagged<T>` and `Spanned<T>`, which is no longer possible. As a result, this change creates more concrete types instead of using `Tagged<T>`. One notable example: Tagged<Value> became Value, and Value became UntaggedValue. This change clarifies the intent of the code in many places, but it does make it a big change.
2019-11-21 14:33:14 +00:00
&self.item.spanned(self.tag.span),
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
operation.into(),
)),
}
}
}
};
}
ranged_int!(u8 -> to_u8 -> U8);
ranged_int!(u16 -> to_u16 -> U16);
ranged_int!(u32 -> to_u32 -> U32);
ranged_int!(u64 -> to_u64 -> U64);
ranged_int!(i8 -> to_i8 -> I8);
ranged_int!(i16 -> to_i16 -> I16);
ranged_int!(i32 -> to_i32 -> I32);
ranged_int!(i64 -> to_i64 -> I64);
macro_rules! ranged_decimal {
($ty:tt -> $op:tt -> $variant:tt) => {
impl ToExpectedRange for $ty {
fn to_expected_range() -> ExpectedRange {
ExpectedRange::$variant
}
}
Extract nu_source into a crate This commit extracts Tag, Span, Text, as well as source-related debug facilities into a new crate called nu_source. This change is much bigger than one might have expected because the previous code relied heavily on implementing inherent methods on `Tagged<T>` and `Spanned<T>`, which is no longer possible. As a result, this change creates more concrete types instead of using `Tagged<T>`. One notable example: Tagged<Value> became Value, and Value became UntaggedValue. This change clarifies the intent of the code in many places, but it does make it a big change.
2019-11-21 14:33:14 +00:00
impl CoerceInto<$ty> for nu_source::Tagged<BigDecimal> {
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
fn coerce_into(self, operation: impl Into<String>) -> Result<$ty, ShellError> {
match self.$op() {
Some(v) => Ok(v),
None => Err(ShellError::range_error(
$ty::to_expected_range(),
Extract nu_source into a crate This commit extracts Tag, Span, Text, as well as source-related debug facilities into a new crate called nu_source. This change is much bigger than one might have expected because the previous code relied heavily on implementing inherent methods on `Tagged<T>` and `Spanned<T>`, which is no longer possible. As a result, this change creates more concrete types instead of using `Tagged<T>`. One notable example: Tagged<Value> became Value, and Value became UntaggedValue. This change clarifies the intent of the code in many places, but it does make it a big change.
2019-11-21 14:33:14 +00:00
&self.item.spanned(self.tag.span),
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
operation.into(),
)),
}
}
}
Extract nu_source into a crate This commit extracts Tag, Span, Text, as well as source-related debug facilities into a new crate called nu_source. This change is much bigger than one might have expected because the previous code relied heavily on implementing inherent methods on `Tagged<T>` and `Spanned<T>`, which is no longer possible. As a result, this change creates more concrete types instead of using `Tagged<T>`. One notable example: Tagged<Value> became Value, and Value became UntaggedValue. This change clarifies the intent of the code in many places, but it does make it a big change.
2019-11-21 14:33:14 +00:00
impl CoerceInto<$ty> for nu_source::Tagged<&BigDecimal> {
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
fn coerce_into(self, operation: impl Into<String>) -> Result<$ty, ShellError> {
match self.$op() {
Some(v) => Ok(v),
None => Err(ShellError::range_error(
$ty::to_expected_range(),
Extract nu_source into a crate This commit extracts Tag, Span, Text, as well as source-related debug facilities into a new crate called nu_source. This change is much bigger than one might have expected because the previous code relied heavily on implementing inherent methods on `Tagged<T>` and `Spanned<T>`, which is no longer possible. As a result, this change creates more concrete types instead of using `Tagged<T>`. One notable example: Tagged<Value> became Value, and Value became UntaggedValue. This change clarifies the intent of the code in many places, but it does make it a big change.
2019-11-21 14:33:14 +00:00
&self.item.spanned(self.tag.span),
Migrated numerics to BigInt/BigDecimal This commit migrates Value's numeric types to BigInt and BigDecimal. The basic idea is that overflow errors aren't great in a shell environment, and not really necessary. The main immediate consequence is that new errors can occur when serializing Nu values to other formats. You can see this in changes to the various serialization formats (JSON, TOML, etc.). There's a new `CoerceInto` trait that uses the `ToPrimitive` trait from `num_traits` to attempt to coerce a `BigNum` or `BigDecimal` into a target type, and produces a `RangeError` (kind of `ShellError`) if the coercion fails. Another possible future consequence is that certain performance-critical numeric operations might be too slow. If that happens, we can introduce specialized numeric types to help improve the performance of those situations, based on the real-world experience.
2019-09-01 16:20:31 +00:00
operation.into(),
)),
}
}
}
};
}
ranged_decimal!(f32 -> to_f32 -> F32);
ranged_decimal!(f64 -> to_f64 -> F64);
Reference in a new issue Copy permalink