(command pwd) uses the system's implementation of pwd. At least the GNU
coreutils implementation defaults to -P, which resulted in symlinks being
expanded when switching between directories with nextd/prevd.
This did some weird unescaping to try to extract the first word.
So we're now more likely to be *correct*, and the alias benchmark is
about 20% *faster*.
Call it a win-win.
This splits a string into variables according to the shell's
tokenization rules, considering quoting, escaping etc.
This runs an automatic `unescape` on the string so it's presented like
it would be passed to the command. E.g.
printf '%s\n' a\ b
returns the tokens
printf
%s\n
a b
It might be useful to add another mode "--tokenize-raw" that doesn't
do that, but this seems to be the more useful of the two.
Fixes#3823.
This added the function offset *again*, but it's already included in
the line for the current file.
And yes, I have explicitly tested a function file with a function
defined at a later line.
Fixes#6350
Since #6287, bare variable assignments do not parse, which broke
the "Unsupported use of '='" error message.
This commit catches parse errors that occur on bare variable assignments.
When a statement node fails to parse, then we check if there is at least one
prefixing variable assignment. If so, we emit the old error message.
See also #6347
This adds initial support for statements with prefixed variable assignments.
Statments like this are supported:
a=1 b=$a echo $b # outputs 1
Just like in other shells, the left-hand side of each assignment must
be a valid variable identifier (no quoting/escaping). Array indexing
(PATH[1]=/bin ls $PATH) is *not* yet supported, but can be added fairly
easily.
The right hand side may be any valid string token, like a command
substitution, or a brace expansion.
Since `a=* foo` is equivalent to `begin set -lx a *; foo; end`,
the assignment, like `set`, uses nullglob behavior, e.g. below command
can safely be used to check if a directory is empty.
x=/nothing/{,.}* test (count $x) -eq 0
Generic file completion is done after the equal sign, so for example
pressing tab after something like `HOME=/` completes files in the
root directory
Subcommand completion works, so something like
`GIT_DIR=repo.git and command git ` correctly calls git completions
(but the git completion does not use the variable as of now).
The variable assignment is highlighted like an argument.
Closes#6048
Presently the completion engine ignores builtins that are part of the
fish syntax. This can be a problem when completing a string that was
based on the output of `commandline -p`. This changes completions to
treat these builtins like any other command.
This also disables generic (filename) completion inside comments and
after strings that do not tokenize.
Additionally, comments are stripped off the output of `commandline -p`.
Fixes#5415Fixes#2705
PATH and CDPATH have special behavior around empty elements. Express this
directly in env_stack_t::set rather than via variable dispatch; this is
cleaner.
This adds support for `fish_trace`, a new variable intended to serve the
same purpose as `set -x` as in bash. Setting this variable to anything
non-empty causes execution to be traced. In the future we may give more
specific meaning to the value of the variable.
The user's prompt is not traced unless you run it explicitly. Events are
also not traced because it is noisy; however autoloading is.
Fixes#3427
Until now, something like
`math '7 = 2'`
would complain about a "missing" operator.
Now we print an error about logical operators not being supported and
point the user towards `test`.
Fixes#6096
The `--entire` would enable output even though the `--quiet` should
have silenced it. These two don't make any sense together so print an
error, because the user could have just left off the `-q`.
This spewed errors because the `math` invocation got no second
operand:
Testing file checks/sigint.fish ... math: Error: Too few arguments
'1571487730 -'
but only if the `date` didn't do milliseconds, which is the case on
FreeBSD and NetBSD.
(also force the variable to be global - we don't want to have a
universal causing trouble here)
Universal exported variables (created by `set -xU`) used to show up
both as universal and global variable in child instances of fish.
As a result, when changing an exported universal variable, the
new value would only be visible after a new login (or deleting the
variable from global scope in each fish instance).
Additionally, something like `set -xU EDITOR vim -g` would be imported
into the global scope as a single word resulting in failures to
execute $EDITOR in fish.
We cannot simply give precedence to universal variables, because
another process might have exported the same variable. Instead, we
only skip importing a variable when it is equivalent to an exported
universal variable with the same name. We compare their values after
joining with spaces, hence skipping those imports does not change the
environment fish passes to its children. Only the representation in
fish is changed from `"vim -g"` to `vim -g`.
Closes#5258.
This eliminates the issue #5348 for universal variables.
Consider a group of short options, like -xzPARAM, where x and z are options and z takes an argument.
This commit enables completion of the argument to the last option (z), both within the same
token (-xzP) or in the next one (-xz P).
complete -C'-xz' will complete only parameters to z.
complete -C'-xz ' will complete only parameters to z if z requires a parameter
otherwise, it will also complete non-option parameters
To do so this implements a heuristic to differentiate such strings from single long options. To
detect whether our token contains some short options, we only require the first character after the
dash (here x) to be an option. Previously, all characters had to be short options. The last option
in our example is z. Everything after the last option is assumed to be a parameter to the last
option.
Assume there is also a single long option -x-foo, then complete -C'-x' will suggest both -x-foo and
-xy. However, when the single option x requires an argument, this will not suggest -x-foo.
However, I assume this will almost never happen in practise since completions very rarely mix
short and single long options.
Fixes#332
This stops reading argument names after another option appears. It does not break any previous uses and in fact fixes uses like
```fish
function foo --argument-names bar --description baz
```
* `function` command handles options after argument names (Fixes#6186)
* Removed unneccesary test
I got tired of seeing ' ... ok (0 sec)' so now with GNU date/gdate
installed there is millisecond output shown. One can get rough
nanoseconds from gdate.
This sequence can be generatd by control-spacebar. Allow it to be bound
properly.
To do this we must be sure that we never round-trip the key sequence
through a C string.
Meaning empty variables, command substitutions that don't print
anything.
A switch without an argument
```fish
switch
case ...
end
```
is still a syntax error, and more than one argument is still a runtime
error.
The none-argument matches either an empty-string `case ''` or a
catch-all `case '*'`.
Fixes#5677.
Fixes#4943.
This test uses universal variables, and so it can fail when run
multiple times.
It might be a good idea to do this in general, but for now let's just
try it here.
Previously when propagating explicitly separated output, we would early-out
if the buffer was empty, where empty meant contains no characters. However
it may contain one or more empty strings, in which case we should propagate
those strings.
Remove this footgun "empty" function and handle this properly.
Fixes#5987
I tested this manually (`littlecheck.py -s fish=fish tests/checks/eval.fish`) from the base directory, which means I got
"tests/checks/eval", while the real test gets "checks/eval".
I then reran `make test_fishscript`, but that didn't pull in the
updated test - we should really handle that better.
I'm gonna add more tests to this and I don't want to touch the old stuff.
Notice that this needs to have the output of the complete_directories
test adjusted because this one now runs later.
That's something we should take into account in future.
history now often writes to the history file asynchronously, but the history
test expects to find the text in the file immediately after running the
command. Hack a bit in history to make this test more reliable.
This required a bit of thinking.
What we do is we have one test that fakes $HOME, and then we do the
various config tests there.
The fake config we have is reused and we exercise all of the same codepaths.
This prints a green "ok" with the duration, just like the rest of the
tests.
Note that this clashes a bit with
https://github.com/ridiculousfish/littlecheck/pull/3.
(also don't check for python again and again and again)
This is a bit weird sometimes, e.g. to test the return status (that
fish actually *returns $status*), we use a #RUN line with %fish
invoking %fish, so we can use the substitution.
Still much nicer.
The missing scripts are those that rely on config.
Especially as, in this case, the documentation is quite massive.
Caught by porting string's test to littlecheck.
See #3404 - this was already supposed to be included.
This is a nice test (ha!) for how this works and what littlecheck can
do for us.
1. Input is now the actual file, not "Standard Input" anymore. So
any errors mentioning that now include the filename.
2. Regex are really nice for filenames, but especially for line
numbers
3. It's much nicer to have the output where it's created, instead of
needing to follow three files at the same time.
Instead of requiring a flag to enable newline trimming, invert it so the
flag (now `--no-trim-newlines`) disables newline trimming. This way our
default behavior matches that of sh's `"$(cmd)"`.
Also change newline trimming to trim all newlines instead of just one,
again to match sh's behavior.
The `string collect` subcommand behaves quite similarly in practice to
`string split0 -m 0` in that it doesn't split its output, but it also
takes an optional `--trim-newline` flag to trim a single trailing
newline off of the output.
See issue #159.
This adds support for .check files inside the tests directory. .check
files are tests designed to be run with littlecheck.
Port printf test to littlecheck and remove the printf.in test.
It's always a bit annoying that `*` requires quoting.
So we allow "x" as an alternative, only it needs to be followed by
whitespace to distinguish it from "0x" hexadecimal notation.
This makes the following changes:
1. Events in background threads are executed in those threads, instead of
being silently dropped
2. Blocked events are now per-parser instead of global
3. Events are posted in builtin_set instead of within the environment stack
The last one means that we no longer support event handlers for implicit
sets like (example) argv. Instead only the `set` builtin (and also `cd`)
post variable-change events.
Events from universal variable changes are still not fully rationalized.
When setting a variable without a specified scope, we should give priority
to an existing local or global above an existing universal variable with
the same name.
In 16fd780484 there was a regression that
made universal variables have priority.
Fixes#5883
Brace expansion with single words in it is quite useless - `HEAD@{0}`
expanding to `HEAD@0` breaks git.
So we complicate the rule slightly - if there is no variable expansion
or "," inside of braces, they are just treated as literal braces.
Note that this is technically backwards-incompatible, because
echo foo{0}
will now print `foo{0}` instead of `foo0`. However that's a
technicality because the braces were literally useless in that case.
Our tests needed to be adjusted, but that's because they are meant to
exercise this in weird ways.
I don't believe this will break any code in practice.
Fixes#5869.
This read something like `o=!_validate_int`, and the flag modifier
reading kept the pointer after the `!`, so it created a long flag
called `_validate_int`, which meant it would not only error out form
```fish
argparse 'i=!_validate_int' 'o=!_validate_int' -- $argv
```
with "Long flag '_validate_int' already defined", but also set
$_flag_validate_int.
Fixes#5864.
As mentioned in #2900, something like
```fish
test -n "$var"; and set -l foo $var
```
is sufficiently idiomatic that it should be allowable.
Also fixes some additional weirdness with semicolons.
This removes semicolons at the end of the line and collapses
consecutive ones, while replacing meaningful semicolons with newlines.
I.e.
```fish
echo;
```
becomes
```fish
echo
```
but
```fish
echo; echo
```
becomes
```fish
echo
echo
```
Fixes#5859.
This keeps all unknown options in $argv, so
```fish
argparse -i a/alpha -- -a banana -o val -w
```
results in $_flag_a set to banana, and $argv set to `-o val -w`.
This allows users to use multiple argparse passes, or to simply avoid
specifying all options e.g. in completions - `systemctl` has 46 of
them, most not having any effect on the completions.
Fixes#5367.
This is a long-standing issue with how `complete --do-complete` does
its argument parsing: It takes an optional argument, so it has to be
attached to the token like `complete --do-complete=foo` or (worse)
`complete -Cfoo`.
But since `complete` doesn't take any bare arguments otherwise (it
would error with "too many arguments" if you did `complete -C foo`) we
can just take one free argument as the argument to `--do-complete`.
It's more of a command than an option anyway, since it entirely
changes what the `complete` call _does_.
Prior to this fix, a job would only inherit a pgrp from its parent if the
first command were external. There seems to be no reason for this
restriction and this causes tcsetgrp() churn, potentially cuasing SIGTTIN.
Switch to unconditionally inheriting a pgrp from parents.
This should fix most of #5765, the only remaining question is
tcsetpgrp from builtins.
Pursuant to 0be7903859, there still
remained one issue with the test when run from within a symlinked
directory after fish gained support for cding into symlinks.
This change should make the test function OK both when the tests are run
out of a PWD containing a symlink in its hierarchy and when run
otherwise.
The final test in `realpath.in` was based on the no-longer-valid
assumption that $PWD cannot be a symlink. Since the recent changes in
fish 3.0 to allow `cd`ing into "virtual" directories preserving symlinks
as-is, when `make test` was run from a path that contained a symlink
component, this test would fail the `pwd-resolved-to-itself` check.
As the test is not designed to initialize then cd into an absolute path
guaranteed to not be symbolic, so this final check is just wrong.
This has been driving nuts for years. The output of the diff emitted
when a test fails was always reversed, because the diff tool is called
with `${difftool} ${new} ${old}` so all the `-` and `+` contexts are
reversed, and the highlights are all screwed up.
The output of a `make test` run should show what has changed from the
baseline/expected, not how the expected differs from the actual. When
considered from both the perspective of intentional changes to the test
outputs and failed test outputs, it is desirable to see how the test
output has changed from the previously expected, and not the other way
around.
(If you were used to the previous behavior, I apologize. But it was
wrong.)
This was printed basically everywhere.
The user knows what they executed on standard input.
A good example:
```fish
set c (subme 513)
```
used to print
```
fish: Too much data emitted by command substitution so it was discarded
set -l x (string repeat -n $argv x)
^
in function 'subme'
called on standard input
with parameter list '513'
in command substitution
called on standard input
```
and now it is
```
fish: Too much data emitted by command substitution so it was discarded
set -l x (string repeat -n $argv x)
^
in function 'subme' with arguments '513'
in command substitution
```
See #5434.
This printed things like
```
in function 'f'
called on standard input
in function 'd'
called on standard input
in function 'b'
called on standard input
in function 'a'
called on standard input
```
As a first step, it removes the empty lines so it's now
```
in function 'f'
called on standard input
in function 'd'
called on standard input
in function 'b'
called on standard input
in function 'a'
called on standard input
```
See #5434.
If a function process is deferred, allow it to be unbuffered.
This permits certain simple cases where functions are piped to external
commands to execute without buffering.
This is a somewhat-hacky stopgap measure that can't really be extended
to more general concurrent processes. However it is overall an improvement
in user experience that might help flush out some bugs too.
POSIX dictates here that incomplete conversions, like in
printf %d\n 15.2
or
printf %d 14g
are still printed along with any error.
This seems alright, as it allows users to silence stderr to accept incomplete conversions.
This commit implements it, but what's a bit weird is the ordering between stdout and stderr,
causing the error to be printed _after_, like
15
14
15.1: value not completely converted
14,2: value not completely converted
but that seems like a general issue with how we buffer the streams.
(I know that nonfatal_error is a copy of most of fatal_error - I tried
differently, and va_* is weird)
Fixes#5532.
Before this change, - was sorted with other punctuation before
A-Z. Now, it sorts above the rest of the characters.
This has a practical effect on completions, where when there are
both -s and --long with the same description, the short option
is now before the long option in the pager, which is what is now
selected when navigating `foo -<TAB>`. The long options can be
picked out with `foo --<TAB>`. Before, short options which
duplicated a long option literally could not be selected by
any means from the pager.
Fixes#5634
This tweaks wcsfilecmp such that certain punctuation characters will
come after A-Z.
A big win with `set <TAB>` - the __prefixed fish junk now comes
after the stuff users should care about.
This disables an extra round of escaping in the `string replace -r`
replacement string.
Currently, to add a backslash to an a or b (to "escape" it):
string replace -ra '([ab])' '\\\\\\\$1' a
7 backslashes!
This removes one of the layers, so now 3 or 4 works (each one escaped
for the single-quotes, so pcre receives two, which it reads as one literal):
string replace -ra '([ab])' '\\\\$1' a
This is backwards-incompatible as replacement strings will change
meaning, so we put it behind a feature flag.
The name is kinda crappy, though.
Fixes#5474.
As a simple replacement for `wc -l`.
This counts both lines on stdin _and_ arguments.
So if "file" has three lines, then `count a b c < file` will print 6.
And since it counts newlines, like wc, `echo -n foo | count` prints 0.
It turns out that `string split0` didn't actually ever do any
splitting. The arg_iterator_t already split stdin on NUL, and split0 just
performed an additional search that could never succeed (since
arguments from argv already can't contain NUL).
Let the arg_iterator_t not perform any splitting if asked, and then
let split0 split in 0.
One slight wart is that split0 ignores a trailing NUL, which normal
split doesn't.
Fixes#5701.
In a galaxy far, far away, event_blockage_t was intended to block only cetain
events. But it always just blocked everything. Eliminate the event block
mask.
On some systems, this sometimes uses unicode quotation marks.
Not on mine, but on Travis it does.
The only other workaround I can think of is setting locale to C, but
that implies not being able to test anything unicode-related in the
entire invocation tests.
So for now disable this test.
Illumos/OpenIndiana/SunOS/Solaris has an rm/rmdir that tries to
protect the user by not allowing them to delete $PWD.
Normally, this would be a good thing as deleting $PWD is a stupid
thing to do. Except in this case, we absolutely need to do that.
So instead we weasel around it by invoking an sh to cd out of the
directory to then invoke an `rmdir` to delete it. That should throw
off any attempts at protection (we could also have tried $PWD/. or
similar, but that's possibly still protected against).
This is the last failing test on
Illumos/OpenIndiana/SunOS/Solaris/afunnyquip, so:
Fixes#5472.
This tested #1728, where redirecting a directory (`begin; something;
end < .`) would cause `status` to misbehave.
Unfortunately, on Illumos/OpenIndiana/SunOS, this returns a different
error (EINVAL instead of EISDIR), so we can't check that with our test harness, because
we can't redirect it.
Since it's not important that this gives the same error across
systems (and indeed we provide no way of intercepting the error!),
use an invocation test instead, because that allows different output per-uname.
See #5472.
300ms was waaay too long, and even 100ms wasn't necessary.
Emacs' evil mode uses 10ms (0.01s), so let's stay a tad higher in case
some terminals are slow.
If anyone really wants to be able to type alt+h with escape, let them
raise the timeout.
Fixes#3904.
This is a large change to how io_buffers are filled. The essential problem
comes about with code like (example):
echo ( /bin/pwd )
The output of /bin/pwd must go to fish, not the tty. To arrange for this,
fish does the following:
1. Invoke pipe() to create a pipe.
2. Add an io_bufferfill_t redirection that owns the write end of the pipe.
3. After fork (or equiv), call dup2() to replace pwd's stdout with this pipe.
Now when /bin/pwd writes, it will send output to the read end of the pipe.
But who reads it?
Prior to this fix, fish would do the following in a loop:
1. select() on the pipe with a 10 msec timeout
2. waitpid(WNOHANG) on the pwd proc
This polling is ugly and confusing and is what is replaced here.
With this new change, fish now reads from the pipe via a background thread:
1. Spawn a background pthread, which select()s on the pipe's read end with
a long (100 msec) timeout.
2. In the foreground, waitpid() (allowing hanging) on the pwd proc.
The big win here is a major simplification of job_t::continue_job() since
it no longer has to worry about filling buffers. This will make things
easier for concurrent execution.
It may not be obvious why the background thread still needs a poll (100 msec).
The answer is for cases where the write end of the fd escapes, in particular
background processes invoked inside command substitutions. psub is perhaps
the only important case of this (other shells typically just hang here).
Using printf like
printf "The message"
is unsafe, because if the message contains any formatting characters,
they'll be interpreted.
In this case it's not all that important because the message contains
only filenames of our tests and static strings, but still.
A while loop now evaluates to the last executed command in the body, or
zero if the loop body is empty. This matches POSIX semantics.
Add a bunch of tricky tests.
See #4982
For some reason, we have two places where a variable can be read-only:
- By key in env.cpp:is_read_only(), which is checked via set*
- By flag on the actual env_var_t, which is checked e.g. in
parse_execution
The latter didn't happen for non-electric variables like hostname,
because they used the default constructor, because they were
constructed via operator[] (or some such C++-iness).
This caused for-loops to crash on an assert if they used a
non-electric read-only var like $hostname or $SHLVL.
Instead, we explicitly set the flag.
We might want to remove one of the two read-only checks, or something?
Fixes#5548.
There's just waaayy too many things that could go wrong with it, so it
annoys more than it helps, especially since we don't get any
indication what failed.
E.g. on FreeBSD, the test failed without a usable message just because
`tput` couldn't find an attribute (so colors were unset).
The one thing I was missing:
`echo -n` isn't POSIX. In practice, it appears the only shell to encounter this
is macOS' crusty old bash in sh-mode. Just replace it with `touch`.
This reverts commit fc5e8f9fec.
This makes the script worse, but it's good enough.
The required changes are:
- `shopt -s nullglob`, which we simply don't use (we have one glob, but that's
guaranteed to match because we ship the files)
- One array, which we replace with a direct use of the glob (plus it
used `echo` again?)
- The `function` word, which I'm still annoyed is even a thing!
- Variable indirection (`color=${!color_var}` - instead we pass the
value directly - which makes the script uglier!)
- One array, which we replace with a function
- A use of `type -t`, replaced with `command -v`
- A use of `${var:begin:end}` substring expansion, replaced with trickery.
- `set -o pipefail` is replaced with a function
Note that checkbashisms still complains about `command -v`, because
we're not using it with "-p". But we _want_ to check the current
$PATH, and `command -v` is POSIX.
This still uses `local`, which technically isn't in POSIX.
The tests now appear to pass in:
- bash
- dash
- zsh
- mksh
- busybox
Rather than killing the process with close, read EOF after sending the
"exit" command and wait for OS cleanup (per the expect examples).
Not cleaning up with wait caused expect to crash on all 32-bit platforms
including i586 and armv7l with "alloc: invalid block: 0xbf993ccb: 3d 3b".
64-bit platforms were not affected, for reasons that are not clear.
Turns out busybox diff (used on alpine) defaults to unified output,
which we can't use because that prints filenames, and those are
tempfiles made by psub.
Instead, we use builtins to print the first line and compare the others.
This isn't all that important, and it breaks on musl just because the message is different.
Just skip it for now, until we figure out how to better test this.
This `set TERM`. Which, if $TERM is inherited, is already exported,
but not if it isn't.
This is the case on sr.ht's arch images, so we failed without a TERM variable.
