Prior to this fix, an escaped character like \x41 (hex for ascii A)
was interpreted the same was as A, so that $\x41 would be the same
as $A. Fix this by inserting an INTERNAL_SEPARATOR before these escapes,
so that we no longer treat it as part of the variable name.
This also affects brackets; don't treat echo $foo\1331\135 the same as
echo $foo[1].
Fixes#7969
In the named pipe notifier, notifications are broadcast by writing to the
pipe, waiting briefly, and then reading it back. When clients see the pipe
as readable, they report the uvars as potentially changed and fish will
sync against the uvar file.
Prior to this change, we synced repeatedly when the pipe was readable. But
we can do somewhat better by also checking the named pipe's timestamp (via
fstat). If the pipe has not changed, then we can skip the sync even if
there is currently data lingering on it.
With this change we should sync against the variable file less often
(typically once or twice per write); in the next change we refactor this
logic so it's easier to follow.
This removes the "did_visit" message because it doesn't really add
anything.
For example:
```
ast-construction: make job_list 0x55a6d19729f0
ast-construction: make job_conjunction 0x55a6d1971c00
ast-construction: will_visit job_conjunction 0x55a6d1971c00
ast-construction: will_visit job 0x55a6d1971c18
ast-construction: variable_assignment_list size: 0
ast-construction: will_visit statement 0x55a6d1971c48
ast-construction: make decorated_statement 0x55a6d1972650
ast-construction: will_visit decorated_statement 0x55a6d1972650
ast-construction: make argument_or_redirection 0x55a6d1968310
ast-construction: will_visit argument_or_redirection 0x55a6d1968310
ast-construction: make argument 0x55a6d197b0b0
ast-construction: did_visit argument_or_redirection 0x55a6d1968310
ast-construction: argument_or_redirection_list size: 1
ast-construction: did_visit decorated_statement 0x55a6d1972650
ast-construction: did_visit statement 0x55a6d1971c48
ast-construction: job_continuation_list size: 0
ast-construction: did_visit job 0x55a6d1971c18
ast-construction: job_conjunction_continuation_list size: 0
ast-construction: did_visit job_conjunction 0x55a6d1971c00
ast-construction: job_list size: 1
```
those "did_visit" messages all correspond to "will_visit" ones. They
are effectively block delimiters like `end` or `}`.
If we remove them it turns into:
```
ast-construction: make job_list 0x55a6d19729f0
ast-construction: make job_conjunction 0x55a6d1971c00
ast-construction: will_visit job_conjunction 0x55a6d1971c00
ast-construction: will_visit job 0x55a6d1971c18
ast-construction: variable_assignment_list size: 0
ast-construction: will_visit statement 0x55a6d1971c48
ast-construction: make decorated_statement 0x55a6d1972650
ast-construction: will_visit decorated_statement 0x55a6d1972650
ast-construction: make argument_or_redirection 0x55a6d1968310
ast-construction: will_visit argument_or_redirection 0x55a6d1968310
ast-construction: make argument 0x55a6d197b0b0
ast-construction: argument_or_redirection_list size: 1
ast-construction: job_continuation_list size: 0
ast-construction: job_conjunction_continuation_list size: 0
ast-construction: job_list size: 1
```
Which is still unambiguous because of the indentation.
(this is still *super verbose* and we might want to remove it from the
`*` "all" debug category and only allow turning it on explicitly)
This simply checks if the parser requested exit after running any
binding scripts (in read_normal_chars).
I think this means we no longer need the `exit` bind function.
Fixes#7967.
Reproducible at least on Linux, where the "named pipe" universal
variable notifier is used:
rm -rf build/test/xdg_config
XDG_CONFIG_HOME=build/test/xdg_config ./build/fish -c "xterm -e ./build/fish"
The child fish reacts to keyboard input with a noticeable initial
delay. This is because the universal variable file is polled over
a million times, even when I immediately press Control-D. This polling
prevents readb() from handling keyboard input.
Before commit 939aba02d ("Refactor input_common.cpp:readb"), readb()
reacted to keyboard input even when there were universal variable
notifications. Restore this behavior, but make sure to call the
universal variable notifier after the new "prepare_to_select" logic.
Maybe the problem is in the notifier but the old behavior was sane.
Fixes the problems described in
7a556ec6f2 (commitcomment-49773677)
Adding "-d uvars-file" to the reproducesr shows that we are checking
the uvar file repeatedly:
uvar-file: universal log sync
uvar-file: universal log sync elided based on fast stat()
uvar-file: universal log no modifications
This only uses the functions fish ships with, but still doesn't allow
any *customization*, which is the point of no-config.
This makes it a lot more usable, given that the actual normal prompt
and things are there.
This still doesn't set any colors, because we don't run
__fish_config_interactive because we don't read config.fish (any
config.fish), because that would run the snippets.
In many cases we currently discard escaped newlines, since they
are often unnecessary (when used around &|;). Escaped newlines
are useful for structuring argument lists. Allow them for variable
assignments since they are similar.
Closes#7955
This would print the default "Argument is invalid" error string, which
is *true* but not super obvious, because `test` doesn't always perform
numeric conversion, and that's the bit that failed here.
This refactors the behavior of string match with capture groups to
correctly handle multiple arguments. Now the variable capture applies to
the first match, as documented. Fixes#7938.
string match is documented as setting an unset variable if a capture group
is unmatched in an otherwise matched regex, and if the `--all` flag is not
provided. However prior to this fix, it instead set a variable containing
the empty string as a single value. Correct the implementation to match
the documentation.
Note that if the `--all` flag is provided we continue to set empty
strings, which is documented.
This came out of an investigation into making strings immutable.
This code did "lazy" lowercasing but we can simplify it by just
providing our own case-insensitive compare routine, which is good
enough for colors.
job_reap is now called more often. This optimizes it by doing an
early-out if there are no running jobs (common at the prompt) and also
skipping the save/restore status, since by inspection we also save and
restore the status when running event handlers.
This concerns printing status messages for background jobs which have
stopped or finished. Previously fish would do this from two places:
1. Before running a command (including empty string)
2. If a signal is received during select()
So if the job finishes while fish is doing something else (like running an
event handler) then we would not print status messages until the user hit
return. This caused the job_summary.py test to be flaky.
Fix this by splitting the interrupt handler into two parts: a part that
handles signals (e.g. triggering exit from the reader), and a part that
always runs just before blocking in select(). This second part always
reaps jobs and prints their status messages. This narrows the window for a
job exit to be "missed" before fish blocks in select, and should make the
job_summary.py test more reliable.
This concerns the problem of "injecting" fancy fish bits like job reaping
into the "common" input stuff which is also used by fish_key_reader.
Instead of providing a callback, make the input event queue a base class
with virtual functions. This allows for a richer interface and simplifies
some memory management issues.
readb is used to read a single byte from stdin, or maybe update universal
variables, or maybe invoke completion handlers, etc. Previously it
returned char_event_t but this is more complex than necessary; instead we
can just have it return a single byte, or one of a few special error
codes. This makes the readb's role more clear.
"The" interrupt handler is used when we get a signal while waiting at the
prompt. Switch it from a global function pointer to an std::function. This
is a mild refactoring which itself will be replaced soon.
Now that timeouts are stored in the event queue peeker, we can remove the
notion of timeout events altogether. Instead you may ask for an event with
a timeout, and get back none on timeout. This simplifies how input events
work.
Previously, when attempting to match a key binding, we would dequeue
events from the queue and put them back on if the binding fails. The
tricky part is timeouts: distinguishing between an escaped character and
the escape key itself. This was handled with "timeout events" and we had
to be careful to know when to discard them.
Switch to a new model: use event_queue_peeker more pervasively.
Temporarily dequeued events are stored in the peeker, and the peeker
itself remembers when it has seen a timeout. This is in preparation for
removing the idea of "timeout events" altogether.
Make it an ordinary struct wrapping a vector, instead of a template.
This is in preparation for using it more widely, for matching bindings
as well as mouse CSI sequences.
Also add some mouse-disabling tests.
select_wrapper_t wraps up the annoying bits of using select(): keeping
track of the max fd, passing null for boring parameters, and
constructing the timeout. Introduce a wrapper struct for this and
replace the existing uses of select() with the wrapper.
In readch_timed, we were passing 1 as the number of fds. This is correct if
the fd is 0 (stdin) which it typically is; however this will fail if in_ is
not stdin. Switch to in_ + 1.
This could have been one iteration off, e.g.
```fish
function on-winch --on-signal winch
echo $LINES
end
```
Resize the terminal, it'll print e.g.
24
then run `echo $LINES` interactively, it might have a different answer.
This isn't beautiful, but it works. A better solution might be to make
the termsize vars electric and just always update them on read?
With something like
```
history | head -n 1
```
this would error "write: Broken pipe", which is just annoying. There
is no *problem* here, `head` closes this on purpose.
Fixes#7924.
Things like
```fish
complete command -n '__fish_seen_subcommand_from subcommand'
--force-files
```
would not be obeyed because we only checked force-files when there was
an option.
Fixes#7920.
When a terminal in a tiling WM starts, it might start the shell before
it has reached its "final" size. So we get the terminal width,
then the terminal would be resized (to appease the tiling logic),
and then we would print the abandon line with the omitted newline
char, only if the size got smaller (likely!), we would overflow the
line and land on the next.
So what we do is a bit of a hack: We don't abandon the first line.
This means that `printf %s foo; fish` will overwrite the `foo`, but
that's a super small problem and I don't see another way around this.
Fixes#7893.
This isn't helpful, and entirely unreadable. Excerpt:
```
__fish_git_prompt_set_char (set -l user_variable_name "$argv[1]" set -l char $argv[2] if set -q argv[3] and begin set -q __fish_git_prompt_show_informative_status or set -q __fi…)
```
Fixes#7911.
When fish starts, it notices which pgroup owns the tty, and then it
restores that pgroup's tty ownership when it exits. However if fish does
not own the tty, then (on Mac at least) the tcsetpgrp call triggers a
SIGSTOP and fish will hang while trying to exit.
The first change is to ignore SIGTTOU instead of defaulting it. This
prevents the hang; however it risks re-introducing #7060.
The second change somewhat mitigates the risk of the first: only do the
restore if the initial pgroup is different than fish's pgroup. This
prevents some useless calls which might potentially steal the tty from
another process (e.g. in #7060).
If fish launches a program and that program marks stdin as O_ASYNC, then
fish will start receiving SIGIO events on Mac. This occurs even though
the file descriptor itself does not have the O_ASYNC flag set.
SIGIO is reported as interrupting select which then breaks multiple-key
bindings, especially in vi-mode.
As the SIGIO based universal notifier is disabled, remove it and the
SIGIO handler itself. This allows fish to ignore properly ignore SIGIO.
Fixes#7853
This correctly sets $status when a builtin succeeds but its output fails;
for example if the output is redirected to a file and that write fails.
Fixes#7857
The oldschool math.h imports the math functions into the global
namespace, cmath imports them into std::.
Unfortunately, we already use cmath elsewhere, and including math.h
doesn't reimport them in some systems, so now they can't find them
with std::.
Fixes#7882.
* math: Make function parentheses optional
It's a bit annoying to use parentheses here because that requires
quoting or escaping.
This allows the parens to be omitted, so
math sin pi
is the same as
math 'sin(pi)'
Function calls have the lowest precedence, so
math sin 2 + 6
is the same as
math 'sin(2 + 6)'
* Add more tests
* Add a note to the docs
* even moar docs
Moar docca
* moar tests
Call me Nikola Testla
The bell is a mechanism for important notifications. Not having things
to do in response to a keypress isn't important enough, especially
because we're already flashing and the bell might actually be a bell.
Fixes#7875.
It's not super clear what $SHLVL is useful for, but the current
definition is essentially
"number of shells in the parent processes + 1"
which isn't *super useful*?
Bash's behavior here is a bit weird in that it increments $SHLVL
basically always, but since it auto-execs the last process it will
decrement it again, so in practice it's often not incremented.
E.g.
```
> echo $SHLVL
1
> bash -c 'echo $SHLVL; bash'
2
>> echo $SHLVL
2
```
Both bashes here end up having the same $SHLVL because this is
equivalent to `echo $SHLVL; exec bash`. Running `echo $SHLVL` and then
`bash -c 'echo $SHLVL'` in an interactive bash will have a different
result (1 and 2) because that doesn't *exec* the inner bash.
That's not something we want to get into, so what we do is increment
$SHLVL in every interactive fish. Non-interactive fish will simply
import the existing value.
That means if you had e.g. a bash that runs a fish script that ends up
opening a new fish session, you would have a $SHLVL of *2* - one for the
bash, and one for the inner fish.
We key this off is_interactive_session() (which can also be enabled
via `fish -i`) because it's easy and because `fish -i` is asking for
fish to be, in some form, "interactive".
That means most of the time $SHLVL will be "how many shells am I deep,
how often do I have to `exit`", except for when you specifically asked
for a fish to be "interactive". If that's a problem, we can rethink it.
Fixes#7864.
In preparation for concurrent execution, introduce a
`get_performer_for_builtin` function. This function itself returns a
function, which when called will run the builtin. The idea is that the
function may be called on a background thread (but not in this commit).
Several functions including wgetopt and execve operate on null-terminated
arrays of nul-terminated pointers: a list of pointers to C strings where
the last pointer is null. Prior to this change, each process_t stored its
argv in such an array. This had two problems:
1. It was awkward to work with this type, instead of using std::vector,
etc.
2. The process's arguments would be rearranged by builtins which is
surprising
Our null terminated arrays were built around a fancy type that would copy
input strings and also generate an array of pointers to them, in one big
allocation.
Switch to a new model where we construct an array of pointers over
existing strings. So you can supply a `vector<string>` and now
`null_terminated_array_t` will just make a list of pointers to them. Now
processes can just store their argv in a familiar wcstring_list_t.
Prior to this change, builtins would take their arguments as `wchar_t **`.
This implies that the order of the arguments may be changed (which is
true, `wgetopter` does so) but also that the strings themselves may be
changed, which no builtin should do.
Switch them all to take `const wchar_t **` instead: now the arguments may
be rearranged but their contents may no longer be modified.
This cleans up builtin_set a bit, with the meat of the change being
reworking `parse_index` into `split_var_and_indexes`.
`parse_index` was a function that split a string like `foo[1 3..5]` into
its variable name `foo` and the indexes (here `1 3 4 5`). It had a funny
interface where it would modify a C string in-place. Switch it to return a
`split_var_t` which is a little struct wrapping up the split operation.
This simplifies memory management, and also avoids modifying the arguments
to the builtin.
Previously iothread_perform could do something on a background thread, and
then do something on the main thread. But we no longer use that second
part: instead everything goes through debounce. Remove the completion
parameter from iothread_perform.
This fixes the following problem: if a command is entered while the
previous command is still executing, fish will see it all at once and
execute it before syntax highlighting as a chance to start. So the
command will appear wrong on the terminal. Fix this by detecting this
case and performing a fast no-io highlight.
An example of how to reproduce this:
run `sleep 3` and then type `echo foo` while the sleep is still running.
- Check for special characters *before* attempting to parse
- Also ignore lines with `{` and `*`
- Also skip lines with `<<` because that might be a heredoc (or a
- `<<<` herestring)
Fixes#7874.
This cleans up some exit code processing. Previously a failed exec
would produce exit code 125 unconditionally, while a failed posix_spawn
would produce exit code 1 (!).
With this change, fish reports exit code 126 for not-executable, and 127
for file-not-found. This matches bash.
This change modifies the fish safety check surrounding execve / spawn so
it can run shell scripts having concatenated binary content. We're using
the same safety check as FreeBSD /bin/sh [1] and the Z-shell [5]. POSIX
was recently revised to require this behavior:
"The input file may be of any type, but the initial portion of the
file intended to be parsed according to the shell grammar (XREF to
XSH 2.10.2 Shell Grammar Rules) shall consist of characters and
shall not contain the NUL character. The shell shall not enforce
any line length limits."
"Earlier versions of this standard required that input files to the
shell be text files except that line lengths were unlimited.
However, that was overly restrictive in relation to the fact that
shells can parse a script without a trailing newline, and in
relation to a common practice of concatenating a shell script
ending with an 'exit' or 'exec $command' with a binary data payload
to form a single-file self-extracting archive." [2] [3]
One example use case of such scripts, is the Cosmopolitan C Library [4]
which configuse the GNU Linker to output a polyglot shell+binary format
that runs on Linux / Mac / Windows / FreeBSD / OpenBSD / NetBSD / BIOS.
Fixesjart/cosmopolitan#88
[1] 9a1cd36331
[2] http://austingroupbugs.net/view.php?id=1250
[3] http://austingroupbugs.net/view.php?id=1226#c4394
[4] https://justine.lol/cosmopolitan/index.html
[5] 326d9c203b
cd: Just try to cd without checking first
Some filesystems are broken and error out on `stat(3)` of existing and
cd-able directories.
So we just try to `fchdir` and report errors later.
Fixes#7577.
We have no idea why this was even a thing. For now simply set it to
"all"/"full" (why these two names? no idea) at startup and allow
changing it later.
Settting it *immediately* when defining the variable sets it too soon
because we don't have the interactive signal handlers
enabled (including the one for SIGTTOU), so let's first settle for
this little piece of awkwardness.
This needs widespread testing, so we merge it early, immediately after
the release.
Fixes#5036Fixes#5832Fixes#7721
(and probably numerous others)
We const cast these, so they aren't actually const,
and const doesn't actually help with optimization or anything (because const_cast exists),
so I would rather remove this, because const_cast gives me both the heebies and the jeebies.
When fish performs syntax highlighting, it attempts to determine which
arguments are valid paths and underline them. Skip paths whose length
exceeds PATH_MAX. This is an optimization: such strings are almost
certainly not valid paths and checking them may be expensive.
Relevant is #7837
Previously wbasename and wdirname wrapped the system-provided basename
and dirname. But these have thread-safety issues and some surprising
error conditions on Mac. Just reimplement these per the OpenGroup spec.
In particular these no longer trigger a null-dereference if the input
exceeds PATH_MAX.
Add some tests too.
This fixes#7837
Previously fish attempted to block all signals on background threads, so
that they would be delivered to the main thread. But on Mac, SIGSEGV
and probably some others just get silently dropped, leading to potential
infinite loops instead of crashing. So stop blocking these signals.
With this change the null-deref in #7837 will properly crash instead of
spinning.
`streams.out.push_back` for fd_streams_t writes to the
fd *immediately*. We might want to introduce a general buffering
strategy, but in this case writing it in one go is the simplest and
seems acceptable - we already have constrained the argument size, so
just pushing it out should work well enough.
See #7836
013a563ed0 made it so we only try to
adjust terminal modes if we are in the terminal pgroup, but that's not
enough.
Fish starts background jobs in events inside its own pgroup, so
function on-foo --on-event foo
fish -c 'sleep 3' &
end
would have the backgrounded fish try to fiddle with the terminal and
succeed.
Instead, only fiddle with the terminal if we're interactive (this
should probably be extended to other bits, but this is the particular
problematic part)
Fixes#7842.
