Commit graph

3769 commits

Author SHA1 Message Date
Johannes Altmanninger
7fea321b3e
Use the correct case in completion pager (#7744)
Consider

	$ complete -c foo -a 'aab aaB' -f
	$ foo A<TAB>

since 28d67c8 we would insert the common prefix AND show the pager.
Due to case-insensitive comparison, "b/B" was considered to be part
of the prefix. Since the prefix is added to each pager item [1]
we get wrong results. Fix this by removing the insensitive comparison
between completions - I don't think it was of much use anyway.
Commandline tokens are still matched case-insensitively, this is
just about completions.

Test this by running interactive fish inside tmux (pexpect's terminal
emulation not have enough capabilities).  Also add tests for recent
interactive regressions #7526 and #7738.

Closes #3978

[1]: b38a23a would solve this differently by giving every pager item
its own prefix, but was reverted since it needs more fixes.
2021-03-21 09:25:29 +01:00
Fabian Homborg
d79f864072 Add a note for help thing to the missing doc error message
This should cover most cases - the user didn't install the docs and is
trying to view the man page via __fish_print_help, so we don't have a
way to show anything.

But `help thing` will fall back to the online version of the docs,
which should work if there's an internet connection.

See #7824.
2021-03-19 17:39:36 +01:00
Fabian Homborg
02699d1acc Reject empty variable names
This allowed `set "" foo`, which is bogus and results in an unusable variable.
2021-03-18 19:36:57 +01:00
Johannes Altmanninger
f9e131aa93 Show an error when "builtin -h" fails to find a man page
Prior to this commit "builtin -h" would silently fail when no
documentation is installed. This happens when running fish without
installing it, or when the docs are not installed.

See #7824
2021-03-15 23:07:08 +01:00
Johannes Altmanninger
865abebd11 Simplify highlight tests by changing into dedicated test dir 2021-03-13 17:51:49 +01:00
Fabian Homborg
29ebd4a5ff tests: Don't break when a file unexpectedly exists
Creating a file called "xfoo" could break the highlight tests because
we'd suddenly get a color with valid_path set to true.

So what we do is simply compare foreground/background and forced
underline, but only check for path validity if we're expecting a valid
path.

If we're not expecting a valid path, we don't fail whether it is there
or not.

This means that we can't check for a non-valid path, but we don't
currently do that anyway and we can just burn that bridge when we get
to it.

cc @siteshwar @krobelus, who both came across this
2021-03-13 17:25:23 +01:00
ridiculousfish
3cb105adbd Restore terminal modes after running key bindings with external commands
This concerns the behavior when running an external command from a key
binding. The history is:

Prior to 5f16a299a7, fish would run these external commands in shell
modes. This meant that fish would pick up any tty changes from external
commands (see #2114).

After 5f16a299a7, fish would save and restore its shell modes around
these external commands. This introduced a regression where anything the
user typed while a bound external command was executing would be echoed,
because external command mode has ECHO set in c_lflag. (This can be
reproed easily with `bind -q 'sleep 1'` and then pressing q and typing).
So 5f16a299a7 was reverted in fd9355966.

This commit partially reverts fd9355966. It has it both ways: external
commands are launched with shell modes, but/and shell modes are restored
after the external command completes. This allows commands to muck with
the tty, as long as they can handle getting shell modes; but it does not
enable ECHO mode so it fixes the regression found in #7770.

Fixes #7770. Fixes #2114 (for the third time!)

This partially reverts commit fd9355966e.
2021-03-10 22:36:33 -08:00
Fabian Homborg
fd9355966e Stop donating the terminal for bind functions
Unfortunately this causes input coming in while bind functions are
running to show up on screen.

Since the cure is worse than the disease let's just stop doing it.

My guess is this needs to *only* be done while running an external
command.

Fixes #7770
Reintroduces #2114

Partially reverts 5f16a299a7
2021-03-10 21:21:19 +01:00
Fabian Homborg
013a563ed0 Only modify terminal modes after config.fish if we have the terminal
This tried fiddling with the terminal even if fish didn't own it, e.g.
in

    fish -c 'sleep 5' &

Fixes #7808.
2021-03-10 09:38:16 +01:00
Fabian Homborg
daa3cc17c4 Fix crash in string pad
Try:

    string pad -w 8 he \eh
2021-03-09 18:36:02 +01:00
Fabian Homborg
a7df92e187 Fix crash with set_color --print-colors --background normal
Found in conjunction with #7805.
2021-03-09 13:46:08 +01:00
Fabian Homborg
4762d52e52 output: A background is set if it's not a special non-color
For reasons unclear to me, fish enables bold mode unconditionally if
the background is set.

However, this called a background "set" if it wasn't exactly the
"normal" color, whereas set_color --print-colors would set a color
of *none*.

We have three special non-color colors:

- "normal"
- "reset"
- "none"

All of these specify some form of absence of background color, so all
of them should be checked.

Fixes #7805
2021-03-09 13:25:00 +01:00
Ilya Grigoriev
762f3aa0ce
Rewrite the real file if history file is a symlink (#7754)
* Rewrite the real file if history file is a symlink

When the history file is a symbolic link, `fish` used to overwrite
the link with a real file whenever it saved history. This makes
it follow the symlink and overwrite the real file instead.

The same issue was fixed for the `fish_variables` file in 622f2868e
from https://github.com/fish-shell/fish-shell/pull/7728.
This makes `fish_history` behave in the same way. The implementation
is nearly identical.

Since the tests for the two issues are so similar, I combined them
together and slightly expanded the older test.

This also addresses https://github.com/fish-shell/fish-shell/issues/7553.

* Add user-facing error when history renaming fails

Currently, when history file renaming fails, no message is shown to the
user. This happens, for instance, if the history file is a symlink
pointing to another filesystem.

This copies code (with a bit of variation, after reviewer comments) from

589eb34571/src/env_universal_common.cpp (L486-L491)

into `history.cpp`, so that a message is shown to the user.

* fixup! Rewrite the real file if history file is a symlink
2021-03-08 17:46:17 +01:00
Fabian Homborg
dbb74f87ba Re-read terminal modes after config.fish
Before now, we would be getting the terminal modes before config.fish,
then running config.fish without any of the term "stealing" and modes
copying. This meant that changes made to the terminal modes in there
were simply lost.

So, what we do is simply set the modes before config and then copy
them after, once.

Note that this does *not* turn off flow control again - if you turn it
on in config.fish that should work.

Fixes #7783.
2021-03-07 11:23:59 +01:00
Fabian Homborg
371516382d Create job-exit caller events inside event handlers
f7e2e7d26b forbid any job exit events
from happening inside jobs that were themselves event handlers, but
that causes e.g.

```fish
function f --on-event fish_prompt
source (echo "echo hello world" | psub)
end
```

to not trigger psub's cleanup, so it leaves files in $TMPDIR behind.

This was hit by pyenv, because that still uses `source (thing |
psub)`.

Fixes #7792.
2021-03-07 10:00:52 +01:00
Fabian Homborg
25d85bdc64 path_get_path: Remove error on unknown errno
This seems like a good idea, but there isn't anything we or anyone
else can *do* in this case. All we ever do is pile on additional
errors on the ignore pile, we can't handle any of them differently.
The command isn't a thing, so we check the next path.

The impetus for this is Cygwin apparently returning a wonderfully
useless 0, and it's not even the first one to do so.

Fixes #7785
2021-03-06 08:12:27 +01:00
Mahmoud Al-Qudsi
11951a245f Optimize pruning of job/proc exit handlers
Pre-emptively delete the handler while we have possession of the lock
before calling the event itself. It's crude, but it works.
2021-03-05 22:40:06 -06:00
Mahmoud Al-Qudsi
e7398c0248 Prune job exit handlers after running
While pid values may be reused, it is logical to assume that fish event
handlers coded against a particular job or process id mean just the job
that is currently referred to be any given pid/pgrp rather than in
perpetuity.

This trims the list of registered event handlers nice and early, and as
a bonus avoids the issue described in #7721.

The cleanup song-and-dance is extremely ugly due to the repeated locking
and unlocking of the event handler list.

Closes #7221.
2021-03-05 22:32:57 -06:00
Fabian Homborg
6bc0064a2a demangled_backtrace: Cast a thing to const char* instead of char*
Apparently this is const char* on NetBSD, so it complains.

Since it should be harmless to allow this one to be const, just do
that.
2021-03-05 19:40:44 +01:00
Fabian Homborg
c96a07dc96 Revert "Prevent redirecting internal processes to file descriptors above 2"
FDs are inherited, and redirecting those is harmless, and forbidding
that is worse than allowing all.

Fixes #7769.

This reverts commit 11a373f121.
2021-03-03 22:26:33 +01:00
Fabian Homborg
791b42f065 Disable SIGIO notifier
It doesn't work on WSL, Solaris and Archlinux (and presumably that
means future versions of other linux distros).

In its current state I don't trust it enough to enable it anywhere by
default, especially since I'm not aware of an actual issue with the
named pipe (other than that the code is ugly).

Fixes #7774
2021-03-03 22:26:28 +01:00
Fabian Homborg
955c0003ca fd_monitor: Explicity include <thread>
Might fix issues with gcc 11.0.1.

See #7764.
2021-03-02 15:27:13 +01:00
Fabian Homborg
119b978cbc type: Add --quiet back
It's supposed to be *deprecated*, not removed. The documentation even
specifically calls it out.

Fixes #7766
2021-03-02 14:53:02 +01:00
Érico Rolim
d948b34420 Include <termios.h> instead of <sys/termios.h>.
Slipped by with ffa24eb361. Given
daf5ef1bbd, fish should be using
<termios.h> in all cases.
2021-03-02 12:05:07 +01:00
Fabian Homborg
abaa057e5c Replace our only dynamic_cast with old-school casting
dynamic_cast requires rtti to be enabled. Now, this isn't a big
problem, but since this is our only dynamic_cast in the entire
codebase, and it's not serving an important function, we can just
replace it.

See #7764
2021-03-02 09:44:23 +01:00
Fabian Homborg
6b6f8d2295 Avoid changing how custom fish_titles are called
This half-reverts commit a3cb1e2dcd,
avoiding the bit that passed arguments differently.

Note that this means the initial bug is kept in the hardcoded fallback title.

Fixes #7749.
2021-02-28 08:40:39 +01:00
Fabian Homborg
459ac2b566 Reset the readline loop state when setting the buffer
Fixes #7740.
2021-02-25 16:19:28 +01:00
Mahmoud Al-Qudsi
42d9f33e16 Use a platform whitelist for opting into the SIGIO notifier
My bet is that the Illumos, Cygwin, and WSL are not the only Unix-like
systems where the SIGIO notifier won't work, and since we have a good
enough and portable alternative that we can use be default on other
platforms where we don't specifically know it'll work, it doesn't make
sense not to go with that one instead.

Even if this patch is reverted at some point and we go back to
blacklisting platforms that *don't* support the SIGIO strategy, this is
almost certainly the right choice for inclusion in a minor release like
3.2.0.

See discussion in #6585.
2021-02-24 14:40:17 -06:00
Mahmoud Al-Qudsi
ffa24eb361 Fix build on Solaris (and hopefully OpenIndiana)
* memset/memzero needs cstring/string.h (again)
 * winsize_t requires an impl from <sys/termios.h>

With this patch, I was able to get fish master to build on Solaris 11.4
without any additional dependencies (after installing gcc 7, git, and
cmake). I think the ncurses dependency can be dropped from the
OpenIndiana package?
2021-02-23 11:02:55 -06:00
Mahmoud Al-Qudsi
d95e99f2d4 Fix missing memset/memzero on Illumos
Tested w/ GCC 9 under OpenIndiana 2020.
2021-02-22 20:58:42 -06:00
Mahmoud Al-Qudsi
17926a9197 Allow low-level tests to at least run to completion under Cygwin
They still don't pass due to path differences, but at least they run to
completion so we can tell just how broken everything is.
2021-02-22 16:27:45 -06:00
Mahmoud Al-Qudsi
d769582023 Ignore intentional redundant move under GCC
This bubbled up after Wredundant-move was added to the build script and
was observed under the CI environment running GCC 9.3.0.
2021-02-22 15:55:40 -06:00
Fabian Homborg
d0a8493844 Don't open pager with only one completion
Since smartcase, we could land in a situation where we offer one
option in the pager, which is awkward.

So detect this and just insert the option directly, we can add any
more smartness later.

Fixes #7738.
2021-02-22 22:51:52 +01:00
Mahmoud Al-Qudsi
1305a0899c Fix comparison warnings on UTF-16 platforms
Without true handling of UTF-16 surrogate pairs, all we can do is
properly detect the BMP range in UTF-16 environments and bail if the
input is in a non-BMP region.

There isn't much else we can do as it is incorrect to encode the
surrogate pairs themselves (fish doesn't know what to do with them and
they're illegal under either of UTF-8 or UTF-32).

(I'm not aware of fish being used in any UTF-16 platforms other than
Cygwin.)
2021-02-22 15:03:49 -06:00
Mahmoud Al-Qudsi
215df7eec6 Extend PCRE2 wchar_t interop support to 16-bit wchar_t impls
Previously, the interop glue for more friendly access to PCRE2's
fixed-size values was only used when char32_t/wchar_t were
interchangeable and PCRE2 was used with a global 32-bit unit width set;
this patch extends the same to char16_t when wchar_t is also 16-bits
(namely on Cygwin) to avoid compilation fpermissive warnings about casts
between types of potentially different sizes.

Reported in #6585.
2021-02-22 13:51:29 -06:00
Mahmoud Al-Qudsi
2cf5fd3d5d Avoid hard compilation errors on platforms w/out O_ASYNC
Those platforms should not be using the sigio notifier in the first
place, this just stops them from failing to be able to compile fish
altogether.

See #6585
2021-02-21 22:39:32 -06:00
ridiculousfish
11a373f121 Prevent redirecting internal processes to file descriptors above 2
The user may write for example:

    echo foo >&5

and fish would try to output to file descriptor 5, within the fish process
itself. This has unpredictable effects and isn't useful. Make this an
error.

Note that the reverse is "allowed" but ignored:

    echo foo 5>&1

this conceptually dup2s stdout to fd 5, but since no builtin writes to fd
5 we ignore it.
2021-02-20 16:16:45 -08:00
Ilya Grigoriev
622f2868e1 Fix set -U when fish_variables is a symlink
Previously, `set -U` would overwrite the symlink with a
regular file.

Fixes https://github.com/fish-shell/fish-shell/issues/7466
2021-02-20 14:24:11 -08:00
ridiculousfish
8648f52857 Use wcs2string when outputting the screen
The screen output code predates the ENCODE_DIRECT scheme, and
directly-encoded bytes were not being properly output. This manifested as
private-user characters being mangled when printed as part of fish_prompt.
Just use str2wcstring instead.

Fixes #7723
2021-02-17 15:38:46 -08:00
ridiculousfish
f1b6a5939d Introduce wcs2string_appending
wcs2string_appending is like wcs2string, but appends to a std::string
instead of creating a new one. This will be more efficient when a string
can be reused, or if we want to accumulate multiple wcstrings into a
single std::string.
2021-02-17 15:36:22 -08:00
ridiculousfish
fce485e6a8 Rename bufferCount_ to buffer_count_
fish uses snake case for naming.
2021-02-17 14:25:32 -08:00
Mahmoud Al-Qudsi
774488686c Correct handling of SGR mouse tracking events
They are of variable length, taking semicolon-separated ASCII characters
and not single chars/bytes as the parameters. Additionally, the global
maximum size for a CSI is 16 characters (NPAR), even though I believe
the maximum possible mouse-tracking CSI is 12 characters.
2021-02-17 15:45:35 -06:00
ridiculousfish
7e77dc8964 Add a test for round-tripping characters in the private use area
I wrote this test believing that fish had a bug, but actually fish is
behaving correctly here. Still the test is nice so I am checking it in.
2021-02-17 12:29:51 -08:00
ridiculousfish
c0b39aaeb7 Do not reset terminal color when donating term for running key bindings
fish maintains two tty modes: one for itself and one for external
commands. The external command mode is also used when executing
fish-script key bindings, which was added in 5f16a299a7 (note that
commit had the wrong issue, the correct issue is #2114).

Prior to this fix, when switching to external modes, we would also reset
the tty's foreground color. This bumped tty's timestamp, causing us to
believe that the tty had been modified, and then repainting the prompt. If
the prompt were multi-line, we would repaint the whole prompt starting
from its second line, leaving a trailing line above it.

It would be reasonable to save the tty timestamp after resetting the
color, but given that using external modes for keybindings is new, it's
better to instead not reset the color in this case. So migrate the color
resetting to only when we run external commands.

Fixes #7722
2021-02-17 11:10:51 -08:00
Mahmoud Al-Qudsi
75af89699a Patch fish_tests to work with changed const_strlen requirements 2021-02-17 12:55:21 -06:00
Mahmoud Al-Qudsi
70b06a899d Bypass recursion in case of single trailing nul in const_strlen()
The default case for string literals like `"foo"` is a single trailing
nul, and that's what we have almost everywhere. By checking the
second-to-last index for a non-nul byte, we can skip the recursive
invocation, thus speeding up compilation that teeny, tinsy bit faster.
2021-02-17 12:26:40 -06:00
Mahmoud Al-Qudsi
071b9b13cd Reverse const_strlen() recursion logic
Rather than making the run-time complexity of the algorithm 𝒪(n) where n
is the length of the string, make it 𝒪(k) where k is the number of
trailing nul bytes.

The second parameter `index` with a default non-value is in lieu of a
helper function that would have had a name like `count_trailing_nuls()`.
2021-02-17 12:26:40 -06:00
Fabian Homborg
313b70a0c2 math: Set LC_NUMERIC to C again
e94f86e6d2 removed it in favor of using
fish_wcstod, but this broke the *output* - math currently prints
numbers with "," and then can't read them.

So we partially revert it until we come up with something better.
Maybe set $LC_NUMERIC globally inside fish?
2021-02-17 09:07:40 +01:00
Johannes Altmanninger
c0af8dae20 fish_indent: fix extra indent of continuation lines inside blocks
fish_indent used to increment the indentation level whenever we saw an escaped
newline.  This broke because of recent changes to parse_util_compute_indents().
Since parse_util_compute_indents() function already indents continuations
there is not much to do for fish_indent - we can simply query the indentation
level of the newline.  Reshuffle the code since we need to pass the offset
of the newline. Maybe this can even be simplified further.

Fixes #7720
2021-02-16 18:39:03 +01:00
Johannes Altmanninger
444c05dfb1 Do not indent after escaped newline in comment
We do something similar in fish_indent.  This fixes the spurious indent
after comments in share/completions/emerge.fish.

See #7720
2021-02-16 18:39:03 +01:00
Fabian Homborg
00fabb4fe1 Enhance fallback keybindings a bit
Bind \cc like normal, since we now no longer use a function, and bind
some important control bindings like \cs and the ever-important emacs \cb/f/p/n.

What really kills the usability here is the up-line vs up-or-search.
2021-02-15 20:54:35 +01:00
Fabian Homborg
65a760528a Skip figuring out config with --no-execute
Cuts the time to check all our fish scripts in the
check-all-fish-files.fish test roughly in half, from 3.3s
to 1.8s.
2021-02-15 18:59:54 +01:00
Fabian Homborg
c83436d050 Don't overwrite IXON/IXOFF for startup modes
We actually restore those before exit, so this would force-disable
flow control whenever fish exits.
2021-02-15 17:30:50 +01:00
Fabian Homborg
1426d61bc9 Stop disabling VSTOP/VSTART
Without flow control enabled these won't be interpreted any way.
2021-02-15 17:30:50 +01:00
Fabian Homborg
2be720b6cc Set flow control settings also in the shell
Since, unlike e.g. OPOST, this can sometimes be useful, just copy
whatever flow control settings the terminal ends up with.

We still *default* flow control to off (because it's an awful default
and allows us to bind ctrl-s), but if the user decides to enable it so
be it.

Note that it's _possible_ flow control ends up enabled accidentally, I
doubt this happens much and it won't render the shell unusable (and
good terminals might even tell you you've stopped the app).

Fixes #7704
2021-02-15 17:30:50 +01:00
Fabian Homborg
c27fb9b802 source: Escape filenames in errors
Otherwise this would look weird if you had, say, a tab in there.

See #7716.

(note that this doesn't handle e.g. zero-width-joiners, because those
aren't currently escaped. we might want to add an escape mode for
unprintable characters, but for combining codepoints that's tricky!)
2021-02-15 17:08:26 +01:00
Mahmoud Al-Qudsi
ef5db47cf7 Make const_strlen return an unsigned value 2021-02-14 11:42:26 -06:00
Fabian Homborg
552c7bc40b Don't add a space if completion only added a single "/"
This added a space if only one character was added, e.g.

```fish
cd dev<TAB>
```

would complete to

```fish
cd dev/<SPACE>
```

which makes picking deeper directories awkward.

So just go back to the old behavior of doing it for any length.

This is a regression from e27d97b02e.

cc @krobelus
2021-02-14 13:50:43 +01:00
ridiculousfish
9a165b93fb handle_builtin_output to take io_chain by const reference
There was no reason for this to be a pointer or mutable.
2021-02-13 20:05:33 -08:00
ridiculousfish
ee15bc2a36 Add a variant of valid_var_name which accepts const wchar_t *
This avoids creating some unnecessary strings.
2021-02-13 18:49:43 -08:00
Johannes Altmanninger
5b3a466fa9 Refactor: collapse if statements 2021-02-13 09:01:41 +01:00
Johannes Altmanninger
5e8a248758 Indent escaped newlines
Similar to what fish_indent does. After typing "echo \" and hitting return,
the cursor will be indented.

A possible annoyance is that when you have multiple indented lines

	echo 1 \
	    2 \
	    3 \
	    4 \

If you remove lines in the middle with Control-k, the lines below
the deleted one will start jumping around, as they are disconnected
from and reconnected to "echo".
2021-02-13 09:01:41 +01:00
Johannes Altmanninger
511747d59e Indent only leaf nodes and in-between gaps
Probably not necessary for the next commit, but this way feels more logical
2021-02-13 09:01:41 +01:00
Johannes Altmanninger
7ee4a3b40d Indent empty lines inside block 2021-02-13 09:01:41 +01:00
Johannes Altmanninger
fffcdf8792 Highlight redirection target as valid if it contains a to-be-defined variable
If a variable is undefined, but it looks like it will be defined by the
current command line, assume the user knows what they are doing.
This should cover most real-world occurrences.

Closes #6654
2021-02-13 08:59:54 +01:00
Ethel Morgan
5a0aa7824f Saturate exit codes to 255 for all builtins
After commit 6dd6a57c60, 3 remaining
builtins were affected by uint8_t overflow: `exit`, `return`, and
`functions --query`.

This commit:
- Moves the overflow check from `builtin_set_query` to `builtin_run`.
- Removes a conflicting int -> uint8_t conversion in `builtin_return`.
- Adds tests for the 3 remaining affected builtins.
- Simplifies the wording for the documentation for `set --query`.
- Does not change documentation for `functions --query`, because it does
  not state the exit code in its API.
- Updates the CHANGELOG to reflect the change to all builtins.
2021-02-13 08:41:51 +01:00
Johannes Altmanninger
e27d97b02e Do not add spaces after completions ending in "-"
Some programs use this to separate things in a word, see
https://github.com/spf13/cobra/pull/1249#discussion_r563605048

Require the token to be at least length 2 for the no-space behavior,
for completions of "-" like for python.
2021-02-13 08:13:31 +01:00
Johannes Altmanninger
b5df9a7137 Fix a compiler warning about comparison of different signedness 2021-02-13 08:07:20 +01:00
Fabian Homborg
a503371c57 termsize: Default to 80x24 when the terminal says 0 again
This was lost in
6bdbe732e40c2e325aa15fcf0f28ad0dedb3a551..c7160d7cb4970c2a03df34547f357721cb5e88db.

Note that we only print a term-support flog message for now, the
warning seems a bit much.

Fixes #7709.
2021-02-12 18:25:21 +01:00
Fabian Homborg
056153eddd Remove stray comment
[ci skip]
2021-02-11 18:44:53 +01:00
ridiculousfish
17d6aa054b exec_internal_builtin_proc to stop returning failure
Now that closing stdin is no longer an error for builtins, the function
exec_internal_builtin_proc cannot fail. Make it return void instead.
2021-02-10 17:43:12 -08:00
ridiculousfish
84d59accfc builtins to allow stdin to be closed
Prior to this fix, if stdin were explicitly closed, then builtins would
silently fail. For example:

    count <&-

would just fail with status 1. Remove this limitation and allow each
builtin to handle a closed stdin how it sees fit.
2021-02-10 17:43:10 -08:00
ridiculousfish
f239329f33 Attempt to fix the 32 bit fd_monitor test
Speculatively the fd_monitor thread is not scheduled, or we are awoken
early. Add a loop to ensure it gets run.

This is an attempt at #7699
2021-02-10 12:28:34 -08:00
Johannes Altmanninger
51850e9303 Output human-readable descriptions of some parse errors
"echo > )" provokes one of these.
2021-02-09 22:38:16 +01:00
Johannes Altmanninger
86707378cc Also allow unclosed quotes in some places
See #7693
2021-02-09 22:38:16 +01:00
Johannes Altmanninger
38b95defbd Inside an unclosed subshell, do not report other parse errors
In an interactive shell, typing "for x in (<RET>" would print an error:

	fish: Expected end of the statement, but found a parse_token_type_t::tokenizer_error

Our tokenizer converts "(" into a special error token, hence this message.
Fix two cases by not reporting errors, but only if we allow parsing incomplete
input. I'm not really sure if this is necessary, but it's sufficient.

Fixes #7693
2021-02-09 22:19:42 +01:00
ridiculousfish
e423a58e24 Add a thread yield to topic monitor torture test
This speeds up the test by about 5 msec.
2021-02-08 14:06:47 -08:00
ridiculousfish
4b9a096cf2 builtins to sometimes not buffer when writing to a pipe
Prior to this change, if you pipe a builtin to another process, it would
be buffered. With this fix the builtin will write directly to the pipe if
safe (that is, if the other end of the pipe is owned by some external
process that has been launched).

Most builtins do not produce a lot of output so this is somewhat tricky to
reproduce, but it can be done like so:

     bash -c 'for i in {1..500}; do echo $i ; sleep .5; done' |
	   string match --regex '[02468]' |
	   cat

Here 'string match' is filtering out numbers which contain no even digits.
With this change, the numbers are printed as they come, instead of
buffering all the output.

Note that bcfc54fdaa fixed this for the case where the
builtin outputs to stdout directly. This fix extends it to all pipelines
that include only one fish internal process.
2021-02-08 14:22:02 -08:00
ridiculousfish
171d09288b Rename allow_buffering to piped_output_needs_buffering
This makes the variable's role clear. It controls whether output to a
pipe must be buffered to avoid deadlock.
2021-02-08 14:22:02 -08:00
Mahmoud Al-Qudsi
2d39568ec4 Statically assert the sort order of more lists
Add compile-time checks to ensure list of string subcommands, builtins,
and electric variables are kept in asciibetical order to facilitate
binary search lookups.
2021-02-08 15:31:49 -06:00
Mahmoud Al-Qudsi
cb3ab80cab Use const_strlen in a few different places
This may slightly improve performance by allowing the compiler greater
visibility into what is happing on top of not executing at runtime in
some hot paths, but more importantly, it gets rid of magic constants in a
few different places.
2021-02-08 15:16:21 -06:00
Mahmoud Al-Qudsi
5d4c24bae1 Refactor color.h/color.cpp
* Use `uint8_t` instead of `unsigned char`
* Statically assert the sort order for `named_colors`
* Use constexpr for array lengths
2021-02-08 15:16:21 -06:00
Mahmoud Al-Qudsi
5fda1e05dc Statically assert the sort order of input_function_metadata_t 2021-02-08 15:16:20 -06:00
Mahmoud Al-Qudsi
2df16b53ff Use thread-local vectors for caching peeked events
These functions are called in the event queue hot path every time an
input event takes place. If we could guarantee a maximum length of
non-char (i.e. readline) events in the queue, we could use
`event_queue_peeker_t` with a fixed storage size of, e.g., 32 events,
but I'm not sure what a reasonable number would in fact be, so I'm just
changing these to use a thread-local vector that will re-use its
previous heap allocation in subsequent invocations rather than thrashing
the heap.
2021-02-08 15:16:20 -06:00
Mahmoud Al-Qudsi
2c1764fd45 Convert more event queue push_front loops to insert_front 2021-02-08 15:16:20 -06:00
Mahmoud Al-Qudsi
5c014e129a Reduce input latency searching for readline function mappings
The lookups are executed on all input events, so they are worth
optimizing.

Cache the list of names, use binary search to get a function code from a
name, and stop enumerating mappings after `has_function` and `has_command`
have been determined.
2021-02-08 15:16:20 -06:00
Ethel Morgan
6dd6a57c60 Saturate return value in builtin_set_query
builtin_set_query returns the number of missing variables. Because the
return value passed to the shell is an 8-bit unsigned integer, if the
number of missing variables is a multiple of 256, it would overflow to 0.

This commit saturates the return value at 255 if there are more than 255
missing variables.
2021-02-08 20:38:56 +01:00
Johannes Altmanninger
e40850ab89 Format fish_tests.cpp 2021-02-08 07:31:33 +01:00
ridiculousfish
50a7798041 Elimiate static variables inside builtin_test
builtin_test stashes some variables in statics, to support
the `test -t` expression. However this will cause conflicts with
concurrent execution, where we may want to run two `test` expressions at
once. Do the grunt work of threading the data into all places it needs
to go.
2021-02-07 17:41:36 -08:00
ridiculousfish
40d8e7e983 Correct the sense of a test for builtin stdin fds
fish isn't quite sure what to do if the user specifies an fd redirection
for builtins. For example `source <&5` could potentially just read from
an arbitrary file descriptor internal to fish, like the history file.

fish has some lame code that tries to detect these, but got the sense
wrong. Fix it so that fd redirections for builtins are restricted to
range 0 through 2.
2021-02-07 16:21:33 -08:00
ridiculousfish
17707065b8 Remove the io_pipe_t parameter from exec_internal_builtin_proc
This parameter describes if stdin has a pipe, but that can be easily
inferred from the io_chain. Remove it in the interest of parsimony.
2021-02-07 16:03:58 -08:00
Fabian Homborg
b3626d48e7 Highlight keywords differently
This introduces a new variable $fish_color_keyword that will be used
to highlight keywords. If it's not defined, we fall back on
$fish_color_command as before.

An issue here is that most of our keywords have this weird duality of
also being builtins *if* executed without an argument or with
`--help`.

This means that e.g.

    if

is highlighted as a command until you start typing

    if t

and then it turns keyword.
2021-02-07 21:18:51 +01:00
ridiculousfish
0f1281bec6 Unify thread sanitizer detection
We now have two files that need to know if thread sanitizer is enabled. They
can share the detection code.
2021-02-07 10:59:10 -08:00
ridiculousfish
ced56d492f Disable iothread pool wait-around under TSan
The iothread pool has a feature where, if the thread is emptied, some
threads will choose to wait around in case new work appears, up to a
certain amount of time (500 msec). This prevents thrashing where new
threads are rapidly created and destroyed as the user types. This is
implemented via `std::condition_variable::wait_for`. However this function
is not properly instrumented under Thread Sanitizer (see
https://github.com/google/sanitizers/issues/1259) so TSan reports false
positives. Just disable this feature under TSan.
2021-02-07 10:59:10 -08:00
ridiculousfish
a942df3886 Use fd_event_signaller_t in fd_monitor_t
fd_monitor_t allows observing a collection of fds. It also has its own
fd, which it uses to awaken itself when there are changes. Switch to
using fd_event_signaller_t instead of a pipe; this reduces the number of
file descriptors and is more efficient under Linux.
2021-02-07 10:59:10 -08:00
ridiculousfish
e004930947 Use fd_event_signaller in iothread completions
This simplifies how iothread notices when there are completions ready to
run.
2021-02-07 10:59:10 -08:00
ridiculousfish
8066428feb Add fd_event_signaller_t
fd_event_signaller_t exists to expose eventfd under Linux. This is a
more lightweight way of signalling events than using a pipe.
2021-02-07 10:59:10 -08:00
Fabian Homborg
78358ab351 Make disable_mouse_tracking inaccessible
This isn't something you want to bind, it's only a readline symbol as
a hack, so we shouldn't expose it to the user.
2021-02-07 19:50:56 +01:00
ridiculousfish
aac5862a67 Use vectors, not queues, in iothread main thread requests
queues use std::deque under the hood which is more expensive than a vector.
We always consume the entire queue so there is no advantage to use deque here.
Just use a vector.
2021-02-06 16:19:21 -08:00
ridiculousfish
76833cf6af Use futures in perform_on_main_thread
Replace the complicated implementation which shared a condition variable, with
one which just uses std::future<void>. This may allocate more condition
variables but is much simpler.
2021-02-06 16:19:21 -08:00
Mahmoud Al-Qudsi
eecc223c51 Recognize and disable mouse-tracking CSI events
Fish was previously oblivious to the existence of mouse-tracking ANSI
escapes; this was mostly OK because they're disabled by default and we
don't enable them, but if a TUI application that turned on mouse
reporting crashed or exited without turning mouse reporting off, fish
would be left in an unusable state as all mouse reporting CSI sequences
would be posted to the prompt.

This can be tested by executing `printf '\x1b[?1003h'` at the prompt,
then clicking with any mouse button anywhere within the terminal window.
Previously, this would have resulted in seeming garbage being spewed to
the prompt; now, fish detects the mouse tracking CSIs posted to stdin by
the terminal emulator and a) ignores them to prevent invalid input, as
well as b) posts the CSI needed to disable future mouse tracking events
from being emitted on subsequent mouse interactions (until re-enabled).

Note that since we respond to a mouse tracking CSI rather than
pre-emptively disable mouse reporting, we do not need to do any sort of
feature detection to determine whether or not the terminal supports
mouse reporting (otherwise, if it didn't support it and we posted the
CSI anyway, we'd end up with exactly the kind of cruft posted to the
prompt that we're trying to avoid).

Fixes #4873
2021-02-06 17:22:59 -06:00
Mahmoud Al-Qudsi
cc392b3774 Add RAII-based event_queue_peeker_t helper
This is a stack-allocating utility class to peek up to N
characters/events out of an `event_queue_t` object. The need for a
hard-coded maximum peek length N at each call site is to avoid any heap
allocation, as this would be called in a hot path on every input event.
2021-02-06 17:18:53 -06:00
Mahmoud Al-Qudsi
c203c88c66 Add and use event_queue_t::insert_front()
This allows directly inserting multiple characters/events in one go at
the front of the input queue, instead of needing to add them one-by-one
in reverse order.

In addition to improving performance in case of fragmented dequeue
allocation, this also is less error prone since a dev need not remember
to use reverse iterators when looping over a vector of peeked events.
2021-02-06 17:18:53 -06:00
Mahmoud Al-Qudsi
ea1a4b7932 Explicitly annotate intentional switch fallthrough
This silences a very useful warning in GCC 10.
2021-02-06 17:03:23 -06:00
ridiculousfish
b7e892d545 next_thread_id to use atomics, not locks
We have multiple places where we use std::atomic<uint64_t>, so let's use it
in next_thread_id too.
2021-02-06 14:27:08 -08:00
ridiculousfish
fe334bf620 Remove scoped_rlock
It is unused.
2021-02-06 14:27:08 -08:00
ridiculousfish
9c238385f0 Fix binary_semaphore_t under non-Linux TSan
Under non-Linux builds, binary_semaphore is implemented with a
self-pipe. When TSan is active we mark the pipe as non-blocking as TSan
cannot interrupt read (but can interrupt select). However we weren't
properly testing for EAGAIN leading to an assertion failure.

Allow looping on EAGAIN.
2021-02-06 14:41:50 -08:00
ridiculousfish
736e344727 assert_is_locked to take std::mutex, not void *
It's unclear why this had the void* cast.
2021-02-06 14:24:45 -08:00
ridiculousfish
98b0ef532f io_buffer_t to store a promise, not a future, to satisfy TSan
io_buffer_t is a buffer that fills itself by reading from a file
descriptor (typically a pipe). When the file descriptor is widowed, the
operation completes, and it reports completion by marking a
`std::promise<void>`. The "main thread" waits for this by waiting on the
promise's future. However TSan was reporting that the future's destructor
races with its promise's wait method. It's not obvious if this is valid,
but we can fix it by keeping the promise alive until the io_buffer_t is
deallocated.

This fixes the TSan issues reported under
`complete_background_fillthread_and_take_buffer` for #7681 (but there
are other unresolved issues).
2021-02-06 13:28:01 -08:00
Fabian Homborg
caca4fec22 fds.h: Add missing types.h include
Broke the build on FreeBSD because that defines mode_t there.
2021-02-06 19:59:53 +01:00
ridiculousfish
b5716e97cc Remove fd_set_t
Now that we no longer need to worry about pipes conflicting with
user-specified redirections, we can remove fd_set_t.
2021-02-05 18:14:50 -08:00
ridiculousfish
b79ec0122a Use pipe2 when creating pipes if avaialble
This allows us to avoid marking the pipe as CLOEXEC in some cases,
saving a system call.
2021-02-05 17:58:08 -08:00
ridiculousfish
97f29b1f4d Pipe fds to move to the "high range"
This concerns how fish prevents its own fds from interfering with
user-defined fd redirections, like `echo hi >&5`. fish has historically
done this by tracking all user defined redirections when running a job,
and ensuring that pipes are not assigned the same fds. However this is
annoying to pass around - it means that we have to thread user-defined
redirections into pipe creation.

Take a page from zsh and just ensure that all pipes we create have fds in
the "high range," which here means at least 10. The primary way to do this
is via the F_DUPFD_CLOEXEC syscall, which also sets CLOEXEC, so we aren't
invoking additional syscalls in the common case. This will free us from
having to track which fds are in user-defined redirections.
2021-02-05 17:58:08 -08:00
ridiculousfish
6c4f2622ef iothread's notify pipes to use make_autoclose_pipes
This allows it to take advantage of the upcoming high-range fd changes.
2021-02-05 17:58:08 -08:00
ridiculousfish
4b4bf541d1 Migrate more fd-concerned functions from wutil into fds
Functions like wopen_cloexec have a new home in fds.cpp. This is in
preparation for reworking how internal fds avoid conflict with user fds.
2021-02-05 17:58:08 -08:00
ridiculousfish
6588cf35f4 Move autoclose_pipes_t from io.h to fds.h 2021-02-05 17:58:08 -08:00
ridiculousfish
be9375e914 Migrate autoclose_fd_t to new file fds.h
fds.h will centralize logic around working with file descriptors. In
particular it will be the new home for logic around moving fds to high
unused values, replacing the "avoid conflicts" logic.
2021-02-05 17:58:08 -08:00
Fabian Homborg
b5305ce3d3 Handle backslashes properly in locate_brackets_of_type
This needs to be rewritten, I'm pretty sure we have like 6 of these
kinds of ad-hoc "is this quoted" things lying around.

But for now, at least don't just check if the *previous* character was
a backslash.

Fixes #7685.
2021-02-05 22:03:13 +01:00
ridiculousfish
97bde2f2bf Further refactoring of io_buffer_t
Previously we sometimes wanted to access an io_buffer_t to append to it
directly, but that's no longer true; all we really care about is its
separated_buffer_t. Make io_bufferfill_t::finish return the
separated_buffer directly, simplifying call sites. No user visible changes
expected here.
2021-02-04 17:14:46 -08:00
ridiculousfish
258149fe2e Improve locking discipline in io_buffer_t
Previously we had a lock that was taken in an ad-hoc manner. Switch to
using owning_lock.
2021-02-04 17:03:54 -08:00
ridiculousfish
8bcc8c1a36 Further cleanup of separated_buffer_t and io_buffer_t
Remove some clinging tendrils of life as a template object.
2021-02-04 16:43:47 -08:00
ridiculousfish
cbf10971f0 Reorganize separated_buffer_t
Move private bits to the bottom and do some other mild cleanup.
2021-02-04 16:06:28 -08:00
ridiculousfish
d578f8d136 separated_buffer_t to accept strings by rvalue reference
This saves a copy in some cases.
2021-02-04 16:02:40 -08:00
ridiculousfish
032467f338 separated_buffer_t to stop being a template
Now that we no longer construct wide separated buffers, it doesn't have
to be templatized.
2021-02-04 15:32:11 -08:00
ridiculousfish
7d494eab5c builtins to write to buffers directly
This concerns builtins writing to an io_buffer_t. io_buffer_t is how fish
captures output, especially in command substitutions:

    set STUFF (string upper stuff)

Recall that io_buffer_t fills itself by reading from an fd (typically
connected to stdout of the command). However if our command is a builtin,
then we can write to the buffer directly.

Prior to this change, when a builtin anticipated writing to an
io_buffer_t, it would first write into an internal buffer, and then after
the builtin was finished, we would copy it to the io_buffer_t. This was
because we didn't have a polymorphic receiver for builtin output: we
always buffered it and then directed it to the io_buffer_t or file
descriptor or stdout or whatever.

Now that we have polymorphpic io_streams_t, we can notice ahead of time
that the builtin output is destined for an internal buffer and have it
just write directly to that buffer. This saves a buffering step, which is
a nice simplification.
2021-02-04 15:21:32 -08:00
ridiculousfish
cd9a035f02 Add a string_output_stream_t to collect builtin output
This is used when creating a function; this breaks a dependency on the
more complicated buffered_output_stream_t to ease refactoring.
2021-02-04 14:12:14 -08:00
ridiculousfish
fc97151aec Add a variant of wcs2string which accepts a ptr, length pair
This will be useful when refactoring separated buffers.
2021-02-04 13:28:48 -08:00
ridiculousfish
86a12e1abd separated_buffer_t::append to stop being a template
In preparation for simplifying how builtins write to buffers, make
append an ordinary function rather than a template function.
2021-02-04 13:19:11 -08:00
ridiculousfish
7e2a538300 create_output_stream_for_builtin to accept read limit directly
This avoids requiring passing in a parser.
2021-02-03 19:00:04 -08:00
ridiculousfish
2d78c9a0d9 Poll the uvar notifier when the reader is interrupted by a signal
While the user waits at the prompt, fish is waiting in select(), on stdin.
The sigio based universal notifier interrupts select() by arranging for a
signal to be delivered, which causes select() to return with EINTR.
However we weren't polling the notifier at that point so we would not
notice uvar changes, until we got some real input.

I didn't notice this when testing, because my testing was changing fish
prompt colors which updated the prompt for other reasons.

Fixes #7671.
2021-01-31 15:42:35 -08:00
ridiculousfish
409ed7d6d0 Factor out count_preceding_backslashes
Now that we have multiple clients of count_preceding_backslashes, factor
it out from fish_indent into wcstringutil.h, and then use the shared
implementation.
2021-01-30 16:20:20 -08:00
Shizcow
cff5aa9130 Ensure escaped trailing spaces are not trimmed 2021-01-30 15:57:29 -08:00
Fabian Homborg
594d51e7eb Add a separate --profile-startup option to profile startup
This goes to a separate file because that makes option parsing easier
and allows profiling both at the same time.

The "normal" profile now contains only the profile data of the actual
run, which is much more useful - you can now profile a function by
running

   fish -C 'source /path/to/thing' --profile /tmp/thefunction.prof -c 'thefunction'

and won't need to filter out extraneous information.
2021-01-29 20:46:34 +01:00
Fabian Homborg
005d3a5981 Enable strict-aliasing and implicit-fallthrough warnings
GCC needs to have the comment *right before* the case label... blergh
2021-01-29 18:23:30 +01:00
Fabian Homborg
4e8c0f757d complete: Don't require a parameter with --force-files
A classic fallthrough problem!

This is why I want to enable -Wimplicit-fallthrough
2021-01-29 18:23:29 +01:00
Johannes Altmanninger
062f24d91b builtin set: make slice index range optional, like in slice expansion
Expansion parses slices like "$PATH[1..2]", but so does "set" when assigning
"set PATH[1..2] . .".  Commit be06f842a ("Allow to omit indices in index
range expansions") forgot the latter.
2021-01-28 07:19:38 +01:00
Fabian Homborg
275534b1b3 read: Remove unused short options
This has both "m" and "B" in the short options but did nothing with
them, so it would assert() out.

Fixes #7659.
2021-01-26 07:06:25 +01:00
Fabian Homborg
baa9b21a6f type: Only print function path with "--path"
Fixes #7653.
2021-01-24 15:31:39 +01:00
Fabian Homborg
88a84bd988 reader: Force ONLCR on for fish and external commands
Just like OPOST this just breaks output for anything not prepared for
it. Fish itself might work with it (and #4505 recommends it), but external commands are broken.

You'll see output like

foo
   ⏎

from `echo foo`.

Fixes #4873.

Continuation of #7133.
2021-01-18 21:00:08 +01:00
Fabian Homborg
8b133833fa Don't inherit windows paths for $PWD
If given a windows path like `F:\foo`, this currently ends up
assert()ing in path_normalize_for_cd.

Instead, since these paths violate a bunch of assumptions we make, we
reject them and fall back on getting $PWD via getcwd() (which should
give us a nice proper unixy path).

Fixes #7636.

This isn't tested because it would require a system where a windowsy
path passes paths_are_same_file, and on the unix systems we run our
tests that's impossible as far as I can tell?
2021-01-17 23:08:04 +01:00
Fabian Homborg
932074f06c escape_string_script: Escape DEL as \x7f
This used to print a literal DEL character in the output for `bind`,
which wouldn't actually show up and made it hard to figure out what
the key was.

So we just escape it back to how we actually used it - `\x7f`.

Fixes #7631.
2021-01-16 12:49:49 +01:00
Fabian Homborg
a4f5dd5054 set: Move the new values
A C++ special!

This makes

```fish
set -l var (seq 1 10000)
set -l v
for f in $var
    set -a v $f
end
```

~15% faster by removing allocations.
2021-01-15 21:00:25 +01:00
ridiculousfish
7a0bddfcfa Teach string repeat to handle multiple arguments
Each argument in string repeat is handled independently, except that the
--no-newline option applies only to the last newline.

Fixes #5988
2021-01-11 17:00:06 -08:00
ridiculousfish
290d1f2cd6 Mild refactoring of builtin_string repeat
Preparation for fixing issue 5988; no behavior change expected here.
2021-01-11 16:52:39 -08:00
ridiculousfish
1d4883d810 Remove an unnecessary 'using' declaration
This was just redundant with the struct tag.
2021-01-11 15:23:52 -08:00
ridiculousfish
7207a205f2 Switch history races test to use threads instead of processes
This avoids issues with ASan and TSan whose allocators do not properly
clean up in atfork, leading to deadlocks in child processes.
2021-01-11 12:44:21 -08:00
Fabian Homborg
3fc9c0b38c tests: Increase cancellation delay
This sometimes fails on github actions with ASAN. I am assuming that's
because the ctrl-c happens *before* the process has had a chance to
start.

So we do what we do and increase the delay.
2021-01-11 21:00:33 +01:00
Fabian Homborg
7bf2b9fd43 output: Rename some variables
These are a foreground and a background color. Now I see the point in
not naming them "foreground_color" and "background_color", but at
least "fg" and "bg" should do, right?
2021-01-11 20:56:15 +01:00
Fabian Homborg
f7b2bf8229 output: Simplify some duplicated code
Becomes a bit boring after a while
2021-01-11 20:53:11 +01:00
Fabian Homborg
19efd22468 env: Setup $HOME/$USER *before* the config directories
They are based on $HOME, so setting $HOME has to be done first.

Fixes #7620

(untested because I'm assuming common CI systems have weird $HOME settings)
2021-01-11 18:51:47 +01:00
ridiculousfish
e8c9da100c Track histories with shared_ptr
Prior to this change, histories were immortal and allocated with either
unique_ptr or just leaked via new. But this can result in races in the
path detection test, as the destructor races with the pointer-captured
history. Switch to using shared_ptr.
2021-01-09 17:02:11 -08:00
ridiculousfish
e062a07a97 Revert "Stop using unique_ptr to store histories"
This reverts commit 6f91195f40.
This triggered ASan complaints due to leaks.
2021-01-09 17:02:11 -08:00
ridiculousfish
87dacc0e95 Improve formatting and layout of history path detection test 2021-01-09 17:02:11 -08:00
ridiculousfish
884eb2b198 Remove an unused static variable 2021-01-09 17:02:11 -08:00
ridiculousfish
89687e7db7 Fix a warning building on Linux
Initialize saved_errno
2021-01-09 13:14:54 -08:00
Fabian Homborg
1dd776ec99 echo: Don't interpret and print options
A weird interaction between grouped short options and our weird option
parsing that puts unknown options back:

```
echo "-n foo"
```

would see the `-n`, turn off printing newlines, interpret the " " as
another grouped short option, see that there is no short option for
space and put the entire token back on the arguments pile.

So it would print "-n foo" *without a newline*.

Fix this by keeping an old state of the options around and reverting
it when putting options back.

The alternative is *probably* to forbid the " " short option in
wgetopt, then check if an option group contains it and error out, but
this should only really be a problem in `echo` because that is,
AFAICT, the only thing that puts the options back.

Fixes #7614
2021-01-09 08:50:30 +01:00
ridiculousfish
3c3d09b65f Fix a tsan warning in features_t 2021-01-08 19:36:56 -08:00
ridiculousfish
6f91195f40 Stop using unique_ptr to store histories
These register shutdown dtors, which cause tsan to complain.
2021-01-08 14:14:05 -08:00
ridiculousfish
bee8e8f6f7 Expand more when performing history path detection
When adding a command to history, we first expand its arguments to see
if any arguments are paths which refer to files. If so, we will only
autosuggest that command from history if the files are still valid. For
example, if the user runs `rm ./file.txt` then we will remember that
`./file.txt` referred to a file, and then only autosuggest that if the file
is present again.

Prior to this change we only performed simple expansion relative to the
working directory. This change extends it to variables and tilde
expansion. For example we will now apply the same hinting for
`rm ~/file.txt`

Fixes #7582
2021-01-08 12:58:34 -08:00
ridiculousfish
4faebf74e6 Remove 100 msec timeout from io_buffer_t
This removes the 100 msec timeout from io_buffer_t. We no longer need to
periodically wake up to check if a command substitution is finished,
because we get explicitly poked when that happens.
2021-01-07 12:07:06 -08:00
ridiculousfish
d5d09c993e io_buffer_t to explicitly poke its item when closing
io_buffer_t is used to buffer output from a command substitution, so we
can split it into arguments. Typically io_buffer_t reads from its pipe
until it gets EOF and then stops reading. However it may be that the
cmdsub ends but EOF is not delivered because the stdout of the cmdsub
escaped with a background process.

Prior to this change we would wake up every 100 msec (select timeout) to
check if the cmdsub is finished. However this 100 msec adds latency if a
background process is launched from e.g. fish_prompt.

Switch to the new poke() function. Now when the cmdsub is finished, it
pokes its item, which explicitly wakes it up. This removes the extra
latency.

Fixes #7559
2021-01-07 11:54:31 -08:00
ridiculousfish
fd08b660c0 Add a poke function to fd_monitor
In preparation for fixing #7559, add a function poke_item to fd_monitor.

fd_monitor has a list of file descriptors, and invokes a callback when an
fd becomes readable. With this change, we assign each item a unique ID and
return it when the item is added; the ID may then be used to invoke the
callback explicitly.

The idea is that we can stop reading from the pipe associated with the
cmdsub when the job is finished, even if the pipe is still open.
2021-01-07 11:51:04 -08:00
Mahmoud Al-Qudsi
7669e8e497 Add concept of edit groups
This allows for multiple edits to be undone/redone in one go, as if they
were one edit.

Useful when a function is editing the commandline buffer via scripted
changes or via a keybinding so the internal changes to the buffer can be
abstracted away.

(Having extreme difficulty getting pexpect to play nice with the concept
of undo/redo...)
2021-01-05 15:43:34 -06:00
Fabian Homborg
6eeb8861e7 Add exit bind function
Currently binding `exit` to a key checks too late that it's exitted,
so it leaves the shell hanging around until the user does an execute
or similar.

As I understand it, the `exit` builtin is supposed to only exit the
current "thread" (once that actually becomes a thing), and the
bindings would probably run in a dedicated one, so the simplest
solution here is to just add an `exit` bind function.

Fixes #7604.
2021-01-04 09:45:34 +01:00
Fabian Homborg
85ba2ed790 type: Add missing newline
Otherwise this would print

    # Defined interactivelyfunction foo

for interactively defined functions.
2021-01-03 17:48:25 +01:00
ridiculousfish
118f710e99 Allow fish_private_mode to change at runtime
Prior to this change, `fish_private_mode` worked by just suppressing
history outright. With this change, `fish_private_mode` can be toggled on
and off. Commands entered while `fish_private_mode` is set are stored but
in memory only; they are not written to disk.

Fixes #7590
Fixes #7589
2021-01-02 22:01:47 -08:00
ridiculousfish
9fdc4f903b Explicitly track persistence mode in history_item_t
Commands that start with a space should not be written to the history
file. Prior to this change, that was implemented by simply not adding them
to history. Items with leading spaces were simply dropped.

With this change, we add a 'history_persistence_mode_t' to
history_item_t, which tracks how the item persists. Items with leading
spaces are now marked as "ephemeral": they can be recovered via up arrow,
until the user runs another command, or types a space and hits return.
This matches zsh's HIST_IGNORE_SPACE feature.

Fixes #1383
2021-01-02 21:31:19 -08:00
ridiculousfish
cdf05325ed Reorganize history_item_t
Move the private bits to the bottom of the class and other mild
refactoring. No user visible behavior change expected.
2021-01-02 19:51:16 -08:00
Fabian Homborg
cf8219e3ce Exit if --no-execute is enabled don't interactively read from the terminal
Don't go into implicit interactive mode without ever executing
anything - not even `exit` or reacting to ctrl-d. That just renders
the shell useless and unquittable.
2021-01-01 21:22:52 +01:00
Fabian Homborg
7ea8e20623
argparse: Make short flag names optional (#7585)
It was always a bit ridiculous that argparse required `X-longflag` if
that "X" short flag was never actually used anywhere.

Since the short letter is for getopt's benefit, we can hack around
this with our old friend: Unicode Private Use Areas.

We have a counter, starting at 0xE000 and going to 0xF8FF, that counts
up for all options that don't have a short flag and provides one. This
gives us up to 6400 long-only options.

6.4K should be enough for everybody.
2021-01-01 11:37:25 +01:00
ridiculousfish
792abf61ec Attempt to fix the tsan build
Deliberately leak the shared thread pool to avoid shutdown dtor registration
and tsan complaints at exit.
2020-12-31 17:03:53 -08:00
ridiculousfish
f03ff8cd00 Add a test for history path detection
This will support history path detection improvements in a future
commit.
2020-12-30 00:44:25 -08:00
Johannes Altmanninger
801955851b Workaround clang-tidy incorrectly assuming null
This silences a false positive linter warning about a null dereference.
2020-12-29 16:31:43 +01:00
Johannes Altmanninger
8fc9b9d61b Address some minor lints
A mildly interesting one is the call to test_wchar2utf8 with a non-null
pointer ("u1"/"dst") but 0 length. In this case we relied on malloc(0)
returning non-null which is not guaranteed.

	src/fish_tests.cpp:1619:23: warning: Call to 'malloc' has an allocation
	size of 0 bytes [clang-analyzer-optin.portability.UnixAPI]
	        mem = (char *)malloc(dlen);
	                      ^
	test_wchar2utf8(w1, sizeof(w1) / sizeof(*w1), u1, 0, 0, 0,
			"invalid params, dst is not NULL");
2020-12-29 16:31:43 +01:00
Johannes Altmanninger
69a9785f50 Refactor: pass by value, not reference, to enable move semantics
clang-tidy wrote:
> warning: passing result of std::move() as a const reference argument;
> no move will actually happen [performance-move-const-arg]
2020-12-29 16:31:43 +01:00
ridiculousfish
43505f7077 Allow ** glob segments to match zero directories
Prior to this change, a glob like `**/file.txt` would only match
`file.txt` in subdirectories; the `**` must match at least one directory.
This is historical behavior.

With this change we move a little closer to bash's implementation by
allowing a literal `**` segment to match in the current directory. That
is, `**/foo` will match both `foo` and `bar/foo`, while `b**/foo` will
only match `bar/foo`.

Fixes #7222.
2020-12-28 23:51:18 -08:00
ridiculousfish
df73964ced Clean up some comments around wildcard expansion 2020-12-28 23:51:18 -08:00
Johannes Altmanninger
322ceb7ab4 builtin realpath: use absolute path also with -s/--no-symlinks
The old test needs to be changed because $XDG_DATA_HOME can be relative.

Fixes #7574
2020-12-24 08:53:08 +01:00
ridiculousfish
e43913a547 Stop expanding globs in command position when performing error checking
Before running a command, or before importing a command from bash history,
we perform error checking. As part of error checking we expand commands
including variables and globs. If the glob is very large, like `/**`, then
we could hang expanding it.

One fix would be to limit the amount of expansion from the glob, but
instead let's just not expand command globs when performing error checking.

Fixes #7407
2020-12-22 12:38:51 -08:00
ridiculousfish
a8080e8e6f Allow specifying a limit on number of expansion in operation_context
If the user types something like `/**`, prior to this change we would
attempt to expand it in the background for both highlighting and
autosuggestions. This could thrash your disk and also consume a lot of
memory.

Add a a field to operation_context_t to allow specifying a limit, and add
a "default background" limit of 512 items.
2020-12-22 12:38:51 -08:00
ridiculousfish
0f2d73e4a3 Remove a stale comment 2020-12-22 12:38:51 -08:00
ridiculousfish
c2c729352e Eagerly abort wildcard completions for ** wildcards
Historically fish has not supported tab completing or autosuggesting
wildcards with **. Prior to this fix, we would test every file match,
discover the ** wildcard, and then ignore it. Instead look for **
wildcards at the top level.

This prevents autosuggesting with /** from chewing up your disk.
2020-12-22 12:38:51 -08:00
ridiculousfish
10362a70df Clean up parse_error_offset_source_start
Use range-based for loops and relax the requirement that we have an
error list.
2020-12-22 12:38:51 -08:00
ridiculousfish
38a30d1798 Mark subclasses of io_data_t as final 2020-12-19 20:06:36 -08:00
ridiculousfish
6f2e377fcc Clean up some unnecessary variable names in maybe.h 2020-12-19 16:10:40 -08:00
ridiculousfish
90f4c458e5 Rename insert_line_above to insert_line_over
This is for symmetry with insert_line_under. See #7442.
2020-12-19 14:31:33 -08:00
ridiculousfish
2d2efc8b2e Implement o and O bindings for vi mode
Credit to @joallard for the patch. Fixes #7442
2020-12-19 14:28:00 -08:00
Fabian Homborg
7e7355bde1 Restore $status after expanding completions
When a completion's "--arguments" script ran, it would clobber $status with its value,
so when you repainted your prompt, it would now show the completion
script's status rather than the status of what you last ran.

Solve this by just storing the status and restoring it - other places
do this by calling exec_subshell with apply_exit_status set to false,
which does basically the same thing. We can't use it here because we
don't want to run a "full" script, we only want the arguments to be
expanded, without a "real" command.

No, I have no idea how to test this automatically.

Fixes #7555.
2020-12-19 11:37:01 +01:00
Fabian Homborg
31166f4731 Simplify some duplicated path checks
This has one functional difference, in that we now report non-EACCESS
errors even for relative paths. I consider that to be a plus.

Some other sites might benefit from this, let's look into that later.
2020-12-15 18:15:59 +01:00
Fabian Homborg
0f6669f43c Stop using env_var_t::to_list in a few places
We don't need the entire list in modifiable form here - some just needs
the size, the others can just get a reference.
2020-12-15 15:47:44 +01:00
Fabian Homborg
14908322a9 Also include fallback.h
GRrrrrr
2020-12-14 23:23:00 +01:00
Fabian Homborg
0f5a226e2f math: Use fish_wcstod instead
1. This should be using our wcstod_l on platforms where we need
it (for some reason it wasn't picking it up on FreeBSD?)

2. This purports to have a "fast path". I like fast paths.
2020-12-14 23:09:01 +01:00
Fabian Homborg
3af07e6c6e math: Wcharify the error message
Dunno, this seems to work, but then this is the sort of thing
that *seems* to work.
2020-12-14 23:02:54 +01:00
Fabian Homborg
e94f86e6d2 math: Use wcstod_l
Locale-wise, we're only interested in one thing:

"." is the radix character when interpreting numbers

And for that it's enough to just use our c-locale, like elsewhere.

This saves a bunch of switching locale back and forth, and simplifies
the code.
2020-12-14 22:58:47 +01:00
Fabian Homborg
97cd87f3b2 math: Use wchar
This was doing a bunch of work narrowing strings for no reason.
2020-12-14 22:54:53 +01:00
Fabian Homborg
6e9364ab50 fish_indent: Change --debug-level to --debug with flog categories
The "debug-level" flag makes little sense since we have no more
debug *levels* left.
2020-12-14 19:36:18 +01:00
ridiculousfish
36766ea3d7 Correct $status for certain pipeline-aborting failures
If we refused to launch a job because of a "pipeline aborting" error,
then it's the caller's responsibility to set $status.

Fixes #7540
2020-12-13 17:33:34 -08:00
ridiculousfish
2caeec24f7 Tighten up pipeline-aborting errors
Prior to this change, the functions in exec.cpp would return true or false
and it was not clear what significance that value had.

Switch to an enum to make this more explicit. In particular we have the
idea of a "pipeline breaking" error which should us to skip processes
which have not yet launched; if no process launches then we can bail out
to a different path which avoids reaping processes.
2020-12-13 17:30:26 -08:00
ridiculousfish
e5cff1a2db Fix some warnings from gcc
Use ignored_result instead of void casts, to satisfy the gcc.
2020-12-13 15:35:59 -08:00
Fabian Homborg
b7f47344b0 Print nicer "defined in" for functions defined on stdin/via source
This would tell you a function was "Defined in - @ line 1" for every
function defined via `source`.

Really, ideally we'd figure out where the *source* call was, but that'
much more complicated, so we just give a comprehensible message.
2020-12-11 23:09:16 +01:00