Commit graph

16552 commits

Author SHA1 Message Date
Aaron Gyes
efa2cf0cb6 Replace fallthrough comments with __fallthrough__
Defined in config.h
2022-10-26 21:02:48 -07:00
Aaron Gyes
df546e01f6 IWYU fixup 2022-10-26 20:04:04 -07:00
Aaron Gyes
92698dff48 Unallowed command subst error: add missing newline and simplify
Fixes ommitted newline char shown after complete -n'(foo)'
Also axes the 'contains syntax errors' line before the error.
Update tests

before
> complete -n'(foo)'
complete: Condition '(foo)' contained a syntax error
complete: Command substitutions not allowed⏎

after
> complete -n'(foo)'
complete: -n '(foo)': command substitutions not allowed here
2022-10-26 19:58:40 -07:00
Aaron Gyes
b2a4a50daf Run include-what-you-use 2022-10-26 19:58:40 -07:00
ridiculousfish
a4aaa4f59b Fix the Xenial build
The Xenial build was failing due to a missing default constructor
in maybe_t. Add it.
2022-10-26 14:19:01 -07:00
Mahmoud Al-Qudsi
06d9708d40 Add complete -k group order test
Ensure that multiple `-k` completions intermixed with one or more non-`-k`
completions are produced in the expected order with the order of all completions
in a single `-k` completion respected, non-`-k` completions correctly sorted and
interspersed, and the results of multiple `-k` completions in the
reverse-intuitive order (with chronologically later completions coming before
chronologically earlier `-k` counterparts), as per #9221.
2022-10-26 13:22:45 -05:00
Mahmoud Al-Qudsi
f7da014602 Optimize storage of completion entries
This is a salvage of the "no functional changes" part of #9221, and cherry-picks
storing completion entries in a vector instead of a linked list. The legacy
"reverse intuitive" group ordering is kept by iterating in reverse order.

Tests pass but don't actually cover group order, which needs another test.
2022-10-26 12:48:31 -05:00
Mahmoud Al-Qudsi
5ad0d95694 CHANGELOG: Add status current-commandline 2022-10-26 12:18:05 -05:00
Mahmoud Al-Qudsi
7133285c88 Move parser status vars to their own struct
Instead of using an enum + array, just use a struct and drop the getter and
setter methods from `parser_t`.
2022-10-26 12:15:02 -05:00
Mahmoud Al-Qudsi
125bcb8289 Add pexpect test for status current-commandline 2022-10-26 12:15:02 -05:00
Mahmoud Al-Qudsi
6ac18defd2 Add status current-commandline
Makes it possible to retrieve the currently executing command line as
opposed to the currently executing command (`status current-command`).

Closes #8905.
2022-10-26 12:15:02 -05:00
Mahmoud Al-Qudsi
e01eb2e615 Add proper way of storing value for status current-command
There should be no functional changes in this commit.

The global variable `$_` set in the parser variables by `reader.cpp` and
read by the `status` builtin was deprecated in fish 2.0 but kept around
internally because there's no good way to store/share/forward parser
variables.

A new enum is added that identifies the status variable and they are
stored in a private array in the parser. There is no need for
synchronization because they are only set during job init and never
thereafter. This is currently asserted via ASSERT_IS_MAIN_THREAD() but
that assert can be dropped in the interest of making the parser possible
to clone and use from worker threads.

The old `$_` global variable is still kept for backwards compatibility,
though it will be dropped in a future release.
2022-10-26 12:15:02 -05:00
Johannes Altmanninger
f637fb31b5 highlight: underline prefixes of valid paths only if at cursor
As the user is typing an argument, fish continually checks if the input is
the prefix of a valid file path. If yes, the input is underlined.

The same prefix-logic is used for all tokens on the command line, even for
"finished" tokens. This means we highlight any token that happens to be
a prefix of a valid file path. We actually want this to only apply to the
token that the user is currently typing.

Let's use the prefix-logic only for tokens adjacent to the cursor.  This should
better match user expectations (and reduce IO traffic). I don't think this is
the perfect criteria but I don't know how else we can determine if a token is
"unfinished".
2022-10-26 16:12:43 +02:00
Johannes Altmanninger
6667c9f50c highlighter: pass the cursor position to the highlighter
This allows the next commit to correct highlighting based on the cursor
position.
2022-10-26 16:11:00 +02:00
Johannes Altmanninger
861ac00a61 highlighter: underline valid "cd" arguments also if they come from CDPATH
When visiting the "cd" node, we mark invalid paths as error, but don't
underline valid paths.  This works fine most of the time because we later
underline paths (for any command, not just "cd").
However the latter check fails to honor CDPATH.  Let's correct that, which
also allows to simplify the logic.
2022-10-26 16:11:00 +02:00
Johannes Altmanninger
dfb0c00d72 highlighter: stop performing IO if canceled
The next commit wants to move the "Underline every valid path" logic into the
visit() methods. The logic currently polls the cancel checker before checking
each path. If that's valid, it should probably have the same behavior inside
visit(). Since we currently can't cancel an AST-visitation, the next best
thing seems to suspend all IO operations, the rest should be very fast anyway.

I'm not sure if the motivation is strong enough; a conceivable alternative
would be to stop using the cancel checker altogether for highlighting.
2022-10-26 16:11:00 +02:00
Johannes Altmanninger
9c6f46a808 highlighter: remove redundant check if we can do io
It's done a few lines above.
2022-10-26 16:09:02 +02:00
Johannes Altmanninger
acb47f70d2 history_file.cpp: remove an unused variable
Now that maybe_t<size_t> no longer has a user-defined destructor, the compiler
can better detect an unused variable of this type.
2022-10-26 16:09:02 +02:00
Johannes Altmanninger
9d303a74e3 maybe_t: make maybe_t<T> trivially copyable if T is
When passing a value of type maybe_t<size_t>, clangd complains:

    Parameter 'cursor' is passed by value and only copied once; consider
    moving it to avoid unnecessary copies (fix available)

We get this warning because maybe_t<size_t> is not trivially copyable
because it has a user-defined destructor and copy-constructor.  Let's remove
them if the contained type is trivially copyable, to avoid such warnings.
No functional change.
2022-10-26 16:09:02 +02:00
Johannes Altmanninger
1ce2961561 maybe_t: remove user-defined destructor
The destructor is equivalent to the compiler-generated one.  The user-defined
destructor prevents maybe_t<size_t> from bearing the predicate "trivially
copyable". Let's remove it. No functional change.
2022-10-26 14:54:33 +02:00
Johannes Altmanninger
45da77c5c5 Format some C++ files with clang-format 2022-10-26 14:53:06 +02:00
Mahmoud Al-Qudsi
ee62bee9cd CHANGELOG: Document eval overflow fix
[ci skip]
2022-10-25 13:51:46 -05:00
Mahmoud Al-Qudsi
2c870b305d Add eval recursion overflow regression test
This particular variant must be executed as a pexpect test since it relies on
the interactive-only `$history` to trigger the recursion. Note that recursion is
possible via other means (e.g. reading/writing a file), the usage of history
here is just one such example.
2022-10-25 13:40:21 -05:00
Mahmoud Al-Qudsi
3913b28153 Only retry failed pexpect tests under CI
A false negative while testing locally should be a rare thing, and individual
pexpect tests already take too long in case of a non-match making for a painful
edit-test loop.
2022-10-25 13:40:21 -05:00
Mahmoud Al-Qudsi
21599a49ea Make CALL_STACK_LIMIT_EXCEEDED_ERR_MSG more generic
We're now using this when a stack overflow is detected during eval/substitution
loops, too.
2022-10-25 13:40:21 -05:00
Mahmoud Al-Qudsi
175caab583 Prevent stack overflow from eval/substitution recursion
It seems to have originally been thought that the only possible way a stack
overflow could happen is via function calls, but there are other possibilities.

Issue #9302 reports how `eval` can be abused to recursively execute a string
substitution ad infinitum, triggering a stack overflow in fish.

This patch extends the stack overflow check to also check the current
`eval_level` against a new constant `FISH_MAX_EVAL_DEPTH`, currently set to a
conservative but hopefully still fair limit of 500. For future reference, with
the default stack size for the main/foreground thread of 8 MiB, we actually have
room for a stack depth around 2800, but that's only with extremely minimal state
stored in each stack frame.

I'm not entirely sure why we don't check `eval_depth` regardless of block type;
it can't be for performance reasons since it's just a simple integer comparison
- and a ridiculously easily one for the branch predictor handle, at that - but
maybe it's to try and support non-recursive nested execution blocks of greater
than `FISH_MAX_STACK_DEPTH`? But even without recursion, the stack can still
overflow so may be we should just bump the limit up some (to 500 like the new
`FISH_MAX_EVAL_DEPTH`?) and check it all the time?

Closes #9302.
2022-10-25 13:40:21 -05:00
Fabian Boehm
14ecb63e40
completions/usermod: Fix subu/gid option spelling
It's "subuid", not "sub-uid".

Fixes #9303
2022-10-25 11:09:41 +02:00
Mahmoud Al-Qudsi
e7bf98adc1 Make block_t moveable
The presence of the explicit constructor (even though it did nothing) prevented
the compiler from generating a move constructor for `block_t`.
2022-10-24 22:06:30 -05:00
Mahmoud Al-Qudsi
84b53b4cae Significantly reduce size of block_t
A `block_t` instance is allocated for each live block type in memory when
executing a script or snippet of fish code. While many of the items in a
`block_t` class are specific to a particular type of block, the overhead of
`maybe_t<event_t>` that's unused except in the relatively extremely rare case of
an event block is more significant than the rest, given that 88 out of the 216
bytes of a `block_t` are set aside for this field that is rarely used.

This patch reorders the `block_t` members by order of decreasing alignment,
bringing down the size to 208 bytes, then changes `maybe_t<event_t>` to
`shared_ptr<event_t>` instead of allocating room for the event on the stack.
This brings down the runtime memory size of a `block_t` to 136 bytes for a 37%
reduction in size.

I would like to investigate using inheritance and virtual methods to have a
`block_t` only include the values that actually make sense for the block rather
than always allocating some sort of storage for them and then only sometimes
using it. In addition to further reducing the memory, I think this could also be
a safer and saner approach overall, as it would make it very clear when and
where we can expect each block_type_type_t-dependent member to be present and
hold a value.
2022-10-24 21:04:17 -05:00
Mahmoud Al-Qudsi
44c9c51841 Disable leak detection in test_autosuggest_suggest_special() under CI
This is a false positive as a result of disabling TLS support in LSAN due to an
incompatibility with newer versions of glibc.

Also remove the older workaround (because it didn't work).
2022-10-24 19:02:49 -05:00
Mahmoud Al-Qudsi
bfa172852f Add a workaround for intermittent LSAN crash under CI
LSAN seems to have an issue with glibc's TLS functionality that causes it to
intermittently crash with SIGSEGV when run virtualized, as it is in our CI.

Relevant GitHub issues:
* https://github.com/google/sanitizers/issues/1342
* https://github.com/google/sanitizers/issues/1409
2022-10-24 18:56:55 -05:00
Mahmoud Al-Qudsi
4f8a7c4779 Drop LSAN CI options that break tests
LSAN with verbosity=1 or log_threads=1 adds output to stderr, breaking
littlecheck tests.
2022-10-24 18:56:55 -05:00
Mahmoud Al-Qudsi
de62091b03 Correctly set ASAN/UBSAN/LSAN options for CI
These are NOT build-time defines but rather run-time environment variables! They
have never had any effect and we have effectively never used them to affect
sanitizer behavior under CI with ASAN/UBSAN/LSAN enabled.

(I caught this because the tests don't pass with either of LSAN_OPTIONS
`verbosity=1` or `log_threads=1` because they inject text into the stderr
output, ensuring they never pass littlecheck.)
2022-10-24 18:56:55 -05:00
Mahmoud Al-Qudsi
63a2fdd773 Re-enable tests under ASAN/LSAN/UBSAN in GitHub CI
With the previous workaround skipping `test_autosuggest_suggest_special()` when
LSAN is enabled, the sanitizer seems to run to completion just fine.
2022-10-24 18:56:55 -05:00
Johannes Altmanninger
90b2c95bbc fish_clipboard_copy: bypass tmux, write OSC 52 to the underlying terminal
For security reasons, some terminals require explicit permission from the
user to interpret OSC 52. One of them is [tmux] but that one usually runs
inside another terminal. This means we can usually write directly to the
underlying terminal, bypassing tmux and the need for user configuration.

This only works if the underlying terminal is writable to the fish user,
which may not be the case if we switched user. For this reason, keep writing
to stdout as well, which should work fine if tmux is configured correctly.

[tmux]: https://github.com/tmux/tmux/wiki/Clipboard
2022-10-24 22:45:45 +02:00
Johannes Altmanninger
4de2891507 fish_clipboard_copy: make it work inside SSH/containers via OSC 52
When running inside SSH, Control-X runs a clipboard utility on the remote
system.  For pbcopy (and probably clip.exe too) this means that we write to the
remote system's clipboard. This is usually not what the user wants (although
it is consistent with  fish_clipboard_paste).  When X11 forwarding is used,
xclip/xsel copy to the SSH client's clipboard, which is what most users want.

When we don't have X11 forwarding, we need a different solution. Fortunately,
modern terminal emulators implement the OSC 52 escape sequence for setting
the clipboard of the terminal's system. Use it in fish_clipboard_copy.

Tested in SSH and Docker containers on foot, iTerm2, kitty, tmux and xterm
(this one requires "XTerm.vt100.allowWindowOps: true").

Should also work in GNU screen and Windows Terminal. On terminals that don't
support OSC 52 (like Gnome Terminal or Konsole), it seems to do nothing.

Since there does not seem to be a way to feature-probe OSC 52, let's just
always do both (pbcopy and friends as well as OSC 52).  In future, we should
probably stop calling pbpaste and clip.exe, at least on remote systems.

I think there is also an escape sequence to request pasting the system
clipboard but that's less important and less popular, possibly due to
security concerns.
2022-10-24 22:45:45 +02:00
Mahmoud Al-Qudsi
db0a297b8a Add new line between each trap -p output function
This makes the output a little easier on the eyes.
Tests appear to not need any changes to pass. I always forget whether or not
littlecheck cares about whitespace.
2022-10-24 15:36:02 -05:00
Mahmoud Al-Qudsi
07fc04465f Add regression test for trap -p 2022-10-24 15:36:02 -05:00
Mahmoud Al-Qudsi
c8f92878c3 Fix trap -p
Two different bugs completely broke `trap -p`. First bug broke filtering of
functions with trap handlers (`functions -na` prints functions separated by a
comma, not a new line). Second bug broke showing of function definitions for
traps because a refactor renamed only some call sites but references to `$i`
renamed.

These issues were introduced in a6820cbe and appear to have been caught just in
time: no released version is affected (changes made post-3.5.1).
2022-10-24 15:35:59 -05:00
Fabian Boehm
040591bc47 Actually explain what that macOS error is about
"Intermittent error has been fixed" tells me nothing.
2022-10-24 21:55:29 +02:00
Fabian Boehm
8d5198b9b4 fish_git_prompt: Fish show_upstream
This isn't a boolean option

Fixes #9301
2022-10-24 19:13:08 +02:00
ridiculousfish
74fd66fcbe Use -- before seq for negative numbers
busybox seq was complaining about the command:

    seq -550 -1

because it was trying to interpret -550 as a flag. Use -- to prevent
this.
2022-10-23 13:53:36 -07:00
ridiculousfish
c3274c3579 Fix up Dockerfiles
The Dockerfiles had bitrotted some.

Get them passing again, add libpcre2-dev where we can so we aren't
hitting more servers than necessary, and reformat the bionic files so
they can share more of the same image.
2022-10-23 13:53:36 -07:00
ridiculousfish
180acbbb27 Correct exit status of fish_run_tests.sh
fish_run_tests.sh was failing because its final command was a variable
test which was usually false. Switch to an if statement so the result is
true.
2022-10-23 13:53:36 -07:00
ridiculousfish
54a60d8dab Enable DOCKER_BUILDKIT
It just seems better without any major disadvantages.
2022-10-23 13:53:36 -07:00
Mahmoud Al-Qudsi
e3c67ae229 Reword debugging section in language docs
- Clean up the wording a little.
- Highlight the limitations of the "debugger" more clearly and don't mislead
  people into thinking it's possible to really interactively set/remove
  breakpoints except in select circumstances.

Sidenote: I can't believe we're using a markup language that doesn't support
nested inline markup. What a crying shame, rST!
2022-10-23 12:51:12 -05:00
Fabian Boehm
23b5390a0c docs: Mention --profile 2022-10-23 13:43:02 +02:00
Fabian Boehm
4001e2fe19 docs: Mention function name restriction in "identifiers" 2022-10-23 13:39:44 +02:00
Fabian Boehm
358214938a docs: Don't double-document --argument-names 2022-10-23 13:36:56 +02:00
Fabian Boehm
5ba9c7c2ca docs: Move event documentation to the events section
This was a bit awkward in the function page.
2022-10-23 11:39:13 +02:00