Let's hope this doesn't causes build failures for e.g. musl: I just
know it's good on macOS and our Linux CI.
It's been a long time.
One fix this brings, is I discovered we #include assert.h or cassert
in a lot of places. If those ever happen to be in a file that doesn't
include common.h, or we are before common.h gets included, we're
unawaringly working with the system 'assert' macro again, which
may get disabled for debug builds or at least has different
behavior on crash. We undef 'assert' and redefine it in common.h.
Those were all eliminated, except in one catch-22 spot for
maybe.h: it can't include common.h. A fix might be to
make a fish_assert.h that *usually* common.h exports.
This is a *tiny* commit code-wise, but the explanation is a bit
longer.
When I made string read in chunks, I picked a chunk size from bash's
read, under the assumption that they had picked a good one.
It turns out, on the (linux) systems I've tested, that's simply not
true.
My tests show that a bigger chunk size of up to 4096 is better *across
the board*:
- It's better with very large inputs
- It's equal-to-slightly-better with small inputs
- It's equal-to-slightly-better even if we quit early
My test setup:
0. Create various fish builds with various sizes for
STRING_CHUNK_SIZE, name them "fish-$CHUNKSIZE".
1. Download the npm package names from
https://github.com/nice-registry/all-the-package-names/blob/master/names.json (I
used commit 87451ea77562a0b1b32550124e3ab4a657bf166c, so it's 46.8MB)
2. Extract the names so we get a line-based version:
```fish
jq '.[]' names.json | string trim -c '"' >/tmp/all
```
3. Create various sizes of random extracts:
```fish
for f in 10000 1000 500 50
shuf /tmp/all | head -n $f > /tmp/$f
end
```
(the idea here is to defeat any form of pattern in the input).
4. Run benchmarks:
hyperfine -w 3 ./fish-{128,512,1024,2048,4096}"
-c 'for i in (seq 1000)
string match -re foot < $f
end; true'"
(reduce the seq size for the larger files so you don't have to wait
for hours - the idea here is to have some time running string and not
just fish startup time)
This shows results pretty much like
```
Summary
'./fish-2048 -c 'for i in (seq 1000)
string match -re foot < /tmp/500
end; true'' ran
1.01 ± 0.02 times faster than './fish-4096 -c 'for i in (seq 1000)
string match -re foot < /tmp/500
end; true''
1.02 ± 0.03 times faster than './fish-1024 -c 'for i in (seq 1000)
string match -re foot < /tmp/500
end; true''
1.08 ± 0.03 times faster than './fish-512 -c 'for i in (seq 1000)
string match -re foot < /tmp/500
end; true''
1.47 ± 0.07 times faster than './fish-128 -c 'for i in (seq 1000)
string match -re foot < /tmp/500
end; true''
```
So we see that up to 1024 there's a difference, and after that the
returns are marginal. So we stick with 1024 because of the memory
trade-off.
----
Fun extra:
Comparisons with `grep` (GNU grep 3.7) are *weird*. Because you both
get
```
'./fish-4096 -c 'for i in (seq 100); string match -re foot < /tmp/500; end; true'' ran
11.65 ± 0.23 times faster than 'fish -c 'for i in (seq 100); command grep foot /tmp/500; end''
```
and
```
'fish -c 'for i in (seq 2); command grep foot /tmp/all; end'' ran
66.34 ± 3.00 times faster than './fish-4096 -c 'for i in (seq 2);
string match -re foot < /tmp/all; end; true''
100.05 ± 4.31 times faster than './fish-128 -c 'for i in (seq 2);
string match -re foot < /tmp/all; end; true''
```
Basically, if you *can* give grep a lot of work at once (~40MB in this
case), it'll churn through it like butter. But if you have to call it
a lot, string beats it by virtue of cheating.
Rephrase this to more explicitly indicate that the uvar actually
was successfully set. I believe the prior phrasing can leave some
ambiguity as far as wether set just failed with an error, whether it
has done anything or not.
Now uses the same macro other builtins use for a missing -e arg,
and the error message show the short or long option as it was used.
e.g. before
$ set -e
set: Erase needs a variable name
after
$ set --erase
set: --erase: option requires an argument
$ set -e
set: -e: option requires an argument
Intern'd strings were intended to be "shared" to reduce memory usage but
this optimization doesn't carry its weight. Remove it. No functional
change expected.
We store filenames in function definitions to indicate where the
function comes from. Previously these were intern'd strings. Switch them
to a shared_ptr<wcstring>, intending to remove intern'd strings.
The history pager will show multiline commands in single-line cells.
We escape newline characters as \\n but that looks awkward if the next line
starts with a letter. Let's render control characters using their corresponding
symbol from the Control Pictures Unicode block.
This means there is also no need to escape backslashes, which further improves
the history pager - now the rendering has exactly as many backslashes as
the eventual command.
This means that (multiline) commands in the history pager will be rendered
with the same amount of characters as are in the actual command (unless
they contain funny nonprintables). This makes it easy for the next commit
to highlight multiline commands correctly in the history pager.
The font size for these symbols (for example ␉) is quite small, but that's
okay since for the proposed uses it's not so important that they readable.
The important thing is that the stand out from surrounding text.
This checked specifically for "| and" and "a=b" and then just gave the
error without a caret at all.
E.g. for a /tmp/broken.fish that contains
```fish
echo foo
echo foo | and cat
```
This would print:
```
/tmp/broken.fish (line 3): The 'and' command can not be used in a pipeline
warning: Error while reading file /tmp/broken.fish
```
without any indication other than the line number as to the location
of the error.
Now we do
```
/tmp/broken.fish (line 3): The 'and' command can not be used in a pipeline
echo foo | and cat
^~^
warning: Error while reading file /tmp/broken.fish
```
Another nice one:
```
fish --no-config -c 'echo notprinted; echo foo; a=b'
```
failed to give the error message!
(Note: Is it really a "warning" if we failed to read the one file we
wer told to?)
We should check if we should either centralize these error messages
completely, or always pass them and remove this "code" system, because
it's only used in some cases.
This skipped printing a "^" line if the start or length of the error
was longer than the source.
That seems like the correc thing at first glance, however it means
that the caret line isn't skipped *if the file goes on*.
So, for example
```fish
echo "$abc["
```
by itself, in a file or via `fish -c`, would not print an error, but
```fish
echo "$abc["
true
```
would. That's not a great way to print errors.
So instead we just.. imagine the start was at most at the end.
The underlying issue why `echo "$abc["` causes this is that `wcstol`
didn't move the end pointer for the index value (because there is no
number there). I'd fix this, but apparently some of
our recursive variable calls absolutely rely on this position value.
This makes the awkward case
fish: Unexpected end of string, square brackets do not match
echo f[oo # not valid, no matching ]
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
(that `]` is simply the last character on the line, it's firmly in a comment)
less awkward by only marking the starting brace.
The implementation here is awkward mostly because the tok_t
communicates two things: The error location and how to carry on.
So we need to store the error length separately, and this is the first
time we've done so.
It's possible we can make this simpler.
This makes it so instead of marking the error location with a simple
`^`, we mark it with a caret, then a run of `~`, and then an ending `^`.
This makes it easier to see where exactly an error occured, e.g. which
command substitution was meant.
Note: Because this uses error locations that haven't been exposed like
that, it's likely to shake out weirdnesses and inaccuracies. For that
reason I've not adjusted the tests yet.
This stops us from loading the completions for e.g. `./foo` if there
is no `foo` in path.
This is because the completion scripts will call an unqualified `foo`,
and then error out.
This of course means if the script would work because it never calls
the command, we still don't load it.
Pathed completions via `complete --path` should be unaffected because
they aren't autoloaded anyway.
Workaround for #3117Fixes#9133
This was misguidedly "fixed" in
9e08609f85, which made printf error out
with any "-"-prefixed words as the first argument.
Note: This means currently `printf --help` doesn't print the help.
This also matches `echo`, and we currently don't have anything to make
a literal `--help` execute a builtin help except for keywords. Oh well.
Fixes#9132
* Disclose pager to screen height immediately
This removes that bit where we only show 4 rows at most at first,
instead we disclose between half of terminal height up to the full terminal height (but still at least 4 rows).
This results in less pressing of tab to get the other results, and
better visibility of results.
Unlike moving it to the actual top of the screen, it's not as jarring and doesn't push terminal history off-screen as much.
Fixes#2698
This used to be kept, so e.g. testing it with
fish_read_limit=5 echo (string repeat -n 10 a)
would cause the prompt and such to error as well.
Also there was no good way to get back to the default value
afterwards.
* string repeat: Don't allocate repeated string all at once
This used to allocate one string and fill it with the necessary
repetitions, which could be a very very large string.
Now, it instead uses one buffer and fills it to a chunk size,
and then writes that.
This fixes:
1. We no longer crash with too large max/count values. Before they
caused a bad_alloc because we tried to fill all RAM.
2. We no longer fill all RAM if given a big-but-not-too-big value. You
could've caused fish to eat *most* of your RAM here.
3. It can start writing almost immediately, instead of waiting
potentially minutes to start.
Performance is about the same to slightly faster overall.
This newline apparently dates back to when we required all statements to
be terminated; but our AST no longer requires that so we can remove
this. No functional change expected here.
The previous fix was reverted because it broke another scenario. Add tests
for both scenarios.
The first test exposes another problem: autosuggestions are sometimes not
recomputed after selecting the first completion with Tab Tab. Fix that too.
This makes it easy to see where the individual commands start. Perhaps we
can get rid of this once we have syntax highlighting for the commands in
the history pager, or if we add timestamps as descriptions.
Note that every change to the search field still starts a new search, from
the end of history. We could change this in future but it's unclear to me
what the expected behavior is. I don't find the traditional readline behavior
very intuitive.
This reimplements ridiculousfish/control_r which is a more future-proof
approach than #6686.
Pressing Control+R shows history in our pager and allows to search filter
commands with the pager search field.
On the surface, this works just like in other shells; though there are
some differences.
- Our pager shows multiple results at a time.
- Other shells allow to use up arrow/down arrow to select adjacent entries
in history. Shouldn't be hard to implement but the hidden state might
confuse users and it doesn't play well with up-or-search, so this is
left out.
Users might expect the history pager to use subsequence matching (fuzzy
matching) like the completion pager, however due to the history pager design it
uses substring matching. We could change this in future, however that means
we would also want to change the ordering from "reverse-chronological" to
"longest common subsequence" (e.g. what fuzzy finders do), because otherwise
a query "fis" might give this ordering:
fsck /dev/disk/by-partlabel/Linux\x20filesystem
fish
which is probably not what the user wants.
The pager shows only a small number of history items at a time. This is
because, as explained above, the history pager does not support subsequence
matching, so navigating it does not scale well.
Closes#602
The next patch wants to add state that should be reset when we clear the
pager, which will happen in this function.
This reverts commit b25b291d38.
No functional change.
The pager's rendering_needs_update() function detects some but not all
scenarios where a rendering is stale. In particular, it does not compare
the completion strings.
To make this work, we manually invalidate the pager rendering whenever we
update completion strings. The history pager needs the same functionality,
so let's move it into the pager.
No functional change.
This addresses code review feedback to not couple the purely visual
concept of cursor style with the logical concept of the selection size.
Instead this now uses a dedicated variable
`$fish_select_char_after_cursor` to determine whether to extend the
selection beyond the cursor:
* fish_select_char_after_cursor = 1 or unset -> extend selection
* all other cases -> place the selection end that the cursor
This fixes the handling of the right end of the selection. Currently the
right end is considered to be at the cursor position + 1. When using a
`block` or `underline` cursor this is arguably correct, because the
cursor has a width of 1 and spans from the current position to the next:
```
x x [x x x̲] x
```
This is incorrect though (or at least very unintuitive), when using a
`line` cursor:
```
x x [x x|x] x
```
This commit changes the strategy for determining the end of the
selection in the following way:
* If the current cursor as determined by `$fish_cursor_<bind_mode>` is
set to `line`, then a cursor width of `0` is assumed.
* In all other cases, including `block` and `underscore` as well as when
no value is set we retain the previous behavior of assuming a cursor
width of `1`.
```
x x [x x x̲] x
x x [x x|]x x
```
This change should not affect many users, because the selection is
probably used most by vi-mode users, who are also likely to use a
block cursor.
We use "c > 0" but we actually mean "c != 0". The former looks like the
other code path handles negative c. Yet if c is negative, our code would
print a single escaped byte (\xXY) which is wrong because a negative value
has "sizeof wchar_t" bytes which is at least 2.
I think on platforms with 16-bit wchar_t it's possible that we actually
get a negative value but I haven't checked.
Since the fix for #3892, this escaping style escapes
\n to \\n
as well as
\\ to \\\\
\' to \\'
I believe these two are the only printable characters that are escaped with
ESCAPE_NO_PRINTABLES.
The rationale is probably to keep the encoding unambiguous and reversible.
However that doesn't justify escaping the single quote. Probably this was
an accident, so let's revert that part.
This has the nice effect that single quotes will no longer be escaped
when rendered in the completion pager (which is consistent with other
special characters). Try it:
complete : -a "aaa\'\; aaaa\'\;" -f
Also this makes the error output of builtin bind consistent:
$ bind -e --preset \;
$ bind -e --preset \'
$ bind \;
bind: No binding found for sequence “;”
$ bind \'
bind: No binding found for sequence “'”
the last line is clearly better than the old version:
bind: No binding found for sequence “\'”
In general, the fact that ESCAPE_NO_PRINTABLES escapes the (printable)
backslash is weird but I guess it's fine because it looks more consistent to
users, even though the result is an undocumented subset of the fish language.
ESCAPE_ALL is not really a helpful name. Also it's the most common flag.
Let's make it the default so we can remove this unhelpful name.
While at it, let's add a default value for the flags argument, which helps
most callers.
The absence of ESCAPE_ALL makes it only escape nonprintable characters
(with some exceptions). We use this for displaying strings in the completion
pager as well as for the human-readable output of "set", "set -S", "bind"
and "functions".
No functional change.
When listing variables, "set" tries to escape variable names.
Since variable names cannot have special characters, this doesn't do anything.
The escaping is one of the few places that does not use ESCAPE_ALL. This has
complex behavior; let's alleviate the problem by getting rid of this call.
No functional change.
Or should we stop using it?
I'm fine with either always or never using auto-formatting but our current
way of using it only sometimes is confusing.
No functional change.
Almost all edits to our commandline are funneled through
reader_data_t::push_edit(). Notable exceptions are undo/redo (which move
across existing edits instead). Due to an oversight, undo/redo fail to
trigger commandline update hooks. Fix that.
Our behavior of triggering hooks only for the search field looks weird. I
reckon that the command line eventually catches up, but this means we trigger
some hooks redundantly. Once we figure that out we can remove the new function.
command_line_has_transient_edit tracks the actual command line, not the
pager search field. We accidentally reset it after modifying the search field
which causes unexpected behavior - the commandline added by the completion
pager remains even after I press Escape.
If the completion pager renders as
foo1 bar1 baz1 qux1
foo2 bar2 baz2
foo3 bar3 baz3
and we go backwards from "foo1" (using left arrow), we'll end up at "baz3",
not "qux1". Pretty smart!
If however we go backwards once more, nothing happens.
The root cause is that there are two different kinds of selection indices:
the one before rendering (9/qux1) and the one after we cleverly subtract
the half-filled last column (8/baz3). The backwards movement ends up
decrementing the first, so it moves from 9 to 8 and nothing changes in
the rendering.
Fix this by using the selection index that we actually rendered.
There is another caller that relies on the old behavior of using the unrendered
selection index. Make it use a dedicated overload that does not depend on
the rendering.
Otherwise realpath would add the cwd, which would be broken if fish
ever cd'd.
We could add the original cwd, but even that isn't enough, because we
need *the parent's* idea of cwd and $PATH.
Or, alternatively, what we need is for the OS to give us the actual
path to ourselves.
get_executable_path says: "This needs to be realpath'd"
So how about we do that? The only other place we use it is fish.cpp,
and we realpath it there already.
See #9085
Our pager computes the selected completion based on its rendering. The number
of rows affect the selection, in particular when moving left from the top
left cell. This computation breaks if the number of rows is zero, which
happens in at least
two scenarios:
1. If the completion pager was not shown (as is the case for complete-or-search)
2. If the search field had filtered away every candidate but not anymore.
I believe in these scenarios the selected completion index is always 0,
so let's fix the selection for that case.
Probably too minor for a changelog entry.
Closes#9080
Posix allows this as an alternative with the same semantics for read.
Found in conjunction with #9067.
Should be no functional difference on other systems.
The wait status value, which we also use internally, is read by a
bunch of macros.
Unfortunately because we want to *create* such a value, and some
systems lack the "W_EXITCODE" macro to do that, we need to figure out
how it's encoded.
So we simply check a specific value, and assume the encoding from
that.
On Haiku the return status is in the lower byte, on other systems it's
typically the upper byte.
TODO: Test on musl (that's the other system without W_EXITCODE).
Fixes#9067
This was an inadvertent change from
cc632d6ae9.
Because we used wgetcwd directly before, we always got the "physical"
resolved $PWD.
There's an argument to be made to use the logical $PWD here as well
but I prefer not to make changes lik that in a random commit without
good reason.
This can be used to print the modification time, like `stat` with some
options.
The reason is that `stat` has caused us a number of portability
headaches:
1. It's not available everywhere by default
2. The versions are quite different
For instance, with GNU stat it's `stat -c '%Y'`, with macOS it's `stat
-f %m`.
So now checking a cache file can be done just with builtins.
These are non-POSIX extensions other test(1) utilities implement,
which compares the modification time of two files as proposed for
fish in #3589: testing if one file is newer than another file.
-ef is a common extension to test(1) which checks if two paths refer
to the same file, by comparing the dev and inode numbers.
As explained by the comment, this was dead code. If it were ever executed,
it would cause very weird behavior because it would make some completions
randomly affect others.
Let's just print a warning (maybe this is better than crashing?).
Previously, the search text is used to find out which part of the
updated command line should be highlighted during a history search. This
approach will cause the incorrect part to be highlighted when the line
contains multiple instances of the search text.
To address this, we have to find out exactly where to highlight, i.e.
the offset of the current token in the command line (0 if not a token
search) plus the offset of the search text in the match.
This function is supposed to return "the next directory". Because this
is imperfect, it only tries to.
Except it went to all the trouble of figuring out the type and then
just... returned it anyway.
This has nice speedups in globs with directory components like `*/` or
`**`. I have observed 1.1x to 2.0x.
We could also return when we know it's definitely a directory and then
skip a stat() later, but preliminary testing seemed to show that's not
worth much.
Take advantage of additional cleanup unlocked by this refactoring,
including eliminating unneeded error returns and simplifying some
control flow.
No user-visible behavior change expected here.
This switches builtin_string from using PCRE2 directly, to using the new re
component. This simplifies some code and removes redundancy.
No user-visible behavior change expected here.
This migrates our PCRE2 dependency from builtin/string.cpp to new files
re.h/re.cpp, allowing regexes to be used in other places in fish.
No user-visible behavior change expected here.
This switches the flag_to_function from a map to just an ordinary switch
statement. This saves some memory/startup time and removes some
relocations. No functional change here.
This adds a line to `set --show`s output like
```
$PATH: originally inherited as |/home/alfa/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/var/lib/flatpak/exports/bin|
```
to help with debugging.
Note that this means keeping an additional copy of the original
environment around. At most this would be one ARG_MAX's worth, which
is about 2M.
It's still useful without, for instance to implement a command that
takes no options, or to check min-args or max-args.
(technically no optspecs, no min/max args and --ignore-unknown does
nothing, but that's a very specific error that we don't need to forbid)
Fixes#9006
Resolves this warning:
> warning: 'sprintf' is deprecated: This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
If you run an initial command via `fish -c`, and that command is
cancelled e.g. via control-C, then ensure that the cancellation signal
is cleared before running config files.
Fixes#9024
This concerns what happens if the user types e.g. `grep --i` and grep or
its completions have not yet been loaded. Previously we would "bounce to
the main thread" from within the autosuggestion thread to load grep's
completions. However under concurrent execution, this may deadlock as the
main thread is waiting for something else.
In the new implementation, complete simply records the commands that it
would autoload, and returns them back to the caller, where the caller can
decide how to handle them.
In general iothread_perform_on_main risks deadlock under concurrent
execution and we should try to get rid of it.
There should be no user-visible change from this fix.
The last remnant of the old debug system, this was only used in
show_stackframe.
Because that's only ever called with an "E" level currently I've
removed the level argument entirely. If it's needed we'd have to pass
a flog category here.
The fix for #3481 caused us to save the screen status after external
commands were run, fixing an unnecessary abandon-line when switching
modes. But we may also run commands not directly as part of a binding,
but instead via an on-variable event, e.g. for fish_bind_mode.
Extend this fix to all bindings, guarded by changes to exec_count. Now
any time an external command runs as part of a binding we should pick up
changes to the tty and not abandon the line.
Fixes#3481 again.
In b0084c3fc4, we refactored out event handlers get removed. But this
also caused us to remove "one-shot" handlers even if they have not yet
been fired. Fix this.
This concerns running a key binding which invokes a command. If that
command modifies the tty, then fish will spot the modification later and
then react to it by redrawing the prompt. However tty modifications may
be benign or desirable; for example switching the cursor from a line to
a block. Fix this by re-fstating the tty after running external
commands.
Fixes#3481
Previously, `kill-whole-line` kills the line and its following
newline. This is insufficient when we are on the last line, because
it would not actually clear the line. The cursor would stay on the
line, which is not the correct behavior for bindings like `dd`.
Also, `cc` in vi-mode used `kill-whole-line`, which is not correct
because it should not remove any newlines. We have to introduce
another special input function (`kill-inner-line`) to fix this.
When the user adds a completion for a command, we push it to the front
of the completion list so it appears first; for that reason we don't
want to use a vector. However we can do better than std::list; try using
std::forward_list which is singly linked. No functional change here (but
we will see if this breaks any old platforms in which case it's fine to
revert this).
Prior to this change, the list of completions was stored as a
std::unordered_set, using some funny comparators and suspicious
const_cast to make it map-like. Use a real map instead, simplifying
the code. No functional change here.
Prior to this commit, setting a universal variable may trigger syncing
against the file which will modify other universal variables. But if we
want to support multiple environments we need the parser to decide when to
sync uvars. Shift the decision of when to sync to the parser itself. When a
universal variable is modified, now we just set a flag and it's up to the
(main) parser when to pick it up. This is hopefully just a refactoring with
no user-visible changes.
This makes it so `complete -c foo -n test1 -n test2` registers *both*
conditions, and when it comes time to check the candidate, tries both,
in that order. If any fails it stops, if all succeed the completion is offered.
The reason for this is that it helps with caching - we have a
condition cache, but conditions like
```fish
test (count (commandline -opc)) -ge 2; and contains -- (commandline -opc)[2] length
test (count (commandline -opc)) -ge 2; and contains -- (commandline -opc)[2] sub
```
defeats it pretty easily, because the cache only looks at the entire
script as a string - it can't tell that the first `test` is the same
in both.
So this means we separate it into
```fish
complete -f -c string -n "test (count (commandline -opc)) -ge 2; and contains -- (commandline -opc)[2] length" -s V -l visible -d "Use the visible width, excluding escape sequences"
+complete -f -c string -n "test (count (commandline -opc)) -ge 2" -n "contains -- (commandline -opc)[2] length" -s V -l visible -d "Use the visible width, excluding escape sequences"
```
which allows the `test` to be cached.
In tests, this improves performance for the string completions by 30%
by reducing all the redundant `test` calls.
The `git` completions can also greatly benefit from this.
This adds a path builtin to deal with paths.
It offers the following subcommands:
filter to go through a list of paths and only print the ones that pass some filter - exist, are a directory, have read permission, ...
is as a shortcut for filter -q to only return true if one of the paths passed the filter
basename, dirname and extension to print certain parts of the path
change-extension to change the extension to a different one (as a string operation)
normalize and resolve to canonicalize the paths in various flavors
sort to sort paths, also only using the basename or dirname as a key
The definition of "extension" here was carefully considered and should line up with how extensions are actually used - ~/.bashrc doesn't have an extension, but ~/.conf.d does (".d").
These subcommands all compose well - they can read from arguments or stdin (like string), they can use null-delimited input or output (input is autodetected - if a NULL happens in the first PATH_MAX bytes it switches automatically).
It is both a failglob exception (so like set if a glob passed to it fails it just doesn't get any arguments for it instead of triggering an error), and passes output to command substitution buffers explicitly split (like string split0) so newlines are easy to handle.
This would still remove non-existent paths, which isn't a strict
inversion and contradicts the docs.
Currently, to only allow paths that exist but don't pass a type check,
you'd have to filter twice:
path filter -Z foo bar | path filter -vfz
If a shortcut for this becomes necessary we can add it later.
This is now added to the two commands that definitely deal with
relative paths.
It doesn't work for e.g. `path basename`, because after removing the
dirname prepending a "./" doesn't refer to the same file, and the
basename is also expected to not contain any slashes.
Because we now count the extension including the ".", we print an
empty entry.
This makes e.g.
```fish
set -l base (path change-extension '' $somefile)
set -l ext (path extension $somefile)
echo $base$ext
```
reconstruct the filename, and makes it easier to deal with files with
no extension.
This means "../" components are cancelled out even after non-existent
paths or files.
(the alternative is to error out, but being able to say `path resolve
/path/to/file/../../` over `path resolve (path dirname
/path/to/file)/../../` seems worth it?)
This sorts paths by basename, dirname or full path - in future
possibly size or age.
It takes --invert to invert the sort and "--what=basename|dirname|..."
to specify what to sort
This can be used to implement better conf.d sorting, with something
like
```fish
set -l sourcelist
for file in (path sort --what=basename $__fish_config_dir/conf.d/*.fish $__fish_sysconf_dir/conf.d/*.fish $vendor_confdirs/*.fish)
```
which will iterate over the files by their basename. Then we keep a
list of their basenames to skip over anything that was already
sourced, like before.
The recent change to skip the newline for `string` changed this, and
it also hit builtin path (which is in development separately, so it's
not like it broke master).
Let's pick a good default here.
This just goes back until it finds an existent path, resolves that,
and adds the normalized rest on top.
So if you try
/bin/foo/bar////../baz
and /bin exists as a symlink to /usr/bin, it would resolve that, and
normalize the rest, giving
/usr/bin/foo/baz
(note: We might want to add this to realpath as well?)
This includes the "." in what `path extension` prints.
This allows distinguishing between an empty extension (just `.`) and a
non-existent extension (no `.` at all).
These are short flags for "--perm=read" and "--type=link" and such.
Not every type or permission has a shorthand - we don't want "-s" for
"suid". So just the big three each get one.
This is needed because you might feasibly give e.g. `path filter`
globs to further match, and they might already present no results.
It's also well-handled since path simply does nothing if given no paths.
These were officially called "--null-input", but I just used
"--null-in" everywhere, which worked because getopt allows unambiguous abbreviations.
But since *I* couldn't keep it straight and the "put" is just
superfluous, let's remove it.
This is theoretically sound, because a path can only be PATH_MAX - 1
bytes long, so at least the PATH_MAXest byte needs to be a NULL.
The one case this could break is when something has a NULL-output mode
but doesn't bother printing the NULL for only one path, and that path
contains a newline. So we leave --null-in there, to force it on.
This adds a "path" builtin that can handle paths.
Implemented so far:
- "path filter PATHS", filters paths according to existence and optionally type and permissions
- "path base" and "path dir", run basename and dirname, respectively
- "path extension PATHS", prints the extension, if any
- "path strip-extension", prints the path without the extension
- "path normalize PATHS", normalizes paths - removing "/./" components
- and such.
- "path real", does realpath - i.e. normalizing *and* link resolution.
Some of these - base, dir, {strip-,}extension and normalize operate on the paths only as strings, so they handle nonexistent paths. filter and real ignore any nonexistent paths.
All output is split explicitly, so paths with newlines in them are
handled correctly. Alternatively, all subcommands have a "--null-input"/"-z" and "--null-output"/"-Z" option to handle null-terminated input and create null-terminated output. So
find . -print0 | path base -z
prints the basename of all files in the current directory,
recursively.
With "-Z" it also prints it null-separated.
(if stdout is going to a command substitution, we probably want to
skip this)
All subcommands also have a "-q"/"--quiet" flag that tells them to skip output. They return true "when something happened". For match/filter that's when a file passed, for "base"/"dir"/"extension"/"strip-extension" that's when something about the path *changed*.
Filtering
---------
`filter` supports all the file*types* `test` has - "dir", "file", "link", "block"..., as well as the permissions - "read", "write", "exec" and things like "suid".
It is missing the tty check and the check for the file being non-empty. The former is best done via `isatty`, the latter I don't think I've ever seen used.
There currently is no way to only get "real" files, i.e. ignore links pointing to files.
Examples
--------
> path real /bin///sh
/usr/bin/bash
> path extension foo.mp4
mp4
> path extension ~/.config
(nothing, because ".config" isn't an extension.)
This teaches `--on-signal SIGINT` (and by extension `trap cmd SIGINT`)
to work properly in scripts, not just interactively. Note any such
function will suppress the default behavior of exiting. Do this for
SIGTERM as well.
s_observed_signals is used to inform the signal handler which signals may
have --on-signal functions attached to them, as an optimization. Prior to
this change it was latched: once we started observing a signal we assume we
will keep observing that signal. Make it properly increment and decrement,
in preparation for making trap work non-interactively.
Like `set` and `read` before it, `eval` can be used to set variables,
and so it can't be shadowed by a function without loss of
functionality.
So this forbids it.
Incidentally, this means we will no longer try to autoload an
`eval.fish` file that's left over from an old version, which would
have helped with #8963.
This concerns what happens if one event handler removes another, when
both are responding to the same event. Previously we had a "double lock"
where we would traverse the list twice. Now track directly in the
handler when it is removed; this simplifies the code a lot. No
functional changes expected here.
Hitting tab on "echo **" will often result in more than 256 matches.
Commit 143757e8c (Expand wildcards on tab, 2021-11-27) describes this scenario
> If the expansion would produce more than 256 items, we flash the command
> line and do nothing, since it would make the commandline overfull.
Yet we actually erase the "**" token, which seems wrong since we already
flash the command line. Fix this, at the cost of making the code a bit uglier.
I tried to write a test in tests/pexpects/wildcard_tab.py but that doesn't
seem to work because pexpect provides only a "dumb" terminal. I wonder if we
can test what we write to the screen without depending on a terminal emulator.
c4fb857dac (in 3.4.1) introduced a regression where process_exit
events would only fire once the job itself is complete. Allow
process_exit events to fire before that. Fixes#8914.
This is after we've tried to find the interpreter, so we would already
have complained about e.g. /usr/bin/pthyon not existing.
Realistically the most common case here is things that don't start
with a shebang like ELFs. Writing special extraction code here is
overkill, and I can't see a good function to do it for us.
But this should point you in the right direction.
Fixes#8938
This gets the passwd entry for $USER (if it is set). If that gives the
same uid that geteuid() gives us, we assume the data is correct.
If not, we reset $USER (and $HOME if it's empty) from the passwd value for our UID.
This allows using $USER in a prompt even if you've `su`d. Bash gets around this by having a special escape in its $PS1 DSL that checks passwd instead.
Fixes#8583
This reverts commit ccb6cb1abe.
CI fails with
/home/runner/work/fish-shell/fish-shell/src/autoload.cpp:148:1: error: function ‘autoload_t::autoload_t(autoload_t&&)’ defaulted on its redeclaration with an exception-specification that differs from the implicit exception-specification ‘’
148 | autoload_t::autoload_t(autoload_t &&) noexcept = default;
| ^~~~~~~~~~
make[2]: *** [CMakeFiles/fishlib.dir/build.make:96: CMakeFiles/fishlib.dir/src/autoload.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:369: CMakeFiles/fishlib.dir/all] Error 2
make: *** [Makefile:139: all] Error 2
Not sure what's wrong - it compiles fine on my machine. Will check later.
Even though we disable exceptions, we use noexcept in some
places to enable certain optimizations in std::vector, see
https://en.cppreference.com/w/cpp/utility/move_if_noexcept.
Some methods have noexcept only at their declaration (or only at the
definition). This will be an error when compiling with "g++ -std=c++17". Make
both signatures match.
This cleans up the path_get_path function which is used to resolve a
command name against $PATH, by removing the dependence on errno and
being explicit about which error is returned.
Should be no user-visible change here.
Curses variables like `enter_italics_mode` are secretly defined to
dereference through the `cur_term` variable. Be sure we do not read or
write these curses variables if cur_term is NULL. See #8873, #8875.
Add a regression test.
Apple's terminfo has missing support for enter_italics_mode,
exit_italics_mode, and enter_dim_mode. Previously we would hack in such
support in set_color; migrate that to init_curses so we do it up-front
instead of opportunistically.
To recap, this means `&` in the middle of a word no longer
backgrounds.
So:
```fish
echo foo&bar # prints foo&bar
echo foo& bar # backgrounds an echo that prints "foo" and runs "bar"
```
This can no longer be changed. If "no-stderr-nocaret" is in
$fish_features it will simply be ignored.
The "^" redirection that was deprecated in fish 3.0 is now gone for good.
Note: For testing reasons, it can still be set _internally_ by running
"feature_flags_t::set". We simply shouldn't do that.
If we get an E2BIG while executing a process, we check how large the
exported variables are. We already did this, but then immediately
added it to the total.
So now we keep the tally just for the variables around, and if it's
over half (which is an atypical value if your system has an ARG_MAX of
2MB), we mention that in the error.
Figuring out which variable is too big (in case it's just one) is probably too complicated,
but we can at least complain if things seem suspect.
Untested because I don't know *how* to do so portably
Prior to this change, if you tab-completed a token with a wildcard (glob), we
would invoke ordinary completions. Instead, expand the wildcard, replacing
the wildcard with the result of expansions. If the wildcard fails to expand,
flash the command line to signal an error and do not modify it.
Example:
> touch file(seq 4)
> echo file*<tab>
becomes:
> echo file1 file2 file3 file4
whereas before the tab would have just added a space.
Some things to note:
1. If the expansion would produce more than 256 items, we flash the command
line and do nothing, since it would make the commandline overfull.
2. The wildcard token can be brought back through Undo (ctrl-Z).
3. This only kicks in if the wildcard is in the "path component
containing the cursor." If the wildcard is in a previous component,
we continue using completions as normal.
Fixes#954.
When fish expands a string that starts with a tilde, like `~/stuff/*`, it
first must resolve the tilde (e.g. to the user's home directory) before
passing it to wildcard expansion. The wildcard expansion will produce full
paths like `/home/user/stuff/file`. fish then "unexpands" the home directory
back to a tilde.
Previously this was only used during completions, but in the next commit
we plan to use it for string expansions as well.
Rationalize this behavior by adding an explicit flag to request it and
explain some subtleties about completions.
This commit was problematic for a few reasons:
1. It silently changed the behavior of argparse, by switching which
characters were replaced with `_` from non-alphanumeric to punctuation.
This is a potentially breaking change and there doesn't appear to be any
justification for it.
2. It combines a one-line if with a multi-line else which we should try
to avoid.
This reverts commit 63bd4eda55.
This reverts commit 4f835a0f0f.
These macros were historically used only in internal error messages which
should never happen! Now we are able to enforce they never happen at
compile time so we can remove them.
No functional change here.
If we ever need any of these... they're in this commit:
fish_wcswidth_visible()
status_cmd_opts_t::feature_name
completion_t::is_naturally_less_than()
parser_t::set_empty_var_and_fire()
parser_t::get_block_desc()
parser_keywords_skip_arguments()
parser_keywords_is_block()
job_t::has_internal_proc()
fish_wcswidth_visible()
When you do
```fish
set foo-bar baz
```
"foo-baz" isn't usable as a variable *name*. When you just say the
"variable" is invalid that could also be interpreted to be a special
type of variable or something.
String tokens are subdivided by command substitutions. Some syntax errors
can occur in the gap between two command substitutions. Make the caret point
to the start of that gap, instead of the token start.
When expanding command substitutions, we use a naïve way of detecting whether
the cmdsub has the optional leading dollar. We check if the last character was
a dollar, which breaks if it's an escaped dollar. We wrongly expand
\$(echo "") to the empty string. Fix this by checking if the dollar was escaped.
The parse_util_* functions have a bunch of output parameters. We should
return a parameter bag instead (I think I tried once and failed).
Given
set var a
echo "$var$(echo b)"
the double-quoted string is expanded right-to-left, so we construct an
intermediate "$varb". Since the variable "varb" is undefined, this wrongly
expands to the empty string (should be "ab"). Fix this by isolating the
expanded command substitution internally. We do the same when handling
unquoted command substitutions.
Fixes#8849
1. Bravely use a real enum for has_arg, despite the warnings.
2. Use some C++11 initializers so we don't have to pass an int for this
parameter.
No functional change expected here.
If the history file is larger than 4GB on a 32 bit system, fish will
refuse to read it. However the check was incorrect because it cast the
file size to size_t, which may be 32 bit. Switch to using uint64.
