Commit graph

617 commits

Author SHA1 Message Date
ridiculousfish
fb92ad946b Rework null terminated arrays
Several functions including wgetopt and execve operate on null-terminated
arrays of nul-terminated pointers: a list of pointers to C strings where
the last pointer is null. Prior to this change, each process_t stored its
argv in such an array. This had two problems:

1. It was awkward to work with this type, instead of using std::vector,
etc.
2. The process's arguments would be rearranged by builtins which is
surprising

Our null terminated arrays were built around a fancy type that would copy
input strings and also generate an array of pointers to them, in one big
allocation.

Switch to a new model where we construct an array of pointers over
existing strings. So you can supply a `vector<string>` and now
`null_terminated_array_t` will just make a list of pointers to them. Now
processes can just store their argv in a familiar wcstring_list_t.
2021-03-28 15:31:25 -07:00
ridiculousfish
e0e4b11dbd Make arguments to builtins const
Prior to this change, builtins would take their arguments as `wchar_t **`.
This implies that the order of the arguments may be changed (which is
true, `wgetopter` does so) but also that the strings themselves may be
changed, which no builtin should do.

Switch them all to take `const wchar_t **` instead: now the arguments may
be rearranged but their contents may no longer be modified.
2021-03-28 15:31:25 -07:00
Fabian Homborg
4e4852c40a history: Improve bash import check
- Check for special characters *before* attempting to parse
- Also ignore lines with `{` and `*`
- Also skip lines with `<<` because that might be a heredoc (or a
- `<<<` herestring)

Fixes #7874.
2021-03-28 20:30:37 +02:00
ridiculousfish
cf35431af9 Reimplement wbasename and wdirname
Previously wbasename and wdirname wrapped the system-provided basename
and dirname. But these have thread-safety issues and some surprising
error conditions on Mac. Just reimplement these per the OpenGroup spec.

In particular these no longer trigger a null-dereference if the input
exceeds PATH_MAX.

Add some tests too.

This fixes #7837
2021-03-21 16:33:04 -07:00
ridiculousfish
6e1b324343 Add some tests for dirname and basename
This is in preparation for replacing our wrappers around the C versions,
with custom versions instead.
2021-03-21 16:32:58 -07:00
Johannes Altmanninger
206543c55b fixup! Pass some parameters by reference/move 2021-03-21 19:46:49 +01:00
Karolina Gontarek
e4eaca1032
Fix wrapping for cd (#7843) 2021-03-21 09:27:19 +01:00
Johannes Altmanninger
865abebd11 Simplify highlight tests by changing into dedicated test dir 2021-03-13 17:51:49 +01:00
Fabian Homborg
29ebd4a5ff tests: Don't break when a file unexpectedly exists
Creating a file called "xfoo" could break the highlight tests because
we'd suddenly get a color with valid_path set to true.

So what we do is simply compare foreground/background and forced
underline, but only check for path validity if we're expecting a valid
path.

If we're not expecting a valid path, we don't fail whether it is there
or not.

This means that we can't check for a non-valid path, but we don't
currently do that anyway and we can just burn that bridge when we get
to it.

cc @siteshwar @krobelus, who both came across this
2021-03-13 17:25:23 +01:00
Mahmoud Al-Qudsi
17926a9197 Allow low-level tests to at least run to completion under Cygwin
They still don't pass due to path differences, but at least they run to
completion so we can tell just how broken everything is.
2021-02-22 16:27:45 -06:00
ridiculousfish
7e77dc8964 Add a test for round-tripping characters in the private use area
I wrote this test believing that fish had a bug, but actually fish is
behaving correctly here. Still the test is nice so I am checking it in.
2021-02-17 12:29:51 -08:00
Mahmoud Al-Qudsi
75af89699a Patch fish_tests to work with changed const_strlen requirements 2021-02-17 12:55:21 -06:00
Johannes Altmanninger
444c05dfb1 Do not indent after escaped newline in comment
We do something similar in fish_indent.  This fixes the spurious indent
after comments in share/completions/emerge.fish.

See #7720
2021-02-16 18:39:03 +01:00
Johannes Altmanninger
5e8a248758 Indent escaped newlines
Similar to what fish_indent does. After typing "echo \" and hitting return,
the cursor will be indented.

A possible annoyance is that when you have multiple indented lines

	echo 1 \
	    2 \
	    3 \
	    4 \

If you remove lines in the middle with Control-k, the lines below
the deleted one will start jumping around, as they are disconnected
from and reconnected to "echo".
2021-02-13 09:01:41 +01:00
Johannes Altmanninger
511747d59e Indent only leaf nodes and in-between gaps
Probably not necessary for the next commit, but this way feels more logical
2021-02-13 09:01:41 +01:00
Johannes Altmanninger
7ee4a3b40d Indent empty lines inside block 2021-02-13 09:01:41 +01:00
Johannes Altmanninger
fffcdf8792 Highlight redirection target as valid if it contains a to-be-defined variable
If a variable is undefined, but it looks like it will be defined by the
current command line, assume the user knows what they are doing.
This should cover most real-world occurrences.

Closes #6654
2021-02-13 08:59:54 +01:00
Johannes Altmanninger
b5df9a7137 Fix a compiler warning about comparison of different signedness 2021-02-13 08:07:20 +01:00
ridiculousfish
f239329f33 Attempt to fix the 32 bit fd_monitor test
Speculatively the fd_monitor thread is not scheduled, or we are awoken
early. Add a loop to ensure it gets run.

This is an attempt at #7699
2021-02-10 12:28:34 -08:00
Johannes Altmanninger
86707378cc Also allow unclosed quotes in some places
See #7693
2021-02-09 22:38:16 +01:00
Johannes Altmanninger
38b95defbd Inside an unclosed subshell, do not report other parse errors
In an interactive shell, typing "for x in (<RET>" would print an error:

	fish: Expected end of the statement, but found a parse_token_type_t::tokenizer_error

Our tokenizer converts "(" into a special error token, hence this message.
Fix two cases by not reporting errors, but only if we allow parsing incomplete
input. I'm not really sure if this is necessary, but it's sufficient.

Fixes #7693
2021-02-09 22:19:42 +01:00
ridiculousfish
e423a58e24 Add a thread yield to topic monitor torture test
This speeds up the test by about 5 msec.
2021-02-08 14:06:47 -08:00
Mahmoud Al-Qudsi
cb3ab80cab Use const_strlen in a few different places
This may slightly improve performance by allowing the compiler greater
visibility into what is happing on top of not executing at runtime in
some hot paths, but more importantly, it gets rid of magic constants in a
few different places.
2021-02-08 15:16:21 -06:00
Johannes Altmanninger
e40850ab89 Format fish_tests.cpp 2021-02-08 07:31:33 +01:00
Fabian Homborg
b3626d48e7 Highlight keywords differently
This introduces a new variable $fish_color_keyword that will be used
to highlight keywords. If it's not defined, we fall back on
$fish_color_command as before.

An issue here is that most of our keywords have this weird duality of
also being builtins *if* executed without an argument or with
`--help`.

This means that e.g.

    if

is highlighted as a command until you start typing

    if t

and then it turns keyword.
2021-02-07 21:18:51 +01:00
ridiculousfish
e004930947 Use fd_event_signaller in iothread completions
This simplifies how iothread notices when there are completions ready to
run.
2021-02-07 10:59:10 -08:00
ridiculousfish
8066428feb Add fd_event_signaller_t
fd_event_signaller_t exists to expose eventfd under Linux. This is a
more lightweight way of signalling events than using a pipe.
2021-02-07 10:59:10 -08:00
ridiculousfish
b5716e97cc Remove fd_set_t
Now that we no longer need to worry about pipes conflicting with
user-specified redirections, we can remove fd_set_t.
2021-02-05 18:14:50 -08:00
ridiculousfish
97f29b1f4d Pipe fds to move to the "high range"
This concerns how fish prevents its own fds from interfering with
user-defined fd redirections, like `echo hi >&5`. fish has historically
done this by tracking all user defined redirections when running a job,
and ensuring that pipes are not assigned the same fds. However this is
annoying to pass around - it means that we have to thread user-defined
redirections into pipe creation.

Take a page from zsh and just ensure that all pipes we create have fds in
the "high range," which here means at least 10. The primary way to do this
is via the F_DUPFD_CLOEXEC syscall, which also sets CLOEXEC, so we aren't
invoking additional syscalls in the common case. This will free us from
having to track which fds are in user-defined redirections.
2021-02-05 17:58:08 -08:00
ridiculousfish
97bde2f2bf Further refactoring of io_buffer_t
Previously we sometimes wanted to access an io_buffer_t to append to it
directly, but that's no longer true; all we really care about is its
separated_buffer_t. Make io_bufferfill_t::finish return the
separated_buffer directly, simplifying call sites. No user visible changes
expected here.
2021-02-04 17:14:46 -08:00
ridiculousfish
258149fe2e Improve locking discipline in io_buffer_t
Previously we had a lock that was taken in an ad-hoc manner. Switch to
using owning_lock.
2021-02-04 17:03:54 -08:00
ridiculousfish
7d494eab5c builtins to write to buffers directly
This concerns builtins writing to an io_buffer_t. io_buffer_t is how fish
captures output, especially in command substitutions:

    set STUFF (string upper stuff)

Recall that io_buffer_t fills itself by reading from an fd (typically
connected to stdout of the command). However if our command is a builtin,
then we can write to the buffer directly.

Prior to this change, when a builtin anticipated writing to an
io_buffer_t, it would first write into an internal buffer, and then after
the builtin was finished, we would copy it to the io_buffer_t. This was
because we didn't have a polymorphic receiver for builtin output: we
always buffered it and then directed it to the io_buffer_t or file
descriptor or stdout or whatever.

Now that we have polymorphpic io_streams_t, we can notice ahead of time
that the builtin output is destined for an internal buffer and have it
just write directly to that buffer. This saves a buffering step, which is
a nice simplification.
2021-02-04 15:21:32 -08:00
ridiculousfish
7207a205f2 Switch history races test to use threads instead of processes
This avoids issues with ASan and TSan whose allocators do not properly
clean up in atfork, leading to deadlocks in child processes.
2021-01-11 12:44:21 -08:00
Fabian Homborg
3fc9c0b38c tests: Increase cancellation delay
This sometimes fails on github actions with ASAN. I am assuming that's
because the ctrl-c happens *before* the process has had a chance to
start.

So we do what we do and increase the delay.
2021-01-11 21:00:33 +01:00
ridiculousfish
e8c9da100c Track histories with shared_ptr
Prior to this change, histories were immortal and allocated with either
unique_ptr or just leaked via new. But this can result in races in the
path detection test, as the destructor races with the pointer-captured
history. Switch to using shared_ptr.
2021-01-09 17:02:11 -08:00
ridiculousfish
87dacc0e95 Improve formatting and layout of history path detection test 2021-01-09 17:02:11 -08:00
ridiculousfish
bee8e8f6f7 Expand more when performing history path detection
When adding a command to history, we first expand its arguments to see
if any arguments are paths which refer to files. If so, we will only
autosuggest that command from history if the files are still valid. For
example, if the user runs `rm ./file.txt` then we will remember that
`./file.txt` referred to a file, and then only autosuggest that if the file
is present again.

Prior to this change we only performed simple expansion relative to the
working directory. This change extends it to variables and tilde
expansion. For example we will now apply the same hinting for
`rm ~/file.txt`

Fixes #7582
2021-01-08 12:58:34 -08:00
ridiculousfish
fd08b660c0 Add a poke function to fd_monitor
In preparation for fixing #7559, add a function poke_item to fd_monitor.

fd_monitor has a list of file descriptors, and invokes a callback when an
fd becomes readable. With this change, we assign each item a unique ID and
return it when the item is added; the ID may then be used to invoke the
callback explicitly.

The idea is that we can stop reading from the pipe associated with the
cmdsub when the job is finished, even if the pipe is still open.
2021-01-07 11:51:04 -08:00
ridiculousfish
9fdc4f903b Explicitly track persistence mode in history_item_t
Commands that start with a space should not be written to the history
file. Prior to this change, that was implemented by simply not adding them
to history. Items with leading spaces were simply dropped.

With this change, we add a 'history_persistence_mode_t' to
history_item_t, which tracks how the item persists. Items with leading
spaces are now marked as "ephemeral": they can be recovered via up arrow,
until the user runs another command, or types a space and hits return.
This matches zsh's HIST_IGNORE_SPACE feature.

Fixes #1383
2021-01-02 21:31:19 -08:00
ridiculousfish
f03ff8cd00 Add a test for history path detection
This will support history path detection improvements in a future
commit.
2020-12-30 00:44:25 -08:00
Johannes Altmanninger
8fc9b9d61b Address some minor lints
A mildly interesting one is the call to test_wchar2utf8 with a non-null
pointer ("u1"/"dst") but 0 length. In this case we relied on malloc(0)
returning non-null which is not guaranteed.

	src/fish_tests.cpp:1619:23: warning: Call to 'malloc' has an allocation
	size of 0 bytes [clang-analyzer-optin.portability.UnixAPI]
	        mem = (char *)malloc(dlen);
	                      ^
	test_wchar2utf8(w1, sizeof(w1) / sizeof(*w1), u1, 0, 0, 0,
			"invalid params, dst is not NULL");
2020-12-29 16:31:43 +01:00
ridiculousfish
e43913a547 Stop expanding globs in command position when performing error checking
Before running a command, or before importing a command from bash history,
we perform error checking. As part of error checking we expand commands
including variables and globs. If the glob is very large, like `/**`, then
we could hang expanding it.

One fix would be to limit the amount of expansion from the glob, but
instead let's just not expand command globs when performing error checking.

Fixes #7407
2020-12-22 12:38:51 -08:00
ridiculousfish
f11a60473a Introduce expansion limits
This adds the ability to limit how many expansions are produced. For
example if $big contains 10 items, and is Cartesian-expanded as
$big$big$big$big... 10 times, we would naviely get 10^10 = 10 billion
results, which fish can't actually handle. Implement this in
completion_receiver_t, which now can return false to indicate an overflow.

The initial expansion limit 'k_default_expansion_limit' is set as 512k
items. There's no way for users to change this at present.
2020-12-05 13:19:07 -08:00
Fabian Homborg
b9b84e63bf Revert "Attempt to simplify how completions get presented in the pager"
The pager cleanup missed that the existing token could already include active (as in unescaped) expansions, and just escaped them all.

This means things like `ls ~/<TAB>` would escape the `~`, which is obviously wrong and makes it awkward to use.

This reverts commit b38a23a46d.

I fully expect that we'll try again, but there's no use in keeping master broken while that happens.

Fixes #7526.
2020-12-04 16:44:48 +01:00
ridiculousfish
2b8d2deb0c Introduces "smartcase" completions
"smartcase" performs case-insensitive matching if the input string is all
lowercase, and case-sensitive matching otherwise. When completing e.g.
files, we will now show both case sensitive and insensitive completions if
the input string does not contain uppercase characters.

This is a delicate fix in an interactive component with low test coverage.
It's likely something will regress here.

Fixes #3978
2020-11-29 12:40:01 -08:00
ridiculousfish
b38a23a46d Attempt to simplify how completions get presented in the pager
This is an attempt to simplfy some completion logic. It mainly refactors
reader_data_t::handle_completions such that all completions have the token
prepended; this attempts to simplify the logic since now all completions
replace the token. It also changes how the pager prefix works. Previously
the pager prefix was an extra string that was prepended to all
completions. In the new model the completions already have the prefix
prepended and the prefix is used only for certain width calculations.

This is a somewhat frightening change in an interactive component with
low test coverage. It tweaks things like how long completions are
ellipsized. Buckle in!
2020-11-29 12:35:18 -08:00
ridiculousfish
4b947e0a23 Refactor string fuzzy matching
In preparation for introducing "smart case", refactor string fuzzy
matching. Specifically split out the case folding and match type into
separate fields, so that we can introduce more case folding types without
a combinatoric explosion.
2020-11-29 12:35:18 -08:00
ridiculousfish
ac1ee6f1fd Make fuzzy_match_type_t an enum class
Also rename it to fuzzy_type_t and shorten some of its values.
2020-11-29 12:35:18 -08:00
Fabian Homborg
2e55e34544 Reformat 2020-11-22 14:39:48 +01:00
ridiculousfish
17fc542082 Revert "Stop caching line breaks in the prompt calculation"
This reverts commit a2ff32d904.

Per comments on the commit, the original code had correct handling of line
breaks inside escapes.
2020-11-12 10:55:11 -08:00
ridiculousfish
c861fdadcf Remove return value from iothread_perform
It was not actually used by any test.
2020-11-05 19:28:26 -08:00
ridiculousfish
a2ff32d904 Stop caching line breaks in the prompt calculation
These are fast enough to find on demand.
2020-11-01 14:45:35 -08:00
Soumya
80aaae5b74 Clear to end of each line in left prompt 2020-11-01 13:29:26 -08:00
Rosen Penev
334be56021 run codebase through clang-tidy
Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-10-25 22:48:49 -07:00
ridiculousfish
e669c41d1b Fix up SIGIO notifier tests
FreeBSD has the behavior where SIGIO is delivered on a read. Teach the
tests how to handle this behavior.
2020-10-25 14:53:31 -07:00
ridiculousfish
558dd6e53d Add sigio-based universal notifier strategy
Introduce a new strategy for notifying other fish processes of universal
variable changes, as a planned replacement for the complex
strategy_named_pipe. The new strategy still uses a named pipe, but instead
of select() on it, it arranges for SIGIO to be delivered when data is
available. If a SIGIO has been seen since the last check, it means the file
needs to be re-read.
2020-10-01 13:19:41 -07:00
Johannes Altmanninger
e8859b4ce2 Do not treat newlines special in bigword movements
Improves on #7328.

I believe this is the correct behavior, simply skip all whitespace before
a word. Try with

	./fish -C 'bind \ef forward-bigword; bind \eb backward-bigword; bind \ed kill-bigword; bind \cw backward-kill-bigword'

Also unrelated formatting fixes. I don't think a CI failure on unformatted
code is warranted but I wish it could do that behind the scenes.
2020-09-27 18:04:09 +02:00
sgrj
ab2cb03189
Consistency-fix for word motions (#7354)
* change word motion test to include start cursor in specification

* add test case for bug

* fix bug
2020-09-27 15:34:34 +02:00
Fabian Homborg
396d7e105f fish_tests: Break if unescape test failed
Otherwise this prints millions of lines of errors, which just seems
like overkill
2020-09-26 10:43:05 +02:00
ridiculousfish
4d5d90d828 Add a simple ASCII conversion test and benchmark
Run `fish_tests perf_convert_ascii` to run the benchmark.
2020-09-19 17:47:12 -07:00
Johannes Altmanninger
7a4fece445 Give reader control of all edits to a command line
So we can do something on every edit, for example repaint the pager (#7318).
This patch fixes pager refiltering and repainting when pressing Control+U
after typing something in the search field.

Implement this by moving the convenience functions from editable_line_t to
the reader, so we have fewer places where we need to refilter.  Essentially we
only have two cases: insertions at the cursor are handled by insert_string(),
and all others go through push_edit().  This should also make it clearer
where we update undo_history.may_coalesce.

This commit was on the history-search-edit-needle branch, so it should
work fine.  I hope it does play well with some recent changes.

In 6d339df61 (Factor repainting decions from readline commands better
in the reader), insert_string() was simplified a lot, mirror that.

The tests for editable_line_t are not that useful anymore since the caller has
to decide whether to coalesce insertions, but I guess they don't hurt either.
We should have more tests for some interactive scenarios like undo and the
pager filtering.
2020-09-08 22:00:48 +02:00
ridiculousfish
57102caba6 Remove the cursor position from highlighting
This used to be used to determine which token contained the cursor, so
as to highlight potential paths. But now we highlight all potential paths,
so we can remove the field.
2020-08-11 17:42:30 -07:00
ridiculousfish
65f7f90433 Correctly highlight =s in var assignments after the first
We were not correctly offsetting the = in the token, it was always from the
start of the string.
2020-08-11 12:27:32 -07:00
ridiculousfish
703e6f571e Highlight the = in variable assignments as an operator 2020-08-10 16:41:56 -07:00
ridiculousfish
c1abb474c2 Remove some dead code and enable a test 2020-08-09 15:05:16 -07:00
ridiculousfish
557fe57deb Close the file descriptor returned by mkstemp 2020-08-06 19:12:15 -07:00
ridiculousfish
206b2d0a26 Simplify topic monitoring
The topic monitor allows a client to wait for multiple events, e.g. sigchld
or an internal process exit. Prior to this change a client had to specify
the list of generations and the list of topics they are interested in.
Simplify this to just the list of generations, with a max-value generation
meaning the topic is not interesting.

Also remove the use of enum_set and enum_array, it was too complex for what
it offered.
2020-08-06 19:01:30 -07:00
Johannes Altmanninger
f7ef91ae2a Use mkstemp over mktemp to silence warning 2020-08-06 21:24:26 +02:00
Johannes Altmanninger
b947e360db Allow newlines after && and ||
We do the same for pipes (#1285). This matches POSIX sh behavior.
2020-08-06 21:24:26 +02:00
Soumya
8dd2d4f15d Change builtins to return maybe_t<int> instead of int 2020-08-05 12:23:49 -07:00
ridiculousfish
7304815736 Make shell highlighting a toggle instead of a function parameter
Remove the ability to specify the "highlight function." The reader
always highlights via shell highlighting, or doesn't.
2020-08-03 14:10:37 -07:00
ridiculousfish
bcfc54fdaa Do not buffer builtin output if avoidable
builtins output to stdout and stderr via io_streams_t. Prior to this fix, it
contained an output_stream_t which just wraps a buffer. So all builtin output
went to this buffer (except for eval).

Switch output_stream_t to become a new abstract class which can output to a
buffer, file descriptor, or nowhere. This allows for example `string` to stream
its output as it is produced, instead of buffering it.
2020-07-30 22:45:44 -07:00
ridiculousfish
f7f35d2373 Fix a warning in fish_tests 2020-07-29 20:12:19 -07:00
ridiculousfish
2809d637db Introduce wwrite_to_fd
wwrite_to_fd() is a function which writes a wide string to a file
descriptor without performing any heap allocations.
2020-07-29 19:38:45 -07:00
ridiculousfish
db086fc5d4 Eliminate wcs2str
Use std::string variants everywhere instead
2020-07-29 16:37:39 -07:00
ridiculousfish
487de1e6c3 Reduce copying in parse_util_detect_errors
Allow parse_util_detect_errors to accept an already-parsed ast. This
eliminates a copy of the source, which is helpful when executing large
scripts.
2020-07-12 16:57:30 -07:00
ridiculousfish
9ee5075fc3 Reformat CPP files 2020-07-12 12:21:25 -07:00
ridiculousfish
2a4c545b21 Rework how signals trigger cancellation
When fish receives a "cancellation inducing" signal (SIGINT in particular)
it has to unwind execution - for example while loops or whatever else that
is executing. There are two ways this may come about:

1. The fish process received the signal
2. A child process received the signal

An example of the second case is:

    some_command | some_function

Here `some_command` is the tty owner and so will receive control-C, but
then fish has to cancel function execution.

Prior to this change, these were handled uniformly: both would just set a
cancellation signal inside the parser. However in the future we will have
multiple parsers and it may not be obvious which one to set the flag in.
So instead distinguish these cases: if a process receives SIGINT we mark
the signal in its job group, and if fish receives it we set a global
variable.
2020-07-12 12:16:01 -07:00
ridiculousfish
12d0afa929 Fix some build warnings in fish_tests 2020-07-12 11:41:06 -07:00
ridiculousfish
35cb449aa1 Make parse_statement_decoration_t a class enum 2020-07-07 16:28:39 -07:00
Mahmoud Al-Qudsi
791c4fb1dd Fix unused return value errors in fish tests
This error only happens on recent versions of gcc, see previous
commit e6bb7fc973 for more info.

Instead of using `ignore_result()` here, I've added a `system_assert`
function/macro that mimics the behavior of all the other `system()`
calls in the file.
2020-07-05 22:18:21 -05:00
ridiculousfish
6976d0ee7e Simplify infinite loop fix when parsing "a="
This reworks the "a=" detection to be simpler.
If we detect a variable assignment that produces an error,
simply consume it.

We also take the opportunity to not highlight it as an error,
and add some tests.

Original commit is 1ca05d32d3.
2020-07-05 12:15:18 -07:00
Mahmoud Al-Qudsi
b8a16a8ba0 Convert highlighted_char_t to a struct 2020-07-04 20:23:50 -05:00
ridiculousfish
0c22f67bde Remove the old parser bits
Now that everything has been migrated to the new AST, remove as much of
the parse_tree bits as possible
2020-07-04 14:58:05 -07:00
ridiculousfish
3534c07584 Adopt the new AST in parse_execution
parse_execution is what turns a parsed tree into jobs, etc. Switch it from
parse_tree to the new AST.
2020-07-04 14:58:05 -07:00
ridiculousfish
6c6088f45c Adopt the new AST in fish_tests
This switches fish_tests from parse_tree to the new AST.
2020-07-04 14:58:05 -07:00
ridiculousfish
202fdfa54a Adopt the new AST in parse_util_detect_errors
This switches parse_util_detect_errors from parsing with parse_tree to
the new ast.
2020-07-04 14:58:05 -07:00
ridiculousfish
7bea5ffa2e Adopt the new AST in parse_util_compute_indents
This switches parse_util_compute_indents from parsing with parse_tree to
the new ast.

It also reworks the parse_util_compute_indents tests, because
parse_util_compute_indents will be the backing for fish_indent.
2020-07-04 14:58:05 -07:00
ridiculousfish
8d9725c301 Adopt the new AST in highlighting
This switches syntax highlighting from parsing with parse_tree to the new
ast.
2020-07-04 14:58:05 -07:00
Johannes Altmanninger
76e0875c8f Apply clang-format 10 and selected lints from "make lint-all" 2020-07-01 00:44:06 +02:00
ridiculousfish
6bdbe732e4 Adopt termsize_t in the pager 2020-06-07 20:00:42 -07:00
ridiculousfish
340c8490f6 Introduce termsize_container_t
fish's handling of terminal sizes is currently rather twisted. The
essential problem is that the terminal size may change at any point from a
SIGWINCH, and common_get_{width,height} may modify it and post variable
change events from arbitrary locations.

Tighten up the semantics. Assign responsibility for managing the tty size
to a new class, `termsize_container_t`. Rationalize locking and reentrancy.

Explicitly nail down the relationship between $COLUMNS/$LINES and the tty
size. The new semantics are: whatever changed most recently takes
precendence.
2020-06-07 20:00:42 -07:00
ridiculousfish
812cc1dbaf Clean up line_t
Use a single allocation instead of two for text and colors.
Comment and tighten up its methods.
2020-06-07 20:00:41 -07:00
Rosen Penev
9636d9f5d3 [clang-tidy] remove pointless string init
Found with readability-redundant-string-init

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-06-07 12:49:35 -07:00
Rosen Penev
67e5473a11 [clang-tidy] remove pointless cstr
Found with readability-convert-member-functions-to-static

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-06-07 12:49:35 -07:00
ridiculousfish
a2ae2d6c36 Attempt to truncate prompts that are too wide
Prior to this change, if the user's prompt was wider than the terminal, we
would reduce it to just `> `. With this change, attempt to truncate the
prompt.

For each line of the prompt, calculate its width. If the width exceeds
COLUMNS, prepend ellipsis to that line, and start removing characters
until it fits. Escape sequences are skipped.

Fixes #904
2020-05-31 09:58:47 -07:00
ridiculousfish
67a4b35838 Migrate cached_layouts into layout_cache_t 2020-05-30 15:00:28 -07:00
Fabian Homborg
02baa321ae Restyle
More of that weird reflowing that clang-format loves to do
2020-04-21 21:11:26 +02:00
Rosen Penev
220f0a132d [clang-tidy] use auto when casting
Found with modernize-use-auto

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-04-05 10:13:13 +02:00
Soumya
8c0ee6ebc9 Use an unlikely to collide username for tests
The prefix 'haha' is short enough, (and phonetic enough), that it could collide with an existing user on the system where the tests are running, causing the test to fail.
2020-03-28 05:41:53 +01:00
Soumya
654a266b22 Fix typo: '\E' is not an escape sequence. 2020-03-26 17:09:52 +01:00
Fabian Homborg
4a9f52466a Set locale during timer test
Fixes #6757
2020-03-15 21:01:11 +01:00
Johannes Altmanninger
3816c4bb39 Add comment about rounding in timer and appease clang-format 2020-03-14 13:34:31 +01:00
afg
f864bd83ad time builtin: align output columns on rare cases
1. When the wall time and cpu time rows has different units
   e.x. running multiple cores
2. When duration is around 1E3 or 1E6 microseconds
   printf("%6.2F", 999.995) gives 1000.00 which is 7 digits
2020-03-14 13:20:14 +01:00
ridiculousfish
bde2f2111d Introduce debounce_t
debounce_t will be used to limit thread creation from background highlighting
and autosuggestion scenarios. This is a one-element queue backed by a
single thread. New requests displace any existing queued request; this
reflects the fact that autosuggestions and highlighting only care about
the most recent result.

A timeout allows for abandoning hung threads, which may happen if you
attempt to e.g. access a dead hard-mounted NFS server. We don't want
this to defeat autosuggestions and highlighting permanently, so allow
spawning a new thread after the timeout (here 500 ms).
2020-03-06 17:15:21 -08:00
ridiculousfish
f6e5a5c521 Thread a cancellation function into is_potential_path
Allows sooner cancellation of redundant operations like highlighting.
2020-03-02 22:10:43 -08:00
ridiculousfish
bd17262e17 Reduce iteration count in thread torture test
Helps the test pass on 32 bit.
2020-03-02 13:46:08 -08:00
Johannes Altmanninger
8a033b9f3d Add undo
Add the input function undo which is bound to `\c_` (control + / on
some terminals). Redoing the most recent chain of undos is supported,
redo is bound to `\e/` for now.

Closes #1367.
This approach should not have the issues discussed in #5897.

Every single modification to the commandline can be undone individually,
except for adjacent single-character inserts, which are coalesced,
so they can be reverted with a single undo. Coalescing is not done for
space characters, so each word can be undone separately.

When moving between history search entries, only the current history
search entry is reachable via the undo history. This allows to go back
to the original search string with a single undo, or by pressing the
escape key.
Similarly, when moving between pager entries, only the most recent
selection in the pager can be undone.
2020-02-07 17:15:17 +01:00
ridiculousfish
057c3a9e75 Introduce fd_monitor
fd_monitor is a new class which can monitor a set of fds, waiting for them
to become readable. When an fd becomes readable, a callback is invoked.
Timeouts are also supported.

This is intended to replace the "bufferfill" threads. Rather than one
thread per bufferfill, we will have a single fd_monitor which can service
multiple bufferfills. This helps today with nested command substitutions,
and will help in the future with concurrent execution.
2020-02-05 12:04:51 -08:00
ridiculousfish
fc4557c784 Eliminate wopen()
It was large and mostly unnecessary. Prefer wopen() followed by
fdopen().
2020-01-28 10:43:37 -08:00
David Adam
dd1e526017 tests: remove an outdated comment
Code removed in af22d6b732
2020-01-28 16:54:29 +08:00
ridiculousfish
af22d6b732 Stop pretending to be interactive during the cancellation tests
Prior to this fix, the cancellation C++ test would mark the parser as
interactive in an effort to install interactive signal handling (so that,
for example, SIGINT would stop the job and return control to the user).

However this flag would also cause fish to attempt to save and restore tty modes
across the job. This would fail since there is no tty, and so the job would fail
with an unexpected error code.

We don't need to mark the parser as interactive, we can just remove that line.

Fixes #6539.
2020-01-27 12:13:46 -08:00
ridiculousfish
7a2a6d76f3 Migrate a test from C++ to littlecheck
This eliminates noisy warnings about tcsetpgrp when the tests are run
without a tty, as reported in #6539
2020-01-26 20:59:08 -08:00
ridiculousfish
38f4330683 Rationalize $status and errors
Prior to this fix, fish was rather inconsistent in when $status gets set
in response to an error. For example, a failed expansion like "$foo["
would not modify $status.

This makes the following inter-related changes:

1. String expansion now directly returns the value to set for $status on
error. The value is always used.

2. parser_t::eval() now directly returns the proc_status_t, which cleans
up a lot of call sites.

3. We expose a new function exec_subshell_for_expand() which ignores
$status but returns errors specifically related to subshell expansion.

4. We reify the notion of "expansion breaking" errors. These include
command-not-found, expand syntax errors, and others.

The upshot is we are more consistent about always setting $status on
errors.
2020-01-25 17:28:41 -08:00
ridiculousfish
81e78c78aa Rename eval_result_t to end_execution_reason_t
We're getting ready to stop returning eval_result_t from parser_t::eval
2020-01-25 14:11:32 -08:00
ridiculousfish
0f7bba5f0e Introduce operation_context_t
This commit recognizes an existing pattern: many operations need some
combination of a set of variables, a way to detect cancellation, and
sometimes a parser. For example, tab completion needs a parser to execute
custom completions, the variable set, should cancel on SIGINT. Background
autosuggestions don't need a parser, but they do need the variables and
should cancel if the user types something new. Etc.

This introduces a new triple operation_context_t that wraps these concepts
up. This simplifies many method signatures and argument passing.
2020-01-16 15:21:28 -08:00
ridiculousfish
db98ee13a9 Make complete() return the completion list directly
Returning it through a pointer was a remnant of pre C++-11 days.
2020-01-16 15:18:50 -08:00
ridiculousfish
4bb18eaf42 Use completion_list_t naming everywhere
std::vector<completion_t> -> completion_list_t
2020-01-16 15:18:49 -08:00
ridiculousfish
f1ce967dfa Do not allow empty items to be added to history
Empty items are used as sentinels to indicate that we've reached the end of
history, so they should not be added as actual items. Enforce this.

Fixes #6032
2020-01-05 12:47:02 -08:00
ridiculousfish
89880891d0 Revert "fish_tests to set HOME and other variables to temporary dir"
This reverts commit cca57a7a87.

The tests target already sets some variables - backing this out.
2020-01-01 17:24:14 -08:00
ridiculousfish
cca57a7a87 fish_tests to set HOME and other variables to temporary dir
Rather than placing files in the user's home directory, have fish_tests
manipulate HOME to be a temporary directory.
2020-01-01 16:34:56 -08:00
ridiculousfish
b784a0caa3 dup2_list_t::resolve_chain to stop returning maybe
It can no longer fail.
2019-12-29 14:49:05 -08:00
Mathieu Duponchelle
15c1b3ed4b Place fish in its own process group when launched with -i
Fixes #5909
2019-12-23 10:32:37 +01:00
ridiculousfish
c19407ab0f Default parser_t::eval()'s block type to top
This is the parameter value at every call site except one. Just make it the
default.
2019-12-22 16:27:03 -08:00
ridiculousfish
a59f35a378 Make block_type_t an enum class 2019-12-22 15:37:14 -08:00
ridiculousfish
0531c02ce4 Remove 'user_supplied' flag for io_fd_t
user_supplied was used to distinguish IO redirections which were
explicit, vs those that came about through "transmogrphication." But
transmogrification is no more. Remove the flag.
2019-12-19 14:14:23 -08:00
ridiculousfish
311f47d683 Remove an errant fprintf from the tests 2019-12-17 18:26:57 -08:00
ridiculousfish
b3d2cdc0ff Unify parse_execution_result_t and eval_result_t again
Do other cleanup to better express the difference between cancellation
and control flow.
2019-12-17 18:12:49 -08:00
ridiculousfish
72de8768d9 Remove RUNNING_IN_XCODE checks
This is never set any more.
2019-12-17 16:19:12 -08:00
ridiculousfish
09f8e05b80 Clean up the return type of parser_t::eval
parser_t::eval indicates whether there was a parse error. It can be
easily confused with the status of the execution. Use a real type to
make it more clear.
2019-12-17 16:16:43 -08:00
ridiculousfish
8bf9f52461 Always detach new pthreads
There are no longer any calls to pthread_join. Just make all pthreads
detached.
2019-12-16 14:08:46 -08:00
ridiculousfish
33aff87c10 Switch io_file_t to store an fd, not a path
Prior to this fix, a file redirection was turned into an io_file_t. This is
annoying because every place where we want to apply the redirection, we
might fail due to open() failing. Switch to opening the file at the point
we resolve the redirection spec. This will simplify a lot of code.
2019-12-12 17:34:40 -08:00
ridiculousfish
be685faeb8 Clean up how pipe fd avoidance works
fish has to ensure that the pipes it creates do not conflict with any
explicit fds named in redirections. Switch this code to using
autoclose_fd_t to make the ownership logic more explicit, and also
introduce fd_set_t to reduce the dependence on io_chain_t.
2019-12-12 14:58:18 -08:00
Johannes Altmanninger
7d5b44e828 Support FOO=bar syntax for passing variables to individual commands
This adds initial support for statements with prefixed variable assignments.
Statments like this are supported:

a=1 b=$a echo $b        # outputs 1

Just like in other shells, the left-hand side of each assignment must
be a valid variable identifier (no quoting/escaping).  Array indexing
(PATH[1]=/bin ls $PATH) is *not* yet supported, but can be added fairly
easily.

The right hand side may be any valid string token, like a command
substitution, or a brace expansion.

Since `a=* foo` is equivalent to `begin set -lx a *; foo; end`,
the assignment, like `set`, uses nullglob behavior, e.g. below command
can safely be used to check if a directory is empty.

x=/nothing/{,.}* test (count $x) -eq 0

Generic file completion is done after the equal sign, so for example
pressing tab after something like `HOME=/` completes files in the
root directory
Subcommand completion works, so something like
`GIT_DIR=repo.git and command git ` correctly calls git completions
(but the git completion does not use the variable as of now).

The variable assignment is highlighted like an argument.

Closes #6048
2019-11-25 09:20:51 +01:00
ridiculousfish
e18fd3cddb Allow unclosed subshells in interactive mode
If the user has an unclosed subshell in interactive mode, break the
line instead of producing an error.

Fixes #6316
2019-11-13 18:01:47 -08:00
ridiculousfish
ec08a50769 Eliminate function_data_t
This struct is now mostly useless and can go.
2019-11-12 11:25:41 -08:00
ridiculousfish
6d7a66592b Make function_add take the filename directly instead of a parser 2019-11-12 10:00:42 -08:00
ridiculousfish
896ef65f8c Rename error_offset to error_offset_within_token
Hopefully clarify the role of this variable.
2019-11-08 16:56:50 -08:00
ridiculousfish
ef8b5e4fa0 Correct certain tokenizer error reporting for unclosed subshells
There was some confusion about the different pointers and offsets
in tokenizer_t::call_error.

Fixes #6281
2019-11-08 16:56:43 -08:00
Johannes Altmanninger
e94b9ccf3e Do import bash history commands containing && or || 2019-11-07 23:33:35 +01:00
ridiculousfish
52e900690b Make history_search_type_t an enum class 2019-11-02 18:56:16 -07:00
ridiculousfish
eeac3333df Correctly highlight input following a tokenizer error 2019-10-27 16:08:49 -07:00
ridiculousfish
afd20b8e1a Correctly report the range of tokenizer errors
This enables proper syntax highlighting of tokenizer errors.
2019-10-27 16:05:37 -07:00
ridiculousfish
2a92e66902 Support for &> and &| as convenience redirections
This adds support for &> and &| syntax, which both redirect stdout, and
also apply a redirection of stderr to stdout.
2019-10-27 15:24:57 -07:00
LawAbidingCactus
305a657694 fix typos 2019-10-23 19:38:44 +02:00
ridiculousfish
35671dd9f0 Clean up and unify pipes and redirections
This cleans up how pipes and redirections are recognized by the parser,
and unifies pipes and redirections into a single type.
2019-10-15 11:26:41 -07:00
ridiculousfish
1a65e18ba8 Clean up some tokenization
Remove TOK_NONE
Turn token_type into an enum class
Make next() turn a maybe_t<tok_t> instead of a bool
2019-10-13 16:06:16 -07:00
ridiculousfish
82eca4bc86 Run clang-format on all files
The main change here is to reorder headers.
2019-10-13 15:50:48 -07:00
ridiculousfish
a19d9b2e73 Add a test for #6130 2019-09-22 14:06:38 -07:00
Johannes Altmanninger
8baea8b157 Do not add a space after completion if one is already there
Example: type `cd --help --help`, move the cursor inside the first `--help` and press tab.
This used to add redundant spaces.
2019-09-18 14:02:24 -05:00
ridiculousfish
535a448d0e Remove an errant fprintf 2019-09-08 15:44:07 -07:00
ridiculousfish
b2fe2f9ff3 Correct escaping and add tests for bracket completion
Add some tests for new bracket completion behavior, and fix an escaping
bug where \\[ was incorrectly marked as escaping.
2019-09-08 15:43:20 -07:00
ridiculousfish
0dc5eaeb97 reader_expand_abbreviation_in_command to return maybe_t<wcstring> 2019-08-06 16:09:23 -07:00
ridiculousfish
822b53c67a Don't color a whole string invalid because of an unclosed quote
When syntax highlighting a quoted string, if the string is not closed,
only show the opening quote as an error, not the whole string.
2019-07-24 12:43:47 -07:00
ridiculousfish
8ec568d21f Add some tests to path_apply_working_directory 2019-07-15 17:56:06 -07:00
ridiculousfish
0e4fe53718 clang-format some files 2019-07-06 13:11:50 -07:00
ridiculousfish
bc0329f775 Add wcstolower
Converts a string to lowercase. Eliminates some loops that did this
explicitly.
2019-07-06 12:19:34 -07:00
ridiculousfish
4a2c709fb1 Eliminate shell_is_interactive
We used to have a global notion of "is the shell interactive" but soon we
will want to have multiple independent execution threads, only some of
which may be interactive. Start tracking this data per-parser.
2019-06-29 11:28:26 -07:00
Fabian Homborg
caedf01c00 Revert "Latch signal handlers"
This reverts commit 7ed1022cf4.

Fixes #5962.
2019-06-25 11:25:09 +02:00
ridiculousfish
be97499106 Revert "Revert "Add a test for autoload_t""
This reverts commit 2507162f80.

Put back the autoload test.
2019-06-07 12:12:43 -07:00
ridiculousfish
fc99d6c7af clang-format all files 2019-06-03 20:30:48 -07:00
ridiculousfish
890c1188ab Instantize and rationalize locking around the input mapping set
Hypothetically a background thread could invoke builtin_bind, etc.
Protect the set of input bindings with a lock.
2019-06-02 22:54:01 -07:00
ridiculousfish
671df14178 Thread a parser into inputter_t 2019-06-02 16:41:13 -07:00
ridiculousfish
a48dbf23b8 Instantize the character event queue
Wrap this stuff up in structs so it's no longer global.
2019-06-02 16:37:05 -07:00
Fabian Homborg
61292b0c6c fish_tests: Fix warnings 2019-05-29 21:11:08 +02:00
David Adam
c848787107 add extra test for wcsfilecmp
Test changes added for #5634 in 6e525cc5d9.
2019-05-28 22:03:03 +08:00
ridiculousfish
7ed1022cf4 Latch signal handlers
Now that our interactive signal handlers are a strict superset of
non-interactive ones, there is no reason to "reset" signals or take action
when becoming non-interactive. Clean up how signal handlers get installed.
2019-05-26 18:04:03 -07:00
Dawid Dziurla
0b3bb0e7c1 Underline every valid entered path (#5872)
* src/highlight: Underline every valid entered path

* update CHANGELOG

* fix highlight test
2019-05-19 10:03:56 +02:00
ridiculousfish
c44dae2d73 Migrate certain runtime flags to atomics hidden behind functions 2019-05-18 18:50:28 -07:00
ridiculousfish
1719d6f136 Make $status and $pipestatus per-parser
Another step towards allowing multiple parsers to execute in parallel.
2019-05-12 14:00:44 -07:00
ridiculousfish
cfddd881ef Make PWD a per-processor variable
Handle this variable specially.
2019-05-11 17:13:34 -07:00
ridiculousfish
8a8b2513b5 Eliminate the global jobs() function
All job lists are attached to a parser now.
2019-05-05 11:33:08 -07:00
Fabian Homborg
c2970f9618 Reformat all files
This runs build_tools/style.fish, which runs clang-format on C++, fish_indent on fish and (new) black on python.

If anything is wrong with the formatting, we should fix the tools, but automated formatting is worth it.
2019-05-05 12:09:25 +02:00
ridiculousfish
f66e010949 Turn a lot of common.h variables into getter functions
Improves thread safety.
2019-05-04 20:58:35 -07:00
ridiculousfish
bffacd2fbf Thread a parser into expansion
Expansion may perform command substitution, which needs to know the parser
to use.
2019-05-04 19:30:00 -07:00
ridiculousfish
923a7ca0f0 Thread the parser into complete()
Eliminates uses of principal_parser
2019-05-04 18:17:18 -07:00
ridiculousfish
63bdc949ab Make completion request flags an enum_set 2019-05-04 17:55:57 -07:00
ridiculousfish
ec45f31ad1 Make debug_level an atomic
Fixes a tsan warning
2019-05-04 15:28:44 -07:00
ridiculousfish
d8ac051f89 Move selection_direction_t to pager.h and make it a class enum 2019-04-28 14:06:03 -07:00
ridiculousfish
2507162f80 Revert "Add a test for autoload_t"
This reverts commit 51c62d6cc6.

Back out the test while I attempt to fix it
2019-04-27 20:14:06 -07:00
ridiculousfish
51c62d6cc6 Add a test for autoload_t 2019-04-27 16:16:48 -07:00
ridiculousfish
96bc8a14ca Promote completion_mode_t to a real type
Eliminate big #defines like NO_COMMON.
2019-04-25 14:21:06 -07:00
ridiculousfish
496529b20a Remove EXPAND prefix from expand_flags and lowercase them 2019-04-25 11:34:49 -07:00
ridiculousfish
d8ab6290e8 Switch expand_flags_t to enum_set 2019-04-25 11:23:03 -07:00
ridiculousfish
dcaac58f45 Rename expand_error_t to expand_result_t and make it class enum
Also lowercase it all.
2019-04-25 10:47:28 -07:00
ridiculousfish
020d4a2848 Adopt env_scoped_t::snapshot() and remove env_var_snapshot_t
Remove the env_var_snapshot_t class and switch everything to the new snapshot
function of env_scoped_t.

Fixes #5658. Fixes #5571.
2019-04-14 15:50:38 -07:00
Mahmoud Al-Qudsi
05f52924c1 Fix bad tests
These tests used raw, unescaped parentheses to perform `test` logical
grouping, but the test failures weren't caught because the parser
evaluation errors were not being propagated (fixed in bdbd173e).
2019-04-13 17:28:16 -05:00
ridiculousfish
e7d7eff0ee Rename all readline commands to lowercase and remove R_ prefix 2019-03-23 23:31:35 -07:00
ridiculousfish
2c56e27d37 Switch readline commands to readline_cmd_t enum class 2019-03-23 23:31:25 -07:00
ridiculousfish
c2be5e8986 Introduce char_event_type_t::readline
Baby steps towards eliminating readline actions as characters.
2019-03-23 23:31:18 -07:00
ridiculousfish
185805641c Remove R_TIMEOUT
Promote timeout to a char_event_type_t, moving it out of the "char" namespace.
This will help simplify the readline implementation.
2019-03-23 20:10:06 -07:00
ridiculousfish
a58662dd46 Make maybe_t conditionally copyable
This allows it to be used with both e.g. unique_ptr and std::vector.
2019-03-17 13:38:18 -07:00
Aaron Gyes
d837eee09d remove some wcstring -> wchar_t* -> wcstring conversions
Mostly related to usage _(L"foo"), keeping in mind the _
macro does a wcstring().c_str() already.

And a smattering of other trivial micro-optimizations certain
to not help tangibly.
2019-03-14 15:21:08 -07:00
Aaron Gyes
477b2e8d7c std::vector<wcstring> is wcstring_list_t 2019-03-14 11:17:26 -07:00
Aaron Gyes
aaacdb89b6 Switches over to cstring from string.h. 2019-03-12 15:09:36 -07:00
Aaron Gyes
d5ac239f68 This commit changes wchar.h includes to cwchar, and uses std::
for everything it provides.
2019-03-12 15:09:36 -07:00
ridiculousfish
9b9c1aa745 Fix some unused variable warnings 2019-03-03 18:06:31 -08:00
ridiculousfish
717ac9a8d5 Switch highlight_spec_t to a struct
Rather than a janky bitmask, use a real struct with real fields.
2019-03-03 18:04:22 -08:00
Aaron Gyes
6fa8b028fc fish_tests.cpp: fixup: I didn't notice the comma here. 2019-02-18 23:19:57 -08:00
Aaron Gyes
c2bc0c67f2 Don't use printf("%d") just to convert an int to a string.
std::to_string, std::to_wstring are more appropriate
2019-02-18 23:15:54 -08:00