Commit graph

3209 commits

Author SHA1 Message Date
ridiculousfish
f1402ac7a3 Eliminate some uses of current_data in the reader 2020-08-03 15:41:12 -07:00
ridiculousfish
272ca37582 Remove most dynamic reader configuration
Have the reader accept a constant configuration object, which controls
whether autosuggestions, etc. are enabled. These things don't change
dynamically.
2020-08-03 15:02:44 -07:00
ridiculousfish
fadd429348 Make the shell test function a toggle
We either perform fish syntax checking or we don't; there's no reason
to specify a function pointer here.
2020-08-03 14:31:09 -07:00
ridiculousfish
7304815736 Make shell highlighting a toggle instead of a function parameter
Remove the ability to specify the "highlight function." The reader
always highlights via shell highlighting, or doesn't.
2020-08-03 14:10:37 -07:00
ridiculousfish
18c7c46657 Remove highlight_universal
This was an attempt to offer syntax highlighting for `read` when shell
highlighting is not enabled, but it hardly did anything.
2020-08-03 13:41:41 -07:00
ridiculousfish
f94a6a74f0 Remove fish_color_match support
fish_color_match is a variable which controls syntax highlighting for
matching quotes and parens, but only with interactive `read` with shell
highlighting disabled. It seems unlikely that anybody cares about this.
2020-08-03 13:36:47 -07:00
Olivier FAURE
e7f0b5801d Add forward-single-char input command
This allows users to add custom keybindings to autocomplete only one
character at a time.

Resolves https://github.com/fish-shell/fish-shell/issues/4984
2020-08-01 12:09:31 +02:00
Olivier FAURE
6778d04aa5 Add or keybind input function 2020-08-01 12:09:31 +02:00
ridiculousfish
81a39be0bb Support explicitly separated output on stderr
In principle this would allow 'string split' or whatever to output to
stderr and not lose the item separation. In practice this is not used
but it fixes a TODO.
2020-07-30 23:00:34 -07:00
ridiculousfish
bcfc54fdaa Do not buffer builtin output if avoidable
builtins output to stdout and stderr via io_streams_t. Prior to this fix, it
contained an output_stream_t which just wraps a buffer. So all builtin output
went to this buffer (except for eval).

Switch output_stream_t to become a new abstract class which can output to a
buffer, file descriptor, or nowhere. This allows for example `string` to stream
its output as it is produced, instead of buffering it.
2020-07-30 22:45:44 -07:00
ridiculousfish
68092c5d21 Bravely have read_blocked return after first read
In commit fd6d814ea4, read_blocked was changed to read until EOF
or the full amount requested is returned. Switch this to returning
as soon as any data is available, which was the behavior prior to
fd6d814ea4.

This will allow builtin_string to output data in a "streaming"
fashion instead of needing to read a large block up-front.
2020-07-30 22:08:54 -07:00
ridiculousfish
81d5a3ea64 Do not add silent mode history items to history
Prior to this fix, if you invoked fish with --private and then used
`read --silent` to read something sensitive, the variable would be
stored in history, with the plain text available through up-arrow.
Fix it to not store items in silent mode.

Note the item was never written to disk; it was only stored in memory.

Fixes #7230
2020-07-30 20:26:01 -07:00
ridiculousfish
f7f35d2373 Fix a warning in fish_tests 2020-07-29 20:12:19 -07:00
ridiculousfish
2809d637db Introduce wwrite_to_fd
wwrite_to_fd() is a function which writes a wide string to a file
descriptor without performing any heap allocations.
2020-07-29 19:38:45 -07:00
ridiculousfish
a0cb23bea5 Introduce wcs2string_callback
This is like wcs2string, but instead of returning a std::string, it invokes
a user-supplied function with each converted character.

The idea is to allow interleaved conversion and output.
2020-07-29 19:36:20 -07:00
ridiculousfish
c9b42c6f1f Stop #include-ing wcstringutil.h in flog.h
This is a header dependency that we can break.
2020-07-29 17:04:18 -07:00
ridiculousfish
db086fc5d4 Eliminate wcs2str
Use std::string variants everywhere instead
2020-07-29 16:37:39 -07:00
ridiculousfish
e4b1fc9f6a Remove stdout_read_limit param from exec_process_in_job
This can always be trivially fetched from the parser - no need to pass it
in separately.
2020-07-29 16:04:29 -07:00
ridiculousfish
3506274ccf Make in_foreground an explicit param to continue_job
This moves us slightly closer towards fish code in the background. The idea is
that a background job may still have "foreground" sub-jobs, example:

    begin ; sleep 5 ; end &

The begin/end job runs in the background but should wait for `sleep`.

Prior to this fix, fish would see the overall job group is in the background
and not wait for any of its processes. With this change we detach waiting from
is_foreground.
2020-07-27 15:56:24 -07:00
ridiculousfish
3382bc70d2 Fix a stale comment
[ci skip]
2020-07-27 15:36:43 -07:00
ridiculousfish
c35fe879c7 Bravely remove reclaim... param from continue_job, and rework tcsetpgrp calls
This changes how fish attempts to protect itself from calling tcsetpgrp() too
aggressively. Recall that tcsetpgrp() will "force" itself, if SIGTTOU is
ignored (which it is in fish when job control is enabled).

Prior to this fix, we avoided SIGTTINs by only transferring the tty ownership
if fish was already the owner. This dated from a time before we had really
nailed down how pgroups should be assigned. Now we more deliberately assign a
job's pgroup so we don't need this conservative check.

However we still need logic to avoid transferring the tty if fish is not the
owner. The bad case is when job control is enabled while fish is running in the
background - here fish would transfer the tty and "steal" from the foreground
process.

So retain the checks of the current tty owner but migrate them to the point of
calling tcsetpgrp() itself.
2020-07-27 14:51:37 -07:00
ridiculousfish
1823f5d95f Remove the send_sigcont from continue_job
We can just send sigcont if the job is stopped; no need to make this an
explicit param.
2020-07-27 10:48:32 -07:00
ridiculousfish
6d77a93cc2 Improve commenting in exec_job
[ci skip]
2020-07-26 10:45:02 -07:00
Mahmoud Al-Qudsi
d46b9ff9be Remove repeated acquire of disowned pid lock in a loop 2020-07-25 20:45:08 -05:00
David Adam
2720f3d2ef proc: disown PIDs, not just PGIDs
add_disowned_pgid skipped jobs that have a PGID equal to the running
process. However, this includes processes started in config.fish or when
job control is turned off, so they never get waited on.

Instead, refactor this function to add_disowned_job, and add either the PGID or
all the PIDs of the job to the list of disowned PIDs/PGIDs.

Fixes #7183.
2020-07-25 20:38:59 -05:00
David Adam
025a0d3cf5 proc: add log message for reaped disowned IDs 2020-07-25 20:35:54 -05:00
ridiculousfish
bae64f8a8a Modest cleanup of profiling
This is a set of miscellaneous cleanup for profiling.

An errant newline has been removed from 'if' statement output, which got
introduced with the new ast.
Switch from storing unique_ptr to a deque, which allocates less.
Collapse "parse" and "exec" times into just a single value "duration". The
"parse" time no longer makes sense, as we now parse ahead of time.
2020-07-24 11:53:07 -07:00
ridiculousfish
7d2d2c97b2 Fix a compiler warning in builtin_printf 2020-07-19 16:51:11 -07:00
ridiculousfish
54b642bc6f Factor job groups into their own file
Migrate out of proc.h, which has become too long.
2020-07-19 16:42:29 -07:00
ridiculousfish
56e250d467 Remove a commented-out field 2020-07-19 15:48:16 -07:00
Johannes Altmanninger
b3485d2457 Have the pager use the correct prefix for case-corrected completions
Follow-up to 28d67c8f which only fixed inserting such completions.
2020-07-20 00:38:01 +02:00
Johannes Altmanninger
a61e97a8b9 Fix case-correcting completion stomping token on common prefix
The prefix has already been case-corrected at this point and the remaining
completions are for the suffix only.

Fixes #7211

Introduced in
28d67c8f Show completion list on Tab also if a common prefix was inserted
2020-07-19 23:40:07 +02:00
ridiculousfish
dff4f140b0 Make the list of blocked events const
These are events that have been queued but not yet fired. There's no
reason to modify the events after creating them. Mark them as const
to ensure that doesn't happen.
2020-07-19 12:03:10 -07:00
ridiculousfish
7f8c00c20a Remove job_t::wants_terminal
This now lives in the job group, not individual jobs.
2020-07-18 12:42:44 -07:00
ridiculousfish
ba8b89873e Teach a job its command at constructor time
No point in allowing this to be set later.
2020-07-18 12:42:44 -07:00
ridiculousfish
f30ce21aaa terminal_maybe_give_to_job to operate on groups, not jobs
Assigning the tty is really a function of a job group, not an individual
job. Reflect that in terminal_maybe_give_to_job_group and also
terminal_return_from_job_group.
2020-07-18 12:42:44 -07:00
ridiculousfish
40c9bda7fd Store the command that produced a job group in the group
This will enable us to replace more uses of jobs with job groups.
2020-07-18 12:42:44 -07:00
ridiculousfish
3571754e06 Only perform universal barriers for the principal env stack
In practice this means that, if fish ever gets multiple variable stacks,
we will only incorporate environment variable changes from other fish
instances on the "main thread."
2020-07-16 16:16:03 -07:00
ridiculousfish
352062219d More clean up of parse_util_detect_errors_in_argument 2020-07-14 15:51:12 -07:00
ridiculousfish
3532a955a6 Use parse_util_locate_cmdsubst_range when validating arguments
Removes another usage of parse_util_locate_cmdsubst().
2020-07-14 15:34:26 -07:00
ridiculousfish
68d256a91c Use parse_util_locate_cmdsubst_range in highlighting 2020-07-14 15:27:14 -07:00
ridiculousfish
9cade52c77 Clean up expand_cmdsubst somewhat
Eliminate some of the pointer arithmetic and switch to
parse_util_locate_cmdsubst_range. It's still a pretty ugly function.
2020-07-14 15:19:10 -07:00
ridiculousfish
37dc554fe1 Revert "Remove unnecessary owning_lock usages"
This reverts commit 3a5585df95.

This reverts a change that removed a lock. It's indeed true that in master,
fish script is bound to the main thread. But I'm working to remove that
limitation and these locks are important in that future.
2020-07-12 18:56:39 -07:00
ridiculousfish
4904e4bc41 Fix autosuggestion validation
When switching to the new ast, commands that were not decorated
statements (like function declarations) would be rejected from
autosuggestion validation because we could not find a command. Stop
rejecting them.
2020-07-12 18:47:33 -07:00
Mahmoud Al-Qudsi
3a5585df95 Remove unnecessary owning_lock usages
The owning locks were added after the original code and decorated with
comments indicating they are thread-safe, even though they're only ever
used from the main thread. Presuming the intent was to make future
manipulation of the code safer rather than to actually make use of any
thread safety guarantees, these have been wrapped in a new
`thread_exclusive` type which always calls ASSERT_IS_MAIN_THREAD.

The benefit is that this does not perform a syscall to lock a mutex
each time the variables are accessed.
2020-07-12 20:21:28 -05:00
Mahmoud Al-Qudsi
6d45dd837f Remove trailing ", " in functions output
When executed interactively and not piped, `functions` adds a comma as a
separator between each result. This removes the separator after the last
item.
2020-07-12 19:14:35 -05:00
Mahmoud Al-Qudsi
f1951fed3a Fix suggestionOK variable name 2020-07-12 19:14:35 -05:00
Mahmoud Al-Qudsi
73bf9dd784 Stop calling path_get_path for builtins and functions
highlight.cpp was blindly calling path_get_path for each head command
typed at the prompt which triggers a lot of syscalls via waccess.

It's still going to do that while commands are being composed, but now
it won't if we can make a cheap lookup to the builtins/functions hash
table and can determine that it's a valid command before inspecting the
filesystem.
2020-07-12 19:14:35 -05:00
ridiculousfish
3319e308d0 Make ast::node_t non-virtual
Eliminate its vtable to save 8 bytes per node, which is a lot!
2020-07-12 16:57:30 -07:00
ridiculousfish
a8eb2a6813 Make union_ptr_t's constructor statically type safe
Ensure it cannot be constructed from the wrong node type.
2020-07-12 16:57:30 -07:00
ridiculousfish
8d37be2916 ast lists to use new[] instead of vector
Because the list is not changed after construction, we do not need
the vector's capacity field. This reduces the size of lists from 48
to 32 bytes.
2020-07-12 16:57:30 -07:00
ridiculousfish
c12ab7a674 Rearrange ast::node_t fields to improve packing 2020-07-12 16:57:30 -07:00
ridiculousfish
487de1e6c3 Reduce copying in parse_util_detect_errors
Allow parse_util_detect_errors to accept an already-parsed ast. This
eliminates a copy of the source, which is helpful when executing large
scripts.
2020-07-12 16:57:30 -07:00
ridiculousfish
dfeec433d8 Reduce allocation churn in parse_util_detect_errors
Reuse a single string for storage.
2020-07-12 16:57:30 -07:00
ridiculousfish
9ee5075fc3 Reformat CPP files 2020-07-12 12:21:25 -07:00
ridiculousfish
2a4c545b21 Rework how signals trigger cancellation
When fish receives a "cancellation inducing" signal (SIGINT in particular)
it has to unwind execution - for example while loops or whatever else that
is executing. There are two ways this may come about:

1. The fish process received the signal
2. A child process received the signal

An example of the second case is:

    some_command | some_function

Here `some_command` is the tty owner and so will receive control-C, but
then fish has to cancel function execution.

Prior to this change, these were handled uniformly: both would just set a
cancellation signal inside the parser. However in the future we will have
multiple parsers and it may not be obvious which one to set the flag in.
So instead distinguish these cases: if a process receives SIGINT we mark
the signal in its job group, and if fish receives it we set a global
variable.
2020-07-12 12:16:01 -07:00
ridiculousfish
12d0afa929 Fix some build warnings in fish_tests 2020-07-12 11:41:06 -07:00
ridiculousfish
2e5222ffe8 Finish renaming job tree to job group
Some "tree" terminology was still there.
2020-07-11 17:05:42 -07:00
ridiculousfish
765c48afa4 Migrate the notion of 'foreground' from job to job group
Whether a job is foreground is a property of its pgid, so it belongs
naturally on the job group.
2020-07-11 17:01:52 -07:00
ridiculousfish
225470493b Make parse_token_type_t an enum class
Improves type safety.
2020-07-09 14:22:04 -07:00
ridiculousfish
7ea396ab3f Remove lrand48 checks and support
lrand48 is no longer used.
2020-07-08 11:00:12 -07:00
ridiculousfish
35cb449aa1 Make parse_statement_decoration_t a class enum 2020-07-07 16:28:39 -07:00
ridiculousfish
71a8eb0aa4 parsed_source_t to hold an ast directly instead of through unique_ptr
We have untangled the dependency loop and so now parsed_source_t no longer
requires indirection.
2020-07-07 16:16:45 -07:00
ridiculousfish
5308223212 Migrate next_parse_token into token_stream_t
Cleaning up parse_tree.cpp with an eye to remove it.
2020-07-07 14:01:01 -07:00
ridiculousfish
72e35af381 Remove preceding_escaped_nl
It's no longer necessary for fish_indent
2020-07-07 13:48:35 -07:00
Mahmoud Al-Qudsi
8d25ed962c Add early abortion of completion match attempt 2020-07-06 23:08:19 -05:00
Mahmoud Al-Qudsi
02d0380e6b Make fish_exit workaround for forced exit not SIGHUP-specific 2020-07-05 23:17:21 -05:00
Mahmoud Al-Qudsi
04c6442dcc Allow fish_exit to run even on fish SIGHUP
We were previously aborting the main event loop before calling fish_exit
in the event of a SIGHUP. This patch causes the SIGHUP to be stored in a
separate state variable from a regular "must exit" condition so the
associated event can be fired before we terminate the loop.

All streams are redirected before the event is called to prevent a
SIGTTIN/SIGTTOU due to the user script reading/writing from a disposed
tty.

Closes #7014
2020-07-05 22:18:21 -05:00
Mahmoud Al-Qudsi
791c4fb1dd Fix unused return value errors in fish tests
This error only happens on recent versions of gcc, see previous
commit e6bb7fc973 for more info.

Instead of using `ignore_result()` here, I've added a `system_assert`
function/macro that mimics the behavior of all the other `system()`
calls in the file.
2020-07-05 22:18:21 -05:00
ridiculousfish
bd4c4a9a9c Stop weak linking pre 10.9 macOS symbols
Previously fish weak-linked wcsdup, wcsncasecmp, and wcscasecmp.
This enabled fish to be used on 10.6. However the minimum Mac version
is now 10.9, where these symbols are available.
2020-07-05 12:49:29 -07:00
ridiculousfish
6976d0ee7e Simplify infinite loop fix when parsing "a="
This reworks the "a=" detection to be simpler.
If we detect a variable assignment that produces an error,
simply consume it.

We also take the opportunity to not highlight it as an error,
and add some tests.

Original commit is 1ca05d32d3.
2020-07-05 12:15:18 -07:00
ridiculousfish
23224f71ce Make some variables local which did not need to be static 2020-07-05 12:15:18 -07:00
Johannes Altmanninger
1ca05d32d3 Fix infinite loop when parsing "a="
Typing that command in an interactive prompt would make the highlighter thread
eat up CPU and memory.  Probably not the right fix; I think the token should
already have been consumed when the error is detected, then there is no need
to consume it when unwinding.
2020-07-05 13:19:48 +02:00
Johannes Altmanninger
ada03d3509 Correct pager size when command line soft-wraps
When selectiong a large completion entry in the pager, it would clobber the
prompt. To reproduce, first run this command

	complete -c : -xa '(
		# completion entries that,  when applied to the commandline
		# need one, two, or three lines respectively
		echo 1
		echo 2(string repeat -n (math $COLUMNS - 5) x)
		echo 3(string repeat -n $COLUMNS x)
		printf %s\n n(seq $LINES)
	)'

then type ": " and hit Tab repeatedly. When cycling through completion
entries, observe that fish always tries to render the pager with the same
size, even though the number of lines occupied by the command line buffer
changes due to soft wrapping.

Fix this by rendering the pager after the command line has been rendered, so
we know how many lines we have left.
2020-07-05 08:55:17 +02:00
Mahmoud Al-Qudsi
e628ba51e7 Remove repeated calculation of fixed string's length in a loop 2020-07-04 20:38:06 -05:00
Mahmoud Al-Qudsi
b8a16a8ba0 Convert highlighted_char_t to a struct 2020-07-04 20:23:50 -05:00
ridiculousfish
0c22f67bde Remove the old parser bits
Now that everything has been migrated to the new AST, remove as much of
the parse_tree bits as possible
2020-07-04 14:58:05 -07:00
ridiculousfish
3534c07584 Adopt the new AST in parse_execution
parse_execution is what turns a parsed tree into jobs, etc. Switch it from
parse_tree to the new AST.
2020-07-04 14:58:05 -07:00
ridiculousfish
6c6088f45c Adopt the new AST in fish_tests
This switches fish_tests from parse_tree to the new AST.
2020-07-04 14:58:05 -07:00
ridiculousfish
886603b2ca Adopt the new AST in fish_indent
This switches fish_indent from parsing with parse_tree
to the new ast.

This is the most difficult transition because the new ast retains less
lexical information than the old parse tree. The strategy is:

1. Use parse_util_compute_indents to compute indenting for each token.

2. Compute the "gap text" between the text of significant tokens. This
contains whitespace, comments, etc.

3. "Fix up" the gap text while leaving the significant tokens alone.
2020-07-04 14:58:05 -07:00
ridiculousfish
6b24edccf6 Adopt the new AST in add_pending_with_file_detection
This switches add_pending_with_file_detection from parsing with parse_tree
to the new ast.
2020-07-04 14:58:05 -07:00
ridiculousfish
202fdfa54a Adopt the new AST in parse_util_detect_errors
This switches parse_util_detect_errors from parsing with parse_tree to
the new ast.
2020-07-04 14:58:05 -07:00
ridiculousfish
7bea5ffa2e Adopt the new AST in parse_util_compute_indents
This switches parse_util_compute_indents from parsing with parse_tree to
the new ast.

It also reworks the parse_util_compute_indents tests, because
parse_util_compute_indents will be the backing for fish_indent.
2020-07-04 14:58:05 -07:00
ridiculousfish
358d7072a2 Adopt the new AST in bash history import
This switches bash history importing from parsing with parse_tree to the
new ast.
2020-07-04 14:58:05 -07:00
ridiculousfish
46c4ec8d68 Adopt the new AST in completion argument lists
This switches completion argument list expansion from parsing with
parse_tree to the new ast.
2020-07-04 14:58:05 -07:00
ridiculousfish
0d4f9c6220 Adopt the new AST in abbreviation expansion
This switches abbreviation expansion from parsing with parse_tree to the
new ast.
2020-07-04 14:58:05 -07:00
ridiculousfish
8d9725c301 Adopt the new AST in highlighting
This switches syntax highlighting from parsing with parse_tree to the new
ast.
2020-07-04 14:58:05 -07:00
ridiculousfish
4d4455007d Introduce a new fish ast
This is the first commit of a series intended to replace the existing
"parse tree" machinery. It adds a new abstract syntax tree and uses a more
normal recursive descent parser.

Initially there are no users of the new ast. The following commits will
replace parse_tree -> ast for all usages.
2020-07-04 14:58:02 -07:00
ridiculousfish
45c9e3b0f1 parsed_source_ref to always make a job_list
Removed an unnecessary param in preparation for more changes.
2020-07-04 14:51:15 -07:00
ridiculousfish
e530163078 Revert "fix unreachable code warning"
This reverts commit 202fe39d34.

If mkostemp is a weak symbol and is null, then the code will
be reachable.
2020-07-04 14:49:05 -07:00
ridiculousfish
90d8df8128 Use _POSIX_VDISABLE instead of \0 to disable control functions
Prior to this commit, fish used NUL ('\0') to disable control
functions (for example, the function that generates SIGTSTP).
However NUL may in fact be bindable and is on macOS via
control-space.

Use instead _POSIX_VDISABLE if defined and not -1.
2020-07-01 22:33:31 -07:00
Fabian Homborg
77f412af1b fixup! Also clear suggestions
Also #7145.
2020-07-01 21:00:16 +02:00
Fabian Homborg
eb35975c0f Make cancel-commandline actual bind function
This was always awkward as fish script, and had problems with
interrupting the autoloading.

Note that we still leave the old function intact to facilitate easier
upgrading for now.

Fixes #7145.
2020-07-01 20:56:56 +02:00
Johannes Altmanninger
76e0875c8f Apply clang-format 10 and selected lints from "make lint-all" 2020-07-01 00:44:06 +02:00
Johannes Altmanninger
25fe353187 Readline command beginning-of-history visits the oldest search match
Previously it would do the same as end-of-history
2020-07-01 00:40:32 +02:00
Johannes Altmanninger
e5b8035fea Fix updating command line on Page-Down
Page-Down seems to deactivate history search, so trying to undo
would leave the command line in an inconsistent state.

Fixes #7162 which was introduced in
12a9cb29 Fix assertion failure on page up / page down
2020-07-01 00:40:32 +02:00
Johannes Altmanninger
2b3b460264 Make reader_history_search_t::matches a vector instead of a deque
It is used exclusively as vector at the moment since we only ever append
at the end.  Making it a deque would be useful when allowing to edit the
search string and subsequently resume the search at an arbitrary position
in the history.
2020-07-01 00:40:32 +02:00
Johannes Altmanninger
971c2eb668 history search: do not move the cursor after failing backward search
When editing a multiline command line and pressing "up" with the cursor at the
first line, fish attempts a hsitory search. If the search fails, don't move
the cursor to the end of the multiline command because this can be annoying
when the user does not actually want to perform a history search.
2020-07-01 00:40:32 +02:00
Johannes Altmanninger
4f796df87c Clarify comment 2020-07-01 00:40:32 +02:00
Mahmoud Al-Qudsi
0b55f08de2 Speed up executable command completions
This brings down the number of syscalls per potential completion result
from three or four to just one.
2020-06-29 09:27:49 -05:00