Commit graph

617 commits

Author SHA1 Message Date
Soumya
654a266b22 Fix typo: '\E' is not an escape sequence. 2020-03-26 17:09:52 +01:00
Fabian Homborg
4a9f52466a Set locale during timer test
Fixes #6757
2020-03-15 21:01:11 +01:00
Johannes Altmanninger
3816c4bb39 Add comment about rounding in timer and appease clang-format 2020-03-14 13:34:31 +01:00
afg
f864bd83ad time builtin: align output columns on rare cases
1. When the wall time and cpu time rows has different units
   e.x. running multiple cores
2. When duration is around 1E3 or 1E6 microseconds
   printf("%6.2F", 999.995) gives 1000.00 which is 7 digits
2020-03-14 13:20:14 +01:00
ridiculousfish
bde2f2111d Introduce debounce_t
debounce_t will be used to limit thread creation from background highlighting
and autosuggestion scenarios. This is a one-element queue backed by a
single thread. New requests displace any existing queued request; this
reflects the fact that autosuggestions and highlighting only care about
the most recent result.

A timeout allows for abandoning hung threads, which may happen if you
attempt to e.g. access a dead hard-mounted NFS server. We don't want
this to defeat autosuggestions and highlighting permanently, so allow
spawning a new thread after the timeout (here 500 ms).
2020-03-06 17:15:21 -08:00
ridiculousfish
f6e5a5c521 Thread a cancellation function into is_potential_path
Allows sooner cancellation of redundant operations like highlighting.
2020-03-02 22:10:43 -08:00
ridiculousfish
bd17262e17 Reduce iteration count in thread torture test
Helps the test pass on 32 bit.
2020-03-02 13:46:08 -08:00
Johannes Altmanninger
8a033b9f3d Add undo
Add the input function undo which is bound to `\c_` (control + / on
some terminals). Redoing the most recent chain of undos is supported,
redo is bound to `\e/` for now.

Closes #1367.
This approach should not have the issues discussed in #5897.

Every single modification to the commandline can be undone individually,
except for adjacent single-character inserts, which are coalesced,
so they can be reverted with a single undo. Coalescing is not done for
space characters, so each word can be undone separately.

When moving between history search entries, only the current history
search entry is reachable via the undo history. This allows to go back
to the original search string with a single undo, or by pressing the
escape key.
Similarly, when moving between pager entries, only the most recent
selection in the pager can be undone.
2020-02-07 17:15:17 +01:00
ridiculousfish
057c3a9e75 Introduce fd_monitor
fd_monitor is a new class which can monitor a set of fds, waiting for them
to become readable. When an fd becomes readable, a callback is invoked.
Timeouts are also supported.

This is intended to replace the "bufferfill" threads. Rather than one
thread per bufferfill, we will have a single fd_monitor which can service
multiple bufferfills. This helps today with nested command substitutions,
and will help in the future with concurrent execution.
2020-02-05 12:04:51 -08:00
ridiculousfish
fc4557c784 Eliminate wopen()
It was large and mostly unnecessary. Prefer wopen() followed by
fdopen().
2020-01-28 10:43:37 -08:00
David Adam
dd1e526017 tests: remove an outdated comment
Code removed in af22d6b732
2020-01-28 16:54:29 +08:00
ridiculousfish
af22d6b732 Stop pretending to be interactive during the cancellation tests
Prior to this fix, the cancellation C++ test would mark the parser as
interactive in an effort to install interactive signal handling (so that,
for example, SIGINT would stop the job and return control to the user).

However this flag would also cause fish to attempt to save and restore tty modes
across the job. This would fail since there is no tty, and so the job would fail
with an unexpected error code.

We don't need to mark the parser as interactive, we can just remove that line.

Fixes #6539.
2020-01-27 12:13:46 -08:00
ridiculousfish
7a2a6d76f3 Migrate a test from C++ to littlecheck
This eliminates noisy warnings about tcsetpgrp when the tests are run
without a tty, as reported in #6539
2020-01-26 20:59:08 -08:00
ridiculousfish
38f4330683 Rationalize $status and errors
Prior to this fix, fish was rather inconsistent in when $status gets set
in response to an error. For example, a failed expansion like "$foo["
would not modify $status.

This makes the following inter-related changes:

1. String expansion now directly returns the value to set for $status on
error. The value is always used.

2. parser_t::eval() now directly returns the proc_status_t, which cleans
up a lot of call sites.

3. We expose a new function exec_subshell_for_expand() which ignores
$status but returns errors specifically related to subshell expansion.

4. We reify the notion of "expansion breaking" errors. These include
command-not-found, expand syntax errors, and others.

The upshot is we are more consistent about always setting $status on
errors.
2020-01-25 17:28:41 -08:00
ridiculousfish
81e78c78aa Rename eval_result_t to end_execution_reason_t
We're getting ready to stop returning eval_result_t from parser_t::eval
2020-01-25 14:11:32 -08:00
ridiculousfish
0f7bba5f0e Introduce operation_context_t
This commit recognizes an existing pattern: many operations need some
combination of a set of variables, a way to detect cancellation, and
sometimes a parser. For example, tab completion needs a parser to execute
custom completions, the variable set, should cancel on SIGINT. Background
autosuggestions don't need a parser, but they do need the variables and
should cancel if the user types something new. Etc.

This introduces a new triple operation_context_t that wraps these concepts
up. This simplifies many method signatures and argument passing.
2020-01-16 15:21:28 -08:00
ridiculousfish
db98ee13a9 Make complete() return the completion list directly
Returning it through a pointer was a remnant of pre C++-11 days.
2020-01-16 15:18:50 -08:00
ridiculousfish
4bb18eaf42 Use completion_list_t naming everywhere
std::vector<completion_t> -> completion_list_t
2020-01-16 15:18:49 -08:00
ridiculousfish
f1ce967dfa Do not allow empty items to be added to history
Empty items are used as sentinels to indicate that we've reached the end of
history, so they should not be added as actual items. Enforce this.

Fixes #6032
2020-01-05 12:47:02 -08:00
ridiculousfish
89880891d0 Revert "fish_tests to set HOME and other variables to temporary dir"
This reverts commit cca57a7a87.

The tests target already sets some variables - backing this out.
2020-01-01 17:24:14 -08:00
ridiculousfish
cca57a7a87 fish_tests to set HOME and other variables to temporary dir
Rather than placing files in the user's home directory, have fish_tests
manipulate HOME to be a temporary directory.
2020-01-01 16:34:56 -08:00
ridiculousfish
b784a0caa3 dup2_list_t::resolve_chain to stop returning maybe
It can no longer fail.
2019-12-29 14:49:05 -08:00
Mathieu Duponchelle
15c1b3ed4b Place fish in its own process group when launched with -i
Fixes #5909
2019-12-23 10:32:37 +01:00
ridiculousfish
c19407ab0f Default parser_t::eval()'s block type to top
This is the parameter value at every call site except one. Just make it the
default.
2019-12-22 16:27:03 -08:00
ridiculousfish
a59f35a378 Make block_type_t an enum class 2019-12-22 15:37:14 -08:00
ridiculousfish
0531c02ce4 Remove 'user_supplied' flag for io_fd_t
user_supplied was used to distinguish IO redirections which were
explicit, vs those that came about through "transmogrphication." But
transmogrification is no more. Remove the flag.
2019-12-19 14:14:23 -08:00
ridiculousfish
311f47d683 Remove an errant fprintf from the tests 2019-12-17 18:26:57 -08:00
ridiculousfish
b3d2cdc0ff Unify parse_execution_result_t and eval_result_t again
Do other cleanup to better express the difference between cancellation
and control flow.
2019-12-17 18:12:49 -08:00
ridiculousfish
72de8768d9 Remove RUNNING_IN_XCODE checks
This is never set any more.
2019-12-17 16:19:12 -08:00
ridiculousfish
09f8e05b80 Clean up the return type of parser_t::eval
parser_t::eval indicates whether there was a parse error. It can be
easily confused with the status of the execution. Use a real type to
make it more clear.
2019-12-17 16:16:43 -08:00
ridiculousfish
8bf9f52461 Always detach new pthreads
There are no longer any calls to pthread_join. Just make all pthreads
detached.
2019-12-16 14:08:46 -08:00
ridiculousfish
33aff87c10 Switch io_file_t to store an fd, not a path
Prior to this fix, a file redirection was turned into an io_file_t. This is
annoying because every place where we want to apply the redirection, we
might fail due to open() failing. Switch to opening the file at the point
we resolve the redirection spec. This will simplify a lot of code.
2019-12-12 17:34:40 -08:00
ridiculousfish
be685faeb8 Clean up how pipe fd avoidance works
fish has to ensure that the pipes it creates do not conflict with any
explicit fds named in redirections. Switch this code to using
autoclose_fd_t to make the ownership logic more explicit, and also
introduce fd_set_t to reduce the dependence on io_chain_t.
2019-12-12 14:58:18 -08:00
Johannes Altmanninger
7d5b44e828 Support FOO=bar syntax for passing variables to individual commands
This adds initial support for statements with prefixed variable assignments.
Statments like this are supported:

a=1 b=$a echo $b        # outputs 1

Just like in other shells, the left-hand side of each assignment must
be a valid variable identifier (no quoting/escaping).  Array indexing
(PATH[1]=/bin ls $PATH) is *not* yet supported, but can be added fairly
easily.

The right hand side may be any valid string token, like a command
substitution, or a brace expansion.

Since `a=* foo` is equivalent to `begin set -lx a *; foo; end`,
the assignment, like `set`, uses nullglob behavior, e.g. below command
can safely be used to check if a directory is empty.

x=/nothing/{,.}* test (count $x) -eq 0

Generic file completion is done after the equal sign, so for example
pressing tab after something like `HOME=/` completes files in the
root directory
Subcommand completion works, so something like
`GIT_DIR=repo.git and command git ` correctly calls git completions
(but the git completion does not use the variable as of now).

The variable assignment is highlighted like an argument.

Closes #6048
2019-11-25 09:20:51 +01:00
ridiculousfish
e18fd3cddb Allow unclosed subshells in interactive mode
If the user has an unclosed subshell in interactive mode, break the
line instead of producing an error.

Fixes #6316
2019-11-13 18:01:47 -08:00
ridiculousfish
ec08a50769 Eliminate function_data_t
This struct is now mostly useless and can go.
2019-11-12 11:25:41 -08:00
ridiculousfish
6d7a66592b Make function_add take the filename directly instead of a parser 2019-11-12 10:00:42 -08:00
ridiculousfish
896ef65f8c Rename error_offset to error_offset_within_token
Hopefully clarify the role of this variable.
2019-11-08 16:56:50 -08:00
ridiculousfish
ef8b5e4fa0 Correct certain tokenizer error reporting for unclosed subshells
There was some confusion about the different pointers and offsets
in tokenizer_t::call_error.

Fixes #6281
2019-11-08 16:56:43 -08:00
Johannes Altmanninger
e94b9ccf3e Do import bash history commands containing && or || 2019-11-07 23:33:35 +01:00
ridiculousfish
52e900690b Make history_search_type_t an enum class 2019-11-02 18:56:16 -07:00
ridiculousfish
eeac3333df Correctly highlight input following a tokenizer error 2019-10-27 16:08:49 -07:00
ridiculousfish
afd20b8e1a Correctly report the range of tokenizer errors
This enables proper syntax highlighting of tokenizer errors.
2019-10-27 16:05:37 -07:00
ridiculousfish
2a92e66902 Support for &> and &| as convenience redirections
This adds support for &> and &| syntax, which both redirect stdout, and
also apply a redirection of stderr to stdout.
2019-10-27 15:24:57 -07:00
LawAbidingCactus
305a657694 fix typos 2019-10-23 19:38:44 +02:00
ridiculousfish
35671dd9f0 Clean up and unify pipes and redirections
This cleans up how pipes and redirections are recognized by the parser,
and unifies pipes and redirections into a single type.
2019-10-15 11:26:41 -07:00
ridiculousfish
1a65e18ba8 Clean up some tokenization
Remove TOK_NONE
Turn token_type into an enum class
Make next() turn a maybe_t<tok_t> instead of a bool
2019-10-13 16:06:16 -07:00
ridiculousfish
82eca4bc86 Run clang-format on all files
The main change here is to reorder headers.
2019-10-13 15:50:48 -07:00
ridiculousfish
a19d9b2e73 Add a test for #6130 2019-09-22 14:06:38 -07:00
Johannes Altmanninger
8baea8b157 Do not add a space after completion if one is already there
Example: type `cd --help --help`, move the cursor inside the first `--help` and press tab.
This used to add redundant spaces.
2019-09-18 14:02:24 -05:00
ridiculousfish
535a448d0e Remove an errant fprintf 2019-09-08 15:44:07 -07:00
ridiculousfish
b2fe2f9ff3 Correct escaping and add tests for bracket completion
Add some tests for new bracket completion behavior, and fix an escaping
bug where \\[ was incorrectly marked as escaping.
2019-09-08 15:43:20 -07:00
ridiculousfish
0dc5eaeb97 reader_expand_abbreviation_in_command to return maybe_t<wcstring> 2019-08-06 16:09:23 -07:00
ridiculousfish
822b53c67a Don't color a whole string invalid because of an unclosed quote
When syntax highlighting a quoted string, if the string is not closed,
only show the opening quote as an error, not the whole string.
2019-07-24 12:43:47 -07:00
ridiculousfish
8ec568d21f Add some tests to path_apply_working_directory 2019-07-15 17:56:06 -07:00
ridiculousfish
0e4fe53718 clang-format some files 2019-07-06 13:11:50 -07:00
ridiculousfish
bc0329f775 Add wcstolower
Converts a string to lowercase. Eliminates some loops that did this
explicitly.
2019-07-06 12:19:34 -07:00
ridiculousfish
4a2c709fb1 Eliminate shell_is_interactive
We used to have a global notion of "is the shell interactive" but soon we
will want to have multiple independent execution threads, only some of
which may be interactive. Start tracking this data per-parser.
2019-06-29 11:28:26 -07:00
Fabian Homborg
caedf01c00 Revert "Latch signal handlers"
This reverts commit 7ed1022cf4.

Fixes #5962.
2019-06-25 11:25:09 +02:00
ridiculousfish
be97499106 Revert "Revert "Add a test for autoload_t""
This reverts commit 2507162f80.

Put back the autoload test.
2019-06-07 12:12:43 -07:00
ridiculousfish
fc99d6c7af clang-format all files 2019-06-03 20:30:48 -07:00
ridiculousfish
890c1188ab Instantize and rationalize locking around the input mapping set
Hypothetically a background thread could invoke builtin_bind, etc.
Protect the set of input bindings with a lock.
2019-06-02 22:54:01 -07:00
ridiculousfish
671df14178 Thread a parser into inputter_t 2019-06-02 16:41:13 -07:00
ridiculousfish
a48dbf23b8 Instantize the character event queue
Wrap this stuff up in structs so it's no longer global.
2019-06-02 16:37:05 -07:00
Fabian Homborg
61292b0c6c fish_tests: Fix warnings 2019-05-29 21:11:08 +02:00
David Adam
c848787107 add extra test for wcsfilecmp
Test changes added for #5634 in 6e525cc5d9.
2019-05-28 22:03:03 +08:00
ridiculousfish
7ed1022cf4 Latch signal handlers
Now that our interactive signal handlers are a strict superset of
non-interactive ones, there is no reason to "reset" signals or take action
when becoming non-interactive. Clean up how signal handlers get installed.
2019-05-26 18:04:03 -07:00
Dawid Dziurla
0b3bb0e7c1 Underline every valid entered path (#5872)
* src/highlight: Underline every valid entered path

* update CHANGELOG

* fix highlight test
2019-05-19 10:03:56 +02:00
ridiculousfish
c44dae2d73 Migrate certain runtime flags to atomics hidden behind functions 2019-05-18 18:50:28 -07:00
ridiculousfish
1719d6f136 Make $status and $pipestatus per-parser
Another step towards allowing multiple parsers to execute in parallel.
2019-05-12 14:00:44 -07:00
ridiculousfish
cfddd881ef Make PWD a per-processor variable
Handle this variable specially.
2019-05-11 17:13:34 -07:00
ridiculousfish
8a8b2513b5 Eliminate the global jobs() function
All job lists are attached to a parser now.
2019-05-05 11:33:08 -07:00
Fabian Homborg
c2970f9618 Reformat all files
This runs build_tools/style.fish, which runs clang-format on C++, fish_indent on fish and (new) black on python.

If anything is wrong with the formatting, we should fix the tools, but automated formatting is worth it.
2019-05-05 12:09:25 +02:00
ridiculousfish
f66e010949 Turn a lot of common.h variables into getter functions
Improves thread safety.
2019-05-04 20:58:35 -07:00
ridiculousfish
bffacd2fbf Thread a parser into expansion
Expansion may perform command substitution, which needs to know the parser
to use.
2019-05-04 19:30:00 -07:00
ridiculousfish
923a7ca0f0 Thread the parser into complete()
Eliminates uses of principal_parser
2019-05-04 18:17:18 -07:00
ridiculousfish
63bdc949ab Make completion request flags an enum_set 2019-05-04 17:55:57 -07:00
ridiculousfish
ec45f31ad1 Make debug_level an atomic
Fixes a tsan warning
2019-05-04 15:28:44 -07:00
ridiculousfish
d8ac051f89 Move selection_direction_t to pager.h and make it a class enum 2019-04-28 14:06:03 -07:00
ridiculousfish
2507162f80 Revert "Add a test for autoload_t"
This reverts commit 51c62d6cc6.

Back out the test while I attempt to fix it
2019-04-27 20:14:06 -07:00
ridiculousfish
51c62d6cc6 Add a test for autoload_t 2019-04-27 16:16:48 -07:00
ridiculousfish
96bc8a14ca Promote completion_mode_t to a real type
Eliminate big #defines like NO_COMMON.
2019-04-25 14:21:06 -07:00
ridiculousfish
496529b20a Remove EXPAND prefix from expand_flags and lowercase them 2019-04-25 11:34:49 -07:00
ridiculousfish
d8ab6290e8 Switch expand_flags_t to enum_set 2019-04-25 11:23:03 -07:00
ridiculousfish
dcaac58f45 Rename expand_error_t to expand_result_t and make it class enum
Also lowercase it all.
2019-04-25 10:47:28 -07:00
ridiculousfish
020d4a2848 Adopt env_scoped_t::snapshot() and remove env_var_snapshot_t
Remove the env_var_snapshot_t class and switch everything to the new snapshot
function of env_scoped_t.

Fixes #5658. Fixes #5571.
2019-04-14 15:50:38 -07:00
Mahmoud Al-Qudsi
05f52924c1 Fix bad tests
These tests used raw, unescaped parentheses to perform `test` logical
grouping, but the test failures weren't caught because the parser
evaluation errors were not being propagated (fixed in bdbd173e).
2019-04-13 17:28:16 -05:00
ridiculousfish
e7d7eff0ee Rename all readline commands to lowercase and remove R_ prefix 2019-03-23 23:31:35 -07:00
ridiculousfish
2c56e27d37 Switch readline commands to readline_cmd_t enum class 2019-03-23 23:31:25 -07:00
ridiculousfish
c2be5e8986 Introduce char_event_type_t::readline
Baby steps towards eliminating readline actions as characters.
2019-03-23 23:31:18 -07:00
ridiculousfish
185805641c Remove R_TIMEOUT
Promote timeout to a char_event_type_t, moving it out of the "char" namespace.
This will help simplify the readline implementation.
2019-03-23 20:10:06 -07:00
ridiculousfish
a58662dd46 Make maybe_t conditionally copyable
This allows it to be used with both e.g. unique_ptr and std::vector.
2019-03-17 13:38:18 -07:00
Aaron Gyes
d837eee09d remove some wcstring -> wchar_t* -> wcstring conversions
Mostly related to usage _(L"foo"), keeping in mind the _
macro does a wcstring().c_str() already.

And a smattering of other trivial micro-optimizations certain
to not help tangibly.
2019-03-14 15:21:08 -07:00
Aaron Gyes
477b2e8d7c std::vector<wcstring> is wcstring_list_t 2019-03-14 11:17:26 -07:00
Aaron Gyes
aaacdb89b6 Switches over to cstring from string.h. 2019-03-12 15:09:36 -07:00
Aaron Gyes
d5ac239f68 This commit changes wchar.h includes to cwchar, and uses std::
for everything it provides.
2019-03-12 15:09:36 -07:00
ridiculousfish
9b9c1aa745 Fix some unused variable warnings 2019-03-03 18:06:31 -08:00
ridiculousfish
717ac9a8d5 Switch highlight_spec_t to a struct
Rather than a janky bitmask, use a real struct with real fields.
2019-03-03 18:04:22 -08:00
Aaron Gyes
6fa8b028fc fish_tests.cpp: fixup: I didn't notice the comma here. 2019-02-18 23:19:57 -08:00
Aaron Gyes
c2bc0c67f2 Don't use printf("%d") just to convert an int to a string.
std::to_string, std::to_wstring are more appropriate
2019-02-18 23:15:54 -08:00
Fabian Homborg
7958e1d5c4 fish_tests: s/rand()/random()/g
As it turns out, NetBSD's rand(3) is awful - it's possible that in any
given run it'll only return odd numbers, which means

    while (rand() % 10)

will never stop.

Since random(3) is also standardized and works, let's use that!
2019-02-18 14:46:11 +01:00
ridiculousfish
ada8ea954e Use "internal" processes to write buffered output
This introduces "internal processes" which are backed by a pthread instead
of a normal process. Internal processes are reaped using the topic
machinery, plugging in neatly alongside the sigchld topic; this means that
process_mark_finished_children() can wait for internal and external
processes simultaneously.

Initially internal processes replace the forked process that fish uses to
write out the output of blocks and functions.
2019-02-17 13:05:20 -08:00
ridiculousfish
061f8f49c6 Add dup2_list_t::fd_for_target_fd
This adds an "in-process" interpretation of dup2s, allowing for fish to
output directly to the correct file descriptor without having to perform
an in-kernel dup2 sequence.
2019-02-17 13:01:59 -08:00
ridiculousfish
a4dc04a28e Add sighupint topic
This corresponds to SIGHUP and SIGINT. This will be used to break out of
process_mark_finished_children().
2019-02-17 13:01:59 -08:00
ridiculousfish
fc9d238642 Introduce topic monitoring
topic_monitor allows for querying changes posted to one or more topics,
initially sigchld. This will eventually replace the waitpid logic in
process_mark_finished_children().

Comment from the new header:

Topic monitoring support. Topics are conceptually "a thing that can
happen." For example, delivery of a SIGINT, a child process exits, etc. It
is possible to post to a topic, which means that that thing happened.

Associated with each topic is a current generation, which is a 64 bit
value. When you query a topic, you get back a generation. If on the next
query the generation has increased, then it indicates someone posted to
the topic.

For example, if you are monitoring a child process, you can query the
sigchld topic. If it has increased since your last query, it is possible
that your child process has exited.

Topic postings may be coalesced. That is there may be two posts to a given
topic, yet the generation only increases by 1. The only guarantee is that
after a topic post, the current generation value is larger than any value
previously queried.

Tying this all together is the topic_monitor_t. This provides the current
topic generations, and also provides the ability to perform a blocking
wait for any topic to change in a particular topic set. This is the real
power of topics: you can wait for a sigchld signal OR a thread exit.
2019-02-17 13:01:59 -08:00
ridiculousfish
ccc45235b0 Introduce enum_array_t
Allows for indexing an array via an enum class.
2019-02-17 13:01:59 -08:00
ridiculousfish
78ed659151 Fancify enum_set and introduce enum_iter_t
Allow iterating over the values of an enum class.
2019-02-17 13:01:59 -08:00
Fabian Homborg
fdbbe9f69d fish_tests: Initialize some collections
For some reason this'd crash on NetBSD otherwise.
2019-02-16 16:40:13 +01:00
David Adam
a042a4cb62 low level tests: set pwd from getcwd before starting
Fixes the tests in Debian pbuilder environments. Closes #5599.
2019-02-06 06:51:33 +08:00
ridiculousfish
6f682c8405 Fill io_buffer via background thread
This is a large change to how io_buffers are filled. The essential problem
comes about with code like (example):

    echo ( /bin/pwd )

The output of /bin/pwd must go to fish, not the tty. To arrange for this,
fish does the following:

1. Invoke pipe() to create a pipe.
2. Add an io_bufferfill_t redirection that owns the write end of the pipe.
3. After fork (or equiv), call dup2() to replace pwd's stdout with this  pipe.

Now when /bin/pwd writes, it will send output to the read end of the pipe.
But who reads it?

Prior to this fix, fish would do the following in a loop:

1. select() on the pipe with a 10 msec timeout
2. waitpid(WNOHANG) on the pwd proc

This polling is ugly and confusing and is what is replaced here.

With this new change, fish now reads from the pipe via a background thread:

1. Spawn a background pthread, which select()s on the pipe's read end with
a long (100 msec) timeout.
2. In the foreground, waitpid() (allowing hanging) on the pwd proc.

The big win here is a major simplification of job_t::continue_job() since
it no longer has to worry about filling buffers. This will make things
easier for concurrent execution.

It may not be obvious why the background thread still needs a poll (100 msec).
The answer is for cases where the write end of the fd escapes, in particular
background processes invoked inside command substitutions. psub is perhaps
the only important case of this (other shells typically just hang here).
2019-02-03 01:58:49 -08:00
ridiculousfish
ec29a5b913 Introduce make_pthread
This allows creating a pthread directly, which can be joined.
iothread_spawn wraps this.
2019-02-03 01:58:49 -08:00
ridiculousfish
178b72b2fd io_buffer_t becomes io_bufferfill_t
This makes some significant architectual improvements to io_pipe_t and
io_buffer_t.

Prior to this fix, io_buffer_t subclassed io_pipe_t. io_buffer_t is now
replaced with a class io_bufferfill_t, which does not subclass pipe.

io_pipe_t no longer remembers both fds. Instead it has an autoclose_fd_t,
so that the file descriptor ownership is clear.
2019-02-03 01:58:49 -08:00
ridiculousfish
dbe906b79e Introduce dup2_list_t
This represents a "resolved" io_chain_t, where all of the different io_data_t
types have been reduced to a sequence of dup2() and close(). This will
eliminate a lot of the logic duplication around posix_spawn vs fork, and pave
the way for in-process redirections.
2019-02-03 01:58:49 -08:00
ridiculousfish
6ba0d4c88a Revert io_bufferfill_t stack
This reverts commit 88dc484858 onwards.
2019-02-02 17:53:40 -08:00
ridiculousfish
9a4153f5e2 Fill io_buffer via background thread
This is a large change to how io_buffers are filled. The essential problem
comes about with code like (example):

    echo ( /bin/pwd )

The output of /bin/pwd must go to fish, not the tty. To arrange for this,
fish does the following:

1. Invoke pipe() to create a pipe.
2. Add an io_bufferfill_t redirection that owns the write end of the pipe.
3. After fork (or equiv), call dup2() to replace pwd's stdout with this  pipe.

Now when /bin/pwd writes, it will send output to the read end of the pipe.
But who reads it?

Prior to this fix, fish would do the following in a loop:

1. select() on the pipe with a 10 msec timeout
2. waitpid(WNOHANG) on the pwd proc

This polling is ugly and confusing and is what is replaced here.

With this new change, fish now reads from the pipe via a background thread:

1. Spawn a background pthread, which select()s on the pipe's read end with
a long (100 msec) timeout.
2. In the foreground, waitpid() (allowing hanging) on the pwd proc.

The big win here is a major simplification of job_t::continue_job() since
it no longer has to worry about filling buffers. This will make things
easier for concurrent execution.

It may not be obvious why the background thread still needs a poll (100 msec).
The answer is for cases where the write end of the fd escapes, in particular
background processes invoked inside command substitutions. psub is perhaps
the only important case of this (other shells typically just hang here).
2019-02-02 14:21:46 -08:00
ridiculousfish
6e0dd06f43 Introduce make_pthread
This allows creating a pthread directly, which can be joined.
iothread_spawn wraps this.
2019-02-02 14:21:46 -08:00
ridiculousfish
78bbcef356 io_buffer_t becomes io_bufferfill_t
This makes some significant architectual improvements to io_pipe_t and
io_buffer_t.

Prior to this fix, io_buffer_t subclassed io_pipe_t. io_buffer_t is now
replaced with a class io_bufferfill_t, which does not subclass pipe.

io_pipe_t no longer remembers both fds. Instead it has an autoclose_fd_t,
so that the file descriptor ownership is clear.
2019-02-02 14:21:46 -08:00
ridiculousfish
88dc484858 Introduce dup2_list_t
This represents a "resolved" io_chain_t, where all of the different io_data_t
types have been reduced to a sequence of dup2() and close(). This will
eliminate a lot of the logic duplication around posix_spawn vs fork, and pave
the way for in-process redirections.
2019-02-02 14:21:46 -08:00
ridiculousfish
b00f039489 Clean up the io_chain_t interface 2019-01-31 18:49:52 -08:00
ridiculousfish
a333c2f01d Fix some compile warnings 2019-01-10 20:59:47 -08:00
ridiculousfish
77884bc21a Instantize env_get
This removes env_get(). All fish variable accesses must go through an
environment_t.
2019-01-10 20:08:06 -08:00
ridiculousfish
6f52e6bb1c Instantize contents of exec.cpp and others 2019-01-10 20:07:47 -08:00
ridiculousfish
038f3cca6d Remove the abbreviation cache
Read abbreviations directly from the environment.
2019-01-10 20:07:41 -08:00
ridiculousfish
9f62a53077 Instantize env_get inside highlighting 2019-01-10 20:07:35 -08:00
ridiculousfish
c1dd284b3e Instantize env_set
Switch env_set to an instance method on environmnet_t.
2019-01-10 20:05:45 -08:00
ridiculousfish
421fbdd52a Instantize env_get_pwd_slash
This requires threading environment_t through many places, such as completions
and history. We introduce null_environment_t for when the environment isn't
important.
2019-01-10 20:01:28 -08:00
ridiculousfish
a00de96a57 Instance env_remove 2019-01-10 20:01:15 -08:00
ridiculousfish
5055621e02 Eliminate env_push and env_pop 2019-01-10 20:29:10 -08:00
ridiculousfish
94adb53b1f Eliminate complete_set_variable_names 2019-01-10 20:29:10 -08:00
ridiculousfish
e6b13c6bac Begin to thread environments explicitly through completions 2019-01-10 20:29:10 -08:00
ridiculousfish
8d7cae63ff Introduce env_stack_t
This will instance environment variable stacks.
2019-01-10 20:29:10 -08:00
ridiculousfish
895c2c4af0 Minor cleanup of parser interface 2019-01-10 20:29:10 -08:00
David Adam
d518b01281 fish_tests.cpp: mock the home directory
Removes the dependency on the current user's home directory, instead
overriding it to be within the current hierarchy.

Fixes the tests on Debian buildd, where the home directory is
deliberately unwriteable to pick up errors in builds.
2019-01-10 21:22:44 +08:00
Fabian Homborg
b8b0c39c77 fish_tests: Use std::isnan
Fixes the tests on Ubuntu 16.04 "xenial".
2019-01-05 12:58:52 +01:00
Fabian Homborg
742fde0dd6 Don't use less in highlighting test
It doesn't have to be installed.

`cat` is in our dependencies, so we can assume it's there.

Fixes #5436.
2018-12-28 17:57:53 +01:00
Aaron Gyes
1adcd2d591 builtin_test: don't exit 1 for eval errors, add tests for big args
Return STATUS_INVALID_ARGS when failing due to evaluation errors,
so we can tell the difference between an error and falseness.

Add a test for the ERANGE error
2018-12-15 22:05:19 -08:00
ridiculousfish
a8ce7bad7b Always pass in the working directory in path_get_cdpath
If the user is in a directory which has been unlinked, it is possible
for the path .. to not exist, relative to the working directory.
Always pass in the working directory (potentially virtual) to
path_get_cdpath; this ensures we check absolute paths and are immune
from issues if the working directory has been unlinked.

Also introduce a new function path_normalize_for_cd which normalizes the
"join point" of a path and a working directory. This allows us to 'cd' out of
a non-existent directory, but not cd into such a directory.

Fixes #5341
2018-11-18 14:36:42 -08:00
Mahmoud Al-Qudsi
31d17f4559 Rename string escape --style=pcre2 to string escape --style=regex 2018-11-16 20:22:06 -06:00
Mahmoud Al-Qudsi
e160cde606 Implement PCRE2 escaping
Closes #5309.
2018-11-15 12:00:56 -06:00
ridiculousfish
bfd50863b8 Correct fish_wcstod and fix Linux build failure
Limit the fish_wcstod fast path to ASCII digits only, to fix the problem
observed in the discussion for a700acadfa
where LANG=de_DE.UTF-8 would cause `test` to interpret commas instead of
periods inside floating point values.
2018-11-06 23:17:41 -08:00
ridiculousfish
a700acadfa Implement fish_wcstod and adopt it in builtin_test
wcstod_l is enormously slow on the Mac. This makes arithmetic comparisons
using builtin_test about 250% as fast on macOS.
2018-11-04 20:28:10 -08:00
Mahmoud Al-Qudsi
1015e74480 Treat _ and - alike for case-insensitive fuzzy matching
Closes #3584.
2018-10-28 10:35:32 -05:00
ridiculousfish
5899694233 Allow setting universal path variables
Support for path and unpath in universal variables.
Fixes #5271
2018-10-27 01:05:00 -07:00
ridiculousfish
7b5296817a Teach universal variables to not overwrite future file formats 2018-10-26 16:06:50 -07:00
ridiculousfish
d18e2d970c Switch to new universal variable format
Example line:

SETUVAR --export foo:bar
2018-10-26 16:06:50 -07:00
ridiculousfish
adc69f94da Support parsing the new universal variable format 2018-10-26 16:06:50 -07:00
ridiculousfish
d98874bd08 Improve testability factoring of env_universal_t 2018-10-26 16:06:47 -07:00
ridiculousfish
11d523e61a Build out support for multiple file formats in uvars
This is in preparation for adjusting the file format to support path
variables.
2018-10-21 15:56:05 -07:00
ridiculousfish
ce1463bde6 Add line_iterator_t
Adds support for splitting a collection into lines.
2018-10-21 15:56:05 -07:00
ridiculousfish
a23894ca37 Simplify callback_data_t
SET_EXPORT no longer makes sense; remove it.
2018-10-21 15:56:05 -07:00
ridiculousfish
ac241b7132 Simplify and add tests for ifind 2018-10-21 15:53:58 -07:00
ridiculousfish
fcd4a44b98 Correct check and add a basic test for fuzzy_match_substring_case_insensitive 2018-10-21 12:02:38 -07:00
ridiculousfish
5fa4e0d2ee Highlight %self as an operator 2018-10-19 16:17:30 -07:00
ridiculousfish
202bf0bede Tab complete abbreviations
This allows abbreviations to be expanded by tab completions.

Fixes #3233
2018-10-16 16:15:27 -07:00
ridiculousfish
c76de58758 Normalize "/" path to "/"
This is one obnoxious function.

Fixes #5250
2018-10-12 22:15:16 -07:00
Mahmoud Al-Qudsi
8c3481a921 Block the history race test from running under WSL
It's always failing, and the current test scripts refuse to progress
after the first failure, making it impossible to test fish under WSL.
2018-10-11 18:54:32 -05:00
ridiculousfish
e363171b8d Switch from strdup to std::string in fish_tests 2018-10-11 09:58:16 -07:00
ridiculousfish
0f0bb1e10f Don't resolve symlinks with cd
This switches fish to a "virtual" PWD, where it no longer uses getcwd to
discover its PWD but instead synthesizes it based on normalizing cd against
the $PWD variable.

Both pwd and $PWD contain the virtual path. pwd is taught about -P to
return the physical path, and -L the logical path (which is the default).

Fixes #3350
2018-10-06 17:03:18 -07:00
ridiculousfish
6ad4d94e12 Introduce path_normalize
This new function performs normalization of paths including dropping
/./ segments, and resolving /../ segments, in preparation for switching
fish to a "virtual" PWD.
2018-10-06 17:03:18 -07:00
Mahmoud Al-Qudsi
f702c42068 Sanitize history item whitespace
Coalesces commands with leading (if even possible) and trailing
whitespace into the same item, improving the experience when iterating
over history entries.

Closes #4908.
2018-10-01 17:12:18 -05:00
ridiculousfish
a1728d61af Report errors on invalid replacements in string replace
If the replacement in `string replace` is invalid, prior to this fix we would
enter into an infinite loop trying to parse it. Instead report errors correctly.

Fixes #3381
2018-09-27 22:28:39 -04:00
ridiculousfish
ae54b66799 Fix string 2018-09-27 22:23:03 -04:00
ridiculousfish
cc99e8d510 Switch tokenizer_error back to just an error code
Rather than having tokenizer_error as pointers to objects, switch it back
to just an error code value. This makes reasoning about it easier since
it's immutable values instead of mutable objects, and it avoids allocation
during startup.
2018-09-27 21:40:51 -04:00
ridiculousfish
f3da54d99c Convert some iterators to C++-11 range-based loops 2018-09-16 15:49:18 -07:00
ridiculousfish
1a9e3761ae Add highlighting tests for empty variables 2018-09-01 12:05:52 -07:00
ridiculousfish
7bd26f9ff0 Teach syntax highlighting about variables in commands 2018-09-01 12:00:56 -07:00
ridiculousfish
865a4647ae Allow variables in commands
Syntax highlighting for these coming in next commit.

Fixes #154
2018-09-01 12:00:56 -07:00
ridiculousfish
59d78e8afa Clean up syntax highlighting test 2018-09-01 10:17:57 -07:00
ridiculousfish
d87f00bdc9 Simplify history search
Remove features related to navigating forwards in history that are no
longer used.
2018-08-12 02:41:09 -07:00
ridiculousfish
e51e854d8d Clean up reader history search
Factor the history search fields into a new class.
As a side effect, this shares the deduplication logic, so that token search
no longer returns duplicates.

Fixes #4795
2018-08-12 02:40:15 -07:00
ridiculousfish
9c957eeef3 Prefer to not autosuggest existing arguments
This teaches autosuggestions to demote completions whose text matches an already
present argument.
2018-08-07 02:04:43 -07:00
ridiculousfish
2b0c54dfc8 Ignore return value of system() in tests
Fixes some warnings on Linux
2018-08-04 16:33:31 -07:00
ridiculousfish
acff2516d4 Straighten out some wchar_t** casts
Embrace the fact that builtins expect to modify their argv array and get rid
of a bunch of const.
2018-08-04 16:25:51 -07:00
ridiculousfish
40e37c4a87 Teach test to handle floating point values 2018-08-03 00:48:50 -07:00
ridiculousfish
57cc3d950d Remove some unnecessary test output 2018-07-21 15:31:14 -07:00
ridiculousfish
f998afaa23 Adopt separated_buffer_t in io_buffer_t 2018-07-01 15:56:33 -07:00
ridiculousfish
90a4af5112 Add separated_buffer_t and adopt it in output_stream_t
separated_buffer_t encapsulates the logic around discarding (which
was previously duplicated between output_stream_t and io_buffer_t),
and will also encapsulate the logic around explicitly separated
output.
2018-07-01 15:56:33 -07:00
Mahmoud Al-Qudsi
dcced5f1bc Replace \e with \x1B, as the former is a gcc extension
While supported by gcc and clang, \e is a gcc-specific extension and not
formally defined in the C or C++ standards.

See [0] for a list of valid escapes.

[0]: https://stackoverflow.com/a/10220539/17027
2018-06-18 00:01:47 -05:00
ridiculousfish
762c31be87 Feature flag support for ? wildcard
This partially reverts 6e56637cf0 and #4520
by bringing back the ? wildcard, guarded by the qmark-noglob feature flag.
2018-05-06 11:20:15 -07:00
ridiculousfish
902af26253 Bring back caret redirections under a feature flag
This partially reverts 5b489ca30f, with
carets acting as redirections unless the stderr-nocaret flag is set.
This flag is off by default but may be enabled on the command line:

fish --features stderr-nocaret
2018-05-06 11:20:14 -07:00
ridiculousfish
8a96f283ba Allow setting feature flags on the command line
This introduces a new command line option --features which can be used for
enabling or disabling features for a particular fish session.

Examples:
  fish --features stderr-nocaret
  fish --features 3.0,no-stderr-nocaret
  fish --features all

Note that the feature set cannot be changed in an existing session.
2018-05-06 11:20:14 -07:00
ridiculousfish
14f766b66d Add support for feature flags
This introduces a new type features_t that exposes feature flags. The intent
is to allow a deprecation/incremental adoption path. This is not a general
purpose configuration mechanism, but instead allows for compatibility during
the transition as features are added/removed.

Each feature has a user-presentable short name and a short description. Their
values are tracked in a struct features_t.

We start with one feature stderr_nocaret, but it's not hooked up yet.
2018-05-06 11:20:14 -07:00
ridiculousfish
ff10e504a1 Remove MAC address from universal variables file
This switches the universal variables file from a machine-specific
name to the fixed '.config/fish/fish_universal_variables'. The old file
name is migrated if necessary.

Fixes #1912
2018-04-01 17:43:12 -07:00
ridiculousfish
222a45f07a Add acquire() to maybe_t
Easy way to pull the value out.
2018-04-01 16:10:59 -07:00
Peter Ammon
5b489ca30f Remove caret redirection
This removes the caret as a shorthand for redirecting stderr.

Note that stderr may be redirected to a file via 2>/some/path...
and may be redirected with a pipe via 2>|.

Fixes #4394
2018-04-01 13:48:21 -07:00
ridiculousfish
6e56637cf0 Remove support for the ? wildcard
Fixes #4520
2018-03-31 16:54:50 -07:00
Mahmoud Al-Qudsi
7447432471 Fix tests for new ) token error 2018-03-11 17:16:53 -05:00
ridiculousfish
7764f27170 Correct failure to set 'filled' flag in maybe_t constructors 2018-03-10 02:17:35 -08:00
Mahmoud Al-Qudsi
f7e0cbc7a4 Drop % test for illegal commands
% is perfectly valid in commands, now :)
2018-03-09 04:02:29 -06:00
ridiculousfish
c7f16439bf Add support for ! as an analog to 'not'
! and not are effectively interchangeable now.
Mark them both as operators for syntax highlighting.
2018-03-05 14:04:49 -08:00
ridiculousfish
f83742d579 Highlight && and || as operators
This also switches 'and' and 'or' to operators as well.
2018-03-05 13:51:05 -08:00
ridiculousfish
23d4f93556 Add && and || to fish grammar
This teaches the parser about && and ||, implemented via a new
production "job_conjunction".

These do not yet have execution support.
2018-03-05 12:20:56 -08:00
ridiculousfish
8ded041352 Add && and || support to tokenizer 2018-03-05 12:20:56 -08:00
ridiculousfish
a5dd96558f Remove special && and || error detection
Soon these will no longer be errors.
2018-03-05 12:20:56 -08:00
ridiculousfish
bb7b649132 Wrapping completions to allow injecting arguments
This enables some limited use of arguments for wrapping completions. The
simplest example is that complete gco -w 'git checkout' now works like
you would want: `gco <tab>` now invokes git's completions with the
`checkout` argument prepended.

Fixes #1976
2018-02-27 14:12:44 -08:00
Fabian Homborg
c793570f2c Fix punctuation movement with one char tokens
Previously, in

    ls ^a bcd

(with "^" as the cursor), kill-word would delete the "a" and then go
on, remove the space and the "bcd".

With this, it will only kill the "a".

Fixes #4747.
2018-02-27 22:56:15 +01:00
ridiculousfish
99fb7bb6aa Refactor how redirections are represented by the tokenizer
Prior to this fix, each redirection type was a separate token_type.
Unify these under a single type TOK_REDIRECT and break the redirection
type out into a new sub-type redirection_type_t.
2018-02-23 15:19:58 -08:00
ridiculousfish
6673fe5457 Clean up tokenizer implementation
Rather than storing a bunch of "next_foo" fields, simply populate the
tok_t directly.
2018-02-23 14:31:13 -08:00
ridiculousfish
b13ee818d2 Some early cleanup of tokenizer
Prior to this the tokenizer ran "one ahead", where tokenizer_t::next()
would in fact return the last-parsed token. Switch to parsing on demand
instead of running one ahead; this is simpler and prepares for tokenizer
changes.
2018-02-19 15:10:59 -08:00
ridiculousfish
74e6a82849 Remove explicit 'void' parameters. 2018-02-18 19:12:45 -08:00
ridiculousfish
fd2a0dffa9 Reflect newlines after pipes in fish grammar
The previous attempt to support newlines after pipes changed the lexer to
swallow newlines after encountering a pipe. This has two problems that are
difficult to fix:

1. comments cannot be placed after the pipe
2. fish_indent won't know about the newlines, so it will erase them

Address these problems by removing the lexer behavior, and replacing it
with a new parser symbol "optional_newlines" allowing the newlines to be
reflected directly in the fish grammar.
2018-02-18 14:44:08 -08:00
ridiculousfish
ddd1afc57c Teach parse_util_detect_errors about unterminated pipelines
Allow it to return PARSER_TEST_INCOMPLETE for code like
`echo | `
2018-02-18 13:13:58 -08:00
ridiculousfish
81b3baaa9c Correct handling of unescapable entities in quotes
Prior to this fix, if you attempt to complete from inside a quote and the
completion contained an entity that cannot be represented inside quotes
(i.e. \n \r \t \b), the result would be a broken mess of quotes. Rewrite
the implementation so that it exits the quotes, emits the correct unquoted
escape, and then re-enters the quotes.
2018-02-17 15:18:46 -08:00
ridiculousfish
01d87455e1 Teach parse_util_escape_string_with_quote about tildes
Properly escape literal tildes in tab completion results. Currently we
always escape tildes in unquoted arguments; in the future we may escape
only leading tildes.

Fixes #2274
2018-02-17 15:18:43 -08:00
ridiculousfish
de23ce6ac1 Functions to store nodes
Prior to this fix, functions stored a string representation of their
contents. Switch them to storing a parsed source reference and the
tnode of the contents. This is part of an effort to avoid reparsing
a function's contents every time it executes.
2018-02-12 10:49:22 -08:00
ridiculousfish
ba7b8a9584 Remove various empty or useless functions
In particular remove init()/destroy() functions that do nothing, or
destroy functions that only free memory.
2018-02-10 17:21:55 -08:00
ridiculousfish
1bd5946d23 Add prompt layout caching to layout_cache_t 2018-02-04 16:20:55 -08:00
ridiculousfish
d1436486e2 Rename cached_esc_sequences_t to layout_cache_t
Preparation for migrating the prompt cache into this struct.
2018-02-04 16:20:55 -08:00
ridiculousfish
f025269195 env_var_t to forget its name
Store properties associated with the name via flags instead
2018-01-30 12:36:50 -08:00
Fabian Homborg
cddd0d7538 [Pager] Adjust tests for changes in behavior
Since moving west no longer gets stuck in the top row (but instead
wraps around to the bottom row), this needs to have some indices
changed.
2018-01-25 13:26:53 +01:00
ridiculousfish
a39c57c1b6 Report errors for arguments to 'end'
For example, `begin ; end arg` will now report an error.

Fixes #986
2018-01-22 13:31:39 -08:00
ridiculousfish
ae9b5871fb Fix a tnode-related crash in syntax highlighting
Adds a new test too.
2018-01-21 02:17:21 -08:00
ridiculousfish
094e853a20 Migrate tnode_t into new header tnode.h 2018-01-20 12:15:28 -08:00
ridiculousfish
194f7f34d9 Eliminate parse_node_tree::find_nodes 2018-01-20 11:45:43 -08:00
ridiculousfish
f0f56a6910 Teach decoration_for_plain_statement about tnode, rename it get_decoration 2018-01-20 11:31:40 -08:00
ridiculousfish
18a120d00e Migrate command_for_plain_statement to tnode_t 2018-01-20 11:31:40 -08:00
ridiculousfish
f16ae9348f Continued adoption of tnode_t in complete.cpp and highlight.cpp 2018-01-20 11:31:40 -08:00
ridiculousfish
55fc10ea6e Migrate parse_util_detect_errors_in_argument to tnode_t 2018-01-20 11:31:40 -08:00
ridiculousfish
f69055b5e9 Adopt tnode_t in parse_util_detect_errors 2018-01-20 11:31:40 -08:00
ridiculousfish
4d68877f51 tnode_t adoption of find_nodes 2018-01-20 11:31:40 -08:00
Matthew Brock
bf63e061c9 Fix overzealous cd tab completion
Changed cd completion to differentiate between cd autosuggest and cd tab
completion. cd autosuggest will find deepest unique hierarchy and cd tab
completion will not.

Issue #4402
2018-01-08 22:32:06 -08:00
ridiculousfish
b688deb33e Reduce number of threads in history race test
Our lock-breaking timeout means this test may spuriously fail.
Reduce the torture element to make the test more likely to pass.
2017-09-11 22:34:59 -07:00
ridiculousfish
cb352317bd Simplify the cached_esc_sequences_t structure
The type cached_esc_sequences_t caches escape sequences, and is tasked
with finding an escape sequence that prefixes a given string. Before
this fix, it did so by storing the lengths of cached escape sequences,
and searching for substrings of that length. The new implementation
instead stores all cached escape sequences in a sorted vector, and uses
binary search to find the shortest escape sequence that is a prefix of
the input. This is a substantial simplification that also reduces
allocations.
2017-09-01 14:36:16 -07:00
ridiculousfish
3d40292c00 Switch env_var to using maybe_t
This eliminates the "missing" notion of env_var_t. Instead
env_get returns a maybe_t<env_var_t>, which forces callers to
handle the possibility that the variable is missing.
2017-09-01 00:14:42 -07:00
ridiculousfish
18203a081c Add maybe_t template class
maybe_t is an implementation of the Maybe/Optional type, allowing
for an optional value to be stored. This will enable a more
principled approach for functions that return values or failure,
such as env_get.
2017-09-01 00:14:14 -07:00
ridiculousfish
c8cf8a6669 Clean up fish_uvars_test directory in tests
Allows running fish_tests directly without an initialization phase.
2017-08-30 01:02:18 -07:00
ridiculousfish
4baada25b9 Use move semantics instead of swap in env_set
This commit backs out certain optimizations around setting environment
variables, and replaces them with move semantics. env_set accepts a
list,  by value, permitting callers to use std::move to transfer
ownership.
2017-08-30 00:59:45 -07:00
Kurtis Rader
99d2a344c7 Fix indexing of $history
cherry-picked from krader1961/fish-shell commit b69df4fe72

Fixes #4353 (regression in indexing of history contents) and introduces
new unit tests to catch bad $history indexing in the future.
2017-08-25 20:28:45 -05:00
Kurtis Rader
11400fb313 Another fish var performance improvement
Make setting fish vars more efficient by avoiding creating a
wcstring_list_t for the case where we're setting one value. For the case
where we're passing a list of values swap it with the list in the var
rather than copying it. This makes the benchmark in #4200 approximately
6% faster.
2017-08-19 20:01:06 -07:00
Mahmoud Al-Qudsi
e76c1fd139 Remove custom lock types in favor of native C++11 mutexes
No longer using RAII wrappers around pthread_mutex_t and pthread_cond_t
in favor of the C++11 std::mutex, std::recursive_mutex, and
std::condition_variable data types.
2017-08-18 23:09:31 -05:00
Kurtis Rader
55b2d36028 Remove unused vars identified by lint 2017-08-18 16:52:39 -07:00
Kurtis Rader
f872f25f5b change env_var_t to a vector of strings
Internally fish should store vars as a vector of elements. The current
flat string representation is a holdover from when the code was written
in C.

Fixes #4200
2017-08-18 16:24:30 -07:00
Kurtis Rader
42ddab0cb4 make missing_var a singleton
Make the `env_var_t::missing_var()` object a singleton rather than a
dynamically constructed object. This requires some discipline in its use
since C++ doesn't directly support immutable objects. But it is slightly
more efficient and helps identify code that incorrectly mutates `env_var_t`
objects that should not be modified.
2017-08-14 18:18:10 -07:00
Kurtis Rader
58b604c5ba change order of env_set() args
It's bugged me forever that the scope is the second arg to `env_get()`
but not `env_set()`. And since I'll be introducing some helper functions
that wrap `env_set()` now is a good time to change the order of its
arguments.
2017-08-14 18:18:09 -07:00
Kurtis Rader
728a4634a1 replace var_entry_t with env_var_t
This is a step to storing fish vars as actual vectors rather than flat
strings.
2017-08-14 18:18:09 -07:00
Kurtis Rader
4197420f39 implement limits on command substitution output
This makes command substitutions impose the same limit on the amount
of data they accept as the `read` builtin. It does not limit output of
external commands or builtins in other contexts.

Fixes #3822
2017-08-03 17:40:25 -07:00
Kurtis Rader
17dff8c569 rewrite abbr function
Rewrite the `abbr` function to store each abbreviation in a separate
variable. This greatly improves the efficiency. For the common case
it is 5x faster. For pathological cases it is upwards of 100x faster.
Most people should be able to unconditionally define abbreviations in
their config.fish without a noticable slow down.

Fixes #4048
2017-08-03 14:35:06 -07:00
Kurtis Rader
8f548962b7 fix regression how fish_escape_delay_ms is handled
Fish 2.6.0 introduced a regression that keeps setting
`fish_escape_delay_ms` as a uvar from working. This also fixes a related
problem: callbacks generated from the initial loading of universal vars
were not being acted on.

Fixes #4196
2017-07-19 19:09:55 -07:00
Kurtis Rader
cf808674bf fix stupid bug in previous commit
This fixes a stupid bug in my previous commit to standardize on a new
`list_to_array_val()` function. This adds a unit test to keep this from
regressing.
2017-07-08 20:41:11 -07:00
Kurtis Rader
a9aa234a64 implement helper functions for fish script vars
This is the first step in implementing a better abstraction for handling
fish script vars in the C++ code. It implements a new function (with two
signatures) to provide a standard method for construct the flag string
representation of a fish script array.

Partial fix for #4200
2017-07-08 13:14:30 -07:00
Kurtis Rader
bd299e96b2 implement status is-breakpoint
This implements `status is-breakpoint` that returns true if the current
shell prompt is displayed in the context of a `breakpoint` command.

This also fixes several bugs. Most notably making `breakpoint` a no-op if
the shell isn't interactive. Also, typing `breakpoint` at an interactive
prompt should be an error rather than creating a new nested debugging
context.

Partial fix for #1310
2017-06-20 17:18:59 -07:00
Kurtis Rader
d910b7af7d fix some lint warnings 2017-06-16 21:01:57 -07:00
Kurtis Rader
6c4a51d56e improve bash history importing
Reject more invalid commands from the bash history file.

Fixes #3636
2017-05-20 21:03:31 -07:00
Kurtis Rader
0b976a1843 fix wcsfilecmp()
This started out as a refactoring to eliminate the lint warnings. Adding
unit tests revealed the current implementation does not behave as
implied. So this is a complete rewrite of the implementation. With the
addition of unit tests so that it doesn't break in the future and anyone
who thinks this new version behaves wrong can update the unit tests to
help ensure we're testing for the correct behavior.

Fixes #4027
2017-05-19 21:09:07 -07:00
Kurtis Rader
e6e1805c5f another step in fixing issue #3985
This changes all of the builtins to behave like `string` to return
STATUS_INVALID_ARGS (121) if the args passed to the command don't make
sense. Also change several of the builtins to use the existing symbols
(e.g., STATUS_CMD_OK and STATUS_CMD_ERROR) rather than hardcoded "0"
and "1" for consistency and to make it easier to find such values in
the future.

Fixes #3985
2017-05-07 21:20:31 -07:00
Kurtis Rader
be2b6bfdc9 fix lint errors that have crept in 2017-05-06 22:08:07 -07:00
Kurtis Rader
4c38867768 another step in fixing issue #3985
This primarily replaces "STATUS_BUILTIN_OK" with "STATUS_CMD_OK" and
"STATUS_BUILTIN_ERROR" with "STATUS_CMD_ERROR". That is because we want
to make it clear these status codes are applicable to fish functions as
well as builtins. Future changes will make it easier to use these
symbols and values in functions.
2017-05-04 00:18:02 -07:00
Kurtis Rader
23978aee81 codify string retval for invalid arguments
This is the first, tiny, step in addressing issue #3985.
2017-05-03 22:18:36 -07:00
Kurtis Rader
48d5342601 style cleanups
Time for another `make style-all`.
2017-05-01 22:05:35 -07:00
Kurtis Rader
e332573fe1 remove transitory test directory
Reinstatate a command to remove a temporary directory that inadvertently
removed by commit 49518b71.
2017-04-15 22:28:59 -07:00
Kurtis Rader
49518b71ab low level tests should not depend on /tmp
The low level tests should not depend on /tmp being a valid directory.
2017-04-08 21:13:34 -07:00
Kurtis Rader
8efe88201e correct escape_string corner case
Fixes #3892
2017-03-13 20:38:57 -07:00
Kurtis Rader
6d02bec4c7 fix setting up and using the terminfo data
There should be just one place that calls `setupterm()`. While refactoring
the code I also decided to not make initializing the curses subsystem a
fatal error. We now try two fallback terminal names ("ansi" and "dumb")
and if those can't be used we still end up with a usable shell.

Fixes #3850
2017-03-08 20:54:37 -08:00
Kurtis Rader
509ee64fc9 implement our own assert() function
I recently upgraded the software on my macOS server and was dismayed to
see that cppcheck reported a huge number of format string errors due to
mismatches between the format string and its arguments from calls to
`assert()`. It turns out they are due to the macOS header using `%lu`
for the line number which is obviously wrong since it is using the C
preprocessor `__LINE__` symbol which evaluates to a signed int.

I also noticed that the macOS implementation writes to stdout, rather
than stderr. It also uses `printf()` which can be a problem on some
platforms if the stream is already in wide mode which is the normal case
for fish.

So implement our own `assert()` implementation. This also eliminates
double-negative warnings that we get from some of our calls to
`assert()` on some platforms by oclint.

Also reimplement the `DIE()` macro in terms of our internal
implementation.

Rewrite `assert(0 && msg)` statements to `DIE(msg)` for clarity and to
eliminate oclint warnings about constant expressions.

Fixes #3276, albeit not in the fashion I originally envisioned.
2017-02-14 18:48:27 -08:00
Kurtis Rader
7fc1994339 some trivial lint cleanups 2017-02-13 18:48:59 -08:00
Kurtis Rader
4ffb0adb78 lint cleanups 2017-02-11 21:30:38 -08:00
Kurtis Rader
0541a34201 improve bash command history importing
Fixes #2091
2017-02-08 20:50:57 -08:00
Kurtis Rader
0970cc8736 time for another make style-all 2017-02-07 21:52:35 -08:00
ridiculousfish
162053ed8d Make history file rewriting be more lock savvy
We now are stingier with taking history file locks - if the lock
is held too long we may just break it. But the current file save
architecture holds the lock for the duration of the save. It also
has some not-quite-right checks that can cause spurious failures in
the history stress test.

Reimplement the history save to retry. Rather than holding the lock,
rewrite the file to a temporary location and then take the lock. If
the history file has changed, start all over.

This is going to be slower under contention, but the advantage is that
the lock is only held for a brief period (stat + rename) rather than
across calls to write().

Some updated logic also fixes spurious failures that were easy to observe
when tsan was enabled. These failures were due to failing to check if the
file at the path was the same file we opened.

The next step is to move the history file saving to a background thread
to reduce the chances of it impacting user's typing.
2017-02-06 11:04:07 -08:00
ridiculousfish
2a472af7b8 Add sanity checking to LRU cache 2017-01-30 12:59:01 -08:00
ridiculousfish
927a678056 Add support for sorting in LRU caches
Performs an in-place merge sort
2017-01-30 10:25:17 -08:00
ridiculousfish
cb70ac6932 Use a std::atomic in test_iothread
Reduces noise from thread-sanitizer
2017-01-29 21:35:22 -08:00
Kurtis Rader
973097f025 style fixups 2017-01-29 18:00:48 -08:00
ridiculousfish
e52a04e341 Cleanup of LRU cache implementation
Switch to CRTP from virtual functions and improve ownership semantics.
It's no longer necessary for clients to use new and delete.
2017-01-29 12:16:42 -08:00
Kurtis Rader
5b6cc5af6d time for another make style-all
Gotta keep the entropy (i.e., disorder) from increasing.
2017-01-26 20:05:46 -08:00
ridiculousfish
1d9cc12984 Repair control-C cancellation of loops
Commit ab189a75 introduced a regression where we stop breaking out
of loops in response to a child death via a signal. Fix that regression.

Also introduces a test to help ensure we don't regress in the future.

Fixes #3780
2017-01-26 13:41:56 -08:00
ridiculousfish
699d294ac1 Adopt iothread_perform-lambdas in fish_tests.cpp 2017-01-23 11:45:28 -08:00
ridiculousfish
144e7b0616 Adopt lamba-style perform_on_main in the tests 2017-01-23 10:45:48 -08:00
ridiculousfish
a91dad35db Use unique_ptr instead of new for history tests 2017-01-21 17:14:56 -08:00
ridiculousfish
6f745762bb Make universal_notifier_t use unique_ptr instead of raw pointers 2017-01-21 16:56:45 -08:00
ridiculousfish
9efa897d0d Early steps towards rationalizing SIGINT handling
Previously we would try to walk all the blocks (from within the
signal handler!) and mark them as skipped. Stop doing that, it's
wildly unsafe.

Also rationalize how the skip flag is set per block. Remove places
that shouldn't set it (e.g. break and continue shouldn't set skip
on the loop block).
2017-01-21 14:15:03 -08:00
ridiculousfish
bb686a2236 Enable some test_illegal_command_exit_code tests
A comment suggests these cause bad_alloc, but this
doesn't seem to happen. If it does happen, we want to hit
it so we can track it down!
2017-01-21 12:56:22 -08:00
ridiculousfish
fc803c75a6 Remove some errant newlines in fish_tests.cpp 2017-01-21 12:55:01 -08:00
Kurtis Rader
2e9a349dd0 normalize use of stdio functions taking a stream
We should never use stdio functions that use stdout implicitly. Saving a
few characters isn't worth the inconsistency. Too, using the forms such
as `fwprintf()` which take an explicit stream makes it easier to find
the places we write to stdout versus stderr.

Fixes #3728
2017-01-13 20:48:55 -08:00
Fabian Homborg
177e06808c Replace all printf invocations with wprintf
Remember, GNU doesn't allow mixing narrow and wide IO.
2017-01-07 14:05:42 +01:00
Kurtis Rader
b118ed69d3 convert narrow stderr output to wide forms
On some platforms, notably GNU libc, you cannot mix narrow and wide
stdio functions on a stream like stdout or stderr. Doing so will drop
the output of one or the other. This change makes all output to the
stderr stream consistently use the wide forms.

This change also converts some fprintf(stderr,...) calls to debug()
calls where appropriate.

Fixes #3692
2017-01-03 16:14:42 -08:00
Kurtis Rader
1621fa43d8 don't mix narrow and wide stdio
Fixes #3686
2016-12-29 12:24:56 -08:00
Kurtis Rader
574424dc5a fix uvar tests
I noticed that universal variable tests were failing on Cygwin and
Dragonfly BSD. The failures were because we are attempting to verify the
correct behavior of mechanisms that are known to be broken on those
platforms. There are still uvar test failures on those platforms with
this change but they are due to actual problems rather than bugs in the
tests.

Fixes #3587
2016-12-25 21:17:27 -08:00
Kurtis Rader
7c40abe4a6 switch from \1xb to \e in the code
Using `\e` is clearer and shorter than `\x1b`. It's also consistent with how
we write related control chars; e.g., we don't write `\x0a` we write '\n'.
2016-12-24 12:15:28 -08:00
Kurtis Rader
9443a415bf time for another make style-all 2016-12-03 20:12:53 -08:00
ridiculousfish
85d697f13d Add some pager layout test cases
Helps ensure correct truncation logic
2016-12-03 13:29:16 -08:00
ridiculousfish
754b78a748 Suppress certain stderr-printing during tests
Tests that exercise error paths may result in output to
stderr. This may make it look like the test failed when it did
not. Introduce should_suppress_stderr_for_tests() to suppress
this output so the test output looks clean.
2016-12-03 13:27:50 -08:00
Radomír Bosák
254762f30f Fix status code when bad command name is entered
This commit fixes a bug which causes that

   fish -c ')'; echo $status

("Illegal command name" error) returns 0. This is inconsistent with
e.g. when trying to run non-existent command:

   fish -c 'invalid-command'; echo $status

("Unknown command" error) which correctly returns 127.

A new status code,

    STATUS_ILLEGAL_CMD = 123

is introduced - which is returned whenever the 'Illegal command name *'
message is printed.

This commit also adds a test which checks if valid commands return 0,
while commands with illegal name return status code 123.

Fixes #3606.
2016-12-03 13:14:40 -08:00
Kurtis Rader
2f33c24a07 fix handling of odd strings by test builtin
The `test` builtin currently has unexpected behavior with respect to
expressions such as `'' -eq 0`. That currently evaluates to true with a
return status of zero. This change addresses that oddity while also
ensuring that other unusual strings (e.g., numbers with leading and
trailing whitespace) are handled consistently.

Fixes #3346
2016-11-27 20:37:21 -08:00
Fredrik Fornwall
df12ac24b2 Fix build on systems without shm_open()
Notably, this fixes building on Android.
2016-11-26 16:14:15 -05:00
Kurtis Rader
a928517e95 improve converting strings to ints/longs
The existing code is inconsistent, and in a couple of cases wrong, about
dealing with strings that are not valid ints. For example, there are
locations that call wcstol() and check errno without first setting errno
to zero. Normalize the code to a consistent pattern.  This is mostly to
deal with inconsistencies between BSD, GNU, and other UNIXes.

This does make some syntax more liberal. For example `echo $PATH[1 .. 3]`
is now valid due to uniformly allowing leading and trailing whitespace
around numbers. Whereas prior to this change you would get a "Invalid
index value" error. Contrast this with `echo $PATH[ 1.. 3 ]` which was
valid and still is.
2016-11-25 18:52:15 -08:00
Kurtis Rader
45c7836bec remove is_wchar_ucs2()
My previous change removed one place where is_wchar_ucs2() was used and
replaced it with compile time tests. This change does the same for the
other uses.
2016-11-20 18:25:49 -08:00
Kurtis Rader
b8778ba4a2 make fish_tests work on MS Windows with C++11
On Cygwin there are two narrowing conversions at line 931 in
src/fish_tests.cpp due to the code assuming a wchar_t is four bytes.
Obviously that's wrong but only became an issue with the pending change to
switch to C++11. The problematic values aren't actually used on Windows
because the tests that would use them are bypassed if is_wchar_ucs2()
returns true. This change predicates that code on a compile time rather
than a run time test.
2016-11-20 18:05:34 -08:00
ridiculousfish
9b4310b10f Ensure we clear first_unwritten_new_item_index after history::merge
Prevents an issue where we think we've written out history items,
but we haven't, and so they get lost. Fixes #3496
2016-11-19 22:40:50 -08:00
Kurtis Rader
5284a133b0 silence "parameter 'notifier' is unused." 2016-11-11 21:07:30 -08:00
Kurtis Rader
1fb8f4e277 lint: misc cleanups
Earlier lint cleanups overlooked a couple of modules because on macOS at
the moment oclint ignores them. I noticed this when I ran `make lint-all`
on Ubuntu.
2016-11-04 20:12:51 -07:00
Aaron Gyes
d437a84828 Fix two xcode warnings
An unused macro and a documentation error.
2016-11-04 18:11:37 -07:00
Kurtis Rader
f490b56378 make history searching case insensitive by default
Fixes #3236
2016-10-16 20:18:16 -07:00
Kurtis Rader
4ffc6e02b7 fix a couple of "unused parameter" warnings
This fixes two of the three "unused parameter" compiler warnings in the
fish_tests.cpp module. The third I'm deferring to issue #3439.
2016-10-09 19:52:02 -07:00
Kurtis Rader
d389b22afc make style-all C++ style cleanup
It's time once again to run `make style-all` to fix some style bogosities
that have crept in.
2016-10-04 20:06:14 -07:00
ridiculousfish
c0c2e1d4fa Don't modify valid directory segments in fuzzy completions
When performing fuzzy completion, if a directory segment is
valid, then don't consider it for a fuzzy match even if
the literal match produces no results.

Fixes #3211
2016-10-01 17:58:15 -07:00
Kurtis Rader
10ae6d8b26 fix some style problems that have crept in 2016-07-21 19:36:32 -07:00
ridiculousfish
f08ac969e9 Correctly handle trailing escaped spaces when completing
Fixes #2447
2016-07-11 14:03:53 -07:00
ridiculousfish
8b06d2ad66 Untangle some fuzzy completion logic
Prior to this fix, when performing completions, we would prepend
the wildcard to the resulting files. When doing fuzzy completions,
we would take some wildcard segment, attempt to locate it in the
final completion, and then replace it with our fuzzy-matched directory.

With this fix, we pass along the "resolved so far" path, and prepend
that instead of doing "surgery" on the completion. This simplifies the
logic.

Fixes #3185
2016-07-09 15:27:10 -07:00
Aaron Gyes
0045b46af8 Rename things to avoid conflicts with headers
Was breaking builds

 * ncurses.h: can declare `char *const key_name'.
 * netbsd term.h: has `newline', `lines' macros.
2016-07-01 02:48:50 -07:00