This is part of an effort to improve fish's Unicode handling. This commit
attempts to grapple with the fact that, certain characters (principally
emoji) were considered to have a wcwidth of 1 in Unicode 8, but a width of
2 in Unicode 9.
The system wcwidth() here cannot be trusted; terminal emulators do not
respect it. iTerm2 even allows this to be set in preferences.
This commit introduces a new function is_width_2_in_Uni9_but_1_in_Uni8() to
detect characters of version-ambiguous width. For these characters, it
returns a width guessed based on the value of TERM_PROGRAM and
TERM_VERSION, defaulting to 1. This value can be overridden by setting the
value of a new variable fish_emoji_width (presumably either to 1 or 2).
Fixes#4539, #2652.
Prior to this fix, autoloads like function and completion autoloads
would check their path variable (like fish_function_path) on every
autoload request. Switch to invalidating it in response to the variable
changing.
This improves time on a microbenchmark:
for i in (seq 50000)
setenv test_env val$i
end
from ~11 seconds to ~6.5 seconds.
Add a fish-specific wrapper around std::mutex that records whether it is
locked in a bool. This is to make ASSERT_IS_LOCKED() simpler (it can just
check the boolean instead of relying on try_lock) which will make Coverity
Scan happier.
Some details: Coverity Scan was complaining about an apparent double-unlock
because it's unaware of the semantics of try_lock(). Specifically fish
asserts that a lock is locked by asserting that try_lock fails; if it
succeeds fish prints an error and then unlocks the lock (so as not to leave
it locked). This unlock is of course correct, but it confused Coverity Scan.
Use wcstring/string instead of a character array. The variable
`term_env` was not being freed before the function exited.
Fixes defect 7520324 in coverity scan.
There were several issues with the way that the include tests for curses.h
were being done that were ultimately causing fish to use the headers from
ncurses but link against curses on platforms that provide an actual
libcurses.so that isn't just a symlink to libncurses.so
In particular, the old code was first testing for curses's cureses.h and then
falling back to libncurses's implementation of the same - but that logic was
reversed when it came to including term.h, in which case it was testing for
the ncurses term.h and falling back to the curses.h header. Long story short,
while cmake will link against libcurses.so if both libcurses.so and
libncurses.so are present (unless CURSES_NEED_NCURSES evaluates to TRUE, but
that makes ncurses a hard requirement), but we were brining in some of the
defines from the ncurses headers, causing SIGSEGV panics when fish ultimately
tried to access variables that weren't exported or were mapped to undefined
areas of memory in the other library.
Additionally it is an error to include termios.h prior to including the plain
Jane curses.h (not ncurses/curses.h), causing errors about unimplemented types
SGTTY/chtype. So far as I can tell, both curses.h and ncurses/curses.h pull in
termios.h themselves so it shouldn't even be necessary to manually include it,
but I have just moved its #include below that of curses.h
* Hoist `for` loop control var to enclosing scope
It should be possible to reference the last value assigned to a `for`
loop control var when the loop terminates. This makes it easier to detect
if we broke out of the loop among other things. This change makes fish
`for` loops behave like most other shells.
Fixes#1935
* Remove redundant line
This eliminates the "missing" notion of env_var_t. Instead
env_get returns a maybe_t<env_var_t>, which forces callers to
handle the possibility that the variable is missing.
This commit backs out certain optimizations around setting environment
variables, and replaces them with move semantics. env_set accepts a
list, by value, permitting callers to use std::move to transfer
ownership.
Commit f872f25f introduced a freed memory access regression on line 460
of env.cpp, where an environment variable was converted to a temporary
string, the .c_str() address of which was stored while the string
temporary was destroyed.
This commit keeps a reference to the original string lying around so
that the c_str() pointer does not point to freed memory.
cherry-picked from krader1961/fish-shell commit b69df4fe72
Fixes#4353 (regression in indexing of history contents) and introduces
new unit tests to catch bad $history indexing in the future.
Make setting fish vars more efficient by avoiding creating a
wcstring_list_t for the case where we're setting one value. For the case
where we're passing a list of values swap it with the list in the var
rather than copying it. This makes the benchmark in #4200 approximately
6% faster.
Since we are including XXHash32/64 anyway for the wchar_t* hashing,
we might as well use it.
Use arch-specific hash size and xxhash for all wcstring hashing
Instead of using XXHash64 for all platforms, use the 32-bit version
when running on 32-bit platforms where XXHash64 is significantly slower
than XXHash32 (and the additional precision will not be used).
Additionally, manually specify wcstring_hash as hashing method for
non-const wcstring unordered_set/map instances (the const varieties
don't have an in-library hash and so already use our xxhash-based
specialization when calling std::hash<const wcstring>).
commit 50f414a45d58fcab664ff662dd27befcfa0fdd95
Author: Mahmoud Al-Qudsi <mqudsi@neosmart.net>
Date: Sat Aug 19 13:43:35 2017 -0500
Converted file_id_t set to unordered_set with custom hash
commit 83ef2dd7cc1bc3e4fdf0b2d3546d6811326cc3c9
Author: Mahmoud Al-Qudsi <mqudsi@neosmart.net>
Date: Sat Aug 19 13:43:14 2017 -0500
Converted remaining set<wcstring> to unordered_set<wcstring>
commit 053da88f933f27505b3cf4810402e2a2be070203
Author: Mahmoud Al-Qudsi <mqudsi@neosmart.net>
Date: Sat Aug 19 13:29:21 2017 -0500
Switched function sets to unordered_set
commit d469742a14ac99599022a9258cda8255178826b5
Author: Mahmoud Al-Qudsi <mqudsi@neosmart.net>
Date: Sat Aug 19 13:21:32 2017 -0500
Converted list of modified variables to an unordered set
commit 5c06f866beeafb23878b1a932c7cd2558412c283
Author: Mahmoud Al-Qudsi <mqudsi@neosmart.net>
Date: Sat Aug 19 13:15:20 2017 -0500
Convert const_string_set_t to std::unordered_set
As it is a readonly-list of raw character pointer strings (not
wcstring), this necessitated the addition of a hashing function since
the C++ standard library does not come with a char pointer hash
function.
To that end, a zlib-licensed [0] port of the excellent, lightweight
XXHash family of 32- and 64-bit hashing algorithms in the form of a C++
header-only include library has been included. XXHash32/64 is pretty
much universally the fastest hashing library for general purpose
applications, and has been thoroughly vetted and is used in countless
open source projects. The single-header version of this library makes it
a lot simpler to include in the fish project, and the license
compatibility with fish' GPLv2 and the zero-lib nature should make it an
easy decision.
std::unordered_set brings a massive speedup as compared to the default
std::set, and the further use of the fast XXHash library to provide the
string hashing should make all forms of string lookups in fish
significantly faster (to a user-noticeable extent).
0: http://create.stephan-brumme.com/about.html
commit 30d7710be8f0c23a4d42f7e713fcb7850f99036e
Author: Mahmoud Al-Qudsi <mqudsi@neosmart.net>
Date: Sat Aug 19 12:29:39 2017 -0500
Using std::unordered_set for completions backing store
While the completions shown to the user are sorted, their storage in
memory does not need to be since they are re-sorted before they are
shown in completions.cpp.
commit 695e83331d7a60ba188e57f6ea0d9b6da54860c6
Author: Mahmoud Al-Qudsi <mqudsi@neosmart.net>
Date: Sat Aug 19 12:06:53 2017 -0500
Updated is_loading to use unordered_set
No longer using RAII wrappers around pthread_mutex_t and pthread_cond_t
in favor of the C++11 std::mutex, std::recursive_mutex, and
std::condition_variable data types.
The `react_to_variable_change()` function is called whenever a fish var
is set. Even as a consequence of statements like `for x in a b c`. It is
therefore critical that that function be as fast as possible. Especially
when setting the var doesn't have any side-effects which is true something
like 99.9999% of the time.
This change reduces the overhead of `react_to_variable_change()` to
unmeasurable levels. Making the synthetic benchmark in issue #4341
36% faster.
Fixes#4341
Internally fish should store vars as a vector of elements. The current
flat string representation is a holdover from when the code was written
in C.
Fixes#4200
Make the `env_var_t::missing_var()` object a singleton rather than a
dynamically constructed object. This requires some discipline in its use
since C++ doesn't directly support immutable objects. But it is slightly
more efficient and helps identify code that incorrectly mutates `env_var_t`
objects that should not be modified.
It's bugged me forever that the scope is the second arg to `env_get()`
but not `env_set()`. And since I'll be introducing some helper functions
that wrap `env_set()` now is a good time to change the order of its
arguments.
My previous change to eliminate `class var_entry_t` caused me to notice
that `env_get()` turned a set but empty var into a missing var. Which
is wrong. Fixing that brought to light several other pieces of code that
were wrong as a consequence of the aforementioned bug.
Another step to fixing issue #4200.
This is the first step to implementing issue #4200 is to stop subclassing
env_var_t from wcstring. Not too surprisingly doing this identified
several places that were incorrectly treating env_var_t and wcstring as
interchangeable types. I'm not talking about those places that passed
an env_var_t instance to a function that takes a wcstring. I'm talking
about doing things like assigning the former to the latter type, relying
on the implicit conversion, and thus losing information.
We also rename `env_get_string()` to `env_get()` for symmetry with
`env_set()` and to make it clear the function does not return a string.
Rewrite the `abbr` function to store each abbreviation in a separate
variable. This greatly improves the efficiency. For the common case
it is 5x faster. For pathological cases it is upwards of 100x faster.
Most people should be able to unconditionally define abbreviations in
their config.fish without a noticable slow down.
Fixes#4048
When executing a function, local-exported (`set -lx`) variables
previously were not accessible at all. This is weird e.g. in case of
aliases, since
```fish
set -lx PAGER cat
git something # which will call $PAGER
```
would not work if `git` were a function, even if that ends up calling
`command git`.
Now, we copy these variables, so functions get a local-exported copy.
```fish
function x
echo $var
set var wurst
echo $var
end
set -lx var banana
x # prints "banana" and "wurst"
echo $var # prints "banana"
```
One weirdness here is that, if a variable is both local and global,
the local-copy takes precedence:
```fish
set -gx var banana
set -lx var pineapple
echo $var # prints "pineapple"
x # from above, prints "pineapple" and "wurst"
echo $var # still prints "pineapple"
set -el var # deletes local version
echo $var # "banana" again
```
I don't think there is any more consistent way to handle this - the
local version is the one that is accessed first, so it should also be
written to first.
Global-exported variables are _not_ copied, instead they still offer
full read-write access.
In the rare case that we don't inherit $HOME _and_ can't read it from
/etc/passwd, this makes it so instead of triggering an assert() $HOME
is set to the empty list.
Tilde-expansion expands to nothing in such a case (and a string-empty
$HOME), `cd` errors out.
Fixes#4229.
Fish 2.6.0 introduced a regression that keeps setting
`fish_escape_delay_ms` as a uvar from working. This also fixes a related
problem: callbacks generated from the initial loading of universal vars
were not being acted on.
Fixes#4196
This fixes a stupid bug in my previous commit to standardize on a new
`list_to_array_val()` function. This adds a unit test to keep this from
regressing.
This is the first step in implementing a better abstraction for handling
fish script vars in the C++ code. It implements a new function (with two
signatures) to provide a standard method for construct the flag string
representation of a fish script array.
Partial fix for #4200
Don't import the bash history if the user has specified that a non-default
fish history file should be used. Also, rename the var that specifies
the fish history session ID from `FISH_HISTFILE` to `FISH_HISTORY`.
Fixes#4172
Using the FISH_HISTFILE variable will let people customise the session
to use for the history file. The resulting history file is:
`$XDG_DATA_HOME/fish/name_history`
Where `name` is the name of the session. The default value is `fish`
which results in the current history file.
If it's set to an empty string, the history will not be stored to a
file.
Fixes#102
Switch from null terminated arrays to `wcstring_list_t` for lists of
special env var names. Rename `list_contains_string` to `contains` and
modify the latter interface to not rely on a `#define`.
Rename `list_contains_string()` to `contains()` and eliminate the
current variadic implementation. Update all callers of the removed
version to use the string list version.
There are at least three env vars describing a sequence of paths to be
searched where an empty path element is implicitly equivalent to ".".
This change converts the implicit "." to explicit whenever the variable
is imported or set. This makes the variable much easier to use in fish
scripts.
Fixes#3914
There should be just one place that calls `setupterm()`. While refactoring
the code I also decided to not make initializing the curses subsystem a
fatal error. We now try two fallback terminal names ("ansi" and "dumb")
and if those can't be used we still end up with a usable shell.
Fixes#3850
I recently upgraded the software on my macOS server and was dismayed to
see that cppcheck reported a huge number of format string errors due to
mismatches between the format string and its arguments from calls to
`assert()`. It turns out they are due to the macOS header using `%lu`
for the line number which is obviously wrong since it is using the C
preprocessor `__LINE__` symbol which evaluates to a signed int.
I also noticed that the macOS implementation writes to stdout, rather
than stderr. It also uses `printf()` which can be a problem on some
platforms if the stream is already in wide mode which is the normal case
for fish.
So implement our own `assert()` implementation. This also eliminates
double-negative warnings that we get from some of our calls to
`assert()` on some platforms by oclint.
Also reimplement the `DIE()` macro in terms of our internal
implementation.
Rewrite `assert(0 && msg)` statements to `DIE(msg)` for clarity and to
eliminate oclint warnings about constant expressions.
Fixes#3276, albeit not in the fashion I originally envisioned.
This puts a hard upper bound of 10 MiB on the amount of data that read
will consume. This is to avoid having the shell consume an unreasonable
amount of memory, possibly causing the system to enter a OOM condition,
if the user does something non-sensical.
Fixes#3712
Cache the escape sequences we've seen when checking for those which
don't take any visual space when writing the prompt or similar strings.
This reduces the cost of determining the true cost of such strings by a
full order of magnitude if they include lots of such escape sequences.
Periodically sort the cached escape sequence lengths based on feedback
from cache hits so that we're always checking for the most likely
sequence lengths first.
Also cache the prompt layouts to avoid doing the calculations if the
prompt doesn't change.
Fixes#3793
If the kernel reports a size of zero for the rows or columns (i.e., what
`stty -a` reports) fall back to the `COLUMNS` and `LINES` variables. If
the resulting values are not reasonable fallback to using 80x24.
Fixes#3740
Update our implementation of the PROMPT_SP heuristic to match current
zsh behavior. This makes it behave better on terminals like ConEmu and
the native MS Windows console which automatically insert a newline when
writing to the last column of the line.
Fixes#789
This change increases the amount of useful information when fish is
unable to create or use its config or data directory. We now make it
clear when neither var is set or one is set to an unusable location.
Fixes#3545
A couple things went wrong with `env -u HOME USER=x ./fish -c ''`
We failed to check that `pw` isn't NULL leading to a crash when USER is
bogus. After fixing that we were not left with both variables in a
correct state still.
We now go back and force fish to dig up a working USER when we notice
this and then get both set successfully. Fixes#3599
The existing code is inconsistent, and in a couple of cases wrong, about
dealing with strings that are not valid ints. For example, there are
locations that call wcstol() and check errno without first setting errno
to zero. Normalize the code to a consistent pattern. This is mostly to
deal with inconsistencies between BSD, GNU, and other UNIXes.
This does make some syntax more liberal. For example `echo $PATH[1 .. 3]`
is now valid due to uniformly allowing leading and trailing whitespace
around numbers. Whereas prior to this change you would get a "Invalid
index value" error. Contrast this with `echo $PATH[ 1.. 3 ]` which was
valid and still is.
This fixes some of the IWYU and cppcheck lint warnings. And only on
macOS (formerly OS X). Fixing these types of warnings on a broader set
of platforms should be done but this is a baby step to making `make
lint-all` have few, if any, warnings. This reduces the number of lines
in the `make lint-all` output on macOS by over 500 lines.
Earlier lint cleanups overlooked a couple of modules because on macOS at
the moment oclint ignores them. I noticed this when I ran `make lint-all`
on Ubuntu.
Where we already manage to cover an enum entirely in a switch
statement such that default: cannot be reached, help ensure
it stays that way by condemning that route.
Also adjust a 'const' I came across that is ignored.
Fix test setup bogosities. Specifically, they weren't hermetic with respect to
locale env vars.
Rewrite the handling of locale vars to simplify the code and make it more like
the pattern most programs employ.
Fixes#3110
We need to actually export the curses/terminfo env vars in order for
`setupterm()` to be able to use them. While fixing this I reworked the
fallback logic implemented by @zanchey in response to issue #1060 in
order to simplify the logic and clarify the error messages.
This does not allow someone to change the curses/terminfo env vars after
the first prompt is displayed (you can but it won't affect the current
fish process). It only makes it possible to set `TERM`, `TERMINFO`, and
`TERMINFO_DIRS` in *config.fish* or similar config file and have them be
honored by fish.
This only eliminates errors reported by `make lint`. It shouldn't cause any
functional changes.
This change does remove several functions that are unused. It also removes the
`desc_arr` variable which is both unused and out of date with reality.
The autoconf-generated config.h contains a number of directives which
may alter the behaviour of system headers on certain platforms. Always
include it in every C++ file as the first include.
Closes#2993.
The fork (create new process) related debugging messages rely on an
undocumented env var and use `printf()` rather than `debug()`. There are
also errors in how the fork count is tracked that this fixes.
Fixes#2995
Remove the "make iwyu" build target. Move the functionality into the
recently introduced lint.fish script. Fix a lot, but not all, of the
include-what-you-use errors. Specifically, it fixes all of the IWYU errors
on my OS X server but only removes some of them on my Ubuntu 14.04 server.
Fixes#2957
The relevant standards allow the mbtowc/mbrtowc functions to reject
non-ASCII characters (i.e., chars with the high bit set) when the locale
is C or POSIX. The BSD libraries (e.g., on OS X) don't do this but
the GNU libraries (e.g., on Linux) do. Like most programs we need the
C/POSIX locales to allow arbitrary bytes. So explicitly check if we're
in a single-byte locale (which would also include ISO-8859 variants)
and simply pass-thru the chars without encoding or decoding.
Fixes#2802.
Introduce a "fish_escape_delay_ms" variable to allow the user to configure the
delay used when seeing a bare escape before assuming no other characters will
be received that might match a bound character sequence. This is primarily
useful for vi mode so that a bare escape character (i.e., keystroke) can
switch to vi "insert" to "normal" mode in a timely fashion.
Previously 'set -ql' would only look for variables in the
immediate local scope. This was not very useful. It's also
arguably surprising, since a 'set -l' in a function, followed
by a 'set -ql' in a child block, would fail. There was also no
way to check for a function-scoped variable, since omitting the
scope would also pull in global variables.
We could revisit this and introduce an explicit function scope.
Fixes#2502
This change moves source files into a src/ directory,
and puts object files into an obj/ directory. The Makefile
and xcode project are updated accordingly.
Fixes#1866
