I needed to rename some types already ported to rust so they don't clash with
their still-extant cpp counterparts. Helper ffi functions added to avoid needing
to dynamically allocate an FdMonitorItem for every fd (we use dozens per basic
prompt).
I ported some functions from cpp to rust that are used only in the backend but
without removing their existing cpp counterparts so cpp code can continue to use
their version of them (`wperror` and `make_detached_pthread`).
I ran into issues porting line-by-line logic because rust inverts the behavior
of `std::remove_if(..)` by making it (basically) `Vec::retain_if(..)` so I
replaced bools with an explict enum to make everything clearer.
I'll port the cpp tests for this separately, for now they're using ffi.
Porting closures was ugly. It's nothing hard, but it's very ugly as now each
capturing lambda has been changed into an explicit struct that contains its
parameters (that needs to be dynamically allocated), a standalone callback
(member) function to replace the lambda contents, and a separate trampoline
function to call it from rust over the shared C abi (not really relevant to
x86_64 w/ its single calling convention but probably needed on other platforms).
I don't like that `fd_monitor.rs` has its own `c_void`. I couldn't find a way to
move that to `ffi.rs` but still get cxx bridge to consider it a shared POD.
Every time I moved it to a different module, it would consider it to be an
opaque rust type instead. I worry this means we're going to have multiple
`c_void1`, `c_void2`, etc. types as we continue to port code to use function
pointers.
Also, rust treats raw pointers as foreign so you can't do `impl Send for * const
Foo` even if `Foo` is from the same module. That necessitated a wrapper type
(`void_ptr`) that implements `Send` and `Sync` so we can move stuff between
threads.
The code in fd_monitor_t has been split into two objects, one that is used by
the caller and a separate one associated with the background thread (this is
made nice and clean by rust's ownership model). Objects not needed under the
lock (i.e. accessed by the background thread exclusively) were moved to the
separate `BackgroundFdMonitor` type.
This is early work but I guess there's no harm in pushing it?
Some thoughts on the conventions:
Types that live only inside Rust follow Rust naming convention
("FeatureMetadata").
Types that live on both sides of the language boundary follow the existing
naming ("feature_flag_t").
The alternative is to define a type alias ("using feature_flag_t =
rust::FeatureFlag") but that doesn't seem to be supported in "[cxx::bridge]"
blocks. We could put it in a header ("future_feature_flags.h").
"feature_metadata_t" is a variant of "FeatureMetadata" that can cross
the language boundary. This has the advantage that we can avoid tainting
"FeatureMetadata" with "CxxString" and such. This is an experimental approach,
probably not what we should do in general.
The original implementation without the test took me 3 hours (first time
seriously looking into this)
The functions take "wcharz_t" for smooth integration with existing C++ callers.
This is at the expense of Rust callers, which would prefer "&wstr". Would be
nice to declare a function parameter that accepts both but I don't think
that really works since "wcharz_t" drops the lifetime annotation.
Prior to this change, abbreviations were stored as fish variables, often
universal. However we intend to add additional features to abbreviations
which would be very awkward to shoe-horn into variables.
Re-implement abbreviations using a builtin, managing them internally.
Existing abbreviations stored in universal variables are still imported,
for compatibility. However new abbreviations will need to be added to a
function. A follow-up commit will add it.
Now that abbr is a built-in, remove the abbr function; but leave the
abbr.fish file so that stale files from past installs do not override
the abbr builtin.
This adds preprocessor defines for _LARGEFILE_SOURCE and
_FILE_OFFSET_BITS=64 and a few others, fixing a bug that was reported on
gitter. This prevents issues when running fish on 32 bit systems that
have filesystems with 64 bit inodes.
Intern'd strings were intended to be "shared" to reduce memory usage but
this optimization doesn't carry its weight. Remove it. No functional
change expected.
This migrates our PCRE2 dependency from builtin/string.cpp to new files
re.h/re.cpp, allowing regexes to be used in other places in fish.
No user-visible behavior change expected here.
This adds a "path" builtin that can handle paths.
Implemented so far:
- "path filter PATHS", filters paths according to existence and optionally type and permissions
- "path base" and "path dir", run basename and dirname, respectively
- "path extension PATHS", prints the extension, if any
- "path strip-extension", prints the path without the extension
- "path normalize PATHS", normalizes paths - removing "/./" components
- and such.
- "path real", does realpath - i.e. normalizing *and* link resolution.
Some of these - base, dir, {strip-,}extension and normalize operate on the paths only as strings, so they handle nonexistent paths. filter and real ignore any nonexistent paths.
All output is split explicitly, so paths with newlines in them are
handled correctly. Alternatively, all subcommands have a "--null-input"/"-z" and "--null-output"/"-Z" option to handle null-terminated input and create null-terminated output. So
find . -print0 | path base -z
prints the basename of all files in the current directory,
recursively.
With "-Z" it also prints it null-separated.
(if stdout is going to a command substitution, we probably want to
skip this)
All subcommands also have a "-q"/"--quiet" flag that tells them to skip output. They return true "when something happened". For match/filter that's when a file passed, for "base"/"dir"/"extension"/"strip-extension" that's when something about the path *changed*.
Filtering
---------
`filter` supports all the file*types* `test` has - "dir", "file", "link", "block"..., as well as the permissions - "read", "write", "exec" and things like "suid".
It is missing the tty check and the check for the file being non-empty. The former is best done via `isatty`, the latter I don't think I've ever seen used.
There currently is no way to only get "real" files, i.e. ignore links pointing to files.
Examples
--------
> path real /bin///sh
/usr/bin/bash
> path extension foo.mp4
mp4
> path extension ~/.config
(nothing, because ".config" isn't an extension.)
This separates the list of builtin sources from the list of other
sources, since it seems like a natural cleavage point. The library
structure is unchanged, it's all just one big fishlib.a.
+ No functional change here, just renames and #include changes.
+ CMake can't have slashes in the target names. I'm suspciious of
that weird machinery for test, but I made it work.
+ A couple of builtins did not include their own headers, that
is no longer the case.
Revert the change getting rid of the -UNDEBUG, add some unused-blah
warnings.
We are often using the system assert() because we include other
headers that include assert.h.
I noticed that assert() was being compiled out because I started
getting new warnings printed about unusued variables (that were only
used in the assert()s. Add these warnings to the build.
Allows the compiler to know our bespoke assert functions
are cold paths. This would normally occur somehow for real assert().
Assembly does appear it will save some branches.
Also don't worry about NDEBUG
(This doesn't matter because we rolled our own assert functions.
Thanks @zanchey.)
This was a workaround for an error that has been removed in glibc
2.32 (by removing sys_errlist and friends, which it complained about).
Other than that, it's an attempt at performance optimization that
should just be fixed at the system level - if your linker is bad,
replace it with a better linker. No need for fish to work around it.
Closes#8152
In preparation for using wait handles in --on-process-exit events, factor
wait handles into their own wait handle store. Also switch them to
per-process instead of per-job, which is a simplification.
I'm not entirely sure this *has* to be given via
CMAKE_EXE_LINKER_FLAGS, but this would have stopped at least one
person from trying.
Static linking 1. does not work at the moment, 2. is not *useful*. You
don't get a single-file fish you can just copy somewhere because
you're missing our functions. On glibc systems you also can't
statically link glibc. Given all that, it does not appear to be worth
putting in any effort to make it work (if it's possible at all).
See #7947.
fds.h will centralize logic around working with file descriptors. In
particular it will be the new home for logic around moving fds to high
unused values, replacing the "avoid conflicts" logic.
Like Gold, it doesn't warn about sys_nerr, _sys_errlist, and co.
Unlike Gold, we can use this on all platforms. It's also faster than
both Gold and plain, old ld.
This is too important to not be one.
For one if it couldn't be loaded for any reason it would
break a lot of fish scripts.
Also this is faster by ~20x.
Fixes#7342
This is the first commit of a series intended to replace the existing
"parse tree" machinery. It adds a new abstract syntax tree and uses a more
normal recursive descent parser.
Initially there are no users of the new ast. The following commits will
replace parse_tree -> ast for all usages.
fish's handling of terminal sizes is currently rather twisted. The
essential problem is that the terminal size may change at any point from a
SIGWINCH, and common_get_{width,height} may modify it and post variable
change events from arbitrary locations.
Tighten up the semantics. Assign responsibility for managing the tty size
to a new class, `termsize_container_t`. Rationalize locking and reentrancy.
Explicitly nail down the relationship between $COLUMNS/$LINES and the tty
size. The new semantics are: whatever changed most recently takes
precendence.
Perform an ad-hoc code signing with the hardened runtime.
This ensures that these executables can pass notarization.
The code signing ID is controlled by the MAC_CODESIGN_ID CMake
cache variable.
fd_monitor is a new class which can monitor a set of fds, waiting for them
to become readable. When an fd becomes readable, a callback is invoked.
Timeouts are also supported.
This is intended to replace the "bufferfill" threads. Rather than one
thread per bufferfill, we will have a single fd_monitor which can service
multiple bufferfills. This helps today with nested command substitutions,
and will help in the future with concurrent execution.
This commit recognizes an existing pattern: many operations need some
combination of a set of variables, a way to detect cancellation, and
sometimes a parser. For example, tab completion needs a parser to execute
custom completions, the variable set, should cancel on SIGINT. Background
autosuggestions don't need a parser, but they do need the variables and
should cancel if the user types something new. Etc.
This introduces a new triple operation_context_t that wraps these concepts
up. This simplifies many method signatures and argument passing.
CMake sets these flags to sane defaults depending on which compiler
you're using, so overriding them isn't very nice.
For example:
with g++, I get
-- Debug: -g
-- RelWithDebInfo: -O2 -g -DNDEBUG
-- MinSizeRel: -O2 -g -DNDEBUG
-- Release: -O3 -DNDEBUG
and with MSVC you get something like
-- Debug: /MDd /Zi /Ob0 /Od /RTC1
-- RelWithDebInfo: /MD /Zi /O2 /Ob1 /DNDEBUG
-- MinSizeRel: /MD /Zi /O2 /Ob1 /DNDEBUG
-- Release: /MD /O2 /Ob2 /DNDEBUG
This adds support for `fish_trace`, a new variable intended to serve the
same purpose as `set -x` as in bash. Setting this variable to anything
non-empty causes execution to be traced. In the future we may give more
specific meaning to the value of the variable.
The user's prompt is not traced unless you run it explicitly. Events are
also not traced because it is noisy; however autoloading is.
Fixes#3427
Not -Werror, just the warnings themselves.
Let's just see what happens. It's warning-free on my system.
Worst case we disable it again, better case we refine the set.
This adds a new mechanism for logging, intended to replace debug().
The entry points are FLOG and FLOGF. FLOG can be used to log a sequence of
arguments, FLOGF is for printf-style formatted strings.
Each call to FLOG and FLOGF requires a category. If logging for a category
is not enabled, there is no effect (and arguments are not evaluated).
Categories may be enabled on the command line via the -d option.
I did not realize builtins could safely call into the parser and inject
jobs during execution. This is much cleaner than hacking around the
required shape of a plain_statement.
This new file is supposed to encapsulate all of the logic around
reacting to variable changes, as opposed to the environment core.
This is to help break up the env.cpp monolith.
In tests we would like to arrange for an executable to invoke certain
system calls, e.g. to claim or relinquish control of the terminal. This is
annoying to do portably via e.g. perl. fish_test_helper is a little
program where we can add custom commands to make it act in certain ways.
As it turns out it didn't work much better, and it fell behind in
support when it comes to things that wcwidth traditionally can't
express like variation selectors and hangul combining characters, but
also simply $fish_*_width.
I've had to tell a few people now to rebuild with widecharwidth after
sending them on a fool's errand to set X variable.
So keeping this option is doing our users a disservice.
topic_monitor allows for querying changes posted to one or more topics,
initially sigchld. This will eventually replace the waitpid logic in
process_mark_finished_children().
Comment from the new header:
Topic monitoring support. Topics are conceptually "a thing that can
happen." For example, delivery of a SIGINT, a child process exits, etc. It
is possible to post to a topic, which means that that thing happened.
Associated with each topic is a current generation, which is a 64 bit
value. When you query a topic, you get back a generation. If on the next
query the generation has increased, then it indicates someone posted to
the topic.
For example, if you are monitoring a child process, you can query the
sigchld topic. If it has increased since your last query, it is possible
that your child process has exited.
Topic postings may be coalesced. That is there may be two posts to a given
topic, yet the generation only increases by 1. The only guarantee is that
after a topic post, the current generation value is larger than any value
previously queried.
Tying this all together is the topic_monitor_t. This provides the current
topic generations, and also provides the ability to perform a blocking
wait for any topic to change in a particular topic set. This is the real
power of topics: you can wait for a sigchld signal OR a thread exit.
This represents a "resolved" io_chain_t, where all of the different io_data_t
types have been reduced to a sequence of dup2() and close(). This will
eliminate a lot of the logic duplication around posix_spawn vs fork, and pave
the way for in-process redirections.
This represents a "resolved" io_chain_t, where all of the different io_data_t
types have been reduced to a sequence of dup2() and close(). This will
eliminate a lot of the logic duplication around posix_spawn vs fork, and pave
the way for in-process redirections.
Since fish began resolving symlinks it broke the running-from-build-dir
detection in fish.cpp if the build directory were a symlink (which is
common on some platforms where the default user HOME directory is a
symlink in the first place, e.g. FreeBSD).
Load fish docs and configuration out of the source and/or build
directories rather from the installed paths when running directly out
of the cmake build directory.
Closes#5255.
Default to RelWithDebInfo (-O2 -g) if no custom CMAKE_BUILD_TYPE is
defined. Also add flags for use with CMAKE_BUILD_TYPEs Debug, Release,
and RelWithDebInfo.
This introduces a new type features_t that exposes feature flags. The intent
is to allow a deprecation/incremental adoption path. This is not a general
purpose configuration mechanism, but instead allows for compatibility during
the transition as features are added/removed.
Each feature has a user-presentable short name and a short description. Their
values are tracked in a struct features_t.
We start with one feature stderr_nocaret, but it's not hooked up yet.
* Make wcwidth configurable
This adds the cmake option "INTERNAL_WCWIDTH" (to be set to "ON" or
"OFF") and the configure option --[en,dis]able-internal-wcwidth.
Both default to enabling our fallback, but can be set to use the system's wcwidth again.
Sequel to #4554.
See #4571, #4539, #4609.
On my system, this would fix#4306.
This reverts commit 36a2f2cc01.
This attempted to modify RPATH when building with Ninja, but the CMake if
statement wasn't actually valid so this wasn't doing anything. This check
couldn't really be tested - let's make sure not to accumulate build system
rules that we don't understand.
