mirror of
https://github.com/fish-shell/fish-shell
synced 2025-01-26 03:35:17 +00:00
No description
56da15d11f
[Do NOT cherry-pick to 4.0 - this needs more time to be tested]
fish sometimes needs to capture the output of a command or block of
commands. Examples include fish_prompt or any command substitution
("cmdsubs"). It does this the obvious way: by creating a pipe, using dup2
to replace stdout of the command with the write end of the pipe, and then
reading from the read end into a buffer, until EOF or the command
substitution completes. Importantly, this task also overlaps with waiting
for the process to exit; that is when executing:
set var (some_cmd)
fish needs to both wait on `some_cmd` and ALSO read its output into memory.
This is awkward to do in a portable way in a single thread (though maybe
doable on Linux with pidfd). So we wait and read on different threads.
To make things worse, command substitutions may themselves create
additional command substitutions (recursion, etc). Creating a read thread
for every command substitution would result in excessive threads. So rather
than a thread per cmdsub, we have a single dedicated thread that handles
ALL command substitutions, by multiplexing multiple file descriptors via
select/poll. This is the "fd monitor." You hand it a file descriptor and it
lets you know when it's readable, and then you can read from it (via a
callback). Also, it has a "wakeup" fd: if you write to that then the fd
monitor wakes up, figures out what it has to do, and resumes.
When the command substitution ends, we need to remove the fd from the fd
monitor, because we intend to close it. You might object "the commands in
the cmdsub have all completed so the write end of the pipe has been closed
so the fd monitor can just notice that the pipe is closed" but it's not so:
consider the horrible case of `set var (yes &)` and abandon all hope.
The current mechanism for removing the fd from the monitor is called a
"poke." We tell the fd monitor (through a "control" self-pipe) to
explicitly wake up the item. It then invokes the callback ("pokes") the
item on the dedicated fd monitor thread. The item notices that the command
substitution is complete, and it returns a value meaning "remove me" and
the fd monitor does so. The client thread is stuck waiting for this process
to complete.
So basically removing a fd from the monitor requires a round trip to its
dedicated thread. This is slow and also complicated (Rust doesn't have
futures)!
So let's not do that.
The big idea is to remove this round-trip synchronization. That is, when we
intend to remove the fd from the fd monitor, we _just do it_ and then close
the fd. Use a lock rather than a round-trip to the thread. Crucially that
lock is unlocked while the monitor thread waits in select/poll.
This invites all sorts of races:
1. fish might remove and close the fd right before the monitor polls it. It
will thus attempt to poll a closed fd.
2. fish might remove and close the fd, and then something else opens a file
and receives the same fd. Now the fd monitor will poll an fd that was
never added.
3. fish might remove and close the fd _while the fd monitor is polling it_.
What happens then? (Turns out on macOS we get EBADF, and on Linux the fd is
marked readable).
The Big Idea is that *all of these races are benign*. As long as
poll/select doesn't crash or hang, we don't care *what* it returns, because
the source of truth are the set of items stored in the fd monitor and these
item IDs are never recycled. (This also assumes that it's OK to select/poll
on random file descriptors; there ought to be no side effects).
Not only is this a large simplification since we no longer need that round
trip, it's a substantial performance improvement as well. The
"aliases.fish" benchmark goes from 164 to 154 msec on my Mac, and from 124
to 112 msec on my Linux machine - nearly 10%.
Add some tests to verify our assumptions about the behavior of closing or
replacing a file descriptor during poll. But even if these fail, all we
care about is that poll/select doesn't crash or hang.
|
||
|---|---|---|
| .builds | ||
| .github | ||
| benchmarks | ||
| build_tools | ||
| cmake | ||
| debian | ||
| doc_internal | ||
| doc_src | ||
| docker | ||
| etc | ||
| osx | ||
| po | ||
| printf | ||
| share | ||
| src | ||
| tests | ||
| .cirrus.yml | ||
| .clang-format | ||
| .editorconfig | ||
| .gitattributes | ||
| .gitignore | ||
| BSDmakefile | ||
| build.rs | ||
| Cargo.lock | ||
| Cargo.toml | ||
| CHANGELOG.rst | ||
| CMakeLists.txt | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.rst | ||
| COPYING | ||
| deny.toml | ||
| Dockerfile | ||
| fish.desktop | ||
| fish.pc.in | ||
| fish.png | ||
| fish.spec.in | ||
| GNUmakefile | ||
| README.rst | ||
.. |Cirrus CI| image:: https://api.cirrus-ci.com/github/fish-shell/fish-shell.svg?branch=master
:target: https://cirrus-ci.com/github/fish-shell/fish-shell
:alt: Cirrus CI Build Status
`fish <https://fishshell.com/>`__ - the friendly interactive shell |Build Status| |Cirrus CI|
=============================================================================================
fish is a smart and user-friendly command line shell for macOS, Linux,
and the rest of the family. fish includes features like syntax
highlighting, autosuggest-as-you-type, and fancy tab completions that
just work, with no configuration required.
For downloads, screenshots and more, go to https://fishshell.com/.
Quick Start
-----------
fish generally works like other shells, like bash or zsh. A few
important differences can be found at
https://fishshell.com/docs/current/tutorial.html by searching for the
magic phrase “unlike other shells”.
Detailed user documentation is available by running ``help`` within
fish, and also at https://fishshell.com/docs/current/index.html
Getting fish
------------
macOS
~~~~~
fish can be installed:
- using `Homebrew <http://brew.sh/>`__: ``brew install fish``
- using `MacPorts <https://www.macports.org/>`__:
``sudo port install fish``
- using the `installer from fishshell.com <https://fishshell.com/>`__
- as a `standalone app from fishshell.com <https://fishshell.com/>`__
Note: The minimum supported macOS version is 10.10 "Yosemite".
Packages for Linux
~~~~~~~~~~~~~~~~~~
Packages for Debian, Fedora, openSUSE, and Red Hat Enterprise
Linux/CentOS are available from the `openSUSE Build
Service <https://software.opensuse.org/download.html?project=shells%3Afish&package=fish>`__.
Packages for Ubuntu are available from the `fish
PPA <https://launchpad.net/~fish-shell/+archive/ubuntu/release-3>`__,
and can be installed using the following commands:
::
sudo apt-add-repository ppa:fish-shell/release-3
sudo apt update
sudo apt install fish
Instructions for other distributions may be found at
`fishshell.com <https://fishshell.com>`__.
Windows
~~~~~~~
- On Windows 10/11, fish can be installed under the WSL Windows Subsystem
for Linux with the instructions for the appropriate distribution
listed above under “Packages for Linux”, or from source with the
instructions below.
- fish (4.0 on and onwards) cannot be installed in Cygwin, due to a lack of Rust support.
Building from source
~~~~~~~~~~~~~~~~~~~~
If packages are not available for your platform, GPG-signed tarballs are
available from `fishshell.com <https://fishshell.com/>`__ and
`fish-shell on
GitHub <https://github.com/fish-shell/fish-shell/releases>`__. See the
`Building <#building>`__ section for instructions.
Running fish
------------
Once installed, run ``fish`` from your current shell to try fish out!
Dependencies
~~~~~~~~~~~~
Running fish requires:
- A terminfo database, typically from curses or ncurses (preinstalled on most \*nix systems) - this needs to be the directory tree format, not the "hashed" database.
If this is unavailable, fish uses an included xterm-256color definition.
- some common \*nix system utilities (currently ``mktemp``), in
addition to the basic POSIX utilities (``cat``, ``cut``, ``dirname``,
``file``, ``ls``, ``mkdir``, ``mkfifo``, ``rm``, ``sort``, ``tee``, ``tr``,
``uname`` and ``sed`` at least, but the full coreutils plus ``find`` and
``awk`` is preferred)
- The gettext library, if compiled with
translation support
The following optional features also have specific requirements:
- builtin commands that have the ``--help`` option or print usage
messages require ``nroff`` or ``mandoc`` for
display
- automated completion generation from manual pages requires Python 3.5+
- the ``fish_config`` web configuration tool requires Python 3.5+ and a web browser
- system clipboard integration (with the default Ctrl-V and Ctrl-X
bindings) require either the ``xsel``, ``xclip``,
``wl-copy``/``wl-paste`` or ``pbcopy``/``pbpaste`` utilities
- full completions for ``yarn`` and ``npm`` require the
``all-the-package-names`` NPM module
- ``colorls`` is used, if installed, to add color when running ``ls`` on platforms
that do not have color support (such as OpenBSD)
Building
--------
.. _dependencies-1:
Dependencies
~~~~~~~~~~~~
Compiling fish requires:
- Rust (version 1.70 or later)
- CMake (version 3.15 or later)
- a C compiler (for system feature detection and the test helper binary)
- PCRE2 (headers and libraries) - optional, this will be downloaded if missing
- gettext (headers and libraries) - optional, for translation support
- an Internet connection, as other dependencies will be downloaded automatically
Sphinx is also optionally required to build the documentation from a
cloned git repository.
Additionally, running the full test suite requires Python 3, tmux, and the pexpect package.
Building from source with CMake
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rather than building from source, consider using a packaged build for your platform. Using the
steps below makes fish difficult to uninstall or upgrade. Release packages are available from the
links above, and up-to-date `development builds of fish are available for many platforms
<https://github.com/fish-shell/fish-shell/wiki/Development-builds>`__
To install into ``/usr/local``, run:
.. code:: bash
mkdir build; cd build
cmake ..
cmake --build .
sudo cmake --install .
The install directory can be changed using the
``-DCMAKE_INSTALL_PREFIX`` parameter for ``cmake``.
CMake Build options
~~~~~~~~~~~~~~~~~~~
In addition to the normal CMake build options (like ``CMAKE_INSTALL_PREFIX``), fish's CMake build has some other options available to customize it.
- BUILD_DOCS=ON|OFF - whether to build the documentation. This is automatically set to OFF when Sphinx isn't installed.
- INSTALL_DOCS=ON|OFF - whether to install the docs. This is automatically set to on when BUILD_DOCS is or prebuilt documentation is available (like when building in-tree from a tarball).
- FISH_USE_SYSTEM_PCRE2=ON|OFF - whether to use an installed pcre2. This is normally autodetected.
- MAC_CODESIGN_ID=String|OFF - the codesign ID to use on Mac, or "OFF" to disable codesigning.
- WITH_GETTEXT=ON|OFF - whether to build with gettext support for translations.
- extra_functionsdir, extra_completionsdir and extra_confdir - to compile in an additional directory to be searched for functions, completions and configuration snippets
Building fish as self-installable (experimental)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can also build fish as a self-installing binary.
This will include all the datafiles like the included functions or web configuration tool in the main ``fish`` binary.
On the first interactive run, and whenever it notices they are out of date, it will extract the datafiles to ~/.local/share/fish/install/ (currently, subject to change). You can do this manually by running ``fish --install``.
To install fish as self-installable, just use ``cargo``, like::
cargo install --path /path/to/fish # if you have a git clone
cargo install --git https://github.com/fish-shell/fish-shell --tag 4.0 # to build from git once 4.0 is released
cargo install --git https://github.com/fish-shell/fish-shell # to build the current development snapshot without cloning
This will place the binaries in ``~/.cargo/bin/``, but you can place them wherever you want.
This build won't have the HTML docs (``help`` will open the online version) or translations.
It will try to build the man pages with sphinx-build. If that is not available and you would like to include man pages, you need to install it and retrigger the build script, e.g. by setting FISH_BUILD_DOCS=1::
FISH_BUILD_DOCS=1 cargo install --path .
Setting it to "0" disables the inclusion of man pages.
You can also link this build statically (but not against glibc) and move it to other computers.
Contributing Changes to the Code
--------------------------------
See the `Guide for Developers <CONTRIBUTING.rst>`__.
Contact Us
----------
Questions, comments, rants and raves can be posted to the official fish
mailing list at https://lists.sourceforge.net/lists/listinfo/fish-users
or join us on our `matrix
channel <https://matrix.to/#/#fish-shell:matrix.org>`__. Or use the `fish tag
on Unix & Linux Stackexchange <https://unix.stackexchange.com/questions/tagged/fish>`__.
There is also a fish tag on Stackoverflow, but it is typically a poor fit.
Found a bug? Have an awesome idea? Please `open an
issue <https://github.com/fish-shell/fish-shell/issues/new>`__.
.. |Build Status| image:: https://github.com/fish-shell/fish-shell/workflows/make%20test/badge.svg
:target: https://github.com/fish-shell/fish-shell/actions