Commit graph

3514 commits

Author SHA1 Message Date
ridiculousfish
fe334bf620 Remove scoped_rlock
It is unused.
2021-02-06 14:27:08 -08:00
ridiculousfish
9c238385f0 Fix binary_semaphore_t under non-Linux TSan
Under non-Linux builds, binary_semaphore is implemented with a
self-pipe. When TSan is active we mark the pipe as non-blocking as TSan
cannot interrupt read (but can interrupt select). However we weren't
properly testing for EAGAIN leading to an assertion failure.

Allow looping on EAGAIN.
2021-02-06 14:41:50 -08:00
ridiculousfish
736e344727 assert_is_locked to take std::mutex, not void *
It's unclear why this had the void* cast.
2021-02-06 14:24:45 -08:00
ridiculousfish
98b0ef532f io_buffer_t to store a promise, not a future, to satisfy TSan
io_buffer_t is a buffer that fills itself by reading from a file
descriptor (typically a pipe). When the file descriptor is widowed, the
operation completes, and it reports completion by marking a
`std::promise<void>`. The "main thread" waits for this by waiting on the
promise's future. However TSan was reporting that the future's destructor
races with its promise's wait method. It's not obvious if this is valid,
but we can fix it by keeping the promise alive until the io_buffer_t is
deallocated.

This fixes the TSan issues reported under
`complete_background_fillthread_and_take_buffer` for #7681 (but there
are other unresolved issues).
2021-02-06 13:28:01 -08:00
Fabian Homborg
caca4fec22 fds.h: Add missing types.h include
Broke the build on FreeBSD because that defines mode_t there.
2021-02-06 19:59:53 +01:00
ridiculousfish
b5716e97cc Remove fd_set_t
Now that we no longer need to worry about pipes conflicting with
user-specified redirections, we can remove fd_set_t.
2021-02-05 18:14:50 -08:00
ridiculousfish
b79ec0122a Use pipe2 when creating pipes if avaialble
This allows us to avoid marking the pipe as CLOEXEC in some cases,
saving a system call.
2021-02-05 17:58:08 -08:00
ridiculousfish
97f29b1f4d Pipe fds to move to the "high range"
This concerns how fish prevents its own fds from interfering with
user-defined fd redirections, like `echo hi >&5`. fish has historically
done this by tracking all user defined redirections when running a job,
and ensuring that pipes are not assigned the same fds. However this is
annoying to pass around - it means that we have to thread user-defined
redirections into pipe creation.

Take a page from zsh and just ensure that all pipes we create have fds in
the "high range," which here means at least 10. The primary way to do this
is via the F_DUPFD_CLOEXEC syscall, which also sets CLOEXEC, so we aren't
invoking additional syscalls in the common case. This will free us from
having to track which fds are in user-defined redirections.
2021-02-05 17:58:08 -08:00
ridiculousfish
6c4f2622ef iothread's notify pipes to use make_autoclose_pipes
This allows it to take advantage of the upcoming high-range fd changes.
2021-02-05 17:58:08 -08:00
ridiculousfish
4b4bf541d1 Migrate more fd-concerned functions from wutil into fds
Functions like wopen_cloexec have a new home in fds.cpp. This is in
preparation for reworking how internal fds avoid conflict with user fds.
2021-02-05 17:58:08 -08:00
ridiculousfish
6588cf35f4 Move autoclose_pipes_t from io.h to fds.h 2021-02-05 17:58:08 -08:00
ridiculousfish
be9375e914 Migrate autoclose_fd_t to new file fds.h
fds.h will centralize logic around working with file descriptors. In
particular it will be the new home for logic around moving fds to high
unused values, replacing the "avoid conflicts" logic.
2021-02-05 17:58:08 -08:00
Fabian Homborg
b5305ce3d3 Handle backslashes properly in locate_brackets_of_type
This needs to be rewritten, I'm pretty sure we have like 6 of these
kinds of ad-hoc "is this quoted" things lying around.

But for now, at least don't just check if the *previous* character was
a backslash.

Fixes #7685.
2021-02-05 22:03:13 +01:00
ridiculousfish
97bde2f2bf Further refactoring of io_buffer_t
Previously we sometimes wanted to access an io_buffer_t to append to it
directly, but that's no longer true; all we really care about is its
separated_buffer_t. Make io_bufferfill_t::finish return the
separated_buffer directly, simplifying call sites. No user visible changes
expected here.
2021-02-04 17:14:46 -08:00
ridiculousfish
258149fe2e Improve locking discipline in io_buffer_t
Previously we had a lock that was taken in an ad-hoc manner. Switch to
using owning_lock.
2021-02-04 17:03:54 -08:00
ridiculousfish
8bcc8c1a36 Further cleanup of separated_buffer_t and io_buffer_t
Remove some clinging tendrils of life as a template object.
2021-02-04 16:43:47 -08:00
ridiculousfish
cbf10971f0 Reorganize separated_buffer_t
Move private bits to the bottom and do some other mild cleanup.
2021-02-04 16:06:28 -08:00
ridiculousfish
d578f8d136 separated_buffer_t to accept strings by rvalue reference
This saves a copy in some cases.
2021-02-04 16:02:40 -08:00
ridiculousfish
032467f338 separated_buffer_t to stop being a template
Now that we no longer construct wide separated buffers, it doesn't have
to be templatized.
2021-02-04 15:32:11 -08:00
ridiculousfish
7d494eab5c builtins to write to buffers directly
This concerns builtins writing to an io_buffer_t. io_buffer_t is how fish
captures output, especially in command substitutions:

    set STUFF (string upper stuff)

Recall that io_buffer_t fills itself by reading from an fd (typically
connected to stdout of the command). However if our command is a builtin,
then we can write to the buffer directly.

Prior to this change, when a builtin anticipated writing to an
io_buffer_t, it would first write into an internal buffer, and then after
the builtin was finished, we would copy it to the io_buffer_t. This was
because we didn't have a polymorphic receiver for builtin output: we
always buffered it and then directed it to the io_buffer_t or file
descriptor or stdout or whatever.

Now that we have polymorphpic io_streams_t, we can notice ahead of time
that the builtin output is destined for an internal buffer and have it
just write directly to that buffer. This saves a buffering step, which is
a nice simplification.
2021-02-04 15:21:32 -08:00
ridiculousfish
cd9a035f02 Add a string_output_stream_t to collect builtin output
This is used when creating a function; this breaks a dependency on the
more complicated buffered_output_stream_t to ease refactoring.
2021-02-04 14:12:14 -08:00
ridiculousfish
fc97151aec Add a variant of wcs2string which accepts a ptr, length pair
This will be useful when refactoring separated buffers.
2021-02-04 13:28:48 -08:00
ridiculousfish
86a12e1abd separated_buffer_t::append to stop being a template
In preparation for simplifying how builtins write to buffers, make
append an ordinary function rather than a template function.
2021-02-04 13:19:11 -08:00
ridiculousfish
7e2a538300 create_output_stream_for_builtin to accept read limit directly
This avoids requiring passing in a parser.
2021-02-03 19:00:04 -08:00
ridiculousfish
2d78c9a0d9 Poll the uvar notifier when the reader is interrupted by a signal
While the user waits at the prompt, fish is waiting in select(), on stdin.
The sigio based universal notifier interrupts select() by arranging for a
signal to be delivered, which causes select() to return with EINTR.
However we weren't polling the notifier at that point so we would not
notice uvar changes, until we got some real input.

I didn't notice this when testing, because my testing was changing fish
prompt colors which updated the prompt for other reasons.

Fixes #7671.
2021-01-31 15:42:35 -08:00
ridiculousfish
409ed7d6d0 Factor out count_preceding_backslashes
Now that we have multiple clients of count_preceding_backslashes, factor
it out from fish_indent into wcstringutil.h, and then use the shared
implementation.
2021-01-30 16:20:20 -08:00
Shizcow
cff5aa9130 Ensure escaped trailing spaces are not trimmed 2021-01-30 15:57:29 -08:00
Fabian Homborg
594d51e7eb Add a separate --profile-startup option to profile startup
This goes to a separate file because that makes option parsing easier
and allows profiling both at the same time.

The "normal" profile now contains only the profile data of the actual
run, which is much more useful - you can now profile a function by
running

   fish -C 'source /path/to/thing' --profile /tmp/thefunction.prof -c 'thefunction'

and won't need to filter out extraneous information.
2021-01-29 20:46:34 +01:00
Fabian Homborg
005d3a5981 Enable strict-aliasing and implicit-fallthrough warnings
GCC needs to have the comment *right before* the case label... blergh
2021-01-29 18:23:30 +01:00
Fabian Homborg
4e8c0f757d complete: Don't require a parameter with --force-files
A classic fallthrough problem!

This is why I want to enable -Wimplicit-fallthrough
2021-01-29 18:23:29 +01:00
Johannes Altmanninger
062f24d91b builtin set: make slice index range optional, like in slice expansion
Expansion parses slices like "$PATH[1..2]", but so does "set" when assigning
"set PATH[1..2] . .".  Commit be06f842a ("Allow to omit indices in index
range expansions") forgot the latter.
2021-01-28 07:19:38 +01:00
Fabian Homborg
275534b1b3 read: Remove unused short options
This has both "m" and "B" in the short options but did nothing with
them, so it would assert() out.

Fixes #7659.
2021-01-26 07:06:25 +01:00
Fabian Homborg
baa9b21a6f type: Only print function path with "--path"
Fixes #7653.
2021-01-24 15:31:39 +01:00
Fabian Homborg
88a84bd988 reader: Force ONLCR on for fish and external commands
Just like OPOST this just breaks output for anything not prepared for
it. Fish itself might work with it (and #4505 recommends it), but external commands are broken.

You'll see output like

foo
   ⏎

from `echo foo`.

Fixes #4873.

Continuation of #7133.
2021-01-18 21:00:08 +01:00
Fabian Homborg
8b133833fa Don't inherit windows paths for $PWD
If given a windows path like `F:\foo`, this currently ends up
assert()ing in path_normalize_for_cd.

Instead, since these paths violate a bunch of assumptions we make, we
reject them and fall back on getting $PWD via getcwd() (which should
give us a nice proper unixy path).

Fixes #7636.

This isn't tested because it would require a system where a windowsy
path passes paths_are_same_file, and on the unix systems we run our
tests that's impossible as far as I can tell?
2021-01-17 23:08:04 +01:00
Fabian Homborg
932074f06c escape_string_script: Escape DEL as \x7f
This used to print a literal DEL character in the output for `bind`,
which wouldn't actually show up and made it hard to figure out what
the key was.

So we just escape it back to how we actually used it - `\x7f`.

Fixes #7631.
2021-01-16 12:49:49 +01:00
Fabian Homborg
a4f5dd5054 set: Move the new values
A C++ special!

This makes

```fish
set -l var (seq 1 10000)
set -l v
for f in $var
    set -a v $f
end
```

~15% faster by removing allocations.
2021-01-15 21:00:25 +01:00
ridiculousfish
7a0bddfcfa Teach string repeat to handle multiple arguments
Each argument in string repeat is handled independently, except that the
--no-newline option applies only to the last newline.

Fixes #5988
2021-01-11 17:00:06 -08:00
ridiculousfish
290d1f2cd6 Mild refactoring of builtin_string repeat
Preparation for fixing issue 5988; no behavior change expected here.
2021-01-11 16:52:39 -08:00
ridiculousfish
1d4883d810 Remove an unnecessary 'using' declaration
This was just redundant with the struct tag.
2021-01-11 15:23:52 -08:00
ridiculousfish
7207a205f2 Switch history races test to use threads instead of processes
This avoids issues with ASan and TSan whose allocators do not properly
clean up in atfork, leading to deadlocks in child processes.
2021-01-11 12:44:21 -08:00
Fabian Homborg
3fc9c0b38c tests: Increase cancellation delay
This sometimes fails on github actions with ASAN. I am assuming that's
because the ctrl-c happens *before* the process has had a chance to
start.

So we do what we do and increase the delay.
2021-01-11 21:00:33 +01:00
Fabian Homborg
7bf2b9fd43 output: Rename some variables
These are a foreground and a background color. Now I see the point in
not naming them "foreground_color" and "background_color", but at
least "fg" and "bg" should do, right?
2021-01-11 20:56:15 +01:00
Fabian Homborg
f7b2bf8229 output: Simplify some duplicated code
Becomes a bit boring after a while
2021-01-11 20:53:11 +01:00
Fabian Homborg
19efd22468 env: Setup $HOME/$USER *before* the config directories
They are based on $HOME, so setting $HOME has to be done first.

Fixes #7620

(untested because I'm assuming common CI systems have weird $HOME settings)
2021-01-11 18:51:47 +01:00
ridiculousfish
e8c9da100c Track histories with shared_ptr
Prior to this change, histories were immortal and allocated with either
unique_ptr or just leaked via new. But this can result in races in the
path detection test, as the destructor races with the pointer-captured
history. Switch to using shared_ptr.
2021-01-09 17:02:11 -08:00
ridiculousfish
e062a07a97 Revert "Stop using unique_ptr to store histories"
This reverts commit 6f91195f40.
This triggered ASan complaints due to leaks.
2021-01-09 17:02:11 -08:00
ridiculousfish
87dacc0e95 Improve formatting and layout of history path detection test 2021-01-09 17:02:11 -08:00
ridiculousfish
884eb2b198 Remove an unused static variable 2021-01-09 17:02:11 -08:00
ridiculousfish
89687e7db7 Fix a warning building on Linux
Initialize saved_errno
2021-01-09 13:14:54 -08:00