Commit graph

100 commits

Author SHA1 Message Date
Mahmoud Al-Qudsi
c8d2f7a0da Add trait to convert FFI reference to &wstr
You can now use a reference to CxxWString or an allocated UniquePtr<CxxWString>
to get an &wstr temporary to use without having to allocate again (e.g. via
`from_ffi()`).
2023-03-12 14:55:50 -05:00
ridiculousfish
5197bf75cd Point fish autocxx and similar dependencies at new fish-shell location
These crates have been moved into fish-shell org; update Cargo.toml to
reflect that.
2023-03-09 21:01:49 -08:00
Johannes Altmanninger
c6756e9324 Canonicalize some wide string imports
wchar.rs should not import let alone reexport FFI strings.
Stop re-exporting utf32str! because we use L! instead.

In wchar_ffi.rs, stop re-exporting cxx::CxxWString because that hasn't
seen adoption.

I think we should use re-exports only for aliases like "wstr" or for aliases
into internal modules.
So I'd probably remove `pub use wchar_ffi::wcharz_t = crate::ffi::wcharz_t`
as well.
2023-03-05 10:32:20 +01:00
Johannes Altmanninger
e6994ea3ac Remove obsolete clippy suppression
This type has been extracted to an alias, so it is okay now.
2023-03-05 10:32:20 +01:00
Mahmoud Al-Qudsi
d839fea748 Silence some more clippy lints
bool_assert_comparison is stupid, the reason they give is "it's shorter". Well,
`assert!(!foo)` is nowhere near as readable as `assert_eq!(foo, false)` because
of the ! noise from the macro.

Uninlined format args is a stupid lint that Rust actually walked back when they
made it an official warning because you still have to use a mix of inlined and
un-inlined format args (the latter of which won't complain) since only idents
can be inlined.
2023-03-05 00:54:17 -06:00
Mahmoud Al-Qudsi
4828346f8b Implement and use Read and Write traits for AutoCloseFd
This lets us use any std::io functions that build on top of these, such as
`write_all()` in place of our own `write_loop()`.
2023-03-05 00:33:54 -06:00
Mahmoud Al-Qudsi
455b744bca Port fd_monitor tests to rust
This shows some of the ugliness of the rust borrow checker when it comes to
safely implementing any sort of recursive access and the need to be overly
explicit about which types are actually used across threads and which aren't.

We're forced to use an `Arc` for `ItemMaker` (née `item_maker_t`) because
there's no other way to make it clear that its lifetime will last longer than
the FdMonitor's. But once we've created an `Arc<T>` we can't call
`Arc::get_mut()` to get an `&mut T` once we've created even a single weak
reference to the Arc (because that weak ref could be upgraded to a strong ref at
any time). This means we need to finish configuring any non-atomic properties
(such as `ItemMaker::always_exit`) before we initialize the callback (which
needs an `Arc<ItemMaker>` to do its thing).

Because rust doesn't like self-referential types and because of the fact that we
now need to create both the `ItemMaker` and the `FdMonitorItem` separately
before we set the callback (at which point it becomes impossible to get a
mutable reference to the `ItemMaker`), `ItemMaker::item` is dropped from the
struct and we instead have the "constructor" for `ItemMaker` take a reference to
an `FdMonitor` instance and directly add itself to the monitor's set, meaning we
don't need to move the item out of the `ItemMaker` in order to add it to the
`FdMonitor` set later.
2023-03-05 00:33:53 -06:00
Mahmoud Al-Qudsi
83a220a532 Make fd_monitor types useable from native code
We were only using their ffi implementations which are automatically
exported/public, but the actual functions we would need if we were to use
FdMonitor and co. in native rust code were either private or missing convenient
wrappers.
2023-03-05 00:23:01 -06:00
Mahmoud Al-Qudsi
78a78a834c Port read_loop() and write_loop() to rust
The existing code is kept, but a rusty version of these functions is added for
code that needs them.

These should only be temporarily used when porting 1-to-1 from C++; we should
use the std library's `read()` and `write_all()` methods instead in the future.
2023-03-05 00:22:56 -06:00
Mahmoud Al-Qudsi
f2f7d1d183 Simplify assert_sorted_by_name! macro
By extracting the equivalent of i32::cmp() into its own const function,
it becomes a lot easier to see what is happening and the logic can be
more direct.
2023-03-04 17:05:11 -06:00
Johannes Altmanninger
2c331e9c69 Implement more bitwise operation for parser bitfields
These will be used in the parser.

Maybe this type should be a struct with boolean fields. The current way has
the upside that the usage is exactly the same as in C++.
2023-03-04 22:24:22 +01:00
Johannes Altmanninger
b92313b79d Allow using wgettext_fmt without comma from macros
Otherwise we'd get this error when using it from another macro

        Some(wgettext_fmt!($fmt $(, $args)*))
                               ^ missing tokens in macro arguments
2023-03-04 22:24:22 +01:00
Johannes Altmanninger
7ec27617ae Support widestring macro on non-literal strings
This enables usage in macros like

        L!(stringify!($snake_case_name))

in the upcoming AST port.
2023-03-04 22:24:22 +01:00
Johannes Altmanninger
be89793669 Fix buffer overflow accessing error source in ParseError::describe()
For some reason this error is triggered by tests after the Rust port of
ast.cpp. Might want to get to the bottom of this but moving it back
to match the original C++ logic fixes it.
2023-03-04 22:24:22 +01:00
Johannes Altmanninger
386f952c53 Implement constructors for some parser types 2023-03-04 22:24:22 +01:00
Johannes Altmanninger
913eeffa7e Derive Copy for some parser types 2023-03-04 22:24:22 +01:00
Johannes Altmanninger
bb1c64b202 Make some parser types public 2023-03-04 22:24:22 +01:00
Johannes Altmanninger
d0bda9893b Silence -Wcomment warnings in cxx compiler runs
This is one of the few warnings we disable due to false positives.  Let's also
disable it in the preprocessing steps needed for the Rust build.

Other warnings we ignore are -Wno-address -Wunused-local-typedefs and
-Wunused-macros. I didn't add them here because I don't expect that they
will be triggered by the headers we give to cxx.
2023-03-04 22:24:22 +01:00
Xiretza
8427e05bf7 Move escape_string tests to Rust
This way, both the Rust FFI wrapper and the actual C++ implementation are
tested.
2023-03-04 12:42:06 -08:00
Xiretza
7585ddf926 Port color.cpp to Rust 2023-03-04 11:46:46 -08:00
Xiretza
a23de237a6 Port ASSERT_SORTED_BY_NAME to Rust 2023-03-04 11:46:46 -08:00
ridiculousfish
a3970c1661 Improve FLOG output
Prior to this fix, the Rust FLOG output was regressed from C++, because
it put quotes around strings. However if we used Display, we would fail
to FLOG non-display types like ThreadIDs.

There is apparently no way in Rust to write a function which formats a
value preferentially using Display, falling back to Debug.

Fix this by introducing two new traits, FloggableDisplay and
FloggableDebug. FloggableDisplay is implemented for all Display types,
and FloggableDebug can be "opted into" for any Debug type:

    impl FloggableDebug for MyType {}

Both traits have a 'to_flog_str' function. FLOG brings them both into
scope, and Rust figures out which 'to_flog_str' gets called.
2023-03-04 11:35:21 -08:00
Clemens Wasser
17c1fa9d64
Port bg builtin to Rust (#9621)
* bg: Port bg builtin to Rust
2023-02-28 16:42:12 -06:00
Victor Song
c7ea768a74
Rewrite wrealpath from wutil in Rust (#9613)
* wutil: Rewrite `wrealpath` in Rust

* Reduce use of FFI types in `wrealpath`

* Addressed PR comments regarding allocation

* Replace let binding assignment with regular comparison
2023-02-26 20:13:40 -07:00
Clemens Wasser
6f5be9bae4 block: Port block builtin to Rust
Closes #9612.
2023-02-26 14:16:55 -06:00
Xiretza
dff7db2f16
Run rustfmt and clippy in CI (#9616)
* Add machine-readable MSRV to Cargo.toml
* Fix clippy warnings
* CI: add rustfmt and clippy checks
2023-02-26 13:20:20 -06:00
Mahmoud Al-Qudsi
562eeac43e
Port job_group to rust (#9608)
More ugliness with types that cxx bridge can't recognize as being POD. Using
pointers to get/set `termios` values with an assert to make sure we're using
identical definitions on both sides (in cpp from the system headers and in rust
from the libc crate as exported).

I don't know why cxx bridge doesn't allow `SharedPtr<OpaqueRustType>` but we can
work around it in C++ by converting a `Box<T>` to a `shared_ptr<T>` then convert
it back when it needs to be destructed. I can't find a clean way of doing it
from the cxx bridge wrapper so for now it needs to be done manually in the C++
code.

Types/values that are drop-in ready over ffi are renamed to match the old cpp
names but for types that now differ due to ffi difficulties I've left the `_ffi`
in the function names to indicate that this isn't the "correct" way of using the
types/methods.
2023-02-25 16:42:45 -06:00
Neeraj Jaiswal
f52569a800 abbr: port abbreviation and abbr builtin to rust 2023-02-25 12:24:58 +01:00
Neeraj Jaiswal
b0ed37c2e0 format: support whitespace padding in str formatting 2023-02-25 12:24:58 +01:00
Neeraj Jaiswal
e384e63b24 re: port regex make anchored to rust and helper ffi funtions for regex 2023-02-25 12:24:57 +01:00
Neeraj Jaiswal
6851d52924 env: port env constants to rust 2023-02-25 12:24:32 +01:00
Neeraj Jaiswal
7bab4c4dda common: pass c_str in ffi escape string 2023-02-25 12:24:32 +01:00
Johannes Altmanninger
5394ca1f96 Address clippy lints 2023-02-25 12:24:25 +01:00
Johannes Altmanninger
0d6b53bc3e Address clippy lints
We want to keep the cast because tv_sec is not always 64 bits, see b5ff175b4
(Fix timer.rs cross-platform compilation, 2023-02-14).
It would be nice to avoid the clippy exemption, perhaps using something like

    #[cfg(target_pointer_width = "32")]
    let seconds = val.tv_sec as i64;
    #[cfg(not(target_pointer_width = "32"))]
    let seconds = val.tv_sec;

but I'm not sure if "target_pointer_width" is the right criteria.
2023-02-25 12:24:25 +01:00
Johannes Altmanninger
30d40c1d49 ffi.rs: sort includes in include_cpp
If we sort includes as we add them instead of adding them at the end, we'll
have fewer conflicts.
2023-02-25 12:24:25 +01:00
Neeraj Jaiswal
3b60bc1de0 contains: port contains builtin to rust 2023-02-22 18:32:27 +01:00
Mahmoud Al-Qudsi
aca7dedf33 Fix Tokenizer::parse_fd() on x86
Upsizing to `usize` from `i32` doesn't work if `usize` is only 32-bits.
I changed the code to use the `FromStr` impl on `i32`, but we could have also
just used `u64` instead of `i32`.

Also, we should get in the habit of using the appropriate type aliases where
possible (`i32` should be `RawFd`).
2023-02-20 13:41:11 -06:00
Mahmoud Al-Qudsi
e616de544e Enable rust overflow checks in release mode, at least for now
We want to try and catch as much unexpected/non-deterministic behavior as we
can. We could run the CI explicitly in debug mode, but I think it makes sense to
always have overflow checks on in both debug/release modes everywhere, at least
for the duration of the codebase transition.
2023-02-20 13:11:29 -06:00
Fabian Boehm
e3b04118b1 Revert "random: Do math as unsigned"
This reverts commit 0902e29f49.

Just doesn't work - overflows.
2023-02-20 19:56:34 +01:00
Fabian Boehm
0902e29f49 random: Do math as unsigned
Hahah bits go brrrr
2023-02-20 19:39:55 +01:00
Xiretza
77a474ee37 Move POD components of library_data_t to separate struct
This allows them to be accessed as regular fields from Rust, rather than having
to create setter/getter methods for each of them.
2023-02-20 11:32:12 +01:00
Mahmoud Al-Qudsi
59fe124c40 builtins/random: Don't lock the mutex unnecessarily
The mutex was being locked from the very start, before it was needed and
possibly before it would be needed.

Also rename the static global to stick to rust naming conventions.

Note that `once_cell::sync::Lazy<T>` actually internally uses its own lock
around the value, but in this case it's insufficient because `SmallRng` doesn't
implement `SeedableRng` so we can't reseed it with only an `&mut` reference and
must instead replace its value.

We probably *could* still use `Lazy<SmallRng>` directly and then rely on
`std::mem::swap()` to replace the contents of the shared global static without
reassigning the variable directly with a new `SmallRng` instance, but I'm not
sure that's a great idea. This is just a built-in, there's no real harm in
locking twice (especially while fish remains essentially single-threaded).
2023-02-19 16:54:50 -06:00
Mahmoud Al-Qudsi
51eb5168e8 builtins/random: Fix stale comments and use explicit output type
The old comments about using i128 logic were still there even though we are no
longer using that approach and the output type was very much misleadingly a u64
printed to the console (but via `%d` so it was ultimately shown as an i64). Be
explicit about the resulting being a valid i64 value before passing it to the
sprintf!() macro.

Also add comments about the safety of the final `unwrap()` operation.
2023-02-19 16:54:50 -06:00
Mahmoud Al-Qudsi
05265e7d90 Port (and use) ASSERT_IS_BACKGROUND_THREAD/ASSERT_IS_MAIN_THREAD
Rust doesn't have __FUNCTION__ or __func__ (though you can hack around it with a
proc macro, but that will require a separate crate and slowing down compilation
times with heavy proc macro dependencies), so these are just regular functions
(at least for now). Rust's default stack trace on panic (even in release mode)
should be enough (and the functions themselves are inlined so the calling
function should be the second frame from the top, after the #[cold] panic
functions).
2023-02-19 16:54:50 -06:00
Mahmoud Al-Qudsi
452cd90c6c Add test asserting std::thread's behavior matches pthread's on *nix
This is to allow us to verify some implementation details that aren't explicitly
documented in the rust standard library's documentation.

std::thread uses `pthread_create()` underneath the hood on *nix platforms, so
this *should* merely be a formality.
2023-02-19 15:42:07 -06:00
Mahmoud Al-Qudsi
aaf2d1c19d Use * const u8 instead of * const c_void
The way cxx bridge works, it doesn't recognize any types from another module as
being shared cxx bridge types with generations native to both C++ and Rust,
meaning every module that was going to use function pointers would have to
define its own `c_void` type (because cxx bridge doesn't recognize any of
libc::c_void, std::ffi::c_void, or autocxx::c_void).

FFI on other platforms has long used the equivalent of `uint8_t *` as an
alternative to `void *` for code where `void` was not available or was
undesirable for some reason. We can join the club - this way we can always use
`* {const|mut} u8` in our rust code and `uint8_t *` in our C++ code to pass
around parameters or values over the C abi.
2023-02-19 15:42:07 -06:00
Mahmoud Al-Qudsi
8deaede6c7 Patch a few minor issues in fd_monitor
These differ from the C++ code and are being committed separately.
2023-02-19 15:42:07 -06:00
Mahmoud Al-Qudsi
ce559bc20e Port fd_monitor (and its needed components)
I needed to rename some types already ported to rust so they don't clash with
their still-extant cpp counterparts. Helper ffi functions added to avoid needing
to dynamically allocate an FdMonitorItem for every fd (we use dozens per basic
prompt).

I ported some functions from cpp to rust that are used only in the backend but
without removing their existing cpp counterparts so cpp code can continue to use
their version of them (`wperror` and `make_detached_pthread`).

I ran into issues porting line-by-line logic because rust inverts the behavior
of `std::remove_if(..)` by making it (basically) `Vec::retain_if(..)` so I
replaced bools with an explict enum to make everything clearer.

I'll port the cpp tests for this separately, for now they're using ffi.

Porting closures was ugly. It's nothing hard, but it's very ugly as now each
capturing lambda has been changed into an explicit struct that contains its
parameters (that needs to be dynamically allocated), a standalone callback
(member) function to replace the lambda contents, and a separate trampoline
function to call it from rust over the shared C abi (not really relevant to
x86_64 w/ its single calling convention but probably needed on other platforms).

I don't like that `fd_monitor.rs` has its own `c_void`. I couldn't find a way to
move that to `ffi.rs` but still get cxx bridge to consider it a shared POD.
Every time I moved it to a different module, it would consider it to be an
opaque rust type instead. I worry this means we're going to have multiple
`c_void1`, `c_void2`, etc. types as we continue to port code to use function
pointers.

Also, rust treats raw pointers as foreign so you can't do `impl Send for * const
Foo` even if `Foo` is from the same module. That necessitated a wrapper type
(`void_ptr`) that implements `Send` and `Sync` so we can move stuff between
threads.

The code in fd_monitor_t has been split into two objects, one that is used by
the caller and a separate one associated with the background thread (this is
made nice and clean by rust's ownership model). Objects not needed under the
lock (i.e. accessed by the background thread exclusively) were moved to the
separate `BackgroundFdMonitor` type.
2023-02-19 15:42:03 -06:00
Fabian Boehm
f01a5d2a1b random: Do it in 64-bits
Turns out we can do it without switching to 128-bit wide numbers.

Co-authored-by: Xiretza <xiretza@xiretza.xyz>
2023-02-19 21:01:46 +01:00
Fabian Boehm
4fd1458d85 Port random to rust 2023-02-19 21:01:46 +01:00