The CMake `cargo test` integration was broken if Rust_CARGO_TARGET were used
with `CARGO_FLAGS` set to `-Zbuild-std` (e.g. to target i586 under i686 without
the i586 toolchain installed).
See the changelog additions for user-visible changes.
Since we enable/disable terminal protocols whenever we pass terminal ownership,
tests can no longer run in parallel on the same terminal.
For the same reason, readline shortcuts in the gdb REPL will not work anymore.
As a remedy, use gdbserver, or lobby for CSI u support in libreadline.
Add sleep to some tests, otherwise they fall (both in CI and locally).
There are two weird failures on FreeBSD remaining, disable them for now
https://github.com/fish-shell/fish-shell/pull/10359/checks?check_run_id=23330096362
Design and implementation borrows heavily from Kakoune.
In future, we should try to implement more of the kitty progressive
enhancements.
Closes#10359
They are probably not terribly useful for us but let's see what happens.
Unfortunately cargo does not properly forward the combination of "RUSTFLAGS"
and "--target" that is currently required to build with ASan [1]. Hence doctests
will fail to link on ASan builds. Let's disable doctests when ASan is active.
[1]: https://github.com/rust-lang/cargo/issues/10666 et al
Use Rust for executables
Drops the C++ entry points and restructures the Rust package into a
library and three binary crates.
Renames the fish-rust package to fish.
At least on Ubuntu, "fish_indent" is built before "fish".
Make sure export CURSES_LIBRARY_LIST to all binaries to make sure
that "cached-curses-libnames" is populated.
Closes#10198
This makes "ninja test" write only to the build directory, not to the source
tree. This enables our docker script which mounts the source as read-only.
- Make CMake use the correct target-path
- Make build.rs use the correct target dir
Workspaces place it in the project root by default, the alternative to making
this change is to add a `.cargo/config.toml` file with
```toml
[build]
target-dir = "fish-rust/target"
```
Which I think is unnecessary, as we likely want to use the new location anyways.
Rust has multiple sanitizers available (with llvm integration).
-Zsanitizer=address catches the most likely culprits but we may want to set up a
separate job w/ -Zsanitizer=memory to catch uninitialized reads.
It might be necessary to execute `cargo build` as `cargo build -Zbuild-std` to
get full coverage.
When we're linking against the hybrid C++ codebase, the sanitizer library is
injected into the binary by also include `-fsanitize=address` in CXXFLAGS - we
do *not* want to manually opt-into `-lasan`. We also need to manually specify
the desired target triple as a CMake variable and then explicitly pass it to all
`cargo` invocations if building with ASAN.
Corrosion has been patched to make sure it follows these rules.
The `cargo-test` target is failing to link under ASAN. For some reason it has
autocxx/ffi dependencies even though only rust-native, ffi-free code should be
tested (and one would think the situation wouldn't change depending on the
presence of the sanitizer flag). It's been disabled under ASAN for now.
The FISH_RUST_TARGET_DIR is not set for Tests.cmake, the target_dir will set to
$CARGO_MANIFEST_DIR/target. But if build.target-dir or CARGO_TARGET_DIR is set,
the real target_dir doesn't at the $CARGO_MANIFEST_DIR/target. It causes failure
in cargo test. Then, set --target-dir for cargo test.
Closes#9600
+ No functional change here, just renames and #include changes.
+ CMake can't have slashes in the target names. I'm suspciious of
that weird machinery for test, but I made it work.
+ A couple of builtins did not include their own headers, that
is no longer the case.
Our tests typically run in their own environment, which is great for
normal tests.
However for the coming translation test, we don't want to copy the .po
files into the test environment, so it's nice to have a way out.
This lets us run non-fish targets (such as `fish_tests`) under a clean
test environment without running into the fish-specific payload
configuration now carried out by `test_driver.sh` which expects a
`.fish` payload that it will run under a deterministically configured
instance of fish, running in an environment initialized by
`test_env.sh`.
This should fix the problem with in-tree builds leaving detritus behind
after a `make test` when `fish_tests` would be executed without
`test_driver.sh` - it is now executed under `test_env.sh` instead.
For littlecheck/pexpect this just unconditionally enables color.
I have no idea what happens if you run cmake outside of a terminal
, but the worst that can happen is that *errors* have color
escapes in them.
If someone figures out how to get cmake to tell us if it's running in
a terminal, we can add a check.
This means instead of printing at least two lines per successful test,
we overwrite one line again and again with the current status, and
for *failed* (i.e interesting) tests we print the output.
Makes test failures much more visible.
`test:foo` is not allowed by CMake ("reserved name") and `test/foo`
won't work since CMake doesn't allow targets to have a directory
separator in their name.
Even though we are using CMake's ctest for testing, we still define our
own `make test` target rather than use its default for many reasons:
* CMake doesn't run tests in-proc or even add each tests as an
individual node in the ninja dependency tree, instead it just bundles
all tests into a target called `test` that always just shells out to
`ctest`, so there are no build-related benefits to not doing that
ourselves.
* CMake devs insist that it is appropriate for `make test` to never
depend on `make all`, i.e. running `make test` does not require any
of the binaries to be built before testing.
* The only way to have a test depend on a binary is to add a fake test
with a name like "build_fish" that executes CMake recursively to
build the `fish` target.
* It is not possible to set top-level CTest options/settings such as
CTEST_PARALLEL_LEVEL from within the CMake configuration file.
* Circling back to the point about individual tests not being actual
Makefile targets, CMake does not offer any way to execute a named
test via the `make`/`ninja`/whatever interface; the only way to
manually invoke test `foo` is to to manually run `ctest` and specify
a regex matching `foo` as an argument, e.g. `ctest -R ^foo$`... which
is really crazy.
With this patch, it is now possible to execute any single test by name,
by invoking the build directly, e.g. to run the `universal.fish` check:
`cmake --build build --target universal.fish` or
`ninja -C build universal.fish`. Unfortunately, this is not integrated
into the Makefile wrapper, so `make universal.fish` won't work (although
this can potentially be hacked around).
Instead of compiling `fish_tests.cpp` dynamically with weakly-linked
symbols and asking it to print the list of all available tests, we
use a magic string `#define`'d as a no-op to allow CMake to regex search
for matching test groups. This speeds up configuration somewhat (by not
compiling anything), but more importantly, it's much less brittle and
doesn't involve and linker dark magic.
There's of course still no getting around the fact that it's really ugly.
When executing “make test -jX” (with X > 1) to build and run tests in a
build directory, there is a race condition between the
serial_test_low_level target and the test_prep target (a dependency of
serial_test_fishscript and serial_test_interactive).
As far as I can tell, these events happen in a serial build scenario
(“make test” with the “Unix Makefiles” CMake generator):
1. The fish_tests binary is built and executed.
2. The test_prep target (a dependency of serial_test_fishscript)
cleans up test directories.
3. Tests in test.fish are executed.
In a parallel build scenario, this often happens:
1. Build of the fish_tests binary is started.
2. The test_prep target cleans up test directories.
3. Build of the fish_tests binary is finished.
4. Execution of the fish_tests binary starts.
5. Execution of the fish_tests binary finishes.
6. Tests in test.fish are executed.
However, if building the fish_tests binary is fast enough but not
instant (e.g. when using ccache), this can happen:
1. Build of the fish_tests binary is started.
2. Build of the fish_tests binary is finished.
3. Execution of the fish_tests binary starts.
4. The test_prep target cleans up test directories.
5. fish_tests tests that depend on said test directories may,
depending on timing, fail because they are wiped by test_prep.
Fix this by making test_prep a dependency of serial_test_low_level so
that test_prep can’t interfere with fish_tests execution.
This is far preferable to the per-test XDG overrides that we may or may
not remember to add the next time around.
Also rename all the directories so it is clear via which variable a file
made its way into that path.
This adds a new interactive test framework based on Python's pexpect. This
is intended to supplant the TCL expect-based tests.
New tests go in `tests/pexpects/`. As a proof-of-concept, the
pipeline.expect test and the (gnarly) bind.expect test are ported to the
new framework.
Perform an ad-hoc code signing with the hardened runtime.
This ensures that these executables can pass notarization.
The code signing ID is controlled by the MAC_CODESIGN_ID CMake
cache variable.
This makes test_low_level, test_interactive, test_invocation, and
test_fishscript independent. This allows running a smaller subset of tests.
To prevent all tests running in parallel, we also have new targets
serial_test_low_level, serial_test_interactive, etc. which have the
dependency chain that enforces serial execution.
This adds support for .check files inside the tests directory. .check
files are tests designed to be run with littlecheck.
Port printf test to littlecheck and remove the printf.in test.
