This has been driving nuts for years. The output of the diff emitted
when a test fails was always reversed, because the diff tool is called
with `${difftool} ${new} ${old}` so all the `-` and `+` contexts are
reversed, and the highlights are all screwed up.
The output of a `make test` run should show what has changed from the
baseline/expected, not how the expected differs from the actual. When
considered from both the perspective of intentional changes to the test
outputs and failed test outputs, it is desirable to see how the test
output has changed from the previously expected, and not the other way
around.
(If you were used to the previous behavior, I apologize. But it was
wrong.)
This removes the caret as a shorthand for redirecting stderr.
Note that stderr may be redirected to a file via 2>/some/path...
and may be redirected with a pipe via 2>|.
Fixes#4394
The tests/*.status files aren't useful so eliminate them. Just verify whether
a given test module exited with a success status. There isn't any point in
having a "status" file that indicates the test module should exit with a
success (zero) status.
Closes#3208Closes#3209
I noticed while fixing issue #2702 that the fish program being tested
was sourcing config.fish files outside of the current build. This also
happens when Travis CI runs the tests but isn't an issue there because
of how Travis is configured to execute the tests.
I also noticed that running `make test` was polluting my personal fish
history; which will become a bigger problem if and when the fishd universal
var file is moved from $XDG_CONFIG_HOME to $XDG_DATA_HOME.
This change makes it possible for an individual to run the tests on
their local machine secure in the knowledge that only the config.fish and
related files from their git repository will be used and doing so won't
pollute their personal fish history.
Resolves#469
Update the test runners so they set up their own environment in
test_util.fish. This simplifies the Makefile and paves the way for
adding utility functions for use in the tests themselves.
Split test_interactive off from test_fishscript and add a new target
test_high_level that tests both.
Add some Makefile magic so the tests can be run serially without using
sub-make, which gets rid of a little noise from the make output.
Rewrite interactive tests to look better.
Split `make test` into two targets `make test_low_level` and `make
test_fishscript`, primarily so fishscript tests can be rechecked quickly
after edits.
Reformat the test.fish file and update some of the code to be a little
more straightforward (e.g. `if not cmd` instead of `if cmd; else`).
This commit makes fish_parse_opt stop at first non-option argument,
which expected as fish script name and pass the remaining argument to
that script. I also updated the test cases.
