Unfortunately the previous solution was too naive and misidentified
some errors.
In essence, passing regex-source couldn't work, because those could
not match any other line, so we have to inject regex-matching into the
SequenceMatcher.
Through awful hackery, this is possible.
Updates littlecheck to 0f6841bbc1674e89f512b5f19d1ad4e0227d2934.
This lets littlecheck "diff" the given output with the checks, leading
to easier to understand errors.
E.g. changing some random lines in andandoror.fish yields error output
like:
```
Testing file checks/andandoror.fish ... Failure:
The CHECK on line 36 wants:
if test 4 ok
which failed to match line stdout:9:
if test 3 ok
Context:
[...] from line 17 (stdout:6):
true && false || true: 0
if test 1 ok
if test 2 ok
if test 3 ok <= no check matches this, previous check on line 35
if test 4 ok
0 0 0
1 1 1
2 2 2
3 3 3 <= does not match CHECK '3 5 3' on line 55
4 4 4
0
1
[...] from line 126 (stdout:33):
0
0
0
<= nothing to match CHECK 'banana' on line 135
when running command:
../test/root/bin/fish checks/andandoror.fish
```
This updates littlecheck to b9c24a3.
In #7459, asan printed error output. However, because we had a failure
on stdout already, littlecheck would only print the first unmatched
line from stderr, leading to output like
```
additional output on stderr:1:
=================================================================
```
Which is of course entirely useless.
So in that case we just let it print *all* unmatched stderr lines, so
you'd get the full asan output, which presumably is of more use.
This upgrades littlecheck to 5f7deafcea4e58dd3d369eae069a3781bb6ce75e.
Commit b2f40783a2b5b0663409c4daa90b794b02dd37a6
This has better progress reporting, and the exit status of littlecheck
indicates how many test failures there were.
Mostly line breaks, one instance of tabs!
For some reason clang-format insists on two spaces before a same-line comment?
(I continue to be unimpressed with super-strict line length limits,
but I continue to believe in automatic styling, so it is what it is)
[ci skip]
This allows:
- Running scripts via shebang (not important here)
- Progress output (so we can ditch more of our run script)
- Context (only after, for now) - this is important if there is a test failure
Closes#6435.
close_fds=True is actually the default in Python 2.7 and 3.2, but not in
ancient (but still in production in Red Hat Enterprise Linux 6) Python
2.6. Enable it there as well.
