Commit graph

98 commits

Author SHA1 Message Date
Fabian Boehm
055e40467f github actions: Disable pexpect for ASAN for now
This fails basically every commit, just by blowing the time budget.
2023-05-08 19:05:44 +02:00
Mahmoud Al-Qudsi
7b0cc33f2e Add LSAN suppressions file
Suppress TLS variable leaks caused by outstanding background threads by
suppressing the ASAN interposer functions. This is possible because because
we're now using use_tls=1.

-----------------------

Direct leak of 64 byte(s) in 2 object(s) allocated from:
    #0 0x5627a1f0cc86 in __interceptor_realloc (/home/runner/work/fish-shell/fish-shell/build/fish_tests+0xb9fc86) (BuildId: da87d16730727369ad5fa46052d10337d6941fa9)
    #1 0x7f04d8800f79 in pthread_getattr_np (/lib/x86_64-linux-gnu/libc.so.6+0x95f79) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
    #2 0x5627a1f2f664 in __sanitizer::GetThreadStackTopAndBottom(bool, unsigned long*, unsigned long*) (/home/runner/work/fish-shell/fish-shell/build/fish_tests+0xbc2664) (BuildId: da87d16730727369ad5fa46052d10337d6941fa9)
    #3 0x5627a1f2fb83 in __sanitizer::GetThreadStackAndTls(bool, unsigned long*, unsigned long*, unsigned long*, unsigned long*) (/home/runner/work/fish-shell/fish-shell/build/fish_tests+0xbc2b83) (BuildId: da87d16730727369ad5fa46052d10337d6941fa9)
    #4 0x5627a1f19a0d in __asan::AsanThread::SetThreadStackAndTls(__asan::AsanThread::InitOptions const*) (/home/runner/work/fish-shell/fish-shell/build/fish_tests+0xbaca0d) (BuildId: da87d16730727369ad5fa46052d10337d6941fa9)
    #5 0x5627a1f19615 in __asan::AsanThread::Init(__asan::AsanThread::InitOptions const*) (/home/runner/work/fish-shell/fish-shell/build/fish_tests+0xbac615) (BuildId: da87d16730727369ad5fa46052d10337d6941fa9)
    #6 0x5627a1f19b01 in __asan::AsanThread::ThreadStart(unsigned long long) (/home/runner/work/fish-shell/fish-shell/build/fish_tests+0xbacb01) (BuildId: da87d16730727369ad5fa46052d10337d6941fa9)
    #7 0x7f04d87ffb42  (/lib/x86_64-linux-gnu/libc.so.6+0x94b42) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
    #8 0x7f04d88919ff  (/lib/x86_64-linux-gnu/libc.so.6+0x1269ff) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
2023-05-02 11:52:42 -05:00
Mahmoud Al-Qudsi
905430629d Use ASAN_OPTIONS fast_unwind_on_malloc=0
This is much slower but gives proper stack traces for calls emanating from code
that wasn't compiled with -fno-omit-frame-pointer.
2023-05-02 11:52:42 -05:00
Mahmoud Al-Qudsi
c43e040c7c Fix spurious ASAN __cxa_thread_atexit_impl() leaks
Set use_tls back to its default of 1.

This is required to work around an ASAN/LSAN virtualization bug but seems to be
behind the random __cxa_thread_atexit_impl() leaks?
2023-05-02 11:52:41 -05:00
Mahmoud Al-Qudsi
544bd183da Add and use ASAN blacklist
Blacklist an apparently false positive in the underlying runtime.
2023-04-29 11:02:59 -05:00
ridiculousfish
2d6f752f6e Revert "Add link-asan to RUSTFLAGS in CI"
This reverts commit 8bb1bb8ae1.
2023-04-06 14:29:18 -07:00
ridiculousfish
8bb1bb8ae1 Add link-asan to RUSTFLAGS in CI
This fixes our CI for the new crates we're about to add.
2023-03-26 11:07:15 -07:00
Mahmoud Al-Qudsi
1bdb7dffaf Use cargo build -Z build-std for ASAN
This is recommended and increases coverage.
2023-03-08 11:27:15 -06:00
Mahmoud Al-Qudsi
ce5686edc7 Have ASAN CI use debug build
This catches things that might be optimized away by the compiler.
2023-03-07 13:04:28 -06:00
Mahmoud Al-Qudsi
91cf526d23
Enable rust address sanitizer for asan ci job (#9643)
Rust has multiple sanitizers available (with llvm integration).
-Zsanitizer=address catches the most likely culprits but we may want to set up a
separate job w/ -Zsanitizer=memory to catch uninitialized reads.

It might be necessary to execute `cargo build` as `cargo build -Zbuild-std` to
get full coverage.

When we're linking against the hybrid C++ codebase, the sanitizer library is
injected into the binary by also include `-fsanitize=address` in CXXFLAGS - we
do *not* want to manually opt-into `-lasan`. We also need to manually specify
the desired target triple as a CMake variable and then explicitly pass it to all
`cargo` invocations if building with ASAN.

Corrosion has been patched to make sure it follows these rules.

The `cargo-test` target is failing to link under ASAN. For some reason it has
autocxx/ffi dependencies even though only rust-native, ffi-free code should be
tested (and one would think the situation wouldn't change depending on the
presence of the sanitizer flag). It's been disabled under ASAN for now.
2023-03-06 18:15:36 -06:00
Xiretza
dff7db2f16
Run rustfmt and clippy in CI (#9616)
* Add machine-readable MSRV to Cargo.toml
* Fix clippy warnings
* CI: add rustfmt and clippy checks
2023-02-26 13:20:20 -06:00
Fabian Boehm
ad22bf9387 GH Actions: Use our MSRV as the rust-version
Currently we're at 1.67, I don't want to accidentally introduce 1.68
features once that's released
2023-02-20 19:40:47 +01:00
Mahmoud Al-Qudsi
4f6fe0999e Disable TSAN in CI for now
See issues encountered in #9586 due to TSAN not recognizing valid/safe rust
patterns.
2023-02-19 15:42:07 -06:00
ridiculousfish
d843b67d2d Initial Rust commit 2023-02-02 19:34:47 -07:00
LingMan
0af2a7a9b7 Upgrade GitHub action dessant/lock-threads to v4
In v3 several input parameters where renamed and since v4 it requires Node.js 16.

This resolves warnings about Node.js 12 and `set-output` being deprecated and
slated for removal in the `Lock threads` workflow.
2023-02-01 10:40:46 +08:00
Mahmoud Al-Qudsi
31f63b2f87 Upgrade GitHub CI actions/checkout to v3
This addresses the node v12 deprecation warning in the GitHub CI, caused by the
dependency on actions/checkout@v2.

While actions/checkout@v3 introduces some new features and changes some
defaults, the subset of features that we use should not be affected by this
migration.

The "breaking change" from v2 to v3 can be seen at [0]. Since we are tracking
only v2 without a dot release specified, we are already opting into any breakage
across minor versions, so really the only change of note is the node version
upgrade.

[0]: https://github.com/actions/checkout/compare/v2.4.2...v3.0.0
2023-01-31 11:50:57 -06:00
ridiculousfish
5f23da9939 Add a TSAN workaround and re-enable the test
This reverts commit 865602e8d1.
2022-12-19 15:54:23 -08:00
Fabian Boehm
865602e8d1 Github Actions: Disable tsan once more 2022-12-19 19:55:36 +01:00
ridiculousfish
4159b2a33b Disable shebangless script tests in CI with sanitizers
Sanitizers inject a busted posix_spawn interceptor which mishandles
shebangless scripts. Disable this test under sanitizers.
2022-12-02 17:32:52 -08:00
ridiculousfish
c844eea661 Restore lockthreads.yml
This file should be modified through pull requests.

This reverts commit bc71f0937b.
This reverts commit 99cac0b1b9.
This reverts commit 0f0da3c3d8.
2022-11-12 10:09:17 -08:00
Aaron Gyes
bc71f0937b Revert "Set issue lock timeout back"
This reverts commit 99cac0b1b9.
2022-11-12 05:04:14 -08:00
Fabian Boehm
99cac0b1b9 Set issue lock timeout back
It is 1 whole year, for an already closed issue.

Any "engagement" that happens at that point is irrelevant to the
original issue at hand, and a new issue should be opened instead.
Increasing the grace period even further is even less likely to be helpful.
2022-11-12 12:29:22 +01:00
Aaron Gyes
0f0da3c3d8 lockthreads.yml: decrease PR threshold, increase issue threshold
In my experience we rarely see a PR that may have activity after
365 days; issue reports are a very different story.

Goal: engagement
2022-11-12 03:16:48 -08:00
Mahmoud Al-Qudsi
44c9c51841 Disable leak detection in test_autosuggest_suggest_special() under CI
This is a false positive as a result of disabling TLS support in LSAN due to an
incompatibility with newer versions of glibc.

Also remove the older workaround (because it didn't work).
2022-10-24 19:02:49 -05:00
Mahmoud Al-Qudsi
bfa172852f Add a workaround for intermittent LSAN crash under CI
LSAN seems to have an issue with glibc's TLS functionality that causes it to
intermittently crash with SIGSEGV when run virtualized, as it is in our CI.

Relevant GitHub issues:
* https://github.com/google/sanitizers/issues/1342
* https://github.com/google/sanitizers/issues/1409
2022-10-24 18:56:55 -05:00
Mahmoud Al-Qudsi
4f8a7c4779 Drop LSAN CI options that break tests
LSAN with verbosity=1 or log_threads=1 adds output to stderr, breaking
littlecheck tests.
2022-10-24 18:56:55 -05:00
Mahmoud Al-Qudsi
de62091b03 Correctly set ASAN/UBSAN/LSAN options for CI
These are NOT build-time defines but rather run-time environment variables! They
have never had any effect and we have effectively never used them to affect
sanitizer behavior under CI with ASAN/UBSAN/LSAN enabled.

(I caught this because the tests don't pass with either of LSAN_OPTIONS
`verbosity=1` or `log_threads=1` because they inject text into the stderr
output, ensuring they never pass littlecheck.)
2022-10-24 18:56:55 -05:00
Mahmoud Al-Qudsi
63a2fdd773 Re-enable tests under ASAN/LSAN/UBSAN in GitHub CI
With the previous workaround skipping `test_autosuggest_suggest_special()` when
LSAN is enabled, the sanitizer seems to run to completion just fine.
2022-10-24 18:56:55 -05:00
ridiculousfish
6c81eb14d7 CI: ubuntu-32bit-vendored-pcre2 -> ubuntu-32bit-fetched-pcre2
We no longer vendor PCRE2 sources, instead we fetch them from the
official repo.
2022-07-02 19:00:08 -07:00
naveen
1331b65023 chore: Set permissions for GitHub actions
Restrict the GitHub token permissions only to the required ones; this way, even if the attackers will succeed in compromising your workflow, they won’t be able to do much.

- Included permissions for the action. https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions

https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions

https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs

[Keeping your GitHub Actions and workflows secure Part 1: Preventing pwn requests](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)

Signed-off-by: naveen <172697+naveensrinivasan@users.noreply.github.com>
2022-06-07 14:34:59 +02:00
Fabian Homborg
648fdc18f8 Remove CodeQL
I don't think we've had a lot of actionable errors or warnings from it, and it takes 30 minutes per push to master.
2022-06-02 16:45:05 +02:00
Fabian Homborg
7dce2c0607 Disable ASAN test
The current Github Actions ubuntu-latest image crashes in the
autosuggest_suggest_special test with ASAN.

We have not been able to reproduce this locally, and this is getting
in the way.

I have no idea how to disable this test on ASAN specifically, all my
attempts have failed. So the only recourse I know is to disable the
ASAN tests on GA entirely.
2022-04-18 10:06:25 +02:00
Fabian Homborg
e3c4cde042 Enable LSAN_OPTIONS
Whyyyyy does this not tell me what's broken by *default*?
2022-04-17 13:57:52 +02:00
ridiculousfish
06666f3d1d CI: Set CMAKE_BUILD_PARALLEL_LEVEL to 4.
This speeds up the CI build, since before it was effectively 1.
Build times on ubuntu-latest are reduced by slightly over 2 minutes.

Note Linux CI runners are defined to have 2 cores and Mac runners 3.
2022-01-02 18:01:54 -08:00
Aaron Gyes
124dac4781
Update main.yml 2021-12-02 05:54:29 -08:00
Aaron Gyes
1df7e9802f
CodeQL on master pushes
LGTM uses CodeQL and gives us green lights on PRs. This should live at https://github.com/fish-shell/fish-shell/security/code-scanning.
2021-12-02 04:23:09 -08:00
Fabian Homborg
e2331eb034 ISSUE_TEMPLATE: Add a marker for where the text goes
We've had at least two issues where people put their text into the
comment, making it look like they filled out nothing.

The alternative is to use Github's new YAML-based system, but tbh I'm
not feeling it.
2021-11-28 15:44:10 +01:00
Kid
03f9c3d2e6 Give tests a more generic name 2021-11-16 12:06:55 +01:00
Aaron Gyes
1261b53d96
Revert 2021-11-07 01:06:57 -07:00
Aaron Gyes
61ccf87cee Revert! These were for my repo. Oh no.
Revert "Move the file - it was trying to triggr stuff."

This reverts commit 108560ff55.

Revert "fixup"

This reverts commit fdc0f2f6a7.

Revert "configure more analyzers, skip vendored stuff."

This reverts commit 023f6683f0.

Revert "Update codeql-analysis.yml"

This reverts commit ea25db544e.
2021-11-06 23:06:01 -07:00
Aaron Gyes
108560ff55 Move the file - it was trying to triggr stuff. 2021-11-06 22:52:29 -07:00
Aaron Gyes
fdc0f2f6a7 fixup 2021-11-06 22:18:19 -07:00
Aaron Gyes
023f6683f0 configure more analyzers, skip vendored stuff. 2021-11-06 22:16:28 -07:00
Aaron Gyes
ea25db544e
Update codeql-analysis.yml 2021-11-06 20:09:08 -07:00
Aaron Gyes
39a7f904b4
Update codeql-analysis.yml 2021-11-06 16:34:10 -07:00
Aaron Gyes
2cdc6002d9
Create codeql-analysis.yml 2021-11-06 16:27:00 -07:00
Fabian Homborg
fcb74f236a Lock enhancements and RFCs again
The point here is to let issues be *done*, and have any *new*
discussions happen in *new* issues so you can decouple the context.

This revert pending further discussion.
2021-11-02 18:15:30 +01:00
Aaron Gyes
e53c284753
Update lockthreads.yml
exclude RFCs, enhancement requests, have it run weekly.
2021-11-02 09:43:47 -07:00
Aaron Gyes
e30b6c47f0 Adjust the GitHub thread locker script.
Set locked thread inactivity count to default 365.

Add 'needs more info' as an obvious on its face exception.

The default seems quite an inconventient, very strict thing t do:
it will lock threads that are closed and quiet because they have
been quiet and closed. This seems to make it hard to talk about
issues after they are closed or contribute. I can as a fish-shell
contributor, but that's not really the point.

Practically, right now to reply to any PR or any issue fixed in
July, well you can't.
2021-10-24 04:50:25 -07:00
Mahmoud Al-Qudsi
aa25c4eccc Run tests serially under CI 2021-08-29 08:56:12 +02:00