2024-02-19 17:09:47 +00:00
|
|
|
//! CI script used for Bevy.
|
|
|
|
|
|
2024-04-25 18:52:11 +00:00
|
|
|
mod ci;
|
tools: Refactor CI to use `argh` (#12923)
# Objective
The CI tool currently parses input manually. This has worked fine, but
makes it just a bit more difficult to maintain and extend. Additionally,
it provides no usage help for devs wanting to run the tool locally.
It would be better if parsing was handled by a dedicated CLI library
like [`clap`](https://github.com/clap-rs/clap) or
[`argh`](https://github.com/google/argh).
## Solution
Use `argh` to parse command line input for CI.
`argh` was chosen over `clap` and other tools due to being more
lightweight and already existing in our dependency tree.
Using `argh`, the usage notes are generated automatically:
```
$ cargo run -p ci --quiet -- --help
Usage: ci [--keep-going] [<command>] [<args>]
The CI command line tool for Bevy.
Options:
--keep-going continue running commands even if one fails
--help display usage information
Commands:
lints Alias for running the `format` and `clippy` subcommands.
doc Alias for running the `doc-test` and `doc-check`
subcommands.
compile Alias for running the `compile-fail`, `bench-check`,
`example-check`, `compile-check`, and `test-check`
subcommands.
format Check code formatting.
clippy Check for clippy warnings and errors.
test Runs all tests (except for doc tests).
test-check Checks that all tests compile.
doc-check Checks that all docs compile.
doc-test Runs all doc tests.
compile-check Checks that the project compiles.
cfg-check Checks that the project compiles using the nightly compiler
with cfg checks enabled.
compile-fail Runs the compile-fail tests.
bench-check Checks that the benches compile.
example-check Checks that the examples compile.
```
This PR makes each subcommand more modular, allowing them to be called
from other subcommands. This also makes it much easier to extract them
out of `main.rs` and into their own dedicated modules.
Additionally, this PR improves failure output:
```
$ cargo run -p ci -- lints
...
One or more CI commands failed:
format: Please run 'cargo fmt --all' to format your code.
```
Including when run with the `--keep-going` flag:
```
$ cargo run -p ci -- --keep-going lints
...
One or more CI commands failed:
- format: Please run 'cargo fmt --all' to format your code.
- clippy: Please fix clippy errors in output above.
```
### Future Work
There are a lot of other things we could possibly clean up. I chose to
try and keep the API surface as unchanged as I could (for this PR at
least).
For example, now that each subcommand is an actual command, we can
specify custom arguments for each.
The `format` subcommand could include a `--check` (making the default
fun `cargo fmt` as normal). Or the `compile-fail` subcommand could
include `--ecs`, `--reflect`, and `--macros` flags for specifying which
set of compile fail tests to run.
The `--keep-going` flag could be split so that it doesn't do double-duty
where it also enables `--no-fail-fast` for certain commands. Or at least
make it more explicit via renaming or using alternative flags.
---
## Changelog
- Improved the CI CLI tool
- Now includes usage info with the `--help` option!
- [Internal] Cleaned up and refactored the `tools/ci` crate using the
`argh` crate
## Migration Guide
The CI tool no longer supports running multiple subcommands in a single
call. Users who are currently doing so will need to split them across
multiple commands:
```bash
# BEFORE
cargo run -p ci -- lints doc compile
# AFTER
cargo run -p ci -- lints && cargo run -p ci -- doc && cargo run -p ci -- compile
# or
cargo run -p ci -- lints; cargo run -p ci -- doc; cargo run -p ci -- compile
# or
cargo run -p ci -- lints
cargo run -p ci -- doc
cargo run -p ci -- compile
```
2024-04-13 01:48:37 +00:00
|
|
|
mod commands;
|
2024-04-25 18:52:11 +00:00
|
|
|
mod prepare;
|
|
|
|
|
|
2024-09-24 11:42:59 +00:00
|
|
|
pub use self::{ci::*, prepare::*};
|
Quality-of-life updates for running CI locally (#12242)
# Objective
- Make running CI locally relatively less painful by allowing
continuation after failure
- Fix #12206
## Solution
Previously, `ci` would halt after encounting a failure (or shortly
thereafter). For instance, if you ran `cargo run -p ci` and a single
test within a CI check failed somewhere in the middle, you would never
even reach many of the other tests within the check and certainly none
of the CI checks that came afterward. This was annoying; if I am fixing
issues with CI tests, I want to just see all of the issues up-front
instead of having to rerun CI every time I fix something to see the next
error.
Furthermore, it is not infrequent (because of subtle
configuration/system differences) to encounter spurious CI failures
locally; previously, when these occurred, they would make the process of
running CI largely pointless, since you would have to push your branch
in order to see your actual CI failures from the automated testing
service.
Now, when running `ci` locally, we have a couple new tools to ameliorate
these and provide a smoother experience:
- Firstly, there is a new optional flag `--keep-going` that can be
passed while calling `ci` (e.g. `cargo run -p ci -- doc --keep-going`).
It has the following effects:
- It causes the `--keep-going` flag to be passed to the script's `cargo
doc` and `cargo check` invocations, so that they do not stop when they
encounter an error in a single module; instead, they keep going (!) and
find errors subsequently.
- It causes the `--no-fail-fast` flag to be passed to the script's
`cargo test` invocations, to similar effect.
- Finally, it causes `ci` itself not to abort after a single check
fails; instead, it will run every check that was invoked.
Thus, for instance, `cargo run -p ci -- --keep-going` will run every CI
check even if it encounters intermediate failures, and every such check
will itself be run to completion.
- Secondly, we now allow multiple ordinary arguments to be passed to
`ci`. For instance, `cargo -p ci -- doc test` now executes both the
'doc' checks and the 'test' checks. This allows the caller to run the
tests they care about with fewer invocations of `ci`.
As of this PR, automated testing will remain unchanged.
---
## Changelog
- tools/ci/src/main.rs refactored into staging and execution steps,
since the previous control flow did not naturally support continuing
after failure.
- Added "--keep-going" flag to `ci`.
- Added support for invoking `ci` with multiple arguments.
---
## Discussion
### Design considerations
I had originally split this into separate flags that controlled:
1. whether `--keep-going`/`--no-fail-fast` would be passed to the
constituent commands
2. whether `ci` would continue after a component test failed
However, I decided to merge these two behaviors, since I think that, if
you're in the business of seeing all the errors, you'll probably want to
actually see all of the errors. One slightly annoying thing, however,
about the new behavior with `--keep-going`, is that you will sometimes
find yourself wishing that the script would pause or something, since it
tends to fill the screen with a lot of junk. I have found that sending
stdout to /dev/null helps quite a bit, but I don't think `cargo fmt` or
`cargo clippy` actually write to stderr, so you need to be cognizant of
that (and perhaps invoke the lints separately).
~~Next, I'll mention that I feel somewhat strongly that the current
behavior of `ci` for automated testing should remain the same, since its
job is more like detecting that errors exist rather than finding all of
them.~~ (I was convinced this could have value.) Orthogonally, there is
the question of whether or not we might want to make this (or something
similar) actually the default behavior and make the automated test
script invoke some optional flags — it doesn't have to type with its
hands, after all. I'm not against that, but I don't really want to rock
the boat much more with this PR, since anyone who looks at the diff
might already be a little incensed.
2024-03-04 22:03:02 +00:00
|
|
|
|
2021-02-22 08:42:19 +00:00
|
|
|
fn main() {
|
2024-04-25 18:52:11 +00:00
|
|
|
argh::from_env::<CI>().run();
|
2021-02-22 08:42:19 +00:00
|
|
|
}
|
