# Objective
The CI tool currently parses input manually. This has worked fine, but
makes it just a bit more difficult to maintain and extend. Additionally,
it provides no usage help for devs wanting to run the tool locally.
It would be better if parsing was handled by a dedicated CLI library
like [`clap`](https://github.com/clap-rs/clap) or
[`argh`](https://github.com/google/argh).
## Solution
Use `argh` to parse command line input for CI.
`argh` was chosen over `clap` and other tools due to being more
lightweight and already existing in our dependency tree.
Using `argh`, the usage notes are generated automatically:
```
$ cargo run -p ci --quiet -- --help
Usage: ci [--keep-going] [<command>] [<args>]
The CI command line tool for Bevy.
Options:
--keep-going continue running commands even if one fails
--help display usage information
Commands:
lints Alias for running the `format` and `clippy` subcommands.
doc Alias for running the `doc-test` and `doc-check`
subcommands.
compile Alias for running the `compile-fail`, `bench-check`,
`example-check`, `compile-check`, and `test-check`
subcommands.
format Check code formatting.
clippy Check for clippy warnings and errors.
test Runs all tests (except for doc tests).
test-check Checks that all tests compile.
doc-check Checks that all docs compile.
doc-test Runs all doc tests.
compile-check Checks that the project compiles.
cfg-check Checks that the project compiles using the nightly compiler
with cfg checks enabled.
compile-fail Runs the compile-fail tests.
bench-check Checks that the benches compile.
example-check Checks that the examples compile.
```
This PR makes each subcommand more modular, allowing them to be called
from other subcommands. This also makes it much easier to extract them
out of `main.rs` and into their own dedicated modules.
Additionally, this PR improves failure output:
```
$ cargo run -p ci -- lints
...
One or more CI commands failed:
format: Please run 'cargo fmt --all' to format your code.
```
Including when run with the `--keep-going` flag:
```
$ cargo run -p ci -- --keep-going lints
...
One or more CI commands failed:
- format: Please run 'cargo fmt --all' to format your code.
- clippy: Please fix clippy errors in output above.
```
### Future Work
There are a lot of other things we could possibly clean up. I chose to
try and keep the API surface as unchanged as I could (for this PR at
least).
For example, now that each subcommand is an actual command, we can
specify custom arguments for each.
The `format` subcommand could include a `--check` (making the default
fun `cargo fmt` as normal). Or the `compile-fail` subcommand could
include `--ecs`, `--reflect`, and `--macros` flags for specifying which
set of compile fail tests to run.
The `--keep-going` flag could be split so that it doesn't do double-duty
where it also enables `--no-fail-fast` for certain commands. Or at least
make it more explicit via renaming or using alternative flags.
---
## Changelog
- Improved the CI CLI tool
- Now includes usage info with the `--help` option!
- [Internal] Cleaned up and refactored the `tools/ci` crate using the
`argh` crate
## Migration Guide
The CI tool no longer supports running multiple subcommands in a single
call. Users who are currently doing so will need to split them across
multiple commands:
```bash
# BEFORE
cargo run -p ci -- lints doc compile
# AFTER
cargo run -p ci -- lints && cargo run -p ci -- doc && cargo run -p ci -- compile
# or
cargo run -p ci -- lints; cargo run -p ci -- doc; cargo run -p ci -- compile
# or
cargo run -p ci -- lints
cargo run -p ci -- doc
cargo run -p ci -- compile
```
# Objective
- The CI tool is slowly getting more difficult to maintain and could use
a bit of refactoring.
## Solution
- Do a first pass of small improvements.
## For Reviewers
This PR is best reviewed commit-by-commit, because I separate it into a
collection of small changes. :)
# Objective
- There are several redundant imports in the tests and examples that are
not caught by CI because additional flags need to be passed.
## Solution
- Run `cargo check --workspace --tests` and `cargo check --workspace
--examples`, then fix all warnings.
- Add `test-check` to CI, which will be run in the check-compiles job.
This should catch future warnings for tests. Examples are already
checked, but I'm not yet sure why they weren't caught.
## Discussion
- Should the `--tests` and `--examples` flags be added to CI, so this is
caught in the future?
- If so, #12818 will need to be merged first. It was also a warning
raised by checking the examples, but I chose to split off into a
separate PR.
---------
Co-authored-by: François Mockers <francois.mockers@vleue.com>
# Objective
- Make running CI locally relatively less painful by allowing
continuation after failure
- Fix#12206
## Solution
Previously, `ci` would halt after encounting a failure (or shortly
thereafter). For instance, if you ran `cargo run -p ci` and a single
test within a CI check failed somewhere in the middle, you would never
even reach many of the other tests within the check and certainly none
of the CI checks that came afterward. This was annoying; if I am fixing
issues with CI tests, I want to just see all of the issues up-front
instead of having to rerun CI every time I fix something to see the next
error.
Furthermore, it is not infrequent (because of subtle
configuration/system differences) to encounter spurious CI failures
locally; previously, when these occurred, they would make the process of
running CI largely pointless, since you would have to push your branch
in order to see your actual CI failures from the automated testing
service.
Now, when running `ci` locally, we have a couple new tools to ameliorate
these and provide a smoother experience:
- Firstly, there is a new optional flag `--keep-going` that can be
passed while calling `ci` (e.g. `cargo run -p ci -- doc --keep-going`).
It has the following effects:
- It causes the `--keep-going` flag to be passed to the script's `cargo
doc` and `cargo check` invocations, so that they do not stop when they
encounter an error in a single module; instead, they keep going (!) and
find errors subsequently.
- It causes the `--no-fail-fast` flag to be passed to the script's
`cargo test` invocations, to similar effect.
- Finally, it causes `ci` itself not to abort after a single check
fails; instead, it will run every check that was invoked.
Thus, for instance, `cargo run -p ci -- --keep-going` will run every CI
check even if it encounters intermediate failures, and every such check
will itself be run to completion.
- Secondly, we now allow multiple ordinary arguments to be passed to
`ci`. For instance, `cargo -p ci -- doc test` now executes both the
'doc' checks and the 'test' checks. This allows the caller to run the
tests they care about with fewer invocations of `ci`.
As of this PR, automated testing will remain unchanged.
---
## Changelog
- tools/ci/src/main.rs refactored into staging and execution steps,
since the previous control flow did not naturally support continuing
after failure.
- Added "--keep-going" flag to `ci`.
- Added support for invoking `ci` with multiple arguments.
---
## Discussion
### Design considerations
I had originally split this into separate flags that controlled:
1. whether `--keep-going`/`--no-fail-fast` would be passed to the
constituent commands
2. whether `ci` would continue after a component test failed
However, I decided to merge these two behaviors, since I think that, if
you're in the business of seeing all the errors, you'll probably want to
actually see all of the errors. One slightly annoying thing, however,
about the new behavior with `--keep-going`, is that you will sometimes
find yourself wishing that the script would pause or something, since it
tends to fill the screen with a lot of junk. I have found that sending
stdout to /dev/null helps quite a bit, but I don't think `cargo fmt` or
`cargo clippy` actually write to stderr, so you need to be cognizant of
that (and perhaps invoke the lints separately).
~~Next, I'll mention that I feel somewhat strongly that the current
behavior of `ci` for automated testing should remain the same, since its
job is more like detecting that errors exist rather than finding all of
them.~~ (I was convinced this could have value.) Orthogonally, there is
the question of whether or not we might want to make this (or something
similar) actually the default behavior and make the automated test
script invoke some optional flags — it doesn't have to type with its
hands, after all. I'm not against that, but I don't really want to rock
the boat much more with this PR, since anyone who looks at the diff
might already be a little incensed.
# Objective
Getting this error running ci locally.
```
$ cargo check -Zcheck-cfg --workspace
error: the `-Z` flag is only accepted on the nightly channel of Cargo, but this is the `stable` channel
See https://doc.rust-lang.org/book/appendix-07-nightly-rust.html for more information about Rust release channels.
thread 'main' panicked at tools\ci\src\main.rs:166:14:
Please fix failing cfg checks in output above.: command exited with non-zero code `cargo check -Zcheck-cfg --workspace`: 101
```
## Solution
- Add `+nightly` flag to the `check-cfg` pass.
---
Obviously we shouldn't be running `nightly` Cargo, but at least now
local running of `cargo run -p ci` will pass.
# Objective
- Add the new `-Zcheck-cfg` checks to catch more warnings
- Fixes#12091
## Solution
- Create a new `cfg-check` to the CI that runs `cargo check -Zcheck-cfg
--workspace` using cargo nightly (and fails if there are warnings)
- Fix all warnings generated by the new check
---
## Changelog
- Remove all redundant imports
- Fix cfg wasm32 targets
- Add 3 dead code exceptions (should StandardColor be unused?)
- Convert ios_simulator to a feature (I'm not sure if this is the right
way to do it, but the check complained before)
## Migration Guide
No breaking changes
---------
Co-authored-by: Alice Cecile <alice.i.cecile@gmail.com>
# Objective
- Some workspace members do not inherit the global lints.
## Solution
- Add a `[lints]` entry for all files returned by `rg
--files-without-match -F "[lints]" **/Cargo.toml`, except the compile
failure tests since these aren't part of the workspace.
- Add some docstrings where needed.
---------
Co-authored-by: Alice Cecile <alice.i.cecile@gmail.com>
# Objective
Partially addresses #10612
After moving the initial docs_markdown warning, this is a PR that moves
the rest of the CI clippy lint definitions to the cargo.toml.
## Solution
- the `tools/ci/src/main.rs` clippy lints removed and just the warning
flag remains.
- the warnings moved to the root cargo workspace toml.
# Objective
Partially Addresses #10612
fix: add clippy::doc_markdown linting to cargo workspace
Rather than do all the warnings in `tools/ci/src/main.rs` in one-shot,
just wanted to have an initial pr adding the first one to get the form
correct as some may trigger build errors and require changes to get
merged more easily.
## Solution
- adding the doc_markdown and removing it from the ci check as it'll now
be a build error during normal compilation.
---------
Co-authored-by: François <mockersf@gmail.com>
# Objective
- See fewer warnings when running `cargo clippy` locally.
## Solution
- allow `clippy::type_complexity` in more places, which also signals to
users they should do the same.
# Objective
Replace instances of
```rust
for x in collection.iter{_mut}() {
```
with
```rust
for x in &{mut} collection {
```
This also changes CI to no longer suppress this lint. Note that since
this lint only shows up when using clippy in pedantic mode, it was
probably unnecessary to suppress this lint in the first place.
# Objective
[Rust 1.72.0](https://blog.rust-lang.org/2023/08/24/Rust-1.72.0.html) is
now stable.
# Notes
- `let-else` formatting has arrived!
- I chose to allow `explicit_iter_loop` due to
https://github.com/rust-lang/rust-clippy/issues/11074.
We didn't hit any of the false positives that prevent compilation, but
fixing this did produce a lot of the "symbol soup" mentioned, e.g. `for
image in &mut *image_events {`.
Happy to undo this if there's consensus the other way.
---------
Co-authored-by: François <mockersf@gmail.com>
# Objective
Bevy code tends to make heavy use of the [newtype](
https://doc.rust-lang.org/rust-by-example/generics/new_types.html)
pattern, which is why we have a dedicated derive for
[`Deref`](https://doc.rust-lang.org/std/ops/trait.Deref.html) and
[`DerefMut`](https://doc.rust-lang.org/std/ops/trait.DerefMut.html).
This derive works for any struct with a single field:
```rust
#[derive(Component, Deref, DerefMut)]
struct MyNewtype(usize);
```
One reason for the single-field limitation is to prevent confusion and
footguns related that would arise from allowing multi-field structs:
<table align="center">
<tr>
<th colspan="2">
Similar structs, different derefs
</th>
</tr>
<tr>
<td>
```rust
#[derive(Deref, DerefMut)]
struct MyStruct {
foo: usize, // <- Derefs usize
bar: String,
}
```
</td>
<td>
```rust
#[derive(Deref, DerefMut)]
struct MyStruct {
bar: String, // <- Derefs String
foo: usize,
}
```
</td>
</tr>
<tr>
<th colspan="2">
Why `.1`?
</th>
</tr>
<tr>
<td colspan="2">
```rust
#[derive(Deref, DerefMut)]
struct MyStruct(Vec<usize>, Vec<f32>);
let mut foo = MyStruct(vec![123], vec![1.23]);
// Why can we skip the `.0` here?
foo.push(456);
// But not here?
foo.1.push(4.56);
```
</td>
</tr>
</table>
However, there are certainly cases where it's useful to allow for
structs with multiple fields. Such as for structs with one "real" field
and one `PhantomData` to allow for generics:
```rust
#[derive(Deref, DerefMut)]
struct MyStruct<T>(
// We want use this field for the `Deref`/`DerefMut` impls
String,
// But we need this field so that we can make this struct generic
PhantomData<T>
);
// ERROR: Deref can only be derived for structs with a single field
// ERROR: DerefMut can only be derived for structs with a single field
```
Additionally, the possible confusion and footguns are mainly an issue
for newer Rust/Bevy users. Those familiar with `Deref` and `DerefMut`
understand what adding the derive really means and can anticipate its
behavior.
## Solution
Allow users to opt into multi-field `Deref`/`DerefMut` derives using a
`#[deref]` attribute:
```rust
#[derive(Deref, DerefMut)]
struct MyStruct<T>(
// Use this field for the `Deref`/`DerefMut` impls
#[deref] String,
// We can freely include any other field without a compile error
PhantomData<T>
);
```
This prevents the footgun pointed out in the first issue described in
the previous section, but it still leaves the possible confusion
surrounding `.0`-vs-`.#`. However, the idea is that by making this
behavior explicit with an attribute, users will be more aware of it and
can adapt appropriately.
---
## Changelog
- Added `#[deref]` attribute to `Deref` and `DerefMut` derives
# Objective
I noticed that running the following command didn't actually do anything:
```
cargo run -p ci -- bench-check
```
## Solution
Made it so that running `cargo run -p ci -- bench-check` actually runs a compile check on the `benches` directory.
# Objective
There isn't really a way to test that code using bevy_reflect compiles or doesn't compile for certain scenarios. This would be especially useful for macro-centric PRs like #6511 and #6042.
## Solution
Using `bevy_ecs_compile_fail_tests` as reference, added the `bevy_reflect_compile_fail_tests` crate.
Currently, this crate contains a very simple test case. This is so that we can get the basic foundation of this crate agreed upon and merged so that more tests can be added by other PRs.
### Open Questions
- [x] Should this be added to CI? (Answer: Yes)
---
## Changelog
- Added the `bevy_reflect_compile_fail_tests` crate for testing compilation errors
# Objective
- Have an easy way to compare spans between executions
## Solution
- Add a tool to compare spans from chrome traces
```bash
> cargo run --release -p spancmp -- --help
Compiling spancmp v0.1.0
Finished release [optimized] target(s) in 1.10s
Running `target/release/spancmp --help`
spancmp
USAGE:
spancmp [OPTIONS] <TRACE> [SECOND_TRACE]
ARGS:
<TRACE>
<SECOND_TRACE>
OPTIONS:
-h, --help Print help information
-p, --pattern <PATTERN> Filter spans by name matching the pattern
-t, --threshold <THRESHOLD> Filter spans that have an average shorther than the threshold
[default: 0]
```
for each span, it will display the count, minimum duration, average duration and max duration. It can be filtered by a pattern on the span name or by a minimum average duration.
just displaying a trace
comparing two traces
Co-authored-by: Robert Swain <robert.swain@gmail.com>
# Objective
- Original objective was to add doc build warning check to the ci local execution
- I somewhat deviated and changed other things...
## Solution
`cargo run -p ci` can now take more parameters:
* `format` - cargo fmt
* `clippy` - clippy
* `compile-fail` - bevy_ecs_compile_fail_tests tests
* `test` - tests but not doc tests and do not build examples
* `doc-test` - doc tests
* `doc-check` - doc build and warnings
* `bench-check` - check that benches build
* `example-check` - check that examples build
* `lints` - group - run lints and format and clippy
* `doc` - group - run doc-test and doc-check
* `compile` - group - run compile-fail and bench-check and example-check
* not providing a parameter will run everything
Ci is using those when possible:
* `build` jobs now don't run doc tests and don't build examples. it makes this job faster, but doc tests and examples are not built for each architecture target
* `ci` job doesn't run the `compile-fail` part but only format and clippy, taking less time
* `check-benches` becomes `check-compiles` and runs the `compile` tasks. It takes longer. I also fixed how it was using cache
* `check-doc` job is now independent and also run the doc tests, so it takes longer. I commented out the deadlinks check as it takes 2.5 minutes (to install) and doesn't work
# Objective
- Using the `cargo run -p ci` command locally is unreliable, as it does not run tests.
- This is particularly unreliable for doc tests, as they are not run as part of `cargo test`.
## Solution
- add more steps to the appropriate Rust file.
## Known Problems
This duplicates work done to run tests when run on Github. @mockersf, suggestions on if we care / how we can mitigate it?
# Objective
CI should check for missing backticks in doc comments.
Fixes#3435
## Solution
`clippy` has a lint for this: `doc_markdown`. This enables that lint in the CI script.
Of course, enabling this lint in CI causes a bunch of lint errors, so I've gone through and fixed all of them. This was a huge edit that touched a ton of files, so I split the PR up by crate.
When all of the following are merged, the CI should pass and this can be merged.
+ [x] #3467
+ [x] #3468
+ [x] #3470
+ [x] #3469
+ [x] #3471
+ [x] #3472
+ [x] #3473
+ [x] #3474
+ [x] #3475
+ [x] #3476
+ [x] #3477
+ [x] #3478
+ [x] #3479
+ [x] #3480
+ [x] #3481
+ [x] #3482
+ [x] #3483
+ [x] #3484
+ [x] #3485
+ [x] #3486
# Objective
- Document that the error codes will be rendered on the bevy website (see bevyengine/bevy-website#216)
- Some Cargo.toml files did not include the license or a description field
## Solution
- Readme for the errors crate
- Mark internal/development crates with `publish = false`
- Add missing license/descriptions to some crates
- [x] merge bevyengine/bevy-website#216
# Objective
bevy_ecs has several compile_fail tests that assert lifetime safety. In the past, these tests have been green for the wrong reasons (see e.g. #2984). This PR makes sure, that they will fail if the compiler error changes.
## Solution
Use [trybuild](https://crates.io/crates/trybuild) to assert the compiler errors.
The UI tests are in a separate crate that is not part of the Bevy workspace. This is to ensure that they do not break Bevy's crater builds. The tests get executed by the CI workflow on the stable toolchain.
Objective
During work on #3009 I've found that not all jobs use actions-rs, and therefore, an previous version of Rust is used for them. So while compilation and other stuff can pass, checking markup and Android build may fail with compilation errors.
Solution
This PR adds `action-rs` for any job running cargo, and updates the edition to 2021.
That override was added to support pre 1.45 Versions of Rust, but Bevy requires currently the latest stable rust release.
This means that the reason for the override doesn't apply anymore.
This PR is easiest to review commit by commit.
Followup on https://github.com/bevyengine/bevy/pull/1309#issuecomment-767310084
- [x] Switch from a bash script to an xtask rust workspace member.
- Results in ~30s longer CI due to compilation of the xtask itself
- Enables Bevy contributors on any platform to run `cargo ci` to run linting -- if the default available Rust is the same version as on CI, then the command should give an identical result.
- [x] Use the xtask from official CI so there's only one place to update.
- [x] Bonus: Run clippy on the _entire_ workspace (existing CI setup was missing the `--workspace` flag
- [x] Clean up newly-exposed clippy errors
~#1388 builds on this to clean up newly discovered clippy errors -- I thought it might be nicer as a separate PR.~ Nope, merged it into this one so CI would pass.
Co-authored-by: Carter Anderson <mcanders1@gmail.com>
