# Objective
- Fix bug where `cargo run -p ci` fails due to differing implementations
for default values between `Default` trait and `argh`
- Automatically install target via `rustup` to make first-run simpler.
## Solution
The command will now attempt to install the target via `rustup` by
default, which will provide a cleaner error message if a malformed
target is passed. It will also avoid confusion when people attempt to
use this tool for the first time without the target already installed.
I've added a flag, --skip-install, to disable the attempted installation
just-in-case. (e.g., maybe rustup isn't in their path but cargo is?).
Also fixed a bug where the default value for `target` was different
between the `Default` trait and `argh`, causing `cargo run -p ci` to
fail.
## Testing
- CI
- Subcommand ran directly
## Notes
This issue was originally discovered by @targrub (on Discord):
> Unfortunately, running `cargo run -p ci` still gives that same error
as I initially reported (though `cargo run -p ci --
compile-check-no-std` succeeds). This is after having run `rustup target
add x86_64-unknown-none`.
# Objective
- Fixes#15840
## Solution
Added a new subcommand to the CI tool, `compile-check-no-std`, which
will attempt to compile each `no_std` crate in Bevy with the appropriate
features (no-defaults, `libm`, etc.) for `x86_64-unknown-none`. The
exact target chosen could be changed to any reasonable platform which
does not include the `std` library.
The currently tested crates are:
- `bevy_ptr`
- `bevy_utils`
- `bevy_mikktspace`
As more crates have `no_std` support added, they _should_ be added to
this CI command. Once Bevy itself can be `no_std`, the individual checks
can be replaced with just checking Bevy, since it will transiently check
all other crates as appropriate.
## Testing
- Ran CI. From a clean target directory (`cargo clean`), these new
checks take approximately 10 seconds total.
---------
Co-authored-by: François Mockers <francois.mockers@vleue.com>
# Objective
- Bevy currently has lot of invalid intra-doc links, let's fix them!
- Also make CI test them, to avoid future regressions.
- Helps with #1983 (but doesn't fix it, as there could still be explicit
links to docs.rs that are broken)
## Solution
- Make `cargo r -p ci -- doc-check` check fail on warnings (could also
be changed to just some specific lints)
- Manually fix all the warnings (note that in some cases it was unclear
to me what the fix should have been, I'll try to highlight them in a
self-review)
# Objective
- Follow-up of #13184 :)
- We use `ui_test` to test compiler errors for our custom macros.
- There are four crates related to compile fail tests
- `bevy_ecs_compile_fail_tests`, `bevy_macros_compile_fail_tests`, and
`bevy_reflect_compile_fail_tests`, which actually test the macros.
-
[`bevy_compile_test_utils`](64c1c65783/crates/bevy_compile_test_utils),
which provides helpers and common patterns for these tests.
- All of these crates reside within the `crates` directory.
- This can be confusing, especially for newcomers. All of the other
folders in `crates` are actual published libraries, except for these 4.
## Solution
- Move all compile fail tests to a `compile_fail` folder under their
corresponding crate.
- E.g. `crates/bevy_ecs_compile_fail_tests` would be moved to
`crates/bevy_ecs/compile_fail`.
- Move `bevy_compile_test_utils` to `tools/compile_fail_utils`.
There are a few benefits to this approach:
1. An internal testing detail is less intrusive (and confusing) for
those who just want to browse the public Bevy interface.
2. Follows a pre-existing approach of organizing related crates inside a
larger crate's folder.
- See `bevy_gizmos/macros` for an example.
4. Makes consistent the terms `compile_test`, `compile_fail`, and
`compile_fail_test` in code. It's all just `compile_fail` now, because
we are specifically testing the error messages on compiler failures.
- To be clear it can still be referred to by these terms in comments and
speech, just the names of the crates and the CI command are now
consistent.
## Testing
Run the compile fail CI command:
```shell
cargo run -p ci -- compile-fail
```
If it still passes, then my refactor was successful.
# Objective
- Many of the items in the `ci` tool use `pub(crate)`, which is
functionally equivalent to `pub` when the crate is not a library.
- A few items are missing documentation.
## Solution
- Make all `pub(crate)` items just `pub`.
- `pub` is easier to type and less obscure, and there's not harm from
this change.
- Add / modify documentation on `CI`, `Prepare`, and `PreparedCommand`.
# Objective
- Currently all `ci` commands are in the `subcommands` module. This is
problematic when you want to implement actually subcommands (such as
`cargo r -p ci -- doc check`).
- All command modules include the `_command` suffix, which is redundant.
## Solution
- Move `commands` modules into root crate folder.
- Merge contents of `commands/mod.rs` into `main.rs`.
- Move `commands::subcommands` module into `commands` module.
- Remove the `_command` suffix from all `commands::subcommands` modules.
# Objective
The CI tool currently parses input manually. This has worked fine, but
makes it just a bit more difficult to maintain and extend. Additionally,
it provides no usage help for devs wanting to run the tool locally.
It would be better if parsing was handled by a dedicated CLI library
like [`clap`](https://github.com/clap-rs/clap) or
[`argh`](https://github.com/google/argh).
## Solution
Use `argh` to parse command line input for CI.
`argh` was chosen over `clap` and other tools due to being more
lightweight and already existing in our dependency tree.
Using `argh`, the usage notes are generated automatically:
```
$ cargo run -p ci --quiet -- --help
Usage: ci [--keep-going] [<command>] [<args>]
The CI command line tool for Bevy.
Options:
--keep-going continue running commands even if one fails
--help display usage information
Commands:
lints Alias for running the `format` and `clippy` subcommands.
doc Alias for running the `doc-test` and `doc-check`
subcommands.
compile Alias for running the `compile-fail`, `bench-check`,
`example-check`, `compile-check`, and `test-check`
subcommands.
format Check code formatting.
clippy Check for clippy warnings and errors.
test Runs all tests (except for doc tests).
test-check Checks that all tests compile.
doc-check Checks that all docs compile.
doc-test Runs all doc tests.
compile-check Checks that the project compiles.
cfg-check Checks that the project compiles using the nightly compiler
with cfg checks enabled.
compile-fail Runs the compile-fail tests.
bench-check Checks that the benches compile.
example-check Checks that the examples compile.
```
This PR makes each subcommand more modular, allowing them to be called
from other subcommands. This also makes it much easier to extract them
out of `main.rs` and into their own dedicated modules.
Additionally, this PR improves failure output:
```
$ cargo run -p ci -- lints
...
One or more CI commands failed:
format: Please run 'cargo fmt --all' to format your code.
```
Including when run with the `--keep-going` flag:
```
$ cargo run -p ci -- --keep-going lints
...
One or more CI commands failed:
- format: Please run 'cargo fmt --all' to format your code.
- clippy: Please fix clippy errors in output above.
```
### Future Work
There are a lot of other things we could possibly clean up. I chose to
try and keep the API surface as unchanged as I could (for this PR at
least).
For example, now that each subcommand is an actual command, we can
specify custom arguments for each.
The `format` subcommand could include a `--check` (making the default
fun `cargo fmt` as normal). Or the `compile-fail` subcommand could
include `--ecs`, `--reflect`, and `--macros` flags for specifying which
set of compile fail tests to run.
The `--keep-going` flag could be split so that it doesn't do double-duty
where it also enables `--no-fail-fast` for certain commands. Or at least
make it more explicit via renaming or using alternative flags.
---
## Changelog
- Improved the CI CLI tool
- Now includes usage info with the `--help` option!
- [Internal] Cleaned up and refactored the `tools/ci` crate using the
`argh` crate
## Migration Guide
The CI tool no longer supports running multiple subcommands in a single
call. Users who are currently doing so will need to split them across
multiple commands:
```bash
# BEFORE
cargo run -p ci -- lints doc compile
# AFTER
cargo run -p ci -- lints && cargo run -p ci -- doc && cargo run -p ci -- compile
# or
cargo run -p ci -- lints; cargo run -p ci -- doc; cargo run -p ci -- compile
# or
cargo run -p ci -- lints
cargo run -p ci -- doc
cargo run -p ci -- compile
```
# Objective
- The CI tool is slowly getting more difficult to maintain and could use
a bit of refactoring.
## Solution
- Do a first pass of small improvements.
## For Reviewers
This PR is best reviewed commit-by-commit, because I separate it into a
collection of small changes. :)
# Objective
- There are several redundant imports in the tests and examples that are
not caught by CI because additional flags need to be passed.
## Solution
- Run `cargo check --workspace --tests` and `cargo check --workspace
--examples`, then fix all warnings.
- Add `test-check` to CI, which will be run in the check-compiles job.
This should catch future warnings for tests. Examples are already
checked, but I'm not yet sure why they weren't caught.
## Discussion
- Should the `--tests` and `--examples` flags be added to CI, so this is
caught in the future?
- If so, #12818 will need to be merged first. It was also a warning
raised by checking the examples, but I chose to split off into a
separate PR.
---------
Co-authored-by: François Mockers <francois.mockers@vleue.com>
# Objective
- Make running CI locally relatively less painful by allowing
continuation after failure
- Fix#12206
## Solution
Previously, `ci` would halt after encounting a failure (or shortly
thereafter). For instance, if you ran `cargo run -p ci` and a single
test within a CI check failed somewhere in the middle, you would never
even reach many of the other tests within the check and certainly none
of the CI checks that came afterward. This was annoying; if I am fixing
issues with CI tests, I want to just see all of the issues up-front
instead of having to rerun CI every time I fix something to see the next
error.
Furthermore, it is not infrequent (because of subtle
configuration/system differences) to encounter spurious CI failures
locally; previously, when these occurred, they would make the process of
running CI largely pointless, since you would have to push your branch
in order to see your actual CI failures from the automated testing
service.
Now, when running `ci` locally, we have a couple new tools to ameliorate
these and provide a smoother experience:
- Firstly, there is a new optional flag `--keep-going` that can be
passed while calling `ci` (e.g. `cargo run -p ci -- doc --keep-going`).
It has the following effects:
- It causes the `--keep-going` flag to be passed to the script's `cargo
doc` and `cargo check` invocations, so that they do not stop when they
encounter an error in a single module; instead, they keep going (!) and
find errors subsequently.
- It causes the `--no-fail-fast` flag to be passed to the script's
`cargo test` invocations, to similar effect.
- Finally, it causes `ci` itself not to abort after a single check
fails; instead, it will run every check that was invoked.
Thus, for instance, `cargo run -p ci -- --keep-going` will run every CI
check even if it encounters intermediate failures, and every such check
will itself be run to completion.
- Secondly, we now allow multiple ordinary arguments to be passed to
`ci`. For instance, `cargo -p ci -- doc test` now executes both the
'doc' checks and the 'test' checks. This allows the caller to run the
tests they care about with fewer invocations of `ci`.
As of this PR, automated testing will remain unchanged.
---
## Changelog
- tools/ci/src/main.rs refactored into staging and execution steps,
since the previous control flow did not naturally support continuing
after failure.
- Added "--keep-going" flag to `ci`.
- Added support for invoking `ci` with multiple arguments.
---
## Discussion
### Design considerations
I had originally split this into separate flags that controlled:
1. whether `--keep-going`/`--no-fail-fast` would be passed to the
constituent commands
2. whether `ci` would continue after a component test failed
However, I decided to merge these two behaviors, since I think that, if
you're in the business of seeing all the errors, you'll probably want to
actually see all of the errors. One slightly annoying thing, however,
about the new behavior with `--keep-going`, is that you will sometimes
find yourself wishing that the script would pause or something, since it
tends to fill the screen with a lot of junk. I have found that sending
stdout to /dev/null helps quite a bit, but I don't think `cargo fmt` or
`cargo clippy` actually write to stderr, so you need to be cognizant of
that (and perhaps invoke the lints separately).
~~Next, I'll mention that I feel somewhat strongly that the current
behavior of `ci` for automated testing should remain the same, since its
job is more like detecting that errors exist rather than finding all of
them.~~ (I was convinced this could have value.) Orthogonally, there is
the question of whether or not we might want to make this (or something
similar) actually the default behavior and make the automated test
script invoke some optional flags — it doesn't have to type with its
hands, after all. I'm not against that, but I don't really want to rock
the boat much more with this PR, since anyone who looks at the diff
might already be a little incensed.
# Objective
Getting this error running ci locally.
```
$ cargo check -Zcheck-cfg --workspace
error: the `-Z` flag is only accepted on the nightly channel of Cargo, but this is the `stable` channel
See https://doc.rust-lang.org/book/appendix-07-nightly-rust.html for more information about Rust release channels.
thread 'main' panicked at tools\ci\src\main.rs:166:14:
Please fix failing cfg checks in output above.: command exited with non-zero code `cargo check -Zcheck-cfg --workspace`: 101
```
## Solution
- Add `+nightly` flag to the `check-cfg` pass.
---
Obviously we shouldn't be running `nightly` Cargo, but at least now
local running of `cargo run -p ci` will pass.
# Objective
- Add the new `-Zcheck-cfg` checks to catch more warnings
- Fixes#12091
## Solution
- Create a new `cfg-check` to the CI that runs `cargo check -Zcheck-cfg
--workspace` using cargo nightly (and fails if there are warnings)
- Fix all warnings generated by the new check
---
## Changelog
- Remove all redundant imports
- Fix cfg wasm32 targets
- Add 3 dead code exceptions (should StandardColor be unused?)
- Convert ios_simulator to a feature (I'm not sure if this is the right
way to do it, but the check complained before)
## Migration Guide
No breaking changes
---------
Co-authored-by: Alice Cecile <alice.i.cecile@gmail.com>
# Objective
- Some workspace members do not inherit the global lints.
## Solution
- Add a `[lints]` entry for all files returned by `rg
--files-without-match -F "[lints]" **/Cargo.toml`, except the compile
failure tests since these aren't part of the workspace.
- Add some docstrings where needed.
---------
Co-authored-by: Alice Cecile <alice.i.cecile@gmail.com>
# Objective
Partially addresses #10612
After moving the initial docs_markdown warning, this is a PR that moves
the rest of the CI clippy lint definitions to the cargo.toml.
## Solution
- the `tools/ci/src/main.rs` clippy lints removed and just the warning
flag remains.
- the warnings moved to the root cargo workspace toml.
# Objective
Partially Addresses #10612
fix: add clippy::doc_markdown linting to cargo workspace
Rather than do all the warnings in `tools/ci/src/main.rs` in one-shot,
just wanted to have an initial pr adding the first one to get the form
correct as some may trigger build errors and require changes to get
merged more easily.
## Solution
- adding the doc_markdown and removing it from the ci check as it'll now
be a build error during normal compilation.
---------
Co-authored-by: François <mockersf@gmail.com>
# Objective
- See fewer warnings when running `cargo clippy` locally.
## Solution
- allow `clippy::type_complexity` in more places, which also signals to
users they should do the same.
# Objective
Replace instances of
```rust
for x in collection.iter{_mut}() {
```
with
```rust
for x in &{mut} collection {
```
This also changes CI to no longer suppress this lint. Note that since
this lint only shows up when using clippy in pedantic mode, it was
probably unnecessary to suppress this lint in the first place.
# Objective
[Rust 1.72.0](https://blog.rust-lang.org/2023/08/24/Rust-1.72.0.html) is
now stable.
# Notes
- `let-else` formatting has arrived!
- I chose to allow `explicit_iter_loop` due to
https://github.com/rust-lang/rust-clippy/issues/11074.
We didn't hit any of the false positives that prevent compilation, but
fixing this did produce a lot of the "symbol soup" mentioned, e.g. `for
image in &mut *image_events {`.
Happy to undo this if there's consensus the other way.
---------
Co-authored-by: François <mockersf@gmail.com>
# Objective
Bevy code tends to make heavy use of the [newtype](
https://doc.rust-lang.org/rust-by-example/generics/new_types.html)
pattern, which is why we have a dedicated derive for
[`Deref`](https://doc.rust-lang.org/std/ops/trait.Deref.html) and
[`DerefMut`](https://doc.rust-lang.org/std/ops/trait.DerefMut.html).
This derive works for any struct with a single field:
```rust
#[derive(Component, Deref, DerefMut)]
struct MyNewtype(usize);
```
One reason for the single-field limitation is to prevent confusion and
footguns related that would arise from allowing multi-field structs:
<table align="center">
<tr>
<th colspan="2">
Similar structs, different derefs
</th>
</tr>
<tr>
<td>
```rust
#[derive(Deref, DerefMut)]
struct MyStruct {
foo: usize, // <- Derefs usize
bar: String,
}
```
</td>
<td>
```rust
#[derive(Deref, DerefMut)]
struct MyStruct {
bar: String, // <- Derefs String
foo: usize,
}
```
</td>
</tr>
<tr>
<th colspan="2">
Why `.1`?
</th>
</tr>
<tr>
<td colspan="2">
```rust
#[derive(Deref, DerefMut)]
struct MyStruct(Vec<usize>, Vec<f32>);
let mut foo = MyStruct(vec![123], vec![1.23]);
// Why can we skip the `.0` here?
foo.push(456);
// But not here?
foo.1.push(4.56);
```
</td>
</tr>
</table>
However, there are certainly cases where it's useful to allow for
structs with multiple fields. Such as for structs with one "real" field
and one `PhantomData` to allow for generics:
```rust
#[derive(Deref, DerefMut)]
struct MyStruct<T>(
// We want use this field for the `Deref`/`DerefMut` impls
String,
// But we need this field so that we can make this struct generic
PhantomData<T>
);
// ERROR: Deref can only be derived for structs with a single field
// ERROR: DerefMut can only be derived for structs with a single field
```
Additionally, the possible confusion and footguns are mainly an issue
for newer Rust/Bevy users. Those familiar with `Deref` and `DerefMut`
understand what adding the derive really means and can anticipate its
behavior.
## Solution
Allow users to opt into multi-field `Deref`/`DerefMut` derives using a
`#[deref]` attribute:
```rust
#[derive(Deref, DerefMut)]
struct MyStruct<T>(
// Use this field for the `Deref`/`DerefMut` impls
#[deref] String,
// We can freely include any other field without a compile error
PhantomData<T>
);
```
This prevents the footgun pointed out in the first issue described in
the previous section, but it still leaves the possible confusion
surrounding `.0`-vs-`.#`. However, the idea is that by making this
behavior explicit with an attribute, users will be more aware of it and
can adapt appropriately.
---
## Changelog
- Added `#[deref]` attribute to `Deref` and `DerefMut` derives
# Objective
I noticed that running the following command didn't actually do anything:
```
cargo run -p ci -- bench-check
```
## Solution
Made it so that running `cargo run -p ci -- bench-check` actually runs a compile check on the `benches` directory.
# Objective
There isn't really a way to test that code using bevy_reflect compiles or doesn't compile for certain scenarios. This would be especially useful for macro-centric PRs like #6511 and #6042.
## Solution
Using `bevy_ecs_compile_fail_tests` as reference, added the `bevy_reflect_compile_fail_tests` crate.
Currently, this crate contains a very simple test case. This is so that we can get the basic foundation of this crate agreed upon and merged so that more tests can be added by other PRs.
### Open Questions
- [x] Should this be added to CI? (Answer: Yes)
---
## Changelog
- Added the `bevy_reflect_compile_fail_tests` crate for testing compilation errors
# Objective
- Have an easy way to compare spans between executions
## Solution
- Add a tool to compare spans from chrome traces
```bash
> cargo run --release -p spancmp -- --help
Compiling spancmp v0.1.0
Finished release [optimized] target(s) in 1.10s
Running `target/release/spancmp --help`
spancmp
USAGE:
spancmp [OPTIONS] <TRACE> [SECOND_TRACE]
ARGS:
<TRACE>
<SECOND_TRACE>
OPTIONS:
-h, --help Print help information
-p, --pattern <PATTERN> Filter spans by name matching the pattern
-t, --threshold <THRESHOLD> Filter spans that have an average shorther than the threshold
[default: 0]
```
for each span, it will display the count, minimum duration, average duration and max duration. It can be filtered by a pattern on the span name or by a minimum average duration.
just displaying a trace
comparing two traces
Co-authored-by: Robert Swain <robert.swain@gmail.com>
# Objective
- Original objective was to add doc build warning check to the ci local execution
- I somewhat deviated and changed other things...
## Solution
`cargo run -p ci` can now take more parameters:
* `format` - cargo fmt
* `clippy` - clippy
* `compile-fail` - bevy_ecs_compile_fail_tests tests
* `test` - tests but not doc tests and do not build examples
* `doc-test` - doc tests
* `doc-check` - doc build and warnings
* `bench-check` - check that benches build
* `example-check` - check that examples build
* `lints` - group - run lints and format and clippy
* `doc` - group - run doc-test and doc-check
* `compile` - group - run compile-fail and bench-check and example-check
* not providing a parameter will run everything
Ci is using those when possible:
* `build` jobs now don't run doc tests and don't build examples. it makes this job faster, but doc tests and examples are not built for each architecture target
* `ci` job doesn't run the `compile-fail` part but only format and clippy, taking less time
* `check-benches` becomes `check-compiles` and runs the `compile` tasks. It takes longer. I also fixed how it was using cache
* `check-doc` job is now independent and also run the doc tests, so it takes longer. I commented out the deadlinks check as it takes 2.5 minutes (to install) and doesn't work
# Objective
- Using the `cargo run -p ci` command locally is unreliable, as it does not run tests.
- This is particularly unreliable for doc tests, as they are not run as part of `cargo test`.
## Solution
- add more steps to the appropriate Rust file.
## Known Problems
This duplicates work done to run tests when run on Github. @mockersf, suggestions on if we care / how we can mitigate it?
# Objective
CI should check for missing backticks in doc comments.
Fixes#3435
## Solution
`clippy` has a lint for this: `doc_markdown`. This enables that lint in the CI script.
Of course, enabling this lint in CI causes a bunch of lint errors, so I've gone through and fixed all of them. This was a huge edit that touched a ton of files, so I split the PR up by crate.
When all of the following are merged, the CI should pass and this can be merged.
+ [x] #3467
+ [x] #3468
+ [x] #3470
+ [x] #3469
+ [x] #3471
+ [x] #3472
+ [x] #3473
+ [x] #3474
+ [x] #3475
+ [x] #3476
+ [x] #3477
+ [x] #3478
+ [x] #3479
+ [x] #3480
+ [x] #3481
+ [x] #3482
+ [x] #3483
+ [x] #3484
+ [x] #3485
+ [x] #3486
# Objective
bevy_ecs has several compile_fail tests that assert lifetime safety. In the past, these tests have been green for the wrong reasons (see e.g. #2984). This PR makes sure, that they will fail if the compiler error changes.
## Solution
Use [trybuild](https://crates.io/crates/trybuild) to assert the compiler errors.
The UI tests are in a separate crate that is not part of the Bevy workspace. This is to ensure that they do not break Bevy's crater builds. The tests get executed by the CI workflow on the stable toolchain.
That override was added to support pre 1.45 Versions of Rust, but Bevy requires currently the latest stable rust release.
This means that the reason for the override doesn't apply anymore.
This PR is easiest to review commit by commit.
Followup on https://github.com/bevyengine/bevy/pull/1309#issuecomment-767310084
- [x] Switch from a bash script to an xtask rust workspace member.
- Results in ~30s longer CI due to compilation of the xtask itself
- Enables Bevy contributors on any platform to run `cargo ci` to run linting -- if the default available Rust is the same version as on CI, then the command should give an identical result.
- [x] Use the xtask from official CI so there's only one place to update.
- [x] Bonus: Run clippy on the _entire_ workspace (existing CI setup was missing the `--workspace` flag
- [x] Clean up newly-exposed clippy errors
~#1388 builds on this to clean up newly discovered clippy errors -- I thought it might be nicer as a separate PR.~ Nope, merged it into this one so CI would pass.
Co-authored-by: Carter Anderson <mcanders1@gmail.com>
