This is a squash-and-rebase of @Ku95's documentation of the new renderer onto the latest `pipelined-rendering` branch.
Original PR is #2884.
Co-authored-by: dataphract <dataphract@gmail.com>
Co-authored-by: Carter Anderson <mcanders1@gmail.com>
# Objective
Fixes these issues:
- `WorldId`s currently aren't necessarily unique
- I want to guarantee that they're unique to safeguard my librarified version of https://github.com/bevyengine/bevy/discussions/2805
- There probably hasn't been a collision yet, but they could technically collide
- `SystemId` isn't used for anything
- It's no longer used now that `Locals` are stored within the `System`.
- `bevy_ecs` depends on rand
## Solution
- Instead of randomly generating `WorldId`s, just use an incrementing atomic counter, panicing on overflow.
- Remove `SystemId`
- We do need to allow Locals for exclusive systems at some point, but exclusive systems couldn't access their own `SystemId` anyway.
- Now that these don't depend on rand, move it to a dev-dependency
## Todo
Determine if `WorldId` should be `u32` based instead
This changes how render logic is composed to make it much more modular. Previously, all extraction logic was centralized for a given "type" of rendered thing. For example, we extracted meshes into a vector of ExtractedMesh, which contained the mesh and material asset handles, the transform, etc. We looked up bindings for "drawn things" using their index in the `Vec<ExtractedMesh>`. This worked fine for built in rendering, but made it hard to reuse logic for "custom" rendering. It also prevented us from reusing things like "extracted transforms" across contexts.
To make rendering more modular, I made a number of changes:
* Entities now drive rendering:
* We extract "render components" from "app components" and store them _on_ entities. No more centralized uber lists! We now have true "ECS-driven rendering"
* To make this perform well, I implemented #2673 in upstream Bevy for fast batch insertions into specific entities. This was merged into the `pipelined-rendering` branch here: #2815
* Reworked the `Draw` abstraction:
* Generic `PhaseItems`: each draw phase can define its own type of "rendered thing", which can define its own "sort key"
* Ported the 2d, 3d, and shadow phases to the new PhaseItem impl (currently Transparent2d, Transparent3d, and Shadow PhaseItems)
* `Draw` trait and and `DrawFunctions` are now generic on PhaseItem
* Modular / Ergonomic `DrawFunctions` via `RenderCommands`
* RenderCommand is a trait that runs an ECS query and produces one or more RenderPass calls. Types implementing this trait can be composed to create a final DrawFunction. For example the DrawPbr DrawFunction is created from the following DrawCommand tuple. Const generics are used to set specific bind group locations:
```rust
pub type DrawPbr = (
SetPbrPipeline,
SetMeshViewBindGroup<0>,
SetStandardMaterialBindGroup<1>,
SetTransformBindGroup<2>,
DrawMesh,
);
```
* The new `custom_shader_pipelined` example illustrates how the commands above can be reused to create a custom draw function:
```rust
type DrawCustom = (
SetCustomMaterialPipeline,
SetMeshViewBindGroup<0>,
SetTransformBindGroup<2>,
DrawMesh,
);
```
* ExtractComponentPlugin and UniformComponentPlugin:
* Simple, standardized ways to easily extract individual components and write them to GPU buffers
* Ported PBR and Sprite rendering to the new primitives above.
* Removed staging buffer from UniformVec in favor of direct Queue usage
* Makes UniformVec much easier to use and more ergonomic. Completely removes the need for custom render graph nodes in these contexts (see the PbrNode and view Node removals and the much simpler call patterns in the relevant Prepare systems).
* Added a many_cubes_pipelined example to benchmark baseline 3d rendering performance and ensure there were no major regressions during this port. Avoiding regressions was challenging given that the old approach of extracting into centralized vectors is basically the "optimal" approach. However thanks to a various ECS optimizations and render logic rephrasing, we pretty much break even on this benchmark!
* Lifetimeless SystemParams: this will be a bit divisive, but as we continue to embrace "trait driven systems" (ex: ExtractComponentPlugin, UniformComponentPlugin, DrawCommand), the ergonomics of `(Query<'static, 'static, (&'static A, &'static B, &'static)>, Res<'static, C>)` were getting very hard to bear. As a compromise, I added "static type aliases" for the relevant SystemParams. The previous example can now be expressed like this: `(SQuery<(Read<A>, Read<B>)>, SRes<C>)`. If anyone has better ideas / conflicting opinions, please let me know!
* RunSystem trait: a way to define Systems via a trait with a SystemParam associated type. This is used to implement the various plugins mentioned above. I also added SystemParamItem and QueryItem type aliases to make "trait stye" ecs interactions nicer on the eyes (and fingers).
* RenderAsset retrying: ensures that render assets are only created when they are "ready" and allows us to create bind groups directly inside render assets (which significantly simplified the StandardMaterial code). I think ultimately we should swap this out on "asset dependency" events to wait for dependencies to load, but this will require significant asset system changes.
* Updated some built in shaders to account for missing MeshUniform fields
This updates the `pipelined-rendering` branch to use the latest `bevy_ecs` from `main`. This accomplishes a couple of goals:
1. prepares for upcoming `custom-shaders` branch changes, which were what drove many of the recent bevy_ecs changes on `main`
2. prepares for the soon-to-happen merge of `pipelined-rendering` into `main`. By including bevy_ecs changes now, we make that merge simpler / easier to review.
I split this up into 3 commits:
1. **add upstream bevy_ecs**: please don't bother reviewing this content. it has already received thorough review on `main` and is a literal copy/paste of the relevant folders (the old folders were deleted so the directories are literally exactly the same as `main`).
2. **support manual buffer application in stages**: this is used to enable the Extract step. we've already reviewed this once on the `pipelined-rendering` branch, but its worth looking at one more time in the new context of (1).
3. **support manual archetype updates in QueryState**: same situation as (2).
# Objective
Enable using exact World lifetimes during read-only access . This is motivated by the new renderer's need to allow read-only world-only queries to outlive the query itself (but still be constrained by the world lifetime).
For example:
115b170d1f/pipelined/bevy_pbr2/src/render/mod.rs (L774)
## Solution
Split out SystemParam state and world lifetimes and pipe those lifetimes up to read-only Query ops (and add into_inner for Res). According to every safety test I've run so far (except one), this is safe (see the temporary safety test commit). Note that changing the mutable variants to the new lifetimes would allow aliased mutable pointers (try doing that to see how it affects the temporary safety tests).
The new state lifetime on SystemParam does make `#[derive(SystemParam)]` more cumbersome (the current impl requires PhantomData if you don't use both lifetimes). We can make this better by detecting whether or not a lifetime is used in the derive and adjusting accordingly, but that should probably be done in its own pr.
## Why is this a draft?
The new lifetimes break QuerySet safety in one very specific case (see the query_set system in system_safety_test). We need to solve this before we can use the lifetimes given.
This is due to the fact that QuerySet is just a wrapper over Query, which now relies on world lifetimes instead of `&self` lifetimes to prevent aliasing (but in systems, each Query has its own implied lifetime, not a centralized world lifetime). I believe the fix is to rewrite QuerySet to have its own World lifetime (and own the internal reference). This will complicate the impl a bit, but I think it is doable. I'm curious if anyone else has better ideas.
Personally, I think these new lifetimes need to happen. We've gotta have a way to directly tie read-only World queries to the World lifetime. The new renderer is the first place this has come up, but I doubt it will be the last. Worst case scenario we can come up with a second `WorldLifetimeQuery<Q, F = ()>` parameter to enable these read-only scenarios, but I'd rather not add another type to the type zoo.
# Objective
- Remove all the `.system()` possible.
- Check for remaining missing cases.
## Solution
- Remove all `.system()`, fix compile errors
- 32 calls to `.system()` remains, mostly internals, the few others should be removed after #2446
* bevy_pbr2: Add support for most of the StandardMaterial textures
Normal maps are not included here as they require tangents in a vertex attribute.
* bevy_pbr2: Ensure RenderCommandQueue is ready for PbrShaders init
* texture_pipelined: Add a light to the scene so we can see stuff
* WIP bevy_pbr2: back to front sorting hack
* bevy_pbr2: Uniform control flow for texture sampling in pbr.frag
From 'fintelia' on the Bevy Render Rework Round 2 discussion:
"My understanding is that GPUs these days never use the "execute both branches
and select the result" strategy. Rather, what they do is evaluate the branch
condition on all threads of a warp, and jump over it if all of them evaluate to
false. If even a single thread needs to execute the if statement body, however,
then the remaining threads are paused until that is completed."
* bevy_pbr2: Simplify texture and sampler names
The StandardMaterial_ prefix is no longer needed
* bevy_pbr2: Match default 'AmbientColor' of current bevy_pbr for now
* bevy_pbr2: Convert from non-linear to linear sRGB for the color uniform
* bevy_pbr2: Add pbr_pipelined example
* Fix view vector in pbr frag to work in ortho
* bevy_pbr2: Use a 90 degree y fov and light range projection for lights
* bevy_pbr2: Add AmbientLight resource
* bevy_pbr2: Convert PointLight color to linear sRGB for use in fragment shader
* bevy_pbr2: pbr.frag: Rename PointLight.projection to view_projection
The uniform contains the view_projection matrix so this was incorrect.
* bevy_pbr2: PointLight is an OmniLight as it has a radius
* bevy_pbr2: Factoring out duplicated code
* bevy_pbr2: Implement RenderAsset for StandardMaterial
* Remove unnecessary texture and sampler clones
* fix comment formatting
* remove redundant Buffer:from
* Don't extract meshes when their material textures aren't ready
* make missing textures in the queue step an error
Co-authored-by: Aevyrie <aevyrie@gmail.com>
Co-authored-by: Carter Anderson <mcanders1@gmail.com>
# Objective
- Continue work of #2398 and #2403.
- Make `.system()` syntax optional when using `.config()` API.
## Solution
- Introduce new prelude trait, `ConfigurableSystem`, that shorthands `my_system.system().config(...)` as `my_system.config(...)`.
- Expand `configure_system_local` test to also cover the new syntax.
This can be your 6 months post-christmas present.
# Objective
- Make `.system` optional
- yeet
- It's ugly
- Alternative title: `.system` is dead; long live `.system`
- **yeet**
## Solution
- Use a higher ranked lifetime, and some trait magic.
N.B. This PR does not actually remove any `.system`s, except in a couple of examples. Once this is merged we can do that piecemeal across crates, and decide on syntax for labels.
This enables `SystemParams` to be used outside of function systems. Anything can create and store `SystemState`, which enables efficient "param state cached" access to `SystemParams`.
It adds a `ReadOnlySystemParamFetch` trait, which enables safe `SystemState::get` calls without unique world access.
I renamed the old `SystemState` to `SystemMeta` to enable us to mirror the `QueryState` naming convention (but I'm happy to discuss alternative names if people have other ideas). I initially pitched this as `ParamState`, but given that it needs to include full system metadata, that doesn't feel like a particularly accurate name.
```rust
#[derive(Eq, PartialEq, Debug)]
struct A(usize);
#[derive(Eq, PartialEq, Debug)]
struct B(usize);
let mut world = World::default();
world.insert_resource(A(42));
world.spawn().insert(B(7));
// we get nice lifetime elision when declaring the type on the left hand side
let mut system_state: SystemState<(Res<A>, Query<&B>)> = SystemState::new(&mut world);
let (a, query) = system_state.get(&world);
assert_eq!(*a, A(42), "returned resource matches initial value");
assert_eq!(
*query.single().unwrap(),
B(7),
"returned component matches initial value"
);
// mutable system params require unique world access
let mut system_state: SystemState<(ResMut<A>, Query<&mut B>)> = SystemState::new(&mut world);
let (a, query) = system_state.get_mut(&mut world);
// static lifetimes are required when declaring inside of structs
struct SomeContainer {
state: SystemState<(Res<'static, A>, Res<'static, B>)>
}
// this can be shortened using type aliases, which will be useful for complex param tuples
type MyParams<'a> = (Res<'a, A>, Res<'a, B>);
struct SomeContainer {
state: SystemState<MyParams<'static>>
}
// It is the user's responsibility to call SystemState::apply(world) for parameters that queue up work
let mut system_state: SystemState<(Commands, Query<&B>)> = SystemState::new(&mut world);
{
let (mut commands, query) = system_state.get(&world);
commands.insert_resource(3.14);
}
system_state.apply(&mut world);
```
## Future Work
* Actually use SystemState inside FunctionSystem. This would be trivial, but it requires FunctionSystem to wrap SystemState in Option in its current form (which complicates system metadata lookup). I'd prefer to hold off until we adopt something like the later designs linked in #1364, which enable us to contruct Systems using a World reference (and also remove the need for `.system`).
* Consider a "scoped" approach to automatically call SystemState::apply when systems params are no longer being used (either a container type with a Drop impl, or a function that takes a closure for user logic operating on params).
2021-06-02 19:57:38 +00:00
Renamed from crates/bevy_ecs/src/system/into_system.rs (Browse further)
