Add morph targets (#8158)
# Objective
- Add morph targets to `bevy_pbr` (closes #5756) & load them from glTF
- Supersedes #3722
- Fixes #6814
[Morph targets][1] (also known as shape interpolation, shape keys, or
blend shapes) allow animating individual vertices with fine grained
controls. This is typically used for facial expressions. By specifying
multiple poses as vertex offset, and providing a set of weight of each
pose, it is possible to define surprisingly realistic transitions
between poses. Blending between multiple poses also allow composition.
Morph targets are part of the [gltf standard][2] and are a feature of
Unity and Unreal, and babylone.js, it is only natural to implement them
in bevy.
## Solution
This implementation of morph targets uses a 3d texture where each pixel
is a component of an animated attribute. Each layer is a different
target. We use a 2d texture for each target, because the number of
attribute×components×animated vertices is expected to always exceed the
maximum pixel row size limit of webGL2. It copies fairly closely the way
skinning is implemented on the CPU side, while on the GPU side, the
shader morph target implementation is a relatively trivial detail.
We add an optional `morph_texture` to the `Mesh` struct. The
`morph_texture` is built through a method that accepts an iterator over
attribute buffers.
The `MorphWeights` component, user-accessible, controls the blend of
poses used by mesh instances (so that multiple copy of the same mesh may
have different weights), all the weights are uploaded to a uniform
buffer of 256 `f32`. We limit to 16 poses per mesh, and a total of 256
poses.
More literature:
* Old babylone.js implementation (vertex attribute-based):
https://www.eternalcoding.com/dev-log-1-morph-targets/
* Babylone.js implementation (similar to ours):
https://www.youtube.com/watch?v=LBPRmGgU0PE
* GPU gems 3:
https://developer.nvidia.com/gpugems/gpugems3/part-i-geometry/chapter-3-directx-10-blend-shapes-breaking-limits
* Development discord thread
https://discord.com/channels/691052431525675048/1083325980615114772
https://user-images.githubusercontent.com/26321040/231181046-3bca2ab2-d4d9-472e-8098-639f1871ce2e.mp4
https://github.com/bevyengine/bevy/assets/26321040/d2a0c544-0ef8-45cf-9f99-8c3792f5a258
## Acknowledgements
* Thanks to `storytold` for sponsoring the feature
* Thanks to `superdump` and `james7132` for guidance and help figuring
out stuff
## Future work
- Handling of less and more attributes (eg: animated uv, animated
arbitrary attributes)
- Dynamic pose allocation (so that zero-weighted poses aren't uploaded
to GPU for example, enables much more total poses)
- Better animation API, see #8357
----
## Changelog
- Add morph targets to bevy meshes
- Support up to 64 poses per mesh of individually up to 116508 vertices,
animation currently strictly limited to the position, normal and tangent
attributes.
- Load a morph target using `Mesh::set_morph_targets`
- Add `VisitMorphTargets` and `VisitMorphAttributes` traits to
`bevy_render`, this allows defining morph targets (a fairly complex and
nested data structure) through iterators (ie: single copy instead of
passing around buffers), see documentation of those traits for details
- Add `MorphWeights` component exported by `bevy_render`
- `MorphWeights` control mesh's morph target weights, blending between
various poses defined as morph targets.
- `MorphWeights` are directly inherited by direct children (single level
of hierarchy) of an entity. This allows controlling several mesh
primitives through a unique entity _as per GLTF spec_.
- Add `MorphTargetNames` component, naming each indices of loaded morph
targets.
- Load morph targets weights and buffers in `bevy_gltf`
- handle morph targets animations in `bevy_animation` (previously, it
was a `warn!` log)
- Add the `MorphStressTest.gltf` asset for morph targets testing, taken
from the glTF samples repo, CC0.
- Add morph target manipulation to `scene_viewer`
- Separate the animation code in `scene_viewer` from the rest of the
code, reducing `#[cfg(feature)]` noise
- Add the `morph_targets.rs` example to show off how to manipulate morph
targets, loading `MorpStressTest.gltf`
## Migration Guide
- (very specialized, unlikely to be touched by 3rd parties)
- `MeshPipeline` now has a single `mesh_layouts` field rather than
separate `mesh_layout` and `skinned_mesh_layout` fields. You should
handle all possible mesh bind group layouts in your implementation
- You should also handle properly the new `MORPH_TARGETS` shader def and
mesh pipeline key. A new function is exposed to make this easier:
`setup_moprh_and_skinning_defs`
- The `MeshBindGroup` is now `MeshBindGroups`, cached bind groups are
now accessed through the `get` method.
[1]: https://en.wikipedia.org/wiki/Morph_target_animation
[2]:
https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#morph-targets
---------
Co-authored-by: François <mockersf@gmail.com>
Co-authored-by: Carter Anderson <mcanders1@gmail.com>
2023-06-22 20:00:01 +00:00
|
|
|
//! Enable controls for morph targets detected in a loaded scene.
|
|
|
|
//!
|
2023-07-10 00:11:51 +00:00
|
|
|
//! Collect morph targets and assign keys to them,
|
Add morph targets (#8158)
# Objective
- Add morph targets to `bevy_pbr` (closes #5756) & load them from glTF
- Supersedes #3722
- Fixes #6814
[Morph targets][1] (also known as shape interpolation, shape keys, or
blend shapes) allow animating individual vertices with fine grained
controls. This is typically used for facial expressions. By specifying
multiple poses as vertex offset, and providing a set of weight of each
pose, it is possible to define surprisingly realistic transitions
between poses. Blending between multiple poses also allow composition.
Morph targets are part of the [gltf standard][2] and are a feature of
Unity and Unreal, and babylone.js, it is only natural to implement them
in bevy.
## Solution
This implementation of morph targets uses a 3d texture where each pixel
is a component of an animated attribute. Each layer is a different
target. We use a 2d texture for each target, because the number of
attribute×components×animated vertices is expected to always exceed the
maximum pixel row size limit of webGL2. It copies fairly closely the way
skinning is implemented on the CPU side, while on the GPU side, the
shader morph target implementation is a relatively trivial detail.
We add an optional `morph_texture` to the `Mesh` struct. The
`morph_texture` is built through a method that accepts an iterator over
attribute buffers.
The `MorphWeights` component, user-accessible, controls the blend of
poses used by mesh instances (so that multiple copy of the same mesh may
have different weights), all the weights are uploaded to a uniform
buffer of 256 `f32`. We limit to 16 poses per mesh, and a total of 256
poses.
More literature:
* Old babylone.js implementation (vertex attribute-based):
https://www.eternalcoding.com/dev-log-1-morph-targets/
* Babylone.js implementation (similar to ours):
https://www.youtube.com/watch?v=LBPRmGgU0PE
* GPU gems 3:
https://developer.nvidia.com/gpugems/gpugems3/part-i-geometry/chapter-3-directx-10-blend-shapes-breaking-limits
* Development discord thread
https://discord.com/channels/691052431525675048/1083325980615114772
https://user-images.githubusercontent.com/26321040/231181046-3bca2ab2-d4d9-472e-8098-639f1871ce2e.mp4
https://github.com/bevyengine/bevy/assets/26321040/d2a0c544-0ef8-45cf-9f99-8c3792f5a258
## Acknowledgements
* Thanks to `storytold` for sponsoring the feature
* Thanks to `superdump` and `james7132` for guidance and help figuring
out stuff
## Future work
- Handling of less and more attributes (eg: animated uv, animated
arbitrary attributes)
- Dynamic pose allocation (so that zero-weighted poses aren't uploaded
to GPU for example, enables much more total poses)
- Better animation API, see #8357
----
## Changelog
- Add morph targets to bevy meshes
- Support up to 64 poses per mesh of individually up to 116508 vertices,
animation currently strictly limited to the position, normal and tangent
attributes.
- Load a morph target using `Mesh::set_morph_targets`
- Add `VisitMorphTargets` and `VisitMorphAttributes` traits to
`bevy_render`, this allows defining morph targets (a fairly complex and
nested data structure) through iterators (ie: single copy instead of
passing around buffers), see documentation of those traits for details
- Add `MorphWeights` component exported by `bevy_render`
- `MorphWeights` control mesh's morph target weights, blending between
various poses defined as morph targets.
- `MorphWeights` are directly inherited by direct children (single level
of hierarchy) of an entity. This allows controlling several mesh
primitives through a unique entity _as per GLTF spec_.
- Add `MorphTargetNames` component, naming each indices of loaded morph
targets.
- Load morph targets weights and buffers in `bevy_gltf`
- handle morph targets animations in `bevy_animation` (previously, it
was a `warn!` log)
- Add the `MorphStressTest.gltf` asset for morph targets testing, taken
from the glTF samples repo, CC0.
- Add morph target manipulation to `scene_viewer`
- Separate the animation code in `scene_viewer` from the rest of the
code, reducing `#[cfg(feature)]` noise
- Add the `morph_targets.rs` example to show off how to manipulate morph
targets, loading `MorpStressTest.gltf`
## Migration Guide
- (very specialized, unlikely to be touched by 3rd parties)
- `MeshPipeline` now has a single `mesh_layouts` field rather than
separate `mesh_layout` and `skinned_mesh_layout` fields. You should
handle all possible mesh bind group layouts in your implementation
- You should also handle properly the new `MORPH_TARGETS` shader def and
mesh pipeline key. A new function is exposed to make this easier:
`setup_moprh_and_skinning_defs`
- The `MeshBindGroup` is now `MeshBindGroups`, cached bind groups are
now accessed through the `get` method.
[1]: https://en.wikipedia.org/wiki/Morph_target_animation
[2]:
https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#morph-targets
---------
Co-authored-by: François <mockersf@gmail.com>
Co-authored-by: Carter Anderson <mcanders1@gmail.com>
2023-06-22 20:00:01 +00:00
|
|
|
//! shows on screen additional controls for morph targets.
|
|
|
|
//!
|
|
|
|
//! Illustrates how to access and modify individual morph target weights.
|
|
|
|
//! See the [`update_morphs`] system for details.
|
|
|
|
//!
|
|
|
|
//! Also illustrates how to read morph target names in [`detect_morphs`].
|
|
|
|
|
|
|
|
use crate::scene_viewer_plugin::SceneHandle;
|
|
|
|
use bevy::prelude::*;
|
|
|
|
use std::fmt;
|
|
|
|
|
|
|
|
const WEIGHT_PER_SECOND: f32 = 0.8;
|
|
|
|
const ALL_MODIFIERS: &[KeyCode] = &[KeyCode::ShiftLeft, KeyCode::ControlLeft, KeyCode::AltLeft];
|
|
|
|
const AVAILABLE_KEYS: [MorphKey; 56] = [
|
Update winit dependency to 0.29 (#10702)
# Objective
- Update winit dependency to 0.29
## Changelog
### KeyCode changes
- Removed `ScanCode`, as it was [replaced by
KeyCode](https://github.com/rust-windowing/winit/blob/master/CHANGELOG.md#0292).
- `ReceivedCharacter.char` is now a `SmolStr`, [relevant
doc](https://docs.rs/winit/latest/winit/event/struct.KeyEvent.html#structfield.text).
- Changed most `KeyCode` values, and added more.
KeyCode has changed meaning. With this PR, it refers to physical
position on keyboard rather than the printed letter on keyboard keys.
In practice this means:
- On QWERTY keyboard layouts, nothing changes
- On any other keyboard layout, `KeyCode` no longer reflects the label
on key.
- This is "good". In bevy 0.12, when you used WASD for movement, users
with non-QWERTY keyboards couldn't play your game! This was especially
bad for non-latin keyboards. Now, WASD represents the physical keys. A
French player will press the ZQSD keys, which are near each other,
Kyrgyz players will use "Цфыв".
- This is "bad" as well. You can't know in advance what the label of the
key for input is. Your UI says "press WASD to move", even if in reality,
they should be pressing "ZQSD" or "Цфыв". You also no longer can use
`KeyCode` for text inputs. In any case, it was a pretty bad API for text
input. You should use `ReceivedCharacter` now instead.
### Other changes
- Use `web-time` rather than `instant` crate.
(https://github.com/rust-windowing/winit/pull/2836)
- winit did split `run_return` in `run_onDemand` and `pump_events`, I
did the same change in bevy_winit and used `pump_events`.
- Removed `return_from_run` from `WinitSettings` as `winit::run` now
returns on supported platforms.
- I left the example "return_after_run" as I think it's still useful.
- This winit change is done partly to allow to create a new window after
quitting all windows: https://github.com/emilk/egui/issues/1918 ; this
PR doesn't address.
- added `width` and `height` properties in the `canvas` from wasm
example
(https://github.com/bevyengine/bevy/pull/10702#discussion_r1420567168)
## Known regressions (important follow ups?)
- Provide an API for reacting when a specific key from current layout
was released.
- possible solutions: use winit::Key from winit::KeyEvent ; mapping
between KeyCode and Key ; or .
- We don't receive characters through alt+numpad (e.g. alt + 151 = "ù")
anymore ; reproduced on winit example "ime". maybe related to
https://github.com/rust-windowing/winit/issues/2945
- (windows) Window content doesn't refresh at all when resizing. By
reading https://github.com/rust-windowing/winit/issues/2900 ; I suspect
we should just fire a `window.request_redraw();` from `AboutToWait`, and
handle actual redrawing within `RedrawRequested`. I'm not sure how to
move all that code so I'd appreciate it to be a follow up.
- (windows) unreleased winit fix for using set_control_flow in
AboutToWait https://github.com/rust-windowing/winit/issues/3215 ; ⚠️ I'm
not sure what the implications are, but that feels bad 🤔
## Follow up
I'd like to avoid bloating this PR, here are a few follow up tasks
worthy of a separate PR, or new issue to track them once this PR is
closed, as they would either complicate reviews, or at risk of being
controversial:
- remove CanvasParentResizePlugin
(https://github.com/bevyengine/bevy/pull/10702#discussion_r1417068856)
- avoid mentionning explicitly winit in docs from bevy_window ?
- NamedKey integration on bevy_input:
https://github.com/rust-windowing/winit/pull/3143 introduced a new
NamedKey variant. I implemented it only on the converters but we'd
benefit making the same changes to bevy_input.
- Add more info in KeyboardInput
https://github.com/bevyengine/bevy/pull/10702#pullrequestreview-1748336313
- https://github.com/bevyengine/bevy/pull/9905 added a workaround on a
bug allegedly fixed by winit 0.29. We should check if it's still
necessary.
- update to raw_window_handle 0.6
- blocked by wgpu
- Rename `KeyCode` to `PhysicalKeyCode`
https://github.com/bevyengine/bevy/pull/10702#discussion_r1404595015
- remove `instant` dependency, [replaced
by](https://github.com/rust-windowing/winit/pull/2836) `web_time`), we'd
need to update to :
- fastrand >= 2.0
- [`async-executor`](https://github.com/smol-rs/async-executor) >= 1.7
- [`futures-lite`](https://github.com/smol-rs/futures-lite) >= 2.0
- Verify license, see
[discussion](https://github.com/bevyengine/bevy/pull/8745#discussion_r1402439800)
- we might be missing a short notice or description of changes made
- Consider using https://github.com/rust-windowing/cursor-icon directly
rather than vendoring it in bevy.
- investigate [this
unwrap](https://github.com/bevyengine/bevy/pull/8745#discussion_r1387044986)
(`winit_window.canvas().unwrap();`)
- Use more good things about winit's update
- https://github.com/bevyengine/bevy/pull/10689#issuecomment-1823560428
## Migration Guide
This PR should have one.
2023-12-21 07:40:47 +00:00
|
|
|
MorphKey::new("r", &[], KeyCode::KeyR),
|
|
|
|
MorphKey::new("t", &[], KeyCode::KeyT),
|
|
|
|
MorphKey::new("z", &[], KeyCode::KeyZ),
|
|
|
|
MorphKey::new("i", &[], KeyCode::KeyI),
|
|
|
|
MorphKey::new("o", &[], KeyCode::KeyO),
|
|
|
|
MorphKey::new("p", &[], KeyCode::KeyP),
|
|
|
|
MorphKey::new("f", &[], KeyCode::KeyF),
|
|
|
|
MorphKey::new("g", &[], KeyCode::KeyG),
|
|
|
|
MorphKey::new("h", &[], KeyCode::KeyH),
|
|
|
|
MorphKey::new("j", &[], KeyCode::KeyJ),
|
|
|
|
MorphKey::new("k", &[], KeyCode::KeyK),
|
|
|
|
MorphKey::new("y", &[], KeyCode::KeyY),
|
|
|
|
MorphKey::new("x", &[], KeyCode::KeyX),
|
|
|
|
MorphKey::new("c", &[], KeyCode::KeyC),
|
|
|
|
MorphKey::new("v", &[], KeyCode::KeyV),
|
|
|
|
MorphKey::new("b", &[], KeyCode::KeyB),
|
|
|
|
MorphKey::new("n", &[], KeyCode::KeyN),
|
|
|
|
MorphKey::new("m", &[], KeyCode::KeyM),
|
|
|
|
MorphKey::new("0", &[], KeyCode::Digit0),
|
|
|
|
MorphKey::new("1", &[], KeyCode::Digit1),
|
|
|
|
MorphKey::new("2", &[], KeyCode::Digit2),
|
|
|
|
MorphKey::new("3", &[], KeyCode::Digit3),
|
|
|
|
MorphKey::new("4", &[], KeyCode::Digit4),
|
|
|
|
MorphKey::new("5", &[], KeyCode::Digit5),
|
|
|
|
MorphKey::new("6", &[], KeyCode::Digit6),
|
|
|
|
MorphKey::new("7", &[], KeyCode::Digit7),
|
|
|
|
MorphKey::new("8", &[], KeyCode::Digit8),
|
|
|
|
MorphKey::new("9", &[], KeyCode::Digit9),
|
|
|
|
MorphKey::new("lshift-R", &[KeyCode::ShiftLeft], KeyCode::KeyR),
|
|
|
|
MorphKey::new("lshift-T", &[KeyCode::ShiftLeft], KeyCode::KeyT),
|
|
|
|
MorphKey::new("lshift-Z", &[KeyCode::ShiftLeft], KeyCode::KeyZ),
|
|
|
|
MorphKey::new("lshift-I", &[KeyCode::ShiftLeft], KeyCode::KeyI),
|
|
|
|
MorphKey::new("lshift-O", &[KeyCode::ShiftLeft], KeyCode::KeyO),
|
|
|
|
MorphKey::new("lshift-P", &[KeyCode::ShiftLeft], KeyCode::KeyP),
|
|
|
|
MorphKey::new("lshift-F", &[KeyCode::ShiftLeft], KeyCode::KeyF),
|
|
|
|
MorphKey::new("lshift-G", &[KeyCode::ShiftLeft], KeyCode::KeyG),
|
|
|
|
MorphKey::new("lshift-H", &[KeyCode::ShiftLeft], KeyCode::KeyH),
|
|
|
|
MorphKey::new("lshift-J", &[KeyCode::ShiftLeft], KeyCode::KeyJ),
|
|
|
|
MorphKey::new("lshift-K", &[KeyCode::ShiftLeft], KeyCode::KeyK),
|
|
|
|
MorphKey::new("lshift-Y", &[KeyCode::ShiftLeft], KeyCode::KeyY),
|
|
|
|
MorphKey::new("lshift-X", &[KeyCode::ShiftLeft], KeyCode::KeyX),
|
|
|
|
MorphKey::new("lshift-C", &[KeyCode::ShiftLeft], KeyCode::KeyC),
|
|
|
|
MorphKey::new("lshift-V", &[KeyCode::ShiftLeft], KeyCode::KeyV),
|
|
|
|
MorphKey::new("lshift-B", &[KeyCode::ShiftLeft], KeyCode::KeyB),
|
|
|
|
MorphKey::new("lshift-N", &[KeyCode::ShiftLeft], KeyCode::KeyN),
|
|
|
|
MorphKey::new("lshift-M", &[KeyCode::ShiftLeft], KeyCode::KeyM),
|
|
|
|
MorphKey::new("lshift-0", &[KeyCode::ShiftLeft], KeyCode::Digit0),
|
|
|
|
MorphKey::new("lshift-1", &[KeyCode::ShiftLeft], KeyCode::Digit1),
|
|
|
|
MorphKey::new("lshift-2", &[KeyCode::ShiftLeft], KeyCode::Digit2),
|
|
|
|
MorphKey::new("lshift-3", &[KeyCode::ShiftLeft], KeyCode::Digit3),
|
|
|
|
MorphKey::new("lshift-4", &[KeyCode::ShiftLeft], KeyCode::Digit4),
|
|
|
|
MorphKey::new("lshift-5", &[KeyCode::ShiftLeft], KeyCode::Digit5),
|
|
|
|
MorphKey::new("lshift-6", &[KeyCode::ShiftLeft], KeyCode::Digit6),
|
|
|
|
MorphKey::new("lshift-7", &[KeyCode::ShiftLeft], KeyCode::Digit7),
|
|
|
|
MorphKey::new("lshift-8", &[KeyCode::ShiftLeft], KeyCode::Digit8),
|
|
|
|
MorphKey::new("lshift-9", &[KeyCode::ShiftLeft], KeyCode::Digit9),
|
Add morph targets (#8158)
# Objective
- Add morph targets to `bevy_pbr` (closes #5756) & load them from glTF
- Supersedes #3722
- Fixes #6814
[Morph targets][1] (also known as shape interpolation, shape keys, or
blend shapes) allow animating individual vertices with fine grained
controls. This is typically used for facial expressions. By specifying
multiple poses as vertex offset, and providing a set of weight of each
pose, it is possible to define surprisingly realistic transitions
between poses. Blending between multiple poses also allow composition.
Morph targets are part of the [gltf standard][2] and are a feature of
Unity and Unreal, and babylone.js, it is only natural to implement them
in bevy.
## Solution
This implementation of morph targets uses a 3d texture where each pixel
is a component of an animated attribute. Each layer is a different
target. We use a 2d texture for each target, because the number of
attribute×components×animated vertices is expected to always exceed the
maximum pixel row size limit of webGL2. It copies fairly closely the way
skinning is implemented on the CPU side, while on the GPU side, the
shader morph target implementation is a relatively trivial detail.
We add an optional `morph_texture` to the `Mesh` struct. The
`morph_texture` is built through a method that accepts an iterator over
attribute buffers.
The `MorphWeights` component, user-accessible, controls the blend of
poses used by mesh instances (so that multiple copy of the same mesh may
have different weights), all the weights are uploaded to a uniform
buffer of 256 `f32`. We limit to 16 poses per mesh, and a total of 256
poses.
More literature:
* Old babylone.js implementation (vertex attribute-based):
https://www.eternalcoding.com/dev-log-1-morph-targets/
* Babylone.js implementation (similar to ours):
https://www.youtube.com/watch?v=LBPRmGgU0PE
* GPU gems 3:
https://developer.nvidia.com/gpugems/gpugems3/part-i-geometry/chapter-3-directx-10-blend-shapes-breaking-limits
* Development discord thread
https://discord.com/channels/691052431525675048/1083325980615114772
https://user-images.githubusercontent.com/26321040/231181046-3bca2ab2-d4d9-472e-8098-639f1871ce2e.mp4
https://github.com/bevyengine/bevy/assets/26321040/d2a0c544-0ef8-45cf-9f99-8c3792f5a258
## Acknowledgements
* Thanks to `storytold` for sponsoring the feature
* Thanks to `superdump` and `james7132` for guidance and help figuring
out stuff
## Future work
- Handling of less and more attributes (eg: animated uv, animated
arbitrary attributes)
- Dynamic pose allocation (so that zero-weighted poses aren't uploaded
to GPU for example, enables much more total poses)
- Better animation API, see #8357
----
## Changelog
- Add morph targets to bevy meshes
- Support up to 64 poses per mesh of individually up to 116508 vertices,
animation currently strictly limited to the position, normal and tangent
attributes.
- Load a morph target using `Mesh::set_morph_targets`
- Add `VisitMorphTargets` and `VisitMorphAttributes` traits to
`bevy_render`, this allows defining morph targets (a fairly complex and
nested data structure) through iterators (ie: single copy instead of
passing around buffers), see documentation of those traits for details
- Add `MorphWeights` component exported by `bevy_render`
- `MorphWeights` control mesh's morph target weights, blending between
various poses defined as morph targets.
- `MorphWeights` are directly inherited by direct children (single level
of hierarchy) of an entity. This allows controlling several mesh
primitives through a unique entity _as per GLTF spec_.
- Add `MorphTargetNames` component, naming each indices of loaded morph
targets.
- Load morph targets weights and buffers in `bevy_gltf`
- handle morph targets animations in `bevy_animation` (previously, it
was a `warn!` log)
- Add the `MorphStressTest.gltf` asset for morph targets testing, taken
from the glTF samples repo, CC0.
- Add morph target manipulation to `scene_viewer`
- Separate the animation code in `scene_viewer` from the rest of the
code, reducing `#[cfg(feature)]` noise
- Add the `morph_targets.rs` example to show off how to manipulate morph
targets, loading `MorpStressTest.gltf`
## Migration Guide
- (very specialized, unlikely to be touched by 3rd parties)
- `MeshPipeline` now has a single `mesh_layouts` field rather than
separate `mesh_layout` and `skinned_mesh_layout` fields. You should
handle all possible mesh bind group layouts in your implementation
- You should also handle properly the new `MORPH_TARGETS` shader def and
mesh pipeline key. A new function is exposed to make this easier:
`setup_moprh_and_skinning_defs`
- The `MeshBindGroup` is now `MeshBindGroups`, cached bind groups are
now accessed through the `get` method.
[1]: https://en.wikipedia.org/wiki/Morph_target_animation
[2]:
https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#morph-targets
---------
Co-authored-by: François <mockersf@gmail.com>
Co-authored-by: Carter Anderson <mcanders1@gmail.com>
2023-06-22 20:00:01 +00:00
|
|
|
];
|
|
|
|
|
|
|
|
#[derive(Clone, Copy)]
|
|
|
|
enum WeightChange {
|
|
|
|
Increase,
|
|
|
|
Decrease,
|
|
|
|
}
|
|
|
|
impl WeightChange {
|
|
|
|
fn reverse(&mut self) {
|
|
|
|
*self = match *self {
|
|
|
|
WeightChange::Increase => WeightChange::Decrease,
|
|
|
|
WeightChange::Decrease => WeightChange::Increase,
|
|
|
|
}
|
|
|
|
}
|
|
|
|
fn sign(self) -> f32 {
|
|
|
|
match self {
|
|
|
|
WeightChange::Increase => 1.0,
|
|
|
|
WeightChange::Decrease => -1.0,
|
|
|
|
}
|
|
|
|
}
|
|
|
|
fn change_weight(&mut self, weight: f32, change: f32) -> f32 {
|
|
|
|
let mut change = change * self.sign();
|
|
|
|
let new_weight = weight + change;
|
|
|
|
if new_weight <= 0.0 || new_weight >= 1.0 {
|
|
|
|
self.reverse();
|
|
|
|
change = -change;
|
|
|
|
}
|
|
|
|
weight + change
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
struct Target {
|
|
|
|
entity_name: Option<String>,
|
|
|
|
entity: Entity,
|
|
|
|
name: Option<String>,
|
|
|
|
index: usize,
|
|
|
|
weight: f32,
|
|
|
|
change_dir: WeightChange,
|
|
|
|
}
|
|
|
|
impl fmt::Display for Target {
|
|
|
|
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
|
|
|
match (self.name.as_ref(), self.entity_name.as_ref()) {
|
|
|
|
(None, None) => write!(f, "animation{} of {:?}", self.index, self.entity),
|
|
|
|
(None, Some(entity)) => write!(f, "animation{} of {entity}", self.index),
|
|
|
|
(Some(target), None) => write!(f, "{target} of {:?}", self.entity),
|
|
|
|
(Some(target), Some(entity)) => write!(f, "{target} of {entity}"),
|
|
|
|
}?;
|
|
|
|
write!(f, ": {}", self.weight)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
impl Target {
|
|
|
|
fn text_section(&self, key: &str, style: TextStyle) -> TextSection {
|
|
|
|
TextSection::new(format!("[{key}] {self}\n"), style)
|
|
|
|
}
|
|
|
|
fn new(
|
|
|
|
entity_name: Option<&Name>,
|
|
|
|
weights: &[f32],
|
|
|
|
target_names: Option<&[String]>,
|
|
|
|
entity: Entity,
|
|
|
|
) -> Vec<Target> {
|
|
|
|
let get_name = |i| target_names.and_then(|names| names.get(i));
|
|
|
|
let entity_name = entity_name.map(|n| n.as_str());
|
|
|
|
weights
|
|
|
|
.iter()
|
|
|
|
.enumerate()
|
|
|
|
.map(|(index, weight)| Target {
|
|
|
|
entity_name: entity_name.map(|n| n.to_owned()),
|
|
|
|
entity,
|
|
|
|
name: get_name(index).cloned(),
|
|
|
|
index,
|
|
|
|
weight: *weight,
|
|
|
|
change_dir: WeightChange::Increase,
|
|
|
|
})
|
|
|
|
.collect()
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
#[derive(Resource)]
|
|
|
|
struct WeightsControl {
|
|
|
|
weights: Vec<Target>,
|
|
|
|
}
|
|
|
|
|
|
|
|
struct MorphKey {
|
|
|
|
name: &'static str,
|
|
|
|
modifiers: &'static [KeyCode],
|
|
|
|
key: KeyCode,
|
|
|
|
}
|
|
|
|
impl MorphKey {
|
|
|
|
const fn new(name: &'static str, modifiers: &'static [KeyCode], key: KeyCode) -> Self {
|
|
|
|
MorphKey {
|
|
|
|
name,
|
|
|
|
modifiers,
|
|
|
|
key,
|
|
|
|
}
|
|
|
|
}
|
2023-12-06 20:32:34 +00:00
|
|
|
fn active(&self, inputs: &ButtonInput<KeyCode>) -> bool {
|
Add morph targets (#8158)
# Objective
- Add morph targets to `bevy_pbr` (closes #5756) & load them from glTF
- Supersedes #3722
- Fixes #6814
[Morph targets][1] (also known as shape interpolation, shape keys, or
blend shapes) allow animating individual vertices with fine grained
controls. This is typically used for facial expressions. By specifying
multiple poses as vertex offset, and providing a set of weight of each
pose, it is possible to define surprisingly realistic transitions
between poses. Blending between multiple poses also allow composition.
Morph targets are part of the [gltf standard][2] and are a feature of
Unity and Unreal, and babylone.js, it is only natural to implement them
in bevy.
## Solution
This implementation of morph targets uses a 3d texture where each pixel
is a component of an animated attribute. Each layer is a different
target. We use a 2d texture for each target, because the number of
attribute×components×animated vertices is expected to always exceed the
maximum pixel row size limit of webGL2. It copies fairly closely the way
skinning is implemented on the CPU side, while on the GPU side, the
shader morph target implementation is a relatively trivial detail.
We add an optional `morph_texture` to the `Mesh` struct. The
`morph_texture` is built through a method that accepts an iterator over
attribute buffers.
The `MorphWeights` component, user-accessible, controls the blend of
poses used by mesh instances (so that multiple copy of the same mesh may
have different weights), all the weights are uploaded to a uniform
buffer of 256 `f32`. We limit to 16 poses per mesh, and a total of 256
poses.
More literature:
* Old babylone.js implementation (vertex attribute-based):
https://www.eternalcoding.com/dev-log-1-morph-targets/
* Babylone.js implementation (similar to ours):
https://www.youtube.com/watch?v=LBPRmGgU0PE
* GPU gems 3:
https://developer.nvidia.com/gpugems/gpugems3/part-i-geometry/chapter-3-directx-10-blend-shapes-breaking-limits
* Development discord thread
https://discord.com/channels/691052431525675048/1083325980615114772
https://user-images.githubusercontent.com/26321040/231181046-3bca2ab2-d4d9-472e-8098-639f1871ce2e.mp4
https://github.com/bevyengine/bevy/assets/26321040/d2a0c544-0ef8-45cf-9f99-8c3792f5a258
## Acknowledgements
* Thanks to `storytold` for sponsoring the feature
* Thanks to `superdump` and `james7132` for guidance and help figuring
out stuff
## Future work
- Handling of less and more attributes (eg: animated uv, animated
arbitrary attributes)
- Dynamic pose allocation (so that zero-weighted poses aren't uploaded
to GPU for example, enables much more total poses)
- Better animation API, see #8357
----
## Changelog
- Add morph targets to bevy meshes
- Support up to 64 poses per mesh of individually up to 116508 vertices,
animation currently strictly limited to the position, normal and tangent
attributes.
- Load a morph target using `Mesh::set_morph_targets`
- Add `VisitMorphTargets` and `VisitMorphAttributes` traits to
`bevy_render`, this allows defining morph targets (a fairly complex and
nested data structure) through iterators (ie: single copy instead of
passing around buffers), see documentation of those traits for details
- Add `MorphWeights` component exported by `bevy_render`
- `MorphWeights` control mesh's morph target weights, blending between
various poses defined as morph targets.
- `MorphWeights` are directly inherited by direct children (single level
of hierarchy) of an entity. This allows controlling several mesh
primitives through a unique entity _as per GLTF spec_.
- Add `MorphTargetNames` component, naming each indices of loaded morph
targets.
- Load morph targets weights and buffers in `bevy_gltf`
- handle morph targets animations in `bevy_animation` (previously, it
was a `warn!` log)
- Add the `MorphStressTest.gltf` asset for morph targets testing, taken
from the glTF samples repo, CC0.
- Add morph target manipulation to `scene_viewer`
- Separate the animation code in `scene_viewer` from the rest of the
code, reducing `#[cfg(feature)]` noise
- Add the `morph_targets.rs` example to show off how to manipulate morph
targets, loading `MorpStressTest.gltf`
## Migration Guide
- (very specialized, unlikely to be touched by 3rd parties)
- `MeshPipeline` now has a single `mesh_layouts` field rather than
separate `mesh_layout` and `skinned_mesh_layout` fields. You should
handle all possible mesh bind group layouts in your implementation
- You should also handle properly the new `MORPH_TARGETS` shader def and
mesh pipeline key. A new function is exposed to make this easier:
`setup_moprh_and_skinning_defs`
- The `MeshBindGroup` is now `MeshBindGroups`, cached bind groups are
now accessed through the `get` method.
[1]: https://en.wikipedia.org/wiki/Morph_target_animation
[2]:
https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#morph-targets
---------
Co-authored-by: François <mockersf@gmail.com>
Co-authored-by: Carter Anderson <mcanders1@gmail.com>
2023-06-22 20:00:01 +00:00
|
|
|
let mut modifier = self.modifiers.iter();
|
|
|
|
let mut non_modifier = ALL_MODIFIERS.iter().filter(|m| !self.modifiers.contains(m));
|
|
|
|
|
|
|
|
let key = inputs.pressed(self.key);
|
|
|
|
let modifier = modifier.all(|m| inputs.pressed(*m));
|
|
|
|
let non_modifier = non_modifier.all(|m| !inputs.pressed(*m));
|
|
|
|
key && modifier && non_modifier
|
|
|
|
}
|
|
|
|
}
|
|
|
|
fn update_text(
|
|
|
|
controls: Option<ResMut<WeightsControl>>,
|
|
|
|
mut text: Query<&mut Text>,
|
|
|
|
morphs: Query<&MorphWeights>,
|
|
|
|
) {
|
2023-08-25 12:34:24 +00:00
|
|
|
let Some(mut controls) = controls else {
|
|
|
|
return;
|
|
|
|
};
|
Add morph targets (#8158)
# Objective
- Add morph targets to `bevy_pbr` (closes #5756) & load them from glTF
- Supersedes #3722
- Fixes #6814
[Morph targets][1] (also known as shape interpolation, shape keys, or
blend shapes) allow animating individual vertices with fine grained
controls. This is typically used for facial expressions. By specifying
multiple poses as vertex offset, and providing a set of weight of each
pose, it is possible to define surprisingly realistic transitions
between poses. Blending between multiple poses also allow composition.
Morph targets are part of the [gltf standard][2] and are a feature of
Unity and Unreal, and babylone.js, it is only natural to implement them
in bevy.
## Solution
This implementation of morph targets uses a 3d texture where each pixel
is a component of an animated attribute. Each layer is a different
target. We use a 2d texture for each target, because the number of
attribute×components×animated vertices is expected to always exceed the
maximum pixel row size limit of webGL2. It copies fairly closely the way
skinning is implemented on the CPU side, while on the GPU side, the
shader morph target implementation is a relatively trivial detail.
We add an optional `morph_texture` to the `Mesh` struct. The
`morph_texture` is built through a method that accepts an iterator over
attribute buffers.
The `MorphWeights` component, user-accessible, controls the blend of
poses used by mesh instances (so that multiple copy of the same mesh may
have different weights), all the weights are uploaded to a uniform
buffer of 256 `f32`. We limit to 16 poses per mesh, and a total of 256
poses.
More literature:
* Old babylone.js implementation (vertex attribute-based):
https://www.eternalcoding.com/dev-log-1-morph-targets/
* Babylone.js implementation (similar to ours):
https://www.youtube.com/watch?v=LBPRmGgU0PE
* GPU gems 3:
https://developer.nvidia.com/gpugems/gpugems3/part-i-geometry/chapter-3-directx-10-blend-shapes-breaking-limits
* Development discord thread
https://discord.com/channels/691052431525675048/1083325980615114772
https://user-images.githubusercontent.com/26321040/231181046-3bca2ab2-d4d9-472e-8098-639f1871ce2e.mp4
https://github.com/bevyengine/bevy/assets/26321040/d2a0c544-0ef8-45cf-9f99-8c3792f5a258
## Acknowledgements
* Thanks to `storytold` for sponsoring the feature
* Thanks to `superdump` and `james7132` for guidance and help figuring
out stuff
## Future work
- Handling of less and more attributes (eg: animated uv, animated
arbitrary attributes)
- Dynamic pose allocation (so that zero-weighted poses aren't uploaded
to GPU for example, enables much more total poses)
- Better animation API, see #8357
----
## Changelog
- Add morph targets to bevy meshes
- Support up to 64 poses per mesh of individually up to 116508 vertices,
animation currently strictly limited to the position, normal and tangent
attributes.
- Load a morph target using `Mesh::set_morph_targets`
- Add `VisitMorphTargets` and `VisitMorphAttributes` traits to
`bevy_render`, this allows defining morph targets (a fairly complex and
nested data structure) through iterators (ie: single copy instead of
passing around buffers), see documentation of those traits for details
- Add `MorphWeights` component exported by `bevy_render`
- `MorphWeights` control mesh's morph target weights, blending between
various poses defined as morph targets.
- `MorphWeights` are directly inherited by direct children (single level
of hierarchy) of an entity. This allows controlling several mesh
primitives through a unique entity _as per GLTF spec_.
- Add `MorphTargetNames` component, naming each indices of loaded morph
targets.
- Load morph targets weights and buffers in `bevy_gltf`
- handle morph targets animations in `bevy_animation` (previously, it
was a `warn!` log)
- Add the `MorphStressTest.gltf` asset for morph targets testing, taken
from the glTF samples repo, CC0.
- Add morph target manipulation to `scene_viewer`
- Separate the animation code in `scene_viewer` from the rest of the
code, reducing `#[cfg(feature)]` noise
- Add the `morph_targets.rs` example to show off how to manipulate morph
targets, loading `MorpStressTest.gltf`
## Migration Guide
- (very specialized, unlikely to be touched by 3rd parties)
- `MeshPipeline` now has a single `mesh_layouts` field rather than
separate `mesh_layout` and `skinned_mesh_layout` fields. You should
handle all possible mesh bind group layouts in your implementation
- You should also handle properly the new `MORPH_TARGETS` shader def and
mesh pipeline key. A new function is exposed to make this easier:
`setup_moprh_and_skinning_defs`
- The `MeshBindGroup` is now `MeshBindGroups`, cached bind groups are
now accessed through the `get` method.
[1]: https://en.wikipedia.org/wiki/Morph_target_animation
[2]:
https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#morph-targets
---------
Co-authored-by: François <mockersf@gmail.com>
Co-authored-by: Carter Anderson <mcanders1@gmail.com>
2023-06-22 20:00:01 +00:00
|
|
|
for (i, target) in controls.weights.iter_mut().enumerate() {
|
|
|
|
let Ok(weights) = morphs.get(target.entity) else {
|
|
|
|
continue;
|
|
|
|
};
|
|
|
|
let Some(&actual_weight) = weights.weights().get(target.index) else {
|
|
|
|
continue;
|
|
|
|
};
|
|
|
|
if actual_weight != target.weight {
|
|
|
|
target.weight = actual_weight;
|
|
|
|
}
|
|
|
|
let key_name = &AVAILABLE_KEYS[i].name;
|
|
|
|
let mut text = text.single_mut();
|
|
|
|
text.sections[i + 2].value = format!("[{key_name}] {target}\n");
|
|
|
|
}
|
|
|
|
}
|
|
|
|
fn update_morphs(
|
|
|
|
controls: Option<ResMut<WeightsControl>>,
|
|
|
|
mut morphs: Query<&mut MorphWeights>,
|
2023-12-06 20:32:34 +00:00
|
|
|
input: Res<ButtonInput<KeyCode>>,
|
Add morph targets (#8158)
# Objective
- Add morph targets to `bevy_pbr` (closes #5756) & load them from glTF
- Supersedes #3722
- Fixes #6814
[Morph targets][1] (also known as shape interpolation, shape keys, or
blend shapes) allow animating individual vertices with fine grained
controls. This is typically used for facial expressions. By specifying
multiple poses as vertex offset, and providing a set of weight of each
pose, it is possible to define surprisingly realistic transitions
between poses. Blending between multiple poses also allow composition.
Morph targets are part of the [gltf standard][2] and are a feature of
Unity and Unreal, and babylone.js, it is only natural to implement them
in bevy.
## Solution
This implementation of morph targets uses a 3d texture where each pixel
is a component of an animated attribute. Each layer is a different
target. We use a 2d texture for each target, because the number of
attribute×components×animated vertices is expected to always exceed the
maximum pixel row size limit of webGL2. It copies fairly closely the way
skinning is implemented on the CPU side, while on the GPU side, the
shader morph target implementation is a relatively trivial detail.
We add an optional `morph_texture` to the `Mesh` struct. The
`morph_texture` is built through a method that accepts an iterator over
attribute buffers.
The `MorphWeights` component, user-accessible, controls the blend of
poses used by mesh instances (so that multiple copy of the same mesh may
have different weights), all the weights are uploaded to a uniform
buffer of 256 `f32`. We limit to 16 poses per mesh, and a total of 256
poses.
More literature:
* Old babylone.js implementation (vertex attribute-based):
https://www.eternalcoding.com/dev-log-1-morph-targets/
* Babylone.js implementation (similar to ours):
https://www.youtube.com/watch?v=LBPRmGgU0PE
* GPU gems 3:
https://developer.nvidia.com/gpugems/gpugems3/part-i-geometry/chapter-3-directx-10-blend-shapes-breaking-limits
* Development discord thread
https://discord.com/channels/691052431525675048/1083325980615114772
https://user-images.githubusercontent.com/26321040/231181046-3bca2ab2-d4d9-472e-8098-639f1871ce2e.mp4
https://github.com/bevyengine/bevy/assets/26321040/d2a0c544-0ef8-45cf-9f99-8c3792f5a258
## Acknowledgements
* Thanks to `storytold` for sponsoring the feature
* Thanks to `superdump` and `james7132` for guidance and help figuring
out stuff
## Future work
- Handling of less and more attributes (eg: animated uv, animated
arbitrary attributes)
- Dynamic pose allocation (so that zero-weighted poses aren't uploaded
to GPU for example, enables much more total poses)
- Better animation API, see #8357
----
## Changelog
- Add morph targets to bevy meshes
- Support up to 64 poses per mesh of individually up to 116508 vertices,
animation currently strictly limited to the position, normal and tangent
attributes.
- Load a morph target using `Mesh::set_morph_targets`
- Add `VisitMorphTargets` and `VisitMorphAttributes` traits to
`bevy_render`, this allows defining morph targets (a fairly complex and
nested data structure) through iterators (ie: single copy instead of
passing around buffers), see documentation of those traits for details
- Add `MorphWeights` component exported by `bevy_render`
- `MorphWeights` control mesh's morph target weights, blending between
various poses defined as morph targets.
- `MorphWeights` are directly inherited by direct children (single level
of hierarchy) of an entity. This allows controlling several mesh
primitives through a unique entity _as per GLTF spec_.
- Add `MorphTargetNames` component, naming each indices of loaded morph
targets.
- Load morph targets weights and buffers in `bevy_gltf`
- handle morph targets animations in `bevy_animation` (previously, it
was a `warn!` log)
- Add the `MorphStressTest.gltf` asset for morph targets testing, taken
from the glTF samples repo, CC0.
- Add morph target manipulation to `scene_viewer`
- Separate the animation code in `scene_viewer` from the rest of the
code, reducing `#[cfg(feature)]` noise
- Add the `morph_targets.rs` example to show off how to manipulate morph
targets, loading `MorpStressTest.gltf`
## Migration Guide
- (very specialized, unlikely to be touched by 3rd parties)
- `MeshPipeline` now has a single `mesh_layouts` field rather than
separate `mesh_layout` and `skinned_mesh_layout` fields. You should
handle all possible mesh bind group layouts in your implementation
- You should also handle properly the new `MORPH_TARGETS` shader def and
mesh pipeline key. A new function is exposed to make this easier:
`setup_moprh_and_skinning_defs`
- The `MeshBindGroup` is now `MeshBindGroups`, cached bind groups are
now accessed through the `get` method.
[1]: https://en.wikipedia.org/wiki/Morph_target_animation
[2]:
https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#morph-targets
---------
Co-authored-by: François <mockersf@gmail.com>
Co-authored-by: Carter Anderson <mcanders1@gmail.com>
2023-06-22 20:00:01 +00:00
|
|
|
time: Res<Time>,
|
|
|
|
) {
|
2023-08-25 12:34:24 +00:00
|
|
|
let Some(mut controls) = controls else {
|
|
|
|
return;
|
|
|
|
};
|
Add morph targets (#8158)
# Objective
- Add morph targets to `bevy_pbr` (closes #5756) & load them from glTF
- Supersedes #3722
- Fixes #6814
[Morph targets][1] (also known as shape interpolation, shape keys, or
blend shapes) allow animating individual vertices with fine grained
controls. This is typically used for facial expressions. By specifying
multiple poses as vertex offset, and providing a set of weight of each
pose, it is possible to define surprisingly realistic transitions
between poses. Blending between multiple poses also allow composition.
Morph targets are part of the [gltf standard][2] and are a feature of
Unity and Unreal, and babylone.js, it is only natural to implement them
in bevy.
## Solution
This implementation of morph targets uses a 3d texture where each pixel
is a component of an animated attribute. Each layer is a different
target. We use a 2d texture for each target, because the number of
attribute×components×animated vertices is expected to always exceed the
maximum pixel row size limit of webGL2. It copies fairly closely the way
skinning is implemented on the CPU side, while on the GPU side, the
shader morph target implementation is a relatively trivial detail.
We add an optional `morph_texture` to the `Mesh` struct. The
`morph_texture` is built through a method that accepts an iterator over
attribute buffers.
The `MorphWeights` component, user-accessible, controls the blend of
poses used by mesh instances (so that multiple copy of the same mesh may
have different weights), all the weights are uploaded to a uniform
buffer of 256 `f32`. We limit to 16 poses per mesh, and a total of 256
poses.
More literature:
* Old babylone.js implementation (vertex attribute-based):
https://www.eternalcoding.com/dev-log-1-morph-targets/
* Babylone.js implementation (similar to ours):
https://www.youtube.com/watch?v=LBPRmGgU0PE
* GPU gems 3:
https://developer.nvidia.com/gpugems/gpugems3/part-i-geometry/chapter-3-directx-10-blend-shapes-breaking-limits
* Development discord thread
https://discord.com/channels/691052431525675048/1083325980615114772
https://user-images.githubusercontent.com/26321040/231181046-3bca2ab2-d4d9-472e-8098-639f1871ce2e.mp4
https://github.com/bevyengine/bevy/assets/26321040/d2a0c544-0ef8-45cf-9f99-8c3792f5a258
## Acknowledgements
* Thanks to `storytold` for sponsoring the feature
* Thanks to `superdump` and `james7132` for guidance and help figuring
out stuff
## Future work
- Handling of less and more attributes (eg: animated uv, animated
arbitrary attributes)
- Dynamic pose allocation (so that zero-weighted poses aren't uploaded
to GPU for example, enables much more total poses)
- Better animation API, see #8357
----
## Changelog
- Add morph targets to bevy meshes
- Support up to 64 poses per mesh of individually up to 116508 vertices,
animation currently strictly limited to the position, normal and tangent
attributes.
- Load a morph target using `Mesh::set_morph_targets`
- Add `VisitMorphTargets` and `VisitMorphAttributes` traits to
`bevy_render`, this allows defining morph targets (a fairly complex and
nested data structure) through iterators (ie: single copy instead of
passing around buffers), see documentation of those traits for details
- Add `MorphWeights` component exported by `bevy_render`
- `MorphWeights` control mesh's morph target weights, blending between
various poses defined as morph targets.
- `MorphWeights` are directly inherited by direct children (single level
of hierarchy) of an entity. This allows controlling several mesh
primitives through a unique entity _as per GLTF spec_.
- Add `MorphTargetNames` component, naming each indices of loaded morph
targets.
- Load morph targets weights and buffers in `bevy_gltf`
- handle morph targets animations in `bevy_animation` (previously, it
was a `warn!` log)
- Add the `MorphStressTest.gltf` asset for morph targets testing, taken
from the glTF samples repo, CC0.
- Add morph target manipulation to `scene_viewer`
- Separate the animation code in `scene_viewer` from the rest of the
code, reducing `#[cfg(feature)]` noise
- Add the `morph_targets.rs` example to show off how to manipulate morph
targets, loading `MorpStressTest.gltf`
## Migration Guide
- (very specialized, unlikely to be touched by 3rd parties)
- `MeshPipeline` now has a single `mesh_layouts` field rather than
separate `mesh_layout` and `skinned_mesh_layout` fields. You should
handle all possible mesh bind group layouts in your implementation
- You should also handle properly the new `MORPH_TARGETS` shader def and
mesh pipeline key. A new function is exposed to make this easier:
`setup_moprh_and_skinning_defs`
- The `MeshBindGroup` is now `MeshBindGroups`, cached bind groups are
now accessed through the `get` method.
[1]: https://en.wikipedia.org/wiki/Morph_target_animation
[2]:
https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#morph-targets
---------
Co-authored-by: François <mockersf@gmail.com>
Co-authored-by: Carter Anderson <mcanders1@gmail.com>
2023-06-22 20:00:01 +00:00
|
|
|
for (i, target) in controls.weights.iter_mut().enumerate() {
|
|
|
|
if !AVAILABLE_KEYS[i].active(&input) {
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
let Ok(mut weights) = morphs.get_mut(target.entity) else {
|
|
|
|
continue;
|
|
|
|
};
|
|
|
|
// To update individual morph target weights, get the `MorphWeights`
|
|
|
|
// component and call `weights_mut` to get access to the weights.
|
|
|
|
let weights_slice = weights.weights_mut();
|
|
|
|
let i = target.index;
|
|
|
|
let change = time.delta_seconds() * WEIGHT_PER_SECOND;
|
|
|
|
let new_weight = target.change_dir.change_weight(weights_slice[i], change);
|
|
|
|
weights_slice[i] = new_weight;
|
|
|
|
target.weight = new_weight;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
fn detect_morphs(
|
|
|
|
mut commands: Commands,
|
|
|
|
morphs: Query<(Entity, &MorphWeights, Option<&Name>)>,
|
|
|
|
meshes: Res<Assets<Mesh>>,
|
|
|
|
scene_handle: Res<SceneHandle>,
|
|
|
|
mut setup: Local<bool>,
|
|
|
|
asset_server: Res<AssetServer>,
|
|
|
|
) {
|
|
|
|
let no_morphing = morphs.iter().len() == 0;
|
|
|
|
if no_morphing {
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
if scene_handle.is_loaded && !*setup {
|
|
|
|
*setup = true;
|
|
|
|
} else {
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
let mut detected = Vec::new();
|
|
|
|
|
|
|
|
for (entity, weights, name) in &morphs {
|
|
|
|
let target_names = weights
|
|
|
|
.first_mesh()
|
|
|
|
.and_then(|h| meshes.get(h))
|
|
|
|
.and_then(|m| m.morph_target_names());
|
|
|
|
let targets = Target::new(name, weights.weights(), target_names, entity);
|
|
|
|
detected.extend(targets);
|
|
|
|
}
|
|
|
|
detected.truncate(AVAILABLE_KEYS.len());
|
|
|
|
let style = TextStyle {
|
|
|
|
font: asset_server.load("assets/fonts/FiraMono-Medium.ttf"),
|
|
|
|
font_size: 13.0,
|
2023-11-03 12:57:38 +00:00
|
|
|
..default()
|
Add morph targets (#8158)
# Objective
- Add morph targets to `bevy_pbr` (closes #5756) & load them from glTF
- Supersedes #3722
- Fixes #6814
[Morph targets][1] (also known as shape interpolation, shape keys, or
blend shapes) allow animating individual vertices with fine grained
controls. This is typically used for facial expressions. By specifying
multiple poses as vertex offset, and providing a set of weight of each
pose, it is possible to define surprisingly realistic transitions
between poses. Blending between multiple poses also allow composition.
Morph targets are part of the [gltf standard][2] and are a feature of
Unity and Unreal, and babylone.js, it is only natural to implement them
in bevy.
## Solution
This implementation of morph targets uses a 3d texture where each pixel
is a component of an animated attribute. Each layer is a different
target. We use a 2d texture for each target, because the number of
attribute×components×animated vertices is expected to always exceed the
maximum pixel row size limit of webGL2. It copies fairly closely the way
skinning is implemented on the CPU side, while on the GPU side, the
shader morph target implementation is a relatively trivial detail.
We add an optional `morph_texture` to the `Mesh` struct. The
`morph_texture` is built through a method that accepts an iterator over
attribute buffers.
The `MorphWeights` component, user-accessible, controls the blend of
poses used by mesh instances (so that multiple copy of the same mesh may
have different weights), all the weights are uploaded to a uniform
buffer of 256 `f32`. We limit to 16 poses per mesh, and a total of 256
poses.
More literature:
* Old babylone.js implementation (vertex attribute-based):
https://www.eternalcoding.com/dev-log-1-morph-targets/
* Babylone.js implementation (similar to ours):
https://www.youtube.com/watch?v=LBPRmGgU0PE
* GPU gems 3:
https://developer.nvidia.com/gpugems/gpugems3/part-i-geometry/chapter-3-directx-10-blend-shapes-breaking-limits
* Development discord thread
https://discord.com/channels/691052431525675048/1083325980615114772
https://user-images.githubusercontent.com/26321040/231181046-3bca2ab2-d4d9-472e-8098-639f1871ce2e.mp4
https://github.com/bevyengine/bevy/assets/26321040/d2a0c544-0ef8-45cf-9f99-8c3792f5a258
## Acknowledgements
* Thanks to `storytold` for sponsoring the feature
* Thanks to `superdump` and `james7132` for guidance and help figuring
out stuff
## Future work
- Handling of less and more attributes (eg: animated uv, animated
arbitrary attributes)
- Dynamic pose allocation (so that zero-weighted poses aren't uploaded
to GPU for example, enables much more total poses)
- Better animation API, see #8357
----
## Changelog
- Add morph targets to bevy meshes
- Support up to 64 poses per mesh of individually up to 116508 vertices,
animation currently strictly limited to the position, normal and tangent
attributes.
- Load a morph target using `Mesh::set_morph_targets`
- Add `VisitMorphTargets` and `VisitMorphAttributes` traits to
`bevy_render`, this allows defining morph targets (a fairly complex and
nested data structure) through iterators (ie: single copy instead of
passing around buffers), see documentation of those traits for details
- Add `MorphWeights` component exported by `bevy_render`
- `MorphWeights` control mesh's morph target weights, blending between
various poses defined as morph targets.
- `MorphWeights` are directly inherited by direct children (single level
of hierarchy) of an entity. This allows controlling several mesh
primitives through a unique entity _as per GLTF spec_.
- Add `MorphTargetNames` component, naming each indices of loaded morph
targets.
- Load morph targets weights and buffers in `bevy_gltf`
- handle morph targets animations in `bevy_animation` (previously, it
was a `warn!` log)
- Add the `MorphStressTest.gltf` asset for morph targets testing, taken
from the glTF samples repo, CC0.
- Add morph target manipulation to `scene_viewer`
- Separate the animation code in `scene_viewer` from the rest of the
code, reducing `#[cfg(feature)]` noise
- Add the `morph_targets.rs` example to show off how to manipulate morph
targets, loading `MorpStressTest.gltf`
## Migration Guide
- (very specialized, unlikely to be touched by 3rd parties)
- `MeshPipeline` now has a single `mesh_layouts` field rather than
separate `mesh_layout` and `skinned_mesh_layout` fields. You should
handle all possible mesh bind group layouts in your implementation
- You should also handle properly the new `MORPH_TARGETS` shader def and
mesh pipeline key. A new function is exposed to make this easier:
`setup_moprh_and_skinning_defs`
- The `MeshBindGroup` is now `MeshBindGroups`, cached bind groups are
now accessed through the `get` method.
[1]: https://en.wikipedia.org/wiki/Morph_target_animation
[2]:
https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#morph-targets
---------
Co-authored-by: François <mockersf@gmail.com>
Co-authored-by: Carter Anderson <mcanders1@gmail.com>
2023-06-22 20:00:01 +00:00
|
|
|
};
|
|
|
|
let mut sections = vec![
|
|
|
|
TextSection::new("Morph Target Controls\n", style.clone()),
|
|
|
|
TextSection::new("---------------\n", style.clone()),
|
|
|
|
];
|
|
|
|
let target_to_text =
|
|
|
|
|(i, target): (usize, &Target)| target.text_section(AVAILABLE_KEYS[i].name, style.clone());
|
|
|
|
sections.extend(detected.iter().enumerate().map(target_to_text));
|
|
|
|
commands.insert_resource(WeightsControl { weights: detected });
|
|
|
|
commands.spawn(TextBundle::from_sections(sections).with_style(Style {
|
|
|
|
position_type: PositionType::Absolute,
|
|
|
|
top: Val::Px(10.0),
|
|
|
|
left: Val::Px(10.0),
|
|
|
|
..default()
|
|
|
|
}));
|
|
|
|
}
|
|
|
|
|
|
|
|
pub struct MorphViewerPlugin;
|
|
|
|
|
|
|
|
impl Plugin for MorphViewerPlugin {
|
|
|
|
fn build(&self, app: &mut App) {
|
|
|
|
app.add_systems(
|
|
|
|
Update,
|
|
|
|
(
|
|
|
|
update_morphs,
|
|
|
|
detect_morphs,
|
|
|
|
update_text.after(update_morphs),
|
|
|
|
),
|
|
|
|
);
|
|
|
|
}
|
|
|
|
}
|