bevy/examples/tools/scene_viewer/main.rs

165 lines
5.7 KiB
Rust
Raw Normal View History

//! A simple glTF scene viewer made with Bevy.
//!
//! Just run `cargo run --release --example scene_viewer /path/to/model.gltf`,
//! replacing the path as appropriate.
//! In case of multiple scenes, you can select which to display by adapting the file path: `/path/to/model.gltf#Scene1`.
//! With no arguments it will load the `FlightHelmet` glTF model from the repository assets subdirectory.
use bevy::{
asset::ChangeWatcher,
math::Vec3A,
prelude::*,
render::primitives::{Aabb, Sphere},
utils::Duration,
Windows as Entities (#5589) # Objective Fix https://github.com/bevyengine/bevy/issues/4530 - Make it easier to open/close/modify windows by setting them up as `Entity`s with a `Window` component. - Make multiple windows very simple to set up. (just add a `Window` component to an entity and it should open) ## Solution - Move all properties of window descriptor to ~components~ a component. - Replace `WindowId` with `Entity`. - ~Use change detection for components to update backend rather than events/commands. (The `CursorMoved`/`WindowResized`/... events are kept for user convenience.~ Check each field individually to see what we need to update, events are still kept for user convenience. --- ## Changelog - `WindowDescriptor` renamed to `Window`. - Width/height consolidated into a `WindowResolution` component. - Requesting maximization/minimization is done on the [`Window::state`] field. - `WindowId` is now `Entity`. ## Migration Guide - Replace `WindowDescriptor` with `Window`. - Change `width` and `height` fields in a `WindowResolution`, either by doing ```rust WindowResolution::new(width, height) // Explicitly // or using From<_> for tuples for convenience (1920., 1080.).into() ``` - Replace any `WindowCommand` code to just modify the `Window`'s fields directly and creating/closing windows is now by spawning/despawning an entity with a `Window` component like so: ```rust let window = commands.spawn(Window { ... }).id(); // open window commands.entity(window).despawn(); // close window ``` ## Unresolved - ~How do we tell when a window is minimized by a user?~ ~Currently using the `Resize(0, 0)` as an indicator of minimization.~ No longer attempting to tell given how finnicky this was across platforms, now the user can only request that a window be maximized/minimized. ## Future work - Move `exit_on_close` functionality out from windowing and into app(?) - https://github.com/bevyengine/bevy/issues/5621 - https://github.com/bevyengine/bevy/issues/7099 - https://github.com/bevyengine/bevy/issues/7098 Co-authored-by: Carter Anderson <mcanders1@gmail.com>
2023-01-19 00:38:28 +00:00
window::WindowPlugin,
};
Add morph targets (#8158) # Objective - Add morph targets to `bevy_pbr` (closes #5756) & load them from glTF - Supersedes #3722 - Fixes #6814 [Morph targets][1] (also known as shape interpolation, shape keys, or blend shapes) allow animating individual vertices with fine grained controls. This is typically used for facial expressions. By specifying multiple poses as vertex offset, and providing a set of weight of each pose, it is possible to define surprisingly realistic transitions between poses. Blending between multiple poses also allow composition. Morph targets are part of the [gltf standard][2] and are a feature of Unity and Unreal, and babylone.js, it is only natural to implement them in bevy. ## Solution This implementation of morph targets uses a 3d texture where each pixel is a component of an animated attribute. Each layer is a different target. We use a 2d texture for each target, because the number of attribute×components×animated vertices is expected to always exceed the maximum pixel row size limit of webGL2. It copies fairly closely the way skinning is implemented on the CPU side, while on the GPU side, the shader morph target implementation is a relatively trivial detail. We add an optional `morph_texture` to the `Mesh` struct. The `morph_texture` is built through a method that accepts an iterator over attribute buffers. The `MorphWeights` component, user-accessible, controls the blend of poses used by mesh instances (so that multiple copy of the same mesh may have different weights), all the weights are uploaded to a uniform buffer of 256 `f32`. We limit to 16 poses per mesh, and a total of 256 poses. More literature: * Old babylone.js implementation (vertex attribute-based): https://www.eternalcoding.com/dev-log-1-morph-targets/ * Babylone.js implementation (similar to ours): https://www.youtube.com/watch?v=LBPRmGgU0PE * GPU gems 3: https://developer.nvidia.com/gpugems/gpugems3/part-i-geometry/chapter-3-directx-10-blend-shapes-breaking-limits * Development discord thread https://discord.com/channels/691052431525675048/1083325980615114772 https://user-images.githubusercontent.com/26321040/231181046-3bca2ab2-d4d9-472e-8098-639f1871ce2e.mp4 https://github.com/bevyengine/bevy/assets/26321040/d2a0c544-0ef8-45cf-9f99-8c3792f5a258 ## Acknowledgements * Thanks to `storytold` for sponsoring the feature * Thanks to `superdump` and `james7132` for guidance and help figuring out stuff ## Future work - Handling of less and more attributes (eg: animated uv, animated arbitrary attributes) - Dynamic pose allocation (so that zero-weighted poses aren't uploaded to GPU for example, enables much more total poses) - Better animation API, see #8357 ---- ## Changelog - Add morph targets to bevy meshes - Support up to 64 poses per mesh of individually up to 116508 vertices, animation currently strictly limited to the position, normal and tangent attributes. - Load a morph target using `Mesh::set_morph_targets` - Add `VisitMorphTargets` and `VisitMorphAttributes` traits to `bevy_render`, this allows defining morph targets (a fairly complex and nested data structure) through iterators (ie: single copy instead of passing around buffers), see documentation of those traits for details - Add `MorphWeights` component exported by `bevy_render` - `MorphWeights` control mesh's morph target weights, blending between various poses defined as morph targets. - `MorphWeights` are directly inherited by direct children (single level of hierarchy) of an entity. This allows controlling several mesh primitives through a unique entity _as per GLTF spec_. - Add `MorphTargetNames` component, naming each indices of loaded morph targets. - Load morph targets weights and buffers in `bevy_gltf` - handle morph targets animations in `bevy_animation` (previously, it was a `warn!` log) - Add the `MorphStressTest.gltf` asset for morph targets testing, taken from the glTF samples repo, CC0. - Add morph target manipulation to `scene_viewer` - Separate the animation code in `scene_viewer` from the rest of the code, reducing `#[cfg(feature)]` noise - Add the `morph_targets.rs` example to show off how to manipulate morph targets, loading `MorpStressTest.gltf` ## Migration Guide - (very specialized, unlikely to be touched by 3rd parties) - `MeshPipeline` now has a single `mesh_layouts` field rather than separate `mesh_layout` and `skinned_mesh_layout` fields. You should handle all possible mesh bind group layouts in your implementation - You should also handle properly the new `MORPH_TARGETS` shader def and mesh pipeline key. A new function is exposed to make this easier: `setup_moprh_and_skinning_defs` - The `MeshBindGroup` is now `MeshBindGroups`, cached bind groups are now accessed through the `get` method. [1]: https://en.wikipedia.org/wiki/Morph_target_animation [2]: https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#morph-targets --------- Co-authored-by: François <mockersf@gmail.com> Co-authored-by: Carter Anderson <mcanders1@gmail.com>
2023-06-22 20:00:01 +00:00
#[cfg(feature = "animation")]
mod animation_plugin;
mod camera_controller_plugin;
Add morph targets (#8158) # Objective - Add morph targets to `bevy_pbr` (closes #5756) & load them from glTF - Supersedes #3722 - Fixes #6814 [Morph targets][1] (also known as shape interpolation, shape keys, or blend shapes) allow animating individual vertices with fine grained controls. This is typically used for facial expressions. By specifying multiple poses as vertex offset, and providing a set of weight of each pose, it is possible to define surprisingly realistic transitions between poses. Blending between multiple poses also allow composition. Morph targets are part of the [gltf standard][2] and are a feature of Unity and Unreal, and babylone.js, it is only natural to implement them in bevy. ## Solution This implementation of morph targets uses a 3d texture where each pixel is a component of an animated attribute. Each layer is a different target. We use a 2d texture for each target, because the number of attribute×components×animated vertices is expected to always exceed the maximum pixel row size limit of webGL2. It copies fairly closely the way skinning is implemented on the CPU side, while on the GPU side, the shader morph target implementation is a relatively trivial detail. We add an optional `morph_texture` to the `Mesh` struct. The `morph_texture` is built through a method that accepts an iterator over attribute buffers. The `MorphWeights` component, user-accessible, controls the blend of poses used by mesh instances (so that multiple copy of the same mesh may have different weights), all the weights are uploaded to a uniform buffer of 256 `f32`. We limit to 16 poses per mesh, and a total of 256 poses. More literature: * Old babylone.js implementation (vertex attribute-based): https://www.eternalcoding.com/dev-log-1-morph-targets/ * Babylone.js implementation (similar to ours): https://www.youtube.com/watch?v=LBPRmGgU0PE * GPU gems 3: https://developer.nvidia.com/gpugems/gpugems3/part-i-geometry/chapter-3-directx-10-blend-shapes-breaking-limits * Development discord thread https://discord.com/channels/691052431525675048/1083325980615114772 https://user-images.githubusercontent.com/26321040/231181046-3bca2ab2-d4d9-472e-8098-639f1871ce2e.mp4 https://github.com/bevyengine/bevy/assets/26321040/d2a0c544-0ef8-45cf-9f99-8c3792f5a258 ## Acknowledgements * Thanks to `storytold` for sponsoring the feature * Thanks to `superdump` and `james7132` for guidance and help figuring out stuff ## Future work - Handling of less and more attributes (eg: animated uv, animated arbitrary attributes) - Dynamic pose allocation (so that zero-weighted poses aren't uploaded to GPU for example, enables much more total poses) - Better animation API, see #8357 ---- ## Changelog - Add morph targets to bevy meshes - Support up to 64 poses per mesh of individually up to 116508 vertices, animation currently strictly limited to the position, normal and tangent attributes. - Load a morph target using `Mesh::set_morph_targets` - Add `VisitMorphTargets` and `VisitMorphAttributes` traits to `bevy_render`, this allows defining morph targets (a fairly complex and nested data structure) through iterators (ie: single copy instead of passing around buffers), see documentation of those traits for details - Add `MorphWeights` component exported by `bevy_render` - `MorphWeights` control mesh's morph target weights, blending between various poses defined as morph targets. - `MorphWeights` are directly inherited by direct children (single level of hierarchy) of an entity. This allows controlling several mesh primitives through a unique entity _as per GLTF spec_. - Add `MorphTargetNames` component, naming each indices of loaded morph targets. - Load morph targets weights and buffers in `bevy_gltf` - handle morph targets animations in `bevy_animation` (previously, it was a `warn!` log) - Add the `MorphStressTest.gltf` asset for morph targets testing, taken from the glTF samples repo, CC0. - Add morph target manipulation to `scene_viewer` - Separate the animation code in `scene_viewer` from the rest of the code, reducing `#[cfg(feature)]` noise - Add the `morph_targets.rs` example to show off how to manipulate morph targets, loading `MorpStressTest.gltf` ## Migration Guide - (very specialized, unlikely to be touched by 3rd parties) - `MeshPipeline` now has a single `mesh_layouts` field rather than separate `mesh_layout` and `skinned_mesh_layout` fields. You should handle all possible mesh bind group layouts in your implementation - You should also handle properly the new `MORPH_TARGETS` shader def and mesh pipeline key. A new function is exposed to make this easier: `setup_moprh_and_skinning_defs` - The `MeshBindGroup` is now `MeshBindGroups`, cached bind groups are now accessed through the `get` method. [1]: https://en.wikipedia.org/wiki/Morph_target_animation [2]: https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#morph-targets --------- Co-authored-by: François <mockersf@gmail.com> Co-authored-by: Carter Anderson <mcanders1@gmail.com>
2023-06-22 20:00:01 +00:00
mod morph_viewer_plugin;
mod scene_viewer_plugin;
use camera_controller_plugin::{CameraController, CameraControllerPlugin};
Add morph targets (#8158) # Objective - Add morph targets to `bevy_pbr` (closes #5756) & load them from glTF - Supersedes #3722 - Fixes #6814 [Morph targets][1] (also known as shape interpolation, shape keys, or blend shapes) allow animating individual vertices with fine grained controls. This is typically used for facial expressions. By specifying multiple poses as vertex offset, and providing a set of weight of each pose, it is possible to define surprisingly realistic transitions between poses. Blending between multiple poses also allow composition. Morph targets are part of the [gltf standard][2] and are a feature of Unity and Unreal, and babylone.js, it is only natural to implement them in bevy. ## Solution This implementation of morph targets uses a 3d texture where each pixel is a component of an animated attribute. Each layer is a different target. We use a 2d texture for each target, because the number of attribute×components×animated vertices is expected to always exceed the maximum pixel row size limit of webGL2. It copies fairly closely the way skinning is implemented on the CPU side, while on the GPU side, the shader morph target implementation is a relatively trivial detail. We add an optional `morph_texture` to the `Mesh` struct. The `morph_texture` is built through a method that accepts an iterator over attribute buffers. The `MorphWeights` component, user-accessible, controls the blend of poses used by mesh instances (so that multiple copy of the same mesh may have different weights), all the weights are uploaded to a uniform buffer of 256 `f32`. We limit to 16 poses per mesh, and a total of 256 poses. More literature: * Old babylone.js implementation (vertex attribute-based): https://www.eternalcoding.com/dev-log-1-morph-targets/ * Babylone.js implementation (similar to ours): https://www.youtube.com/watch?v=LBPRmGgU0PE * GPU gems 3: https://developer.nvidia.com/gpugems/gpugems3/part-i-geometry/chapter-3-directx-10-blend-shapes-breaking-limits * Development discord thread https://discord.com/channels/691052431525675048/1083325980615114772 https://user-images.githubusercontent.com/26321040/231181046-3bca2ab2-d4d9-472e-8098-639f1871ce2e.mp4 https://github.com/bevyengine/bevy/assets/26321040/d2a0c544-0ef8-45cf-9f99-8c3792f5a258 ## Acknowledgements * Thanks to `storytold` for sponsoring the feature * Thanks to `superdump` and `james7132` for guidance and help figuring out stuff ## Future work - Handling of less and more attributes (eg: animated uv, animated arbitrary attributes) - Dynamic pose allocation (so that zero-weighted poses aren't uploaded to GPU for example, enables much more total poses) - Better animation API, see #8357 ---- ## Changelog - Add morph targets to bevy meshes - Support up to 64 poses per mesh of individually up to 116508 vertices, animation currently strictly limited to the position, normal and tangent attributes. - Load a morph target using `Mesh::set_morph_targets` - Add `VisitMorphTargets` and `VisitMorphAttributes` traits to `bevy_render`, this allows defining morph targets (a fairly complex and nested data structure) through iterators (ie: single copy instead of passing around buffers), see documentation of those traits for details - Add `MorphWeights` component exported by `bevy_render` - `MorphWeights` control mesh's morph target weights, blending between various poses defined as morph targets. - `MorphWeights` are directly inherited by direct children (single level of hierarchy) of an entity. This allows controlling several mesh primitives through a unique entity _as per GLTF spec_. - Add `MorphTargetNames` component, naming each indices of loaded morph targets. - Load morph targets weights and buffers in `bevy_gltf` - handle morph targets animations in `bevy_animation` (previously, it was a `warn!` log) - Add the `MorphStressTest.gltf` asset for morph targets testing, taken from the glTF samples repo, CC0. - Add morph target manipulation to `scene_viewer` - Separate the animation code in `scene_viewer` from the rest of the code, reducing `#[cfg(feature)]` noise - Add the `morph_targets.rs` example to show off how to manipulate morph targets, loading `MorpStressTest.gltf` ## Migration Guide - (very specialized, unlikely to be touched by 3rd parties) - `MeshPipeline` now has a single `mesh_layouts` field rather than separate `mesh_layout` and `skinned_mesh_layout` fields. You should handle all possible mesh bind group layouts in your implementation - You should also handle properly the new `MORPH_TARGETS` shader def and mesh pipeline key. A new function is exposed to make this easier: `setup_moprh_and_skinning_defs` - The `MeshBindGroup` is now `MeshBindGroups`, cached bind groups are now accessed through the `get` method. [1]: https://en.wikipedia.org/wiki/Morph_target_animation [2]: https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#morph-targets --------- Co-authored-by: François <mockersf@gmail.com> Co-authored-by: Carter Anderson <mcanders1@gmail.com>
2023-06-22 20:00:01 +00:00
use morph_viewer_plugin::MorphViewerPlugin;
use scene_viewer_plugin::{SceneHandle, SceneViewerPlugin};
fn main() {
let mut app = App::new();
app.insert_resource(AmbientLight {
color: Color::WHITE,
brightness: 1.0 / 5.0f32,
})
.add_plugins((
DefaultPlugins
.set(WindowPlugin {
Windows as Entities (#5589) # Objective Fix https://github.com/bevyengine/bevy/issues/4530 - Make it easier to open/close/modify windows by setting them up as `Entity`s with a `Window` component. - Make multiple windows very simple to set up. (just add a `Window` component to an entity and it should open) ## Solution - Move all properties of window descriptor to ~components~ a component. - Replace `WindowId` with `Entity`. - ~Use change detection for components to update backend rather than events/commands. (The `CursorMoved`/`WindowResized`/... events are kept for user convenience.~ Check each field individually to see what we need to update, events are still kept for user convenience. --- ## Changelog - `WindowDescriptor` renamed to `Window`. - Width/height consolidated into a `WindowResolution` component. - Requesting maximization/minimization is done on the [`Window::state`] field. - `WindowId` is now `Entity`. ## Migration Guide - Replace `WindowDescriptor` with `Window`. - Change `width` and `height` fields in a `WindowResolution`, either by doing ```rust WindowResolution::new(width, height) // Explicitly // or using From<_> for tuples for convenience (1920., 1080.).into() ``` - Replace any `WindowCommand` code to just modify the `Window`'s fields directly and creating/closing windows is now by spawning/despawning an entity with a `Window` component like so: ```rust let window = commands.spawn(Window { ... }).id(); // open window commands.entity(window).despawn(); // close window ``` ## Unresolved - ~How do we tell when a window is minimized by a user?~ ~Currently using the `Resize(0, 0)` as an indicator of minimization.~ No longer attempting to tell given how finnicky this was across platforms, now the user can only request that a window be maximized/minimized. ## Future work - Move `exit_on_close` functionality out from windowing and into app(?) - https://github.com/bevyengine/bevy/issues/5621 - https://github.com/bevyengine/bevy/issues/7099 - https://github.com/bevyengine/bevy/issues/7098 Co-authored-by: Carter Anderson <mcanders1@gmail.com>
2023-01-19 00:38:28 +00:00
primary_window: Some(Window {
title: "bevy scene viewer".to_string(),
..default()
Windows as Entities (#5589) # Objective Fix https://github.com/bevyengine/bevy/issues/4530 - Make it easier to open/close/modify windows by setting them up as `Entity`s with a `Window` component. - Make multiple windows very simple to set up. (just add a `Window` component to an entity and it should open) ## Solution - Move all properties of window descriptor to ~components~ a component. - Replace `WindowId` with `Entity`. - ~Use change detection for components to update backend rather than events/commands. (The `CursorMoved`/`WindowResized`/... events are kept for user convenience.~ Check each field individually to see what we need to update, events are still kept for user convenience. --- ## Changelog - `WindowDescriptor` renamed to `Window`. - Width/height consolidated into a `WindowResolution` component. - Requesting maximization/minimization is done on the [`Window::state`] field. - `WindowId` is now `Entity`. ## Migration Guide - Replace `WindowDescriptor` with `Window`. - Change `width` and `height` fields in a `WindowResolution`, either by doing ```rust WindowResolution::new(width, height) // Explicitly // or using From<_> for tuples for convenience (1920., 1080.).into() ``` - Replace any `WindowCommand` code to just modify the `Window`'s fields directly and creating/closing windows is now by spawning/despawning an entity with a `Window` component like so: ```rust let window = commands.spawn(Window { ... }).id(); // open window commands.entity(window).despawn(); // close window ``` ## Unresolved - ~How do we tell when a window is minimized by a user?~ ~Currently using the `Resize(0, 0)` as an indicator of minimization.~ No longer attempting to tell given how finnicky this was across platforms, now the user can only request that a window be maximized/minimized. ## Future work - Move `exit_on_close` functionality out from windowing and into app(?) - https://github.com/bevyengine/bevy/issues/5621 - https://github.com/bevyengine/bevy/issues/7099 - https://github.com/bevyengine/bevy/issues/7098 Co-authored-by: Carter Anderson <mcanders1@gmail.com>
2023-01-19 00:38:28 +00:00
}),
..default()
})
.set(AssetPlugin {
asset_folder: std::env::var("CARGO_MANIFEST_DIR")
.unwrap_or_else(|_| ".".to_string()),
watch_for_changes: ChangeWatcher::with_delay(Duration::from_millis(200)),
}),
CameraControllerPlugin,
SceneViewerPlugin,
Add morph targets (#8158) # Objective - Add morph targets to `bevy_pbr` (closes #5756) & load them from glTF - Supersedes #3722 - Fixes #6814 [Morph targets][1] (also known as shape interpolation, shape keys, or blend shapes) allow animating individual vertices with fine grained controls. This is typically used for facial expressions. By specifying multiple poses as vertex offset, and providing a set of weight of each pose, it is possible to define surprisingly realistic transitions between poses. Blending between multiple poses also allow composition. Morph targets are part of the [gltf standard][2] and are a feature of Unity and Unreal, and babylone.js, it is only natural to implement them in bevy. ## Solution This implementation of morph targets uses a 3d texture where each pixel is a component of an animated attribute. Each layer is a different target. We use a 2d texture for each target, because the number of attribute×components×animated vertices is expected to always exceed the maximum pixel row size limit of webGL2. It copies fairly closely the way skinning is implemented on the CPU side, while on the GPU side, the shader morph target implementation is a relatively trivial detail. We add an optional `morph_texture` to the `Mesh` struct. The `morph_texture` is built through a method that accepts an iterator over attribute buffers. The `MorphWeights` component, user-accessible, controls the blend of poses used by mesh instances (so that multiple copy of the same mesh may have different weights), all the weights are uploaded to a uniform buffer of 256 `f32`. We limit to 16 poses per mesh, and a total of 256 poses. More literature: * Old babylone.js implementation (vertex attribute-based): https://www.eternalcoding.com/dev-log-1-morph-targets/ * Babylone.js implementation (similar to ours): https://www.youtube.com/watch?v=LBPRmGgU0PE * GPU gems 3: https://developer.nvidia.com/gpugems/gpugems3/part-i-geometry/chapter-3-directx-10-blend-shapes-breaking-limits * Development discord thread https://discord.com/channels/691052431525675048/1083325980615114772 https://user-images.githubusercontent.com/26321040/231181046-3bca2ab2-d4d9-472e-8098-639f1871ce2e.mp4 https://github.com/bevyengine/bevy/assets/26321040/d2a0c544-0ef8-45cf-9f99-8c3792f5a258 ## Acknowledgements * Thanks to `storytold` for sponsoring the feature * Thanks to `superdump` and `james7132` for guidance and help figuring out stuff ## Future work - Handling of less and more attributes (eg: animated uv, animated arbitrary attributes) - Dynamic pose allocation (so that zero-weighted poses aren't uploaded to GPU for example, enables much more total poses) - Better animation API, see #8357 ---- ## Changelog - Add morph targets to bevy meshes - Support up to 64 poses per mesh of individually up to 116508 vertices, animation currently strictly limited to the position, normal and tangent attributes. - Load a morph target using `Mesh::set_morph_targets` - Add `VisitMorphTargets` and `VisitMorphAttributes` traits to `bevy_render`, this allows defining morph targets (a fairly complex and nested data structure) through iterators (ie: single copy instead of passing around buffers), see documentation of those traits for details - Add `MorphWeights` component exported by `bevy_render` - `MorphWeights` control mesh's morph target weights, blending between various poses defined as morph targets. - `MorphWeights` are directly inherited by direct children (single level of hierarchy) of an entity. This allows controlling several mesh primitives through a unique entity _as per GLTF spec_. - Add `MorphTargetNames` component, naming each indices of loaded morph targets. - Load morph targets weights and buffers in `bevy_gltf` - handle morph targets animations in `bevy_animation` (previously, it was a `warn!` log) - Add the `MorphStressTest.gltf` asset for morph targets testing, taken from the glTF samples repo, CC0. - Add morph target manipulation to `scene_viewer` - Separate the animation code in `scene_viewer` from the rest of the code, reducing `#[cfg(feature)]` noise - Add the `morph_targets.rs` example to show off how to manipulate morph targets, loading `MorpStressTest.gltf` ## Migration Guide - (very specialized, unlikely to be touched by 3rd parties) - `MeshPipeline` now has a single `mesh_layouts` field rather than separate `mesh_layout` and `skinned_mesh_layout` fields. You should handle all possible mesh bind group layouts in your implementation - You should also handle properly the new `MORPH_TARGETS` shader def and mesh pipeline key. A new function is exposed to make this easier: `setup_moprh_and_skinning_defs` - The `MeshBindGroup` is now `MeshBindGroups`, cached bind groups are now accessed through the `get` method. [1]: https://en.wikipedia.org/wiki/Morph_target_animation [2]: https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#morph-targets --------- Co-authored-by: François <mockersf@gmail.com> Co-authored-by: Carter Anderson <mcanders1@gmail.com>
2023-06-22 20:00:01 +00:00
MorphViewerPlugin,
))
.add_systems(Startup, setup)
.add_systems(PreUpdate, setup_scene_after_load);
Add morph targets (#8158) # Objective - Add morph targets to `bevy_pbr` (closes #5756) & load them from glTF - Supersedes #3722 - Fixes #6814 [Morph targets][1] (also known as shape interpolation, shape keys, or blend shapes) allow animating individual vertices with fine grained controls. This is typically used for facial expressions. By specifying multiple poses as vertex offset, and providing a set of weight of each pose, it is possible to define surprisingly realistic transitions between poses. Blending between multiple poses also allow composition. Morph targets are part of the [gltf standard][2] and are a feature of Unity and Unreal, and babylone.js, it is only natural to implement them in bevy. ## Solution This implementation of morph targets uses a 3d texture where each pixel is a component of an animated attribute. Each layer is a different target. We use a 2d texture for each target, because the number of attribute×components×animated vertices is expected to always exceed the maximum pixel row size limit of webGL2. It copies fairly closely the way skinning is implemented on the CPU side, while on the GPU side, the shader morph target implementation is a relatively trivial detail. We add an optional `morph_texture` to the `Mesh` struct. The `morph_texture` is built through a method that accepts an iterator over attribute buffers. The `MorphWeights` component, user-accessible, controls the blend of poses used by mesh instances (so that multiple copy of the same mesh may have different weights), all the weights are uploaded to a uniform buffer of 256 `f32`. We limit to 16 poses per mesh, and a total of 256 poses. More literature: * Old babylone.js implementation (vertex attribute-based): https://www.eternalcoding.com/dev-log-1-morph-targets/ * Babylone.js implementation (similar to ours): https://www.youtube.com/watch?v=LBPRmGgU0PE * GPU gems 3: https://developer.nvidia.com/gpugems/gpugems3/part-i-geometry/chapter-3-directx-10-blend-shapes-breaking-limits * Development discord thread https://discord.com/channels/691052431525675048/1083325980615114772 https://user-images.githubusercontent.com/26321040/231181046-3bca2ab2-d4d9-472e-8098-639f1871ce2e.mp4 https://github.com/bevyengine/bevy/assets/26321040/d2a0c544-0ef8-45cf-9f99-8c3792f5a258 ## Acknowledgements * Thanks to `storytold` for sponsoring the feature * Thanks to `superdump` and `james7132` for guidance and help figuring out stuff ## Future work - Handling of less and more attributes (eg: animated uv, animated arbitrary attributes) - Dynamic pose allocation (so that zero-weighted poses aren't uploaded to GPU for example, enables much more total poses) - Better animation API, see #8357 ---- ## Changelog - Add morph targets to bevy meshes - Support up to 64 poses per mesh of individually up to 116508 vertices, animation currently strictly limited to the position, normal and tangent attributes. - Load a morph target using `Mesh::set_morph_targets` - Add `VisitMorphTargets` and `VisitMorphAttributes` traits to `bevy_render`, this allows defining morph targets (a fairly complex and nested data structure) through iterators (ie: single copy instead of passing around buffers), see documentation of those traits for details - Add `MorphWeights` component exported by `bevy_render` - `MorphWeights` control mesh's morph target weights, blending between various poses defined as morph targets. - `MorphWeights` are directly inherited by direct children (single level of hierarchy) of an entity. This allows controlling several mesh primitives through a unique entity _as per GLTF spec_. - Add `MorphTargetNames` component, naming each indices of loaded morph targets. - Load morph targets weights and buffers in `bevy_gltf` - handle morph targets animations in `bevy_animation` (previously, it was a `warn!` log) - Add the `MorphStressTest.gltf` asset for morph targets testing, taken from the glTF samples repo, CC0. - Add morph target manipulation to `scene_viewer` - Separate the animation code in `scene_viewer` from the rest of the code, reducing `#[cfg(feature)]` noise - Add the `morph_targets.rs` example to show off how to manipulate morph targets, loading `MorpStressTest.gltf` ## Migration Guide - (very specialized, unlikely to be touched by 3rd parties) - `MeshPipeline` now has a single `mesh_layouts` field rather than separate `mesh_layout` and `skinned_mesh_layout` fields. You should handle all possible mesh bind group layouts in your implementation - You should also handle properly the new `MORPH_TARGETS` shader def and mesh pipeline key. A new function is exposed to make this easier: `setup_moprh_and_skinning_defs` - The `MeshBindGroup` is now `MeshBindGroups`, cached bind groups are now accessed through the `get` method. [1]: https://en.wikipedia.org/wiki/Morph_target_animation [2]: https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#morph-targets --------- Co-authored-by: François <mockersf@gmail.com> Co-authored-by: Carter Anderson <mcanders1@gmail.com>
2023-06-22 20:00:01 +00:00
#[cfg(feature = "animation")]
app.add_plugins(animation_plugin::AnimationManipulationPlugin);
app.run();
}
fn parse_scene(scene_path: String) -> (String, usize) {
if scene_path.contains('#') {
let gltf_and_scene = scene_path.split('#').collect::<Vec<_>>();
if let Some((last, path)) = gltf_and_scene.split_last() {
if let Some(index) = last
.strip_prefix("Scene")
.and_then(|index| index.parse::<usize>().ok())
{
return (path.join("#"), index);
}
}
}
(scene_path, 0)
}
fn setup(mut commands: Commands, asset_server: Res<AssetServer>) {
let scene_path = std::env::args()
.nth(1)
.unwrap_or_else(|| "assets/models/FlightHelmet/FlightHelmet.gltf".to_string());
info!("Loading {}", scene_path);
let (file_path, scene_index) = parse_scene(scene_path);
commands.insert_resource(SceneHandle::new(asset_server.load(file_path), scene_index));
}
fn setup_scene_after_load(
mut commands: Commands,
mut setup: Local<bool>,
mut scene_handle: ResMut<SceneHandle>,
EnvironmentMapLight, BRDF Improvements (#7051) (Before) ![image](https://user-images.githubusercontent.com/47158642/213946111-15ec758f-1f1d-443c-b196-1fdcd4ae49da.png) (After) ![image](https://user-images.githubusercontent.com/47158642/217051179-67381e73-dd44-461b-a2c7-87b0440ef8de.png) ![image](https://user-images.githubusercontent.com/47158642/212492404-524e4ad3-7837-4ed4-8b20-2abc276aa8e8.png) # Objective - Improve lighting; especially reflections. - Closes https://github.com/bevyengine/bevy/issues/4581. ## Solution - Implement environment maps, providing better ambient light. - Add microfacet multibounce approximation for specular highlights from Filament. - Occlusion is no longer incorrectly applied to direct lighting. It now only applies to diffuse indirect light. Unsure if it's also supposed to apply to specular indirect light - the glTF specification just says "indirect light". In the case of ambient occlusion, for instance, that's usually only calculated as diffuse though. For now, I'm choosing to apply this just to indirect diffuse light, and not specular. - Modified the PBR example to use an environment map, and have labels. - Added `FallbackImageCubemap`. ## Implementation - IBL technique references can be found in environment_map.wgsl. - It's more accurate to use a LUT for the scale/bias. Filament has a good reference on generating this LUT. For now, I just used an analytic approximation. - For now, environment maps must first be prefiltered outside of bevy using a 3rd party tool. See the `EnvironmentMap` documentation. - Eventually, we should have our own prefiltering code, so that we can have dynamically changing environment maps, as well as let users drop in an HDR image and use asset preprocessing to create the needed textures using only bevy. --- ## Changelog - Added an `EnvironmentMapLight` camera component that adds additional ambient light to a scene. - StandardMaterials will now appear brighter and more saturated at high roughness, due to internal material changes. This is more physically correct. - Fixed StandardMaterial occlusion being incorrectly applied to direct lighting. - Added `FallbackImageCubemap`. Co-authored-by: IceSentry <c.giguere42@gmail.com> Co-authored-by: James Liu <contact@jamessliu.com> Co-authored-by: Rob Parrett <robparrett@gmail.com>
2023-02-09 16:46:32 +00:00
asset_server: Res<AssetServer>,
meshes: Query<(&GlobalTransform, Option<&Aabb>), With<Handle<Mesh>>>,
) {
if scene_handle.is_loaded && !*setup {
*setup = true;
// Find an approximate bounding box of the scene from its meshes
if meshes.iter().any(|(_, maybe_aabb)| maybe_aabb.is_none()) {
return;
}
let mut min = Vec3A::splat(f32::MAX);
let mut max = Vec3A::splat(f32::MIN);
for (transform, maybe_aabb) in &meshes {
let aabb = maybe_aabb.unwrap();
// If the Aabb had not been rotated, applying the non-uniform scale would produce the
// correct bounds. However, it could very well be rotated and so we first convert to
// a Sphere, and then back to an Aabb to find the conservative min and max points.
let sphere = Sphere {
center: Vec3A::from(transform.transform_point(Vec3::from(aabb.center))),
radius: transform.radius_vec3a(aabb.half_extents),
};
let aabb = Aabb::from(sphere);
min = min.min(aabb.min());
max = max.max(aabb.max());
}
let size = (max - min).length();
let aabb = Aabb::from_min_max(Vec3::from(min), Vec3::from(max));
info!("Spawning a controllable 3D perspective camera");
let mut projection = PerspectiveProjection::default();
projection.far = projection.far.max(size * 10.0);
let camera_controller = CameraController::default();
// Display the controls of the scene viewer
info!("{}", camera_controller);
info!("{}", *scene_handle);
commands.spawn((
Camera3dBundle {
projection: projection.into(),
transform: Transform::from_translation(
Vec3::from(aabb.center) + size * Vec3::new(0.5, 0.25, 0.5),
)
.looking_at(Vec3::from(aabb.center), Vec3::Y),
camera: Camera {
is_active: false,
..default()
},
..default()
},
EnvironmentMapLight, BRDF Improvements (#7051) (Before) ![image](https://user-images.githubusercontent.com/47158642/213946111-15ec758f-1f1d-443c-b196-1fdcd4ae49da.png) (After) ![image](https://user-images.githubusercontent.com/47158642/217051179-67381e73-dd44-461b-a2c7-87b0440ef8de.png) ![image](https://user-images.githubusercontent.com/47158642/212492404-524e4ad3-7837-4ed4-8b20-2abc276aa8e8.png) # Objective - Improve lighting; especially reflections. - Closes https://github.com/bevyengine/bevy/issues/4581. ## Solution - Implement environment maps, providing better ambient light. - Add microfacet multibounce approximation for specular highlights from Filament. - Occlusion is no longer incorrectly applied to direct lighting. It now only applies to diffuse indirect light. Unsure if it's also supposed to apply to specular indirect light - the glTF specification just says "indirect light". In the case of ambient occlusion, for instance, that's usually only calculated as diffuse though. For now, I'm choosing to apply this just to indirect diffuse light, and not specular. - Modified the PBR example to use an environment map, and have labels. - Added `FallbackImageCubemap`. ## Implementation - IBL technique references can be found in environment_map.wgsl. - It's more accurate to use a LUT for the scale/bias. Filament has a good reference on generating this LUT. For now, I just used an analytic approximation. - For now, environment maps must first be prefiltered outside of bevy using a 3rd party tool. See the `EnvironmentMap` documentation. - Eventually, we should have our own prefiltering code, so that we can have dynamically changing environment maps, as well as let users drop in an HDR image and use asset preprocessing to create the needed textures using only bevy. --- ## Changelog - Added an `EnvironmentMapLight` camera component that adds additional ambient light to a scene. - StandardMaterials will now appear brighter and more saturated at high roughness, due to internal material changes. This is more physically correct. - Fixed StandardMaterial occlusion being incorrectly applied to direct lighting. - Added `FallbackImageCubemap`. Co-authored-by: IceSentry <c.giguere42@gmail.com> Co-authored-by: James Liu <contact@jamessliu.com> Co-authored-by: Rob Parrett <robparrett@gmail.com>
2023-02-09 16:46:32 +00:00
EnvironmentMapLight {
diffuse_map: asset_server
.load("assets/environment_maps/pisa_diffuse_rgb9e5_zstd.ktx2"),
specular_map: asset_server
.load("assets/environment_maps/pisa_specular_rgb9e5_zstd.ktx2"),
},
camera_controller,
));
// Spawn a default light if the scene does not have one
if !scene_handle.has_light {
info!("Spawning a directional light");
commands.spawn(DirectionalLightBundle {
directional_light: DirectionalLight {
shadows_enabled: false,
..default()
},
..default()
});
scene_handle.has_light = true;
}
}
}