mirror of
https://github.com/bevyengine/bevy
synced 2024-12-23 03:23:20 +00:00
c6170d48f9
# Objective - Add morph targets to `bevy_pbr` (closes #5756) & load them from glTF - Supersedes #3722 - Fixes #6814 [Morph targets][1] (also known as shape interpolation, shape keys, or blend shapes) allow animating individual vertices with fine grained controls. This is typically used for facial expressions. By specifying multiple poses as vertex offset, and providing a set of weight of each pose, it is possible to define surprisingly realistic transitions between poses. Blending between multiple poses also allow composition. Morph targets are part of the [gltf standard][2] and are a feature of Unity and Unreal, and babylone.js, it is only natural to implement them in bevy. ## Solution This implementation of morph targets uses a 3d texture where each pixel is a component of an animated attribute. Each layer is a different target. We use a 2d texture for each target, because the number of attribute×components×animated vertices is expected to always exceed the maximum pixel row size limit of webGL2. It copies fairly closely the way skinning is implemented on the CPU side, while on the GPU side, the shader morph target implementation is a relatively trivial detail. We add an optional `morph_texture` to the `Mesh` struct. The `morph_texture` is built through a method that accepts an iterator over attribute buffers. The `MorphWeights` component, user-accessible, controls the blend of poses used by mesh instances (so that multiple copy of the same mesh may have different weights), all the weights are uploaded to a uniform buffer of 256 `f32`. We limit to 16 poses per mesh, and a total of 256 poses. More literature: * Old babylone.js implementation (vertex attribute-based): https://www.eternalcoding.com/dev-log-1-morph-targets/ * Babylone.js implementation (similar to ours): https://www.youtube.com/watch?v=LBPRmGgU0PE * GPU gems 3: https://developer.nvidia.com/gpugems/gpugems3/part-i-geometry/chapter-3-directx-10-blend-shapes-breaking-limits * Development discord thread https://discord.com/channels/691052431525675048/1083325980615114772 https://user-images.githubusercontent.com/26321040/231181046-3bca2ab2-d4d9-472e-8098-639f1871ce2e.mp4 https://github.com/bevyengine/bevy/assets/26321040/d2a0c544-0ef8-45cf-9f99-8c3792f5a258 ## Acknowledgements * Thanks to `storytold` for sponsoring the feature * Thanks to `superdump` and `james7132` for guidance and help figuring out stuff ## Future work - Handling of less and more attributes (eg: animated uv, animated arbitrary attributes) - Dynamic pose allocation (so that zero-weighted poses aren't uploaded to GPU for example, enables much more total poses) - Better animation API, see #8357 ---- ## Changelog - Add morph targets to bevy meshes - Support up to 64 poses per mesh of individually up to 116508 vertices, animation currently strictly limited to the position, normal and tangent attributes. - Load a morph target using `Mesh::set_morph_targets` - Add `VisitMorphTargets` and `VisitMorphAttributes` traits to `bevy_render`, this allows defining morph targets (a fairly complex and nested data structure) through iterators (ie: single copy instead of passing around buffers), see documentation of those traits for details - Add `MorphWeights` component exported by `bevy_render` - `MorphWeights` control mesh's morph target weights, blending between various poses defined as morph targets. - `MorphWeights` are directly inherited by direct children (single level of hierarchy) of an entity. This allows controlling several mesh primitives through a unique entity _as per GLTF spec_. - Add `MorphTargetNames` component, naming each indices of loaded morph targets. - Load morph targets weights and buffers in `bevy_gltf` - handle morph targets animations in `bevy_animation` (previously, it was a `warn!` log) - Add the `MorphStressTest.gltf` asset for morph targets testing, taken from the glTF samples repo, CC0. - Add morph target manipulation to `scene_viewer` - Separate the animation code in `scene_viewer` from the rest of the code, reducing `#[cfg(feature)]` noise - Add the `morph_targets.rs` example to show off how to manipulate morph targets, loading `MorpStressTest.gltf` ## Migration Guide - (very specialized, unlikely to be touched by 3rd parties) - `MeshPipeline` now has a single `mesh_layouts` field rather than separate `mesh_layout` and `skinned_mesh_layout` fields. You should handle all possible mesh bind group layouts in your implementation - You should also handle properly the new `MORPH_TARGETS` shader def and mesh pipeline key. A new function is exposed to make this easier: `setup_moprh_and_skinning_defs` - The `MeshBindGroup` is now `MeshBindGroups`, cached bind groups are now accessed through the `get` method. [1]: https://en.wikipedia.org/wiki/Morph_target_animation [2]: https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#morph-targets --------- Co-authored-by: François <mockersf@gmail.com> Co-authored-by: Carter Anderson <mcanders1@gmail.com>
233 lines
7.2 KiB
Rust
233 lines
7.2 KiB
Rust
//! A glTF scene viewer plugin. Provides controls for directional lighting, and switching between scene cameras.
|
|
//! To use in your own application:
|
|
//! - Copy the code for the `SceneViewerPlugin` and add the plugin to your App.
|
|
//! - Insert an initialized `SceneHandle` resource into your App's `AssetServer`.
|
|
|
|
use bevy::{
|
|
asset::LoadState, gltf::Gltf, input::common_conditions::input_just_pressed, prelude::*,
|
|
scene::InstanceId,
|
|
};
|
|
|
|
use std::f32::consts::*;
|
|
use std::fmt;
|
|
|
|
use super::camera_controller_plugin::*;
|
|
|
|
#[derive(Resource)]
|
|
pub struct SceneHandle {
|
|
pub gltf_handle: Handle<Gltf>,
|
|
scene_index: usize,
|
|
instance_id: Option<InstanceId>,
|
|
pub is_loaded: bool,
|
|
pub has_light: bool,
|
|
}
|
|
|
|
impl SceneHandle {
|
|
pub fn new(gltf_handle: Handle<Gltf>, scene_index: usize) -> Self {
|
|
Self {
|
|
gltf_handle,
|
|
scene_index,
|
|
instance_id: None,
|
|
is_loaded: false,
|
|
has_light: false,
|
|
}
|
|
}
|
|
}
|
|
|
|
#[cfg(not(feature = "animation"))]
|
|
const INSTRUCTIONS: &str = r#"
|
|
Scene Controls:
|
|
L - animate light direction
|
|
U - toggle shadows
|
|
C - cycle through the camera controller and any cameras loaded from the scene
|
|
|
|
compile with "--features animation" for animation controls.
|
|
"#;
|
|
|
|
#[cfg(feature = "animation")]
|
|
const INSTRUCTIONS: &str = "
|
|
Scene Controls:
|
|
L - animate light direction
|
|
U - toggle shadows
|
|
B - toggle bounding boxes
|
|
C - cycle through the camera controller and any cameras loaded from the scene
|
|
|
|
Space - Play/Pause animation
|
|
Enter - Cycle through animations
|
|
";
|
|
|
|
impl fmt::Display for SceneHandle {
|
|
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
|
write!(f, "{INSTRUCTIONS}")
|
|
}
|
|
}
|
|
|
|
pub struct SceneViewerPlugin;
|
|
|
|
impl Plugin for SceneViewerPlugin {
|
|
fn build(&self, app: &mut App) {
|
|
app.init_resource::<CameraTracker>()
|
|
.add_systems(PreUpdate, scene_load_check)
|
|
.add_systems(
|
|
Update,
|
|
(
|
|
update_lights,
|
|
camera_tracker,
|
|
toggle_bounding_boxes.run_if(input_just_pressed(KeyCode::B)),
|
|
),
|
|
);
|
|
}
|
|
}
|
|
|
|
fn toggle_bounding_boxes(mut config: ResMut<GizmoConfig>) {
|
|
config.aabb.draw_all ^= true;
|
|
}
|
|
|
|
fn scene_load_check(
|
|
asset_server: Res<AssetServer>,
|
|
mut scenes: ResMut<Assets<Scene>>,
|
|
gltf_assets: Res<Assets<Gltf>>,
|
|
mut scene_handle: ResMut<SceneHandle>,
|
|
mut scene_spawner: ResMut<SceneSpawner>,
|
|
) {
|
|
match scene_handle.instance_id {
|
|
None => {
|
|
if asset_server.get_load_state(&scene_handle.gltf_handle) == LoadState::Loaded {
|
|
let gltf = gltf_assets.get(&scene_handle.gltf_handle).unwrap();
|
|
if gltf.scenes.len() > 1 {
|
|
info!(
|
|
"Displaying scene {} out of {}",
|
|
scene_handle.scene_index,
|
|
gltf.scenes.len()
|
|
);
|
|
info!("You can select the scene by adding '#Scene' followed by a number to the end of the file path (e.g '#Scene1' to load the second scene).");
|
|
}
|
|
|
|
let gltf_scene_handle =
|
|
gltf.scenes
|
|
.get(scene_handle.scene_index)
|
|
.unwrap_or_else(|| {
|
|
panic!(
|
|
"glTF file doesn't contain scene {}!",
|
|
scene_handle.scene_index
|
|
)
|
|
});
|
|
let scene = scenes.get_mut(gltf_scene_handle).unwrap();
|
|
|
|
let mut query = scene
|
|
.world
|
|
.query::<(Option<&DirectionalLight>, Option<&PointLight>)>();
|
|
scene_handle.has_light =
|
|
query
|
|
.iter(&scene.world)
|
|
.any(|(maybe_directional_light, maybe_point_light)| {
|
|
maybe_directional_light.is_some() || maybe_point_light.is_some()
|
|
});
|
|
|
|
scene_handle.instance_id =
|
|
Some(scene_spawner.spawn(gltf_scene_handle.clone_weak()));
|
|
|
|
info!("Spawning scene...");
|
|
}
|
|
}
|
|
Some(instance_id) if !scene_handle.is_loaded => {
|
|
if scene_spawner.instance_is_ready(instance_id) {
|
|
info!("...done!");
|
|
scene_handle.is_loaded = true;
|
|
}
|
|
}
|
|
Some(_) => {}
|
|
}
|
|
}
|
|
fn update_lights(
|
|
key_input: Res<Input<KeyCode>>,
|
|
time: Res<Time>,
|
|
mut query: Query<(&mut Transform, &mut DirectionalLight)>,
|
|
mut animate_directional_light: Local<bool>,
|
|
) {
|
|
for (_, mut light) in &mut query {
|
|
if key_input.just_pressed(KeyCode::U) {
|
|
light.shadows_enabled = !light.shadows_enabled;
|
|
}
|
|
}
|
|
|
|
if key_input.just_pressed(KeyCode::L) {
|
|
*animate_directional_light = !*animate_directional_light;
|
|
}
|
|
if *animate_directional_light {
|
|
for (mut transform, _) in &mut query {
|
|
transform.rotation = Quat::from_euler(
|
|
EulerRot::ZYX,
|
|
0.0,
|
|
time.elapsed_seconds() * PI / 15.0,
|
|
-FRAC_PI_4,
|
|
);
|
|
}
|
|
}
|
|
}
|
|
|
|
#[derive(Resource, Default)]
|
|
struct CameraTracker {
|
|
active_index: Option<usize>,
|
|
cameras: Vec<Entity>,
|
|
}
|
|
|
|
impl CameraTracker {
|
|
fn track_camera(&mut self, entity: Entity) -> bool {
|
|
self.cameras.push(entity);
|
|
if self.active_index.is_none() {
|
|
self.active_index = Some(self.cameras.len() - 1);
|
|
true
|
|
} else {
|
|
false
|
|
}
|
|
}
|
|
|
|
fn active_camera(&self) -> Option<Entity> {
|
|
self.active_index.map(|i| self.cameras[i])
|
|
}
|
|
|
|
fn set_next_active(&mut self) -> Option<Entity> {
|
|
let active_index = self.active_index?;
|
|
let new_i = (active_index + 1) % self.cameras.len();
|
|
self.active_index = Some(new_i);
|
|
Some(self.cameras[new_i])
|
|
}
|
|
}
|
|
|
|
fn camera_tracker(
|
|
mut camera_tracker: ResMut<CameraTracker>,
|
|
keyboard_input: Res<Input<KeyCode>>,
|
|
mut queries: ParamSet<(
|
|
Query<(Entity, &mut Camera), (Added<Camera>, Without<CameraController>)>,
|
|
Query<(Entity, &mut Camera), (Added<Camera>, With<CameraController>)>,
|
|
Query<&mut Camera>,
|
|
)>,
|
|
) {
|
|
// track added scene camera entities first, to ensure they are preferred for the
|
|
// default active camera
|
|
for (entity, mut camera) in queries.p0().iter_mut() {
|
|
camera.is_active = camera_tracker.track_camera(entity);
|
|
}
|
|
|
|
// iterate added custom camera entities second
|
|
for (entity, mut camera) in queries.p1().iter_mut() {
|
|
camera.is_active = camera_tracker.track_camera(entity);
|
|
}
|
|
|
|
if keyboard_input.just_pressed(KeyCode::C) {
|
|
// disable currently active camera
|
|
if let Some(e) = camera_tracker.active_camera() {
|
|
if let Ok(mut camera) = queries.p2().get_mut(e) {
|
|
camera.is_active = false;
|
|
}
|
|
}
|
|
|
|
// enable next active camera
|
|
if let Some(e) = camera_tracker.set_next_active() {
|
|
if let Ok(mut camera) = queries.p2().get_mut(e) {
|
|
camera.is_active = true;
|
|
}
|
|
}
|
|
}
|
|
}
|