Mirrors/bevy
2
0
Fork 0
mirror of https://github.com/bevyengine/bevy synced 2024-11-10 07:04:33 +00:00
Code Issues Projects Releases Packages Wiki Activity

Implement minimal reflection probes (fixed macOS, iOS, and Android). (#11366)

Browse source 
This pull request re-submits #10057, which was backed out for breaking
macOS, iOS, and Android. I've tested this version on macOS and Android
and on the iOS simulator.

# Objective

This pull request implements *reflection probes*, which generalize
environment maps to allow for multiple environment maps in the same
scene, each of which has an axis-aligned bounding box. This is a
standard feature of physically-based renderers and was inspired by [the
corresponding feature in Blender's Eevee renderer].

## Solution

This is a minimal implementation of reflection probes that allows
artists to define cuboid bounding regions associated with environment
maps. For every view, on every frame, a system builds up a list of the
nearest 4 reflection probes that are within the view's frustum and
supplies that list to the shader. The PBR fragment shader searches
through the list, finds the first containing reflection probe, and uses
it for indirect lighting, falling back to the view's environment map if
none is found. Both forward and deferred renderers are fully supported.

A reflection probe is an entity with a pair of components, *LightProbe*
and *EnvironmentMapLight* (as well as the standard *SpatialBundle*, to
position it in the world). The *LightProbe* component (along with the
*Transform*) defines the bounding region, while the
*EnvironmentMapLight* component specifies the associated diffuse and
specular cubemaps.

A frequent question is "why two components instead of just one?" The
advantages of this setup are:

1. It's readily extensible to other types of light probes, in particular
*irradiance volumes* (also known as ambient cubes or voxel global
illumination), which use the same approach of bounding cuboids. With a
single component that applies to both reflection probes and irradiance
volumes, we can share the logic that implements falloff and blending
between multiple light probes between both of those features.

2. It reduces duplication between the existing *EnvironmentMapLight* and
these new reflection probes. Systems can treat environment maps attached
to cameras the same way they treat environment maps applied to
reflection probes if they wish.

Internally, we gather up all environment maps in the scene and place
them in a cubemap array. At present, this means that all environment
maps must have the same size, mipmap count, and texture format. A
warning is emitted if this restriction is violated. We could potentially
relax this in the future as part of the automatic mipmap generation
work, which could easily do texture format conversion as part of its
preprocessing.

An easy way to generate reflection probe cubemaps is to bake them in
Blender and use the `export-blender-gi` tool that's part of the
[`bevy-baked-gi`] project. This tool takes a `.blend` file containing
baked cubemaps as input and exports cubemap images, pre-filtered with an
embedded fork of the [glTF IBL Sampler], alongside a corresponding
`.scn.ron` file that the scene spawner can use to recreate the
reflection probes.

Note that this is intentionally a minimal implementation, to aid
reviewability. Known issues are:

* Reflection probes are basically unsupported on WebGL 2, because WebGL
2 has no cubemap arrays. (Strictly speaking, you can have precisely one
reflection probe in the scene if you have no other cubemaps anywhere,
but this isn't very useful.)

* Reflection probes have no falloff, so reflections will abruptly change
when objects move from one bounding region to another.

* As mentioned before, all cubemaps in the world of a given type
(diffuse or specular) must have the same size, format, and mipmap count.

Future work includes:

* Blending between multiple reflection probes.

* A falloff/fade-out region so that reflected objects disappear
gradually instead of vanishing all at once.

* Irradiance volumes for voxel-based global illumination. This should
reuse much of the reflection probe logic, as they're both GI techniques
based on cuboid bounding regions.

* Support for WebGL 2, by breaking batches when reflection probes are
used.

These issues notwithstanding, I think it's best to land this with
roughly the current set of functionality, because this patch is useful
as is and adding everything above would make the pull request
significantly larger and harder to review.

---

## Changelog

### Added

* A new *LightProbe* component is available that specifies a bounding
region that an *EnvironmentMapLight* applies to. The combination of a
*LightProbe* and an *EnvironmentMapLight* offers *reflection probe*
functionality similar to that available in other engines.

[the corresponding feature in Blender's Eevee renderer]:
https://docs.blender.org/manual/en/latest/render/eevee/light_probes/reflection_cubemaps.html

[`bevy-baked-gi`]: https://github.com/pcwalton/bevy-baked-gi

[glTF IBL Sampler]: https://github.com/KhronosGroup/glTF-IBL-Sampler
This commit is contained in:
Patrick Walton 2024-01-18 23:33:52 -08:00 committed by GitHub
parent f795656d65
commit 83d6600267
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
24 changed files with 1582 additions and 264 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
Cargo.toml
View file

@ -2508,6 +2508,17 @@ name = "fallback_image"
path = "examples/shader/fallback_image.rs"
doc-scrape-examples = true
[[example]]
name = "reflection_probes"
path = "examples/3d/reflection_probes.rs"
doc-scrape-examples = true
[package.metadata.example.reflection_probes]
name = "Reflection Probes"
description = "Demonstrates reflection probes"
category = "3D Rendering"
wasm = false
[package.metadata.example.fallback_image]
hidden = true

BIN
assets/environment_maps/cubes_reflection_probe_specular_rgb9e5_zstd.ktx2 Normal file
View file

Binary file not shown.

BIN
assets/models/cubes/Cubes.glb Normal file
View file

Binary file not shown.

4
crates/bevy_core_pipeline/src/tonemapping/tonemapping_shared.wgsl
View file

@ -7,8 +7,8 @@
@group(0) @binding(3) var dt_lut_texture: texture_3d<f32>;
@group(0) @binding(4) var dt_lut_sampler: sampler;
#else
@group(0) @binding(15) var dt_lut_texture: texture_3d<f32>;
@group(0) @binding(16) var dt_lut_sampler: sampler;
@group(0) @binding(16) var dt_lut_texture: texture_3d<f32>;
@group(0) @binding(17) var dt_lut_sampler: sampler;
#endif
fn sample_current_lut(p: vec3<f32>) -> vec3<f32> {

5
crates/bevy_internal/Cargo.toml
View file

@ -58,7 +58,10 @@ symphonia-vorbis = ["bevy_audio/symphonia-vorbis"]
symphonia-wav = ["bevy_audio/symphonia-wav"]
# Shader formats
shader_format_glsl = ["bevy_render/shader_format_glsl"]
shader_format_glsl = [
"bevy_render/shader_format_glsl",
"bevy_pbr?/shader_format_glsl",
]
shader_format_spirv = ["bevy_render/shader_format_spirv"]
serialize = [

12
crates/bevy_pbr/Cargo.toml
View file

@ -10,6 +10,7 @@ keywords = ["bevy"]
[features]
webgl = []
shader_format_glsl = ["naga_oil/glsl"]
pbr_transmission_textures = []
[dependencies]
@ -34,8 +35,17 @@ fixedbitset = "0.4"
# direct dependency required for derive macro
bytemuck = { version = "1", features = ["derive"] }
radsort = "0.1"
naga_oil = "0.11"
smallvec = "1.6"
thread_local = "1.0"
[target.'cfg(target_arch = "wasm32")'.dependencies]
naga_oil = { version = "0.11" }
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
# Omit the `glsl` feature in non-WebAssembly by default.
naga_oil = { version = "0.11", default-features = false, features = [
"test_shader",
] }
[lints]
workspace = true

44
crates/bevy_pbr/src/deferred/mod.rs
View file

@ -1,4 +1,7 @@
use crate::{MeshPipeline, MeshViewBindGroup, ScreenSpaceAmbientOcclusionSettings};
use crate::{
environment_map::RenderViewEnvironmentMaps, MeshPipeline, MeshViewBindGroup,
ScreenSpaceAmbientOcclusionSettings, ViewLightProbesUniformOffset,
};
use bevy_app::prelude::*;
use bevy_asset::{load_internal_asset, Handle};
use bevy_core_pipeline::{
@ -14,25 +17,17 @@ use bevy_render::{
extract_component::{
ComponentUniforms, ExtractComponent, ExtractComponentPlugin, UniformComponentPlugin,
},
render_asset::RenderAssets,
render_graph::{NodeRunError, RenderGraphContext, ViewNode, ViewNodeRunner},
render_resource::{
binding_types::uniform_buffer, Operations, PipelineCache, RenderPassDescriptor,
},
render_graph::{NodeRunError, RenderGraphApp, RenderGraphContext, ViewNode, ViewNodeRunner},
render_resource::binding_types::uniform_buffer,
render_resource::*,
renderer::{RenderContext, RenderDevice},
texture::Image,
view::{ViewTarget, ViewUniformOffset},
Render, RenderSet,
};
use bevy_render::{
render_graph::RenderGraphApp, render_resource::*, texture::BevyDefault, view::ExtractedView,
RenderApp,
texture::BevyDefault,
view::{ExtractedView, ViewTarget, ViewUniformOffset},
Render, RenderApp, RenderSet,
};
use crate::{
EnvironmentMapLight, MeshPipelineKey, ShadowFilteringMethod, ViewFogUniformOffset,
ViewLightsUniformOffset,
MeshPipelineKey, ShadowFilteringMethod, ViewFogUniformOffset, ViewLightsUniformOffset,
};
pub struct DeferredPbrLightingPlugin;
@ -151,6 +146,7 @@ impl ViewNode for DeferredOpaquePass3dPbrLightingNode {
&'static ViewUniformOffset,
&'static ViewLightsUniformOffset,
&'static ViewFogUniformOffset,
&'static ViewLightProbesUniformOffset,
&'static MeshViewBindGroup,
&'static ViewTarget,
&'static DeferredLightingIdDepthTexture,
@ -165,6 +161,7 @@ impl ViewNode for DeferredOpaquePass3dPbrLightingNode {
view_uniform_offset,
view_lights_offset,
view_fog_offset,
view_light_probes_offset,
mesh_view_bind_group,
target,
deferred_lighting_id_depth_texture,
@ -218,6 +215,7 @@ impl ViewNode for DeferredOpaquePass3dPbrLightingNode {
view_uniform_offset.offset,
view_lights_offset.offset,
view_fog_offset.offset,
**view_light_probes_offset,
],
);
render_pass.set_bind_group(1, &bind_group_1, &[]);
@ -403,7 +401,6 @@ pub fn prepare_deferred_lighting_pipelines(
&ExtractedView,
Option<&Tonemapping>,
Option<&DebandDither>,
Option<&EnvironmentMapLight>,
Option<&ShadowFilteringMethod>,
Has<ScreenSpaceAmbientOcclusionSettings>,
(
@ -411,20 +408,20 @@ pub fn prepare_deferred_lighting_pipelines(
Has<DepthPrepass>,
Has<MotionVectorPrepass>,
),
Has<RenderViewEnvironmentMaps>,
),
With<DeferredPrepass>,
>,
images: Res<RenderAssets<Image>>,
) {
for (
entity,
view,
tonemapping,
dither,
environment_map,
shadow_filter_method,
ssao,
(normal_prepass, depth_prepass, motion_vector_prepass),
has_environment_maps,
) in &views
{
let mut view_key = MeshPipelineKey::from_hdr(view.hdr);
@ -471,11 +468,10 @@ pub fn prepare_deferred_lighting_pipelines(
view_key |= MeshPipelineKey::SCREEN_SPACE_AMBIENT_OCCLUSION;
}
let environment_map_loaded = match environment_map {
Some(environment_map) => environment_map.is_loaded(&images),
None => false,
};
if environment_map_loaded {
// We don't need to check to see whether the environment map is loaded
// because [`gather_light_probes`] already checked that for us before
// adding the [`RenderViewEnvironmentMaps`] component.
if has_environment_maps {
view_key |= MeshPipelineKey::ENVIRONMENT_MAP;
}

50
crates/bevy_pbr/src/environment_map/environment_map.wgsl
View file

@ -1,50 +0,0 @@
#define_import_path bevy_pbr::environment_map
#import bevy_pbr::mesh_view_bindings as bindings;
struct EnvironmentMapLight {
diffuse: vec3<f32>,
specular: vec3<f32>,
};
fn environment_map_light(
perceptual_roughness: f32,
roughness: f32,
diffuse_color: vec3<f32>,
NdotV: f32,
f_ab: vec2<f32>,
N: vec3<f32>,
R: vec3<f32>,
F0: vec3<f32>,
) -> EnvironmentMapLight {
// Split-sum approximation for image based lighting: https://cdn2.unrealengine.com/Resources/files/2013SiggraphPresentationsNotes-26915738.pdf
// Technically we could use textureNumLevels(environment_map_specular) - 1 here, but we use a uniform
// because textureNumLevels() does not work on WebGL2
let radiance_level = perceptual_roughness * f32(bindings::lights.environment_map_smallest_specular_mip_level);
let irradiance = textureSampleLevel(bindings::environment_map_diffuse, bindings::environment_map_sampler, vec3(N.xy, -N.z), 0.0).rgb;
let radiance = textureSampleLevel(bindings::environment_map_specular, bindings::environment_map_sampler, vec3(R.xy, -R.z), radiance_level).rgb;
// No real world material has specular values under 0.02, so we use this range as a
// "pre-baked specular occlusion" that extinguishes the fresnel term, for artistic control.
// See: https://google.github.io/filament/Filament.html#specularocclusion
let specular_occlusion = saturate(dot(F0, vec3(50.0 * 0.33)));
// Multiscattering approximation: https://www.jcgt.org/published/0008/01/03/paper.pdf
// Useful reference: https://bruop.github.io/ibl
let Fr = max(vec3(1.0 - roughness), F0) - F0;
let kS = F0 + Fr * pow(1.0 - NdotV, 5.0);
let Ess = f_ab.x + f_ab.y;
let FssEss = kS * Ess * specular_occlusion;
let Ems = 1.0 - Ess;
let Favg = F0 + (1.0 - F0) / 21.0;
let Fms = FssEss * Favg / (1.0 - Ems * Favg);
let FmsEms = Fms * Ems;
let Edss = 1.0 - (FssEss + FmsEms);
let kD = diffuse_color * Edss;
var out: EnvironmentMapLight;
out.diffuse = (FmsEms + kD) * irradiance * bindings::lights.environment_map_intensity;
out.specular = FssEss * radiance * bindings::lights.environment_map_intensity;
return out;
}

98
crates/bevy_pbr/src/environment_map/mod.rs
View file

@ -1,98 +0,0 @@
use bevy_app::{App, Plugin};
use bevy_asset::{load_internal_asset, Handle};
use bevy_core_pipeline::prelude::Camera3d;
use bevy_ecs::{prelude::Component, query::With};
use bevy_reflect::Reflect;
use bevy_render::{
extract_component::{ExtractComponent, ExtractComponentPlugin},
render_asset::RenderAssets,
render_resource::{
binding_types::{sampler, texture_cube},
*,
},
texture::{FallbackImageCubemap, Image},
};
pub const ENVIRONMENT_MAP_SHADER_HANDLE: Handle<Shader> =
Handle::weak_from_u128(154476556247605696);
pub struct EnvironmentMapPlugin;
impl Plugin for EnvironmentMapPlugin {
fn build(&self, app: &mut App) {
load_internal_asset!(
app,
ENVIRONMENT_MAP_SHADER_HANDLE,
"environment_map.wgsl",
Shader::from_wgsl
);
app.register_type::<EnvironmentMapLight>()
.add_plugins(ExtractComponentPlugin::<EnvironmentMapLight>::default());
}
}
/// Environment map based ambient lighting representing light from distant scenery.
///
/// When added to a 3D camera, this component adds indirect light
/// to every point of the scene (including inside, enclosed areas) based on
/// an environment cubemap texture. This is similar to [`crate::AmbientLight`], but
/// higher quality, and is intended for outdoor scenes.
///
/// The environment map must be prefiltered into a diffuse and specular cubemap based on the
/// [split-sum approximation](https://cdn2.unrealengine.com/Resources/files/2013SiggraphPresentationsNotes-26915738.pdf).
///
/// To prefilter your environment map, you can use `KhronosGroup`'s [glTF-IBL-Sampler](https://github.com/KhronosGroup/glTF-IBL-Sampler).
/// The diffuse map uses the Lambertian distribution, and the specular map uses the GGX distribution.
///
/// `KhronosGroup` also has several prefiltered environment maps that can be found [here](https://github.com/KhronosGroup/glTF-Sample-Environments).
#[derive(Component, Reflect, Clone, ExtractComponent)]
#[extract_component_filter(With<Camera3d>)]
pub struct EnvironmentMapLight {
pub diffuse_map: Handle<Image>,
pub specular_map: Handle<Image>,
/// Scale factor applied to the diffuse and specular light generated by this component.
///
/// After applying this multiplier, the resulting values should
/// be in units of [cd/m^2](https://en.wikipedia.org/wiki/Candela_per_square_metre).
///
/// See also <https://google.github.io/filament/Filament.html#lighting/imagebasedlights/iblunit>.
pub intensity: f32,
}
impl EnvironmentMapLight {
/// Whether or not all textures necessary to use the environment map
/// have been loaded by the asset server.
pub fn is_loaded(&self, images: &RenderAssets<Image>) -> bool {
images.get(&self.diffuse_map).is_some() && images.get(&self.specular_map).is_some()
}
}
pub fn get_bindings<'a>(
environment_map_light: Option<&EnvironmentMapLight>,
images: &'a RenderAssets<Image>,
fallback_image_cubemap: &'a FallbackImageCubemap,
) -> (&'a TextureView, &'a TextureView, &'a Sampler) {
let (diffuse_map, specular_map) = match (
environment_map_light.and_then(|env_map| images.get(&env_map.diffuse_map)),
environment_map_light.and_then(|env_map| images.get(&env_map.specular_map)),
) {
(Some(diffuse_map), Some(specular_map)) => {
(&diffuse_map.texture_view, &specular_map.texture_view)
}
_ => (
&fallback_image_cubemap.texture_view,
&fallback_image_cubemap.texture_view,
),
};
(diffuse_map, specular_map, &fallback_image_cubemap.sampler)
}
pub fn get_bind_group_layout_entries() -> [BindGroupLayoutEntryBuilder; 3] {
[
texture_cube(TextureSampleType::Float { filterable: true }),
texture_cube(TextureSampleType::Float { filterable: true }),
sampler(SamplerBindingType::Filtering),
]
}

12
crates/bevy_pbr/src/lib.rs
View file

@ -3,10 +3,10 @@ pub mod wireframe;
mod alpha;
mod bundle;
pub mod deferred;
mod environment_map;
mod extended_material;
mod fog;
mod light;
mod light_probe;
mod lightmap;
mod material;
mod parallax;
@ -17,10 +17,10 @@ mod ssao;
pub use alpha::*;
pub use bundle::*;
pub use environment_map::EnvironmentMapLight;
pub use extended_material::*;
pub use fog::*;
pub use light::*;
pub use light_probe::*;
pub use lightmap::*;
pub use material::*;
pub use parallax::*;
@ -37,9 +37,12 @@ pub mod prelude {
DirectionalLightBundle, MaterialMeshBundle, PbrBundle, PointLightBundle,
SpotLightBundle,
},
environment_map::EnvironmentMapLight,
fog::{FogFalloff, FogSettings},
light::{AmbientLight, DirectionalLight, PointLight, SpotLight},
light_probe::{
environment_map::{EnvironmentMapLight, ReflectionProbeBundle},
LightProbe,
},
material::{Material, MaterialPlugin},
parallax::ParallaxMappingMethod,
pbr_material::StandardMaterial,
@ -71,7 +74,6 @@ use bevy_render::{
ExtractSchedule, Render, RenderApp, RenderSet,
};
use bevy_transform::TransformSystem;
use environment_map::EnvironmentMapPlugin;
use crate::deferred::DeferredPbrLightingPlugin;
@ -255,12 +257,12 @@ impl Plugin for PbrPlugin {
..Default::default()
},
ScreenSpaceAmbientOcclusionPlugin,
EnvironmentMapPlugin,
ExtractResourcePlugin::<AmbientLight>::default(),
FogPlugin,
ExtractResourcePlugin::<DefaultOpaqueRendererMethod>::default(),
ExtractComponentPlugin::<ShadowFilteringMethod>::default(),
LightmapPlugin,
LightProbePlugin,
))
.configure_sets(
PostUpdate,

370
crates/bevy_pbr/src/light_probe/environment_map.rs Normal file
View file

@ -0,0 +1,370 @@
//! Environment maps and reflection probes.
//!
//! An *environment map* consists of a pair of diffuse and specular cubemaps
//! that together reflect the static surrounding area of a region in space. When
//! available, the PBR shader uses these to apply diffuse light and calculate
//! specular reflections.
//!
//! Environment maps come in two flavors, depending on what other components the
//! entities they're attached to have:
//!
//! 1. If attached to a view, they represent the objects located a very far
//! distance from the view, in a similar manner to a skybox. Essentially, these
//! *view environment maps* represent a higher-quality replacement for
//! [`crate::AmbientLight`] for outdoor scenes. The indirect light from such
//! environment maps are added to every point of the scene, including
//! interior enclosed areas.
//!
//! 2. If attached to a [`LightProbe`], environment maps represent the immediate
//! surroundings of a specific location in the scene. These types of
//! environment maps are known as *reflection probes*.
//! [`ReflectionProbeBundle`] is available as a mechanism to conveniently add
//! these to a scene.
//!
//! Typically, environment maps are static (i.e. "baked", calculated ahead of
//! time) and so only reflect fixed static geometry. The environment maps must
//! be pre-filtered into a pair of cubemaps, one for the diffuse component and
//! one for the specular component, according to the [split-sum approximation].
//! To pre-filter your environment map, you can use the [glTF IBL Sampler] or
//! its [artist-friendly UI]. The diffuse map uses the Lambertian distribution,
//! while the specular map uses the GGX distribution.
//!
//! The Khronos Group has [several pre-filtered environment maps] available for
//! you to use.
//!
//! Currently, reflection probes (i.e. environment maps attached to light
//! probes) use binding arrays (also known as bindless textures) and
//! consequently aren't supported on WebGL2 or WebGPU. Reflection probes are
//! also unsupported if GLSL is in use, due to `naga` limitations. Environment
//! maps attached to views are, however, supported on all platforms.
//!
//! [split-sum approximation]: https://cdn2.unrealengine.com/Resources/files/2013SiggraphPresentationsNotes-26915738.pdf
//!
//! [glTF IBL Sampler]: https://github.com/KhronosGroup/glTF-IBL-Sampler
//!
//! [artist-friendly UI]: https://github.com/pcwalton/gltf-ibl-sampler-egui
//!
//! [several pre-filtered environment maps]: https://github.com/KhronosGroup/glTF-Sample-Environments
use bevy_asset::{AssetId, Handle};
use bevy_ecs::{
bundle::Bundle, component::Component, query::QueryItem, system::lifetimeless::Read,
};
use bevy_reflect::Reflect;
use bevy_render::{
extract_instances::ExtractInstance,
prelude::SpatialBundle,
render_asset::RenderAssets,
render_resource::{
binding_types, BindGroupLayoutEntryBuilder, Sampler, SamplerBindingType, Shader,
TextureSampleType, TextureView,
},
renderer::RenderDevice,
settings::WgpuFeatures,
texture::{FallbackImage, Image},
};
use bevy_utils::HashMap;
use std::num::NonZeroU32;
use std::ops::Deref;
use crate::{LightProbe, MAX_VIEW_REFLECTION_PROBES};
/// A handle to the environment map helper shader.
pub const ENVIRONMENT_MAP_SHADER_HANDLE: Handle<Shader> =
Handle::weak_from_u128(154476556247605696);
/// How many texture bindings are used in the fragment shader, *not* counting
/// environment maps.
const STANDARD_MATERIAL_FRAGMENT_SHADER_MIN_TEXTURE_BINDINGS: usize = 16;
/// A pair of cubemap textures that represent the surroundings of a specific
/// area in space.
///
/// See [`crate::environment_map`] for detailed information.
#[derive(Clone, Component, Reflect)]
pub struct EnvironmentMapLight {
/// The blurry image that represents diffuse radiance surrounding a region.
pub diffuse_map: Handle<Image>,
/// The typically-sharper, mipmapped image that represents specular radiance
/// surrounding a region.
pub specular_map: Handle<Image>,
/// Scale factor applied to the diffuse and specular light generated by this component.
///
/// After applying this multiplier, the resulting values should
/// be in units of [cd/m^2](https://en.wikipedia.org/wiki/Candela_per_square_metre).
///
/// See also <https://google.github.io/filament/Filament.html#lighting/imagebasedlights/iblunit>.
pub intensity: f32,
}
/// Like [`EnvironmentMapLight`], but contains asset IDs instead of handles.
///
/// This is for use in the render app.
#[derive(Clone, Copy, PartialEq, Eq, Hash)]
pub(crate) struct EnvironmentMapIds {
/// The blurry image that represents diffuse radiance surrounding a region.
pub(crate) diffuse: AssetId<Image>,
/// The typically-sharper, mipmapped image that represents specular radiance
/// surrounding a region.
pub(crate) specular: AssetId<Image>,
}
/// A bundle that contains everything needed to make an entity a reflection
/// probe.
///
/// A reflection probe is a type of environment map that specifies the light
/// surrounding a region in space. For more information, see
/// [`crate::environment_map`].
#[derive(Bundle)]
pub struct ReflectionProbeBundle {
/// Contains a transform that specifies the position of this reflection probe in space.
pub spatial: SpatialBundle,
/// Marks this environment map as a light probe.
pub light_probe: LightProbe,
/// The cubemaps that make up this environment map.
pub environment_map: EnvironmentMapLight,
}
/// A component, part of the render world, that stores the mapping from
/// environment map ID to texture index in the diffuse and specular binding
/// arrays.
///
/// Cubemap textures belonging to environment maps are collected into binding
/// arrays, and the index of each texture is presented to the shader for runtime
/// lookup.
///
/// This component is attached to each view in the render world, because each
/// view may have a different set of cubemaps that it considers and therefore
/// cubemap indices are per-view.
#[derive(Component, Default)]
pub struct RenderViewEnvironmentMaps {
/// The list of environment maps presented to the shader, in order.
binding_index_to_cubemap: Vec<EnvironmentMapIds>,
/// The reverse of `binding_index_to_cubemap`: a map from the environment
/// map IDs to the index in `binding_index_to_cubemap`.
cubemap_to_binding_index: HashMap<EnvironmentMapIds, u32>,
}
/// All the bind group entries necessary for PBR shaders to access the
/// environment maps exposed to a view.
pub(crate) enum RenderViewBindGroupEntries<'a> {
/// The version used when binding arrays aren't available on the current
/// platform.
Single {
/// The texture view of the view's diffuse cubemap.
diffuse_texture_view: &'a TextureView,
/// The texture view of the view's specular cubemap.
specular_texture_view: &'a TextureView,
/// The sampler used to sample elements of both `diffuse_texture_views` and
/// `specular_texture_views`.
sampler: &'a Sampler,
},
/// The version used when binding arrays aren't available on the current
/// platform.
Multiple {
/// A texture view of each diffuse cubemap, in the same order that they are
/// supplied to the view (i.e. in the same order as
/// `binding_index_to_cubemap` in [`RenderViewEnvironmentMaps`]).
///
/// This is a vector of `wgpu::TextureView`s. But we don't want to import
/// `wgpu` in this crate, so we refer to it indirectly like this.
diffuse_texture_views: Vec<&'a <TextureView as Deref>::Target>,
/// As above, but for specular cubemaps.
specular_texture_views: Vec<&'a <TextureView as Deref>::Target>,
/// The sampler used to sample elements of both `diffuse_texture_views` and
/// `specular_texture_views`.
sampler: &'a Sampler,
},
}
impl ExtractInstance for EnvironmentMapIds {
type Data = Read<EnvironmentMapLight>;
type Filter = ();
fn extract(item: QueryItem<'_, Self::Data>) -> Option<Self> {
Some(EnvironmentMapIds {
diffuse: item.diffuse_map.id(),
specular: item.specular_map.id(),
})
}
}
impl RenderViewEnvironmentMaps {
pub(crate) fn new() -> Self {
Self::default()
}
}
impl RenderViewEnvironmentMaps {
/// Whether there are no environment maps associated with the view.
pub(crate) fn is_empty(&self) -> bool {
self.binding_index_to_cubemap.is_empty()
}
/// Adds a cubemap to the list of bindings, if it wasn't there already, and
/// returns its index within that list.
pub(crate) fn get_or_insert_cubemap(&mut self, cubemap_id: &EnvironmentMapIds) -> u32 {
*self
.cubemap_to_binding_index
.entry(*cubemap_id)
.or_insert_with(|| {
let index = self.binding_index_to_cubemap.len() as u32;
self.binding_index_to_cubemap.push(*cubemap_id);
index
})
}
}
/// Returns the bind group layout entries for the environment map diffuse and
/// specular binding arrays respectively, in addition to the sampler.
pub(crate) fn get_bind_group_layout_entries(
render_device: &RenderDevice,
) -> [BindGroupLayoutEntryBuilder; 3] {
let mut texture_cube_binding =
binding_types::texture_cube(TextureSampleType::Float { filterable: true });
if binding_arrays_are_usable(render_device) {
texture_cube_binding =
texture_cube_binding.count(NonZeroU32::new(MAX_VIEW_REFLECTION_PROBES as _).unwrap());
}
[
texture_cube_binding,
texture_cube_binding,
binding_types::sampler(SamplerBindingType::Filtering),
]
}
impl<'a> RenderViewBindGroupEntries<'a> {
/// Looks up and returns the bindings for the environment map diffuse and
/// specular binding arrays respectively, as well as the sampler.
pub(crate) fn get(
render_view_environment_maps: Option<&RenderViewEnvironmentMaps>,
images: &'a RenderAssets<Image>,
fallback_image: &'a FallbackImage,
render_device: &RenderDevice,
) -> RenderViewBindGroupEntries<'a> {
if binding_arrays_are_usable(render_device) {
let mut diffuse_texture_views = vec![];
let mut specular_texture_views = vec![];
let mut sampler = None;
if let Some(environment_maps) = render_view_environment_maps {
for &cubemap_id in &environment_maps.binding_index_to_cubemap {
add_texture_view(
&mut diffuse_texture_views,
&mut sampler,
cubemap_id.diffuse,
images,
fallback_image,
);
add_texture_view(
&mut specular_texture_views,
&mut sampler,
cubemap_id.specular,
images,
fallback_image,
);
}
}
// Pad out the bindings to the size of the binding array using fallback
// textures. This is necessary on D3D12 and Metal.
diffuse_texture_views.resize(
MAX_VIEW_REFLECTION_PROBES,
&*fallback_image.cube.texture_view,
);
specular_texture_views.resize(
MAX_VIEW_REFLECTION_PROBES,
&*fallback_image.cube.texture_view,
);
return RenderViewBindGroupEntries::Multiple {
diffuse_texture_views,
specular_texture_views,
sampler: sampler.unwrap_or(&fallback_image.cube.sampler),
};
}
if let Some(environment_maps) = render_view_environment_maps {
if let Some(cubemap) = environment_maps.binding_index_to_cubemap.first() {
if let (Some(diffuse_image), Some(specular_image)) =
(images.get(cubemap.diffuse), images.get(cubemap.specular))
{
return RenderViewBindGroupEntries::Single {
diffuse_texture_view: &diffuse_image.texture_view,
specular_texture_view: &specular_image.texture_view,
sampler: &diffuse_image.sampler,
};
}
}
}
RenderViewBindGroupEntries::Single {
diffuse_texture_view: &fallback_image.cube.texture_view,
specular_texture_view: &fallback_image.cube.texture_view,
sampler: &fallback_image.cube.sampler,
}
}
}
/// Adds a diffuse or specular texture view to the `texture_views` list, and
/// populates `sampler` if this is the first such view.
fn add_texture_view<'a>(
texture_views: &mut Vec<&'a <TextureView as Deref>::Target>,
sampler: &mut Option<&'a Sampler>,
image_id: AssetId<Image>,
images: &'a RenderAssets<Image>,
fallback_image: &'a FallbackImage,
) {
match images.get(image_id) {
None => {
// Use the fallback image if the cubemap isn't loaded yet.
texture_views.push(&*fallback_image.cube.texture_view);
}
Some(image) => {
// If this is the first texture view, populate `sampler`.
if sampler.is_none() {
*sampler = Some(&image.sampler);
}
texture_views.push(&*image.texture_view);
}
}
}
/// Many things can go wrong when attempting to use texture binding arrays
/// (a.k.a. bindless textures). This function checks for these pitfalls:
///
/// 1. If GLSL support is enabled at the feature level, then in debug mode
/// `naga_oil` will attempt to compile all shader modules under GLSL to check
/// validity of names, even if GLSL isn't actually used. This will cause a crash
/// if binding arrays are enabled, because binding arrays are currently
/// unimplemented in the GLSL backend of Naga. Therefore, we disable binding
/// arrays if the `shader_format_glsl` feature is present.
///
/// 2. If there aren't enough texture bindings available to accommodate all the
/// binding arrays, the driver will panic. So we also bail out if there aren't
/// enough texture bindings available in the fragment shader.
///
/// 3. If binding arrays aren't supported on the hardware, then we obviously
/// can't use them.
///
/// If binding arrays aren't usable, we disable reflection probes, as they rely
/// on them.
pub(crate) fn binding_arrays_are_usable(render_device: &RenderDevice) -> bool {
!cfg!(feature = "shader_format_glsl")
&& render_device.limits().max_storage_textures_per_shader_stage
>= (STANDARD_MATERIAL_FRAGMENT_SHADER_MIN_TEXTURE_BINDINGS + MAX_VIEW_REFLECTION_PROBES)
as u32
&& render_device
.features()
.contains(WgpuFeatures::TEXTURE_BINDING_ARRAY)
}

176
crates/bevy_pbr/src/light_probe/environment_map.wgsl Normal file
View file

@ -0,0 +1,176 @@
#define_import_path bevy_pbr::environment_map
#import bevy_pbr::mesh_view_bindings as bindings
#import bevy_pbr::mesh_view_bindings::light_probes
struct EnvironmentMapLight {
diffuse: vec3<f32>,
specular: vec3<f32>,
};
struct EnvironmentMapRadiances {
irradiance: vec3<f32>,
radiance: vec3<f32>,
}
// Define two versions of this function, one for the case in which there are
// multiple light probes and one for the case in which only the view light probe
// is present.
#ifdef MULTIPLE_LIGHT_PROBES_IN_ARRAY
fn compute_radiances(
perceptual_roughness: f32,
N: vec3<f32>,
R: vec3<f32>,
world_position: vec3<f32>,
) -> EnvironmentMapRadiances {
var radiances: EnvironmentMapRadiances;
// Search for a reflection probe that contains the fragment.
//
// TODO: Interpolate between multiple reflection probes.
var cubemap_index: i32 = -1;
var intensity: f32 = 1.0;
for (var reflection_probe_index: i32 = 0;
reflection_probe_index < light_probes.reflection_probe_count;
reflection_probe_index += 1) {
let reflection_probe = light_probes.reflection_probes[reflection_probe_index];
// Unpack the inverse transform.
let inverse_transpose_transform = mat4x4<f32>(
reflection_probe.inverse_transpose_transform[0],
reflection_probe.inverse_transpose_transform[1],
reflection_probe.inverse_transpose_transform[2],
vec4<f32>(0.0, 0.0, 0.0, 1.0));
let inverse_transform = transpose(inverse_transpose_transform);
// Check to see if the transformed point is inside the unit cube
// centered at the origin.
let probe_space_pos = (inverse_transform * vec4<f32>(world_position, 1.0)).xyz;
if (all(abs(probe_space_pos) <= vec3(0.5))) {
cubemap_index = reflection_probe.cubemap_index;
intensity = reflection_probe.intensity;
break;
}
}
// If we didn't find a reflection probe, use the view environment map if applicable.
if (cubemap_index < 0) {
cubemap_index = light_probes.view_cubemap_index;
intensity = light_probes.intensity_for_view;
}
// If there's no cubemap, bail out.
if (cubemap_index < 0) {
radiances.irradiance = vec3(0.0);
radiances.radiance = vec3(0.0);
return radiances;
}
// Split-sum approximation for image based lighting: https://cdn2.unrealengine.com/Resources/files/2013SiggraphPresentationsNotes-26915738.pdf
let radiance_level = perceptual_roughness * f32(textureNumLevels(bindings::specular_environment_maps[cubemap_index]) - 1u);
#ifndef LIGHTMAP
radiances.irradiance = textureSampleLevel(
bindings::diffuse_environment_maps[cubemap_index],
bindings::environment_map_sampler,
vec3(N.xy, -N.z),
0.0).rgb * intensity;
#endif // LIGHTMAP
radiances.radiance = textureSampleLevel(
bindings::specular_environment_maps[cubemap_index],
bindings::environment_map_sampler,
vec3(R.xy, -R.z),
radiance_level).rgb * intensity;
return radiances;
}
#else // MULTIPLE_LIGHT_PROBES_IN_ARRAY
fn compute_radiances(
perceptual_roughness: f32,
N: vec3<f32>,
R: vec3<f32>,
world_position: vec3<f32>,
) -> EnvironmentMapRadiances {
var radiances: EnvironmentMapRadiances;
if (light_probes.view_cubemap_index < 0) {
radiances.irradiance = vec3(0.0);
radiances.radiance = vec3(0.0);
return radiances;
}
// Split-sum approximation for image based lighting: https://cdn2.unrealengine.com/Resources/files/2013SiggraphPresentationsNotes-26915738.pdf
// Technically we could use textureNumLevels(specular_environment_map) - 1 here, but we use a uniform
// because textureNumLevels() does not work on WebGL2
let radiance_level = perceptual_roughness * f32(light_probes.smallest_specular_mip_level_for_view);
let intensity = light_probes.intensity_for_view;
#ifndef LIGHTMAP
radiances.irradiance = textureSampleLevel(
bindings::diffuse_environment_map,
bindings::environment_map_sampler,
vec3(N.xy, -N.z),
0.0).rgb * intensity;
#endif // LIGHTMAP
radiances.radiance = textureSampleLevel(
bindings::specular_environment_map,
bindings::environment_map_sampler,
vec3(R.xy, -R.z),
radiance_level).rgb * intensity;
return radiances;
}
#endif // MULTIPLE_LIGHT_PROBES_IN_ARRAY
fn environment_map_light(
perceptual_roughness: f32,
roughness: f32,
diffuse_color: vec3<f32>,
NdotV: f32,
f_ab: vec2<f32>,
N: vec3<f32>,
R: vec3<f32>,
F0: vec3<f32>,
world_position: vec3<f32>,
) -> EnvironmentMapLight {
let radiances = compute_radiances(perceptual_roughness, N, R, world_position);
// No real world material has specular values under 0.02, so we use this range as a
// "pre-baked specular occlusion" that extinguishes the fresnel term, for artistic control.
// See: https://google.github.io/filament/Filament.html#specularocclusion
let specular_occlusion = saturate(dot(F0, vec3(50.0 * 0.33)));
// Multiscattering approximation: https://www.jcgt.org/published/0008/01/03/paper.pdf
// Useful reference: https://bruop.github.io/ibl
let Fr = max(vec3(1.0 - roughness), F0) - F0;
let kS = F0 + Fr * pow(1.0 - NdotV, 5.0);
let Ess = f_ab.x + f_ab.y;
let FssEss = kS * Ess * specular_occlusion;
let Ems = 1.0 - Ess;
let Favg = F0 + (1.0 - F0) / 21.0;
let Fms = FssEss * Favg / (1.0 - Ems * Favg);
let FmsEms = Fms * Ems;
let Edss = 1.0 - (FssEss + FmsEms);
let kD = diffuse_color * Edss;
var out: EnvironmentMapLight;
// If there's a lightmap, ignore the diffuse component of the reflection
// probe, so we don't double-count light.
#ifdef LIGHTMAP
out.diffuse = vec3(0.0);
#else
out.diffuse = (FmsEms + kD) * radiances.irradiance;
#endif
out.specular = FssEss * radiances.radiance;
return out;
}

438
crates/bevy_pbr/src/light_probe/mod.rs Normal file
View file

@ -0,0 +1,438 @@
//! Light probes for baked global illumination.
use bevy_app::{App, Plugin};
use bevy_asset::load_internal_asset;
use bevy_core_pipeline::core_3d::Camera3d;
use bevy_derive::{Deref, DerefMut};
use bevy_ecs::{
component::Component,
entity::Entity,
query::With,
reflect::ReflectComponent,
schedule::IntoSystemConfigs,
system::{Commands, Local, Query, Res, ResMut, Resource},
};
use bevy_math::{Affine3A, Mat4, Vec3A, Vec4};
use bevy_reflect::{std_traits::ReflectDefault, Reflect};
use bevy_render::{
extract_instances::ExtractInstancesPlugin,
primitives::{Aabb, Frustum},
render_asset::RenderAssets,
render_resource::{DynamicUniformBuffer, Shader, ShaderType},
renderer::{RenderDevice, RenderQueue},
texture::Image,
Extract, ExtractSchedule, Render, RenderApp, RenderSet,
};
use bevy_transform::prelude::GlobalTransform;
use bevy_utils::{EntityHashMap, FloatOrd};
use crate::light_probe::environment_map::{
binding_arrays_are_usable, EnvironmentMapIds, EnvironmentMapLight, RenderViewEnvironmentMaps,
ENVIRONMENT_MAP_SHADER_HANDLE,
};
pub mod environment_map;
/// The maximum number of reflection probes that each view will consider.
///
/// Because the fragment shader does a linear search through the list for each
/// fragment, this number needs to be relatively small.
pub const MAX_VIEW_REFLECTION_PROBES: usize = 8;
/// Adds support for light probes: cuboid bounding regions that apply global
/// illumination to objects within them.
///
/// This also adds support for view environment maps: diffuse and specular
/// cubemaps applied to all objects that a view renders.
pub struct LightProbePlugin;
/// A marker component for a light probe, which is a cuboid region that provides
/// global illumination to all fragments inside it.
///
/// The light probe range is conceptually a unit cube (1×1×1) centered on the
/// origin. The [`bevy_transform::prelude::Transform`] applied to this entity
/// can scale, rotate, or translate that cube so that it contains all fragments
/// that should take this light probe into account.
///
/// Note that a light probe will have no effect unless the entity contains some
/// kind of illumination. At present, the only supported type of illumination is
/// the [`EnvironmentMapLight`].
#[derive(Component, Debug, Clone, Copy, Default, Reflect)]
#[reflect(Component, Default)]
pub struct LightProbe;
/// A GPU type that stores information about a reflection probe.
#[derive(Clone, Copy, ShaderType, Default)]
struct RenderReflectionProbe {
/// The transform from the world space to the model space. This is used to
/// efficiently check for bounding box intersection.
inverse_transpose_transform: [Vec4; 3],
/// The index of the environment map in the diffuse and specular cubemap
/// binding arrays.
cubemap_index: i32,
/// Scale factor applied to the diffuse and specular light generated by this
/// reflection probe.
///
/// See the comment in [`EnvironmentMapLight`] for details.
intensity: f32,
}
/// A per-view shader uniform that specifies all the light probes that the view
/// takes into account.
#[derive(ShaderType)]
pub struct LightProbesUniform {
/// The list of applicable reflection probes, sorted from nearest to the
/// camera to the farthest away from the camera.
reflection_probes: [RenderReflectionProbe; MAX_VIEW_REFLECTION_PROBES],
/// The number of reflection probes in the list.
reflection_probe_count: i32,
/// The index of the diffuse and specular environment maps associated with
/// the view itself. This is used as a fallback if no reflection probe in
/// the list contains the fragment.
view_cubemap_index: i32,
/// The smallest valid mipmap level for the specular environment cubemap
/// associated with the view.
smallest_specular_mip_level_for_view: u32,
/// The intensity of the environment cubemap associated with the view.
///
/// See the comment in [`EnvironmentMapLight`] for details.
intensity_for_view: f32,
}
/// A map from each camera to the light probe uniform associated with it.
#[derive(Resource, Default, Deref, DerefMut)]
struct RenderLightProbes(EntityHashMap<Entity, LightProbesUniform>);
/// A GPU buffer that stores information about all light probes.
#[derive(Resource, Default, Deref, DerefMut)]
pub struct LightProbesBuffer(DynamicUniformBuffer<LightProbesUniform>);
/// A component attached to each camera in the render world that stores the
/// index of the [`LightProbesUniform`] in the [`LightProbesBuffer`].
#[derive(Component, Default, Deref, DerefMut)]
pub struct ViewLightProbesUniformOffset(u32);
/// Information that [`gather_light_probes`] keeps about each light probe.
#[derive(Clone, Copy)]
#[allow(dead_code)]
struct LightProbeInfo {
// The transform from world space to light probe space.
inverse_transform: Mat4,
// The transform from light probe space to world space.
affine_transform: Affine3A,
// The diffuse and specular environment maps associated with this light
// probe.
environment_maps: EnvironmentMapIds,
// Scale factor applied to the diffuse and specular light generated by this
// reflection probe.
//
// See the comment in [`EnvironmentMapLight`] for details.
intensity: f32,
}
impl LightProbe {
/// Creates a new light probe component.
#[inline]
pub fn new() -> Self {
Self
}
}
impl Plugin for LightProbePlugin {
fn build(&self, app: &mut App) {
load_internal_asset!(
app,
ENVIRONMENT_MAP_SHADER_HANDLE,
"environment_map.wgsl",
Shader::from_wgsl
);
app.register_type::<LightProbe>()
.register_type::<EnvironmentMapLight>();
}
fn finish(&self, app: &mut App) {
let Ok(render_app) = app.get_sub_app_mut(RenderApp) else {
return;
};
render_app
.add_plugins(ExtractInstancesPlugin::<EnvironmentMapIds>::new())
.init_resource::<LightProbesBuffer>()
.init_resource::<RenderLightProbes>()
.add_systems(ExtractSchedule, gather_light_probes)
.add_systems(
Render,
upload_light_probes.in_set(RenderSet::PrepareResources),
);
}
}
/// Gathers up all light probes in the scene and assigns them to views,
/// performing frustum culling and distance sorting in the process.
///
/// This populates the [`RenderLightProbes`] resource.
#[allow(clippy::too_many_arguments)]
fn gather_light_probes(
mut render_light_probes: ResMut<RenderLightProbes>,
image_assets: Res<RenderAssets<Image>>,
render_device: Res<RenderDevice>,
light_probe_query: Extract<Query<(&GlobalTransform, &EnvironmentMapLight), With<LightProbe>>>,
view_query: Extract<
Query<
(
Entity,
&GlobalTransform,
&Frustum,
Option<&EnvironmentMapLight>,
),
With<Camera3d>,
>,
>,
mut light_probes: Local<Vec<LightProbeInfo>>,
mut view_light_probes: Local<Vec<LightProbeInfo>>,
mut commands: Commands,
) {
// Create [`LightProbeInfo`] for every light probe in the scene.
light_probes.clear();
light_probes.extend(
light_probe_query
.iter()
.filter_map(|query_row| LightProbeInfo::new(query_row, &image_assets)),
);
// Build up the light probes uniform and the key table.
render_light_probes.clear();
for (view_entity, view_transform, view_frustum, view_environment_maps) in view_query.iter() {
// Cull light probes outside the view frustum.
view_light_probes.clear();
view_light_probes.extend(
light_probes
.iter()
.filter(|light_probe_info| light_probe_info.frustum_cull(view_frustum))
.cloned(),
);
// Sort by distance to camera.
view_light_probes.sort_by_cached_key(|light_probe_info| {
light_probe_info.camera_distance_sort_key(view_transform)
});
// Create the light probes uniform.
let (light_probes_uniform, render_view_environment_maps) = LightProbesUniform::build(
view_environment_maps,
&view_light_probes,
&image_assets,
&render_device,
);
// Record the uniforms.
render_light_probes.insert(view_entity, light_probes_uniform);
// Record the per-view environment maps.
let mut commands = commands.get_or_spawn(view_entity);
if render_view_environment_maps.is_empty() {
commands.remove::<RenderViewEnvironmentMaps>();
} else {
commands.insert(render_view_environment_maps);
}
}
}
/// Uploads the result of [`gather_light_probes`] to the GPU.
fn upload_light_probes(
mut commands: Commands,
light_probes_uniforms: Res<RenderLightProbes>,
mut light_probes_buffer: ResMut<LightProbesBuffer>,
render_device: Res<RenderDevice>,
render_queue: Res<RenderQueue>,
) {
// Get the uniform buffer writer.
let Some(mut writer) =
light_probes_buffer.get_writer(light_probes_uniforms.len(), &render_device, &render_queue)
else {
return;
};
// Send each view's uniforms to the GPU.
for (&view_entity, light_probes_uniform) in light_probes_uniforms.iter() {
commands
.entity(view_entity)
.insert(ViewLightProbesUniformOffset(
writer.write(light_probes_uniform),
));
}
}
impl Default for LightProbesUniform {
fn default() -> Self {
Self {
reflection_probes: [RenderReflectionProbe::default(); MAX_VIEW_REFLECTION_PROBES],
reflection_probe_count: 0,
view_cubemap_index: -1,
smallest_specular_mip_level_for_view: 0,
intensity_for_view: 1.0,
}
}
}
impl LightProbesUniform {
/// Constructs a [`LightProbesUniform`] containing all the environment maps
/// that fragments rendered by a single view need to consider.
///
/// The `view_environment_maps` parameter describes the environment maps
/// attached to the view. The `light_probes` parameter is expected to be the
/// list of light probes in the scene, sorted by increasing view distance
/// from the camera.
fn build(
view_environment_maps: Option<&EnvironmentMapLight>,
light_probes: &[LightProbeInfo],
image_assets: &RenderAssets<Image>,
render_device: &RenderDevice,
) -> (LightProbesUniform, RenderViewEnvironmentMaps) {
let mut render_view_environment_maps = RenderViewEnvironmentMaps::new();
// Find the index of the cubemap associated with the view, and determine
// its smallest mip level.
let mut view_cubemap_index = -1;
let mut smallest_specular_mip_level_for_view = 0;
let mut intensity_for_view = 1.0;
if let Some(EnvironmentMapLight {
diffuse_map: diffuse_map_handle,
specular_map: specular_map_handle,
intensity,
}) = view_environment_maps
{
if let (Some(_), Some(specular_map)) = (
image_assets.get(diffuse_map_handle),
image_assets.get(specular_map_handle),
) {
view_cubemap_index =
render_view_environment_maps.get_or_insert_cubemap(&EnvironmentMapIds {
diffuse: diffuse_map_handle.id(),
specular: specular_map_handle.id(),
}) as i32;
smallest_specular_mip_level_for_view = specular_map.mip_level_count - 1;
intensity_for_view = *intensity;
}
};
// Initialize the uniform to only contain the view environment map, if
// applicable.
let mut uniform = LightProbesUniform {
reflection_probes: [RenderReflectionProbe::default(); MAX_VIEW_REFLECTION_PROBES],
reflection_probe_count: light_probes.len().min(MAX_VIEW_REFLECTION_PROBES) as i32,
view_cubemap_index,
smallest_specular_mip_level_for_view,
intensity_for_view,
};
// Add reflection probes from the scene, if supported by the current
// platform.
uniform.maybe_gather_reflection_probes(
&mut render_view_environment_maps,
light_probes,
render_device,
);
(uniform, render_view_environment_maps)
}
/// Gathers up all reflection probes in the scene and writes them into this
/// uniform and `render_view_environment_maps`.
fn maybe_gather_reflection_probes(
&mut self,
render_view_environment_maps: &mut RenderViewEnvironmentMaps,
light_probes: &[LightProbeInfo],
render_device: &RenderDevice,
) {
if !binding_arrays_are_usable(render_device) {
return;
}
for (reflection_probe, light_probe) in self
.reflection_probes
.iter_mut()
.zip(light_probes.iter().take(MAX_VIEW_REFLECTION_PROBES))
{
// Determine the index of the cubemap in the binding array.
let cubemap_index = render_view_environment_maps
.get_or_insert_cubemap(&light_probe.environment_maps)
as i32;
// Transpose the inverse transform to compress the structure on the
// GPU (from 4 `Vec4`s to 3 `Vec4`s). The shader will transpose it
// to recover the original inverse transform.
let inverse_transpose_transform = light_probe.inverse_transform.transpose();
// Write in the reflection probe data.
*reflection_probe = RenderReflectionProbe {
inverse_transpose_transform: [
inverse_transpose_transform.x_axis,
inverse_transpose_transform.y_axis,
inverse_transpose_transform.z_axis,
],
cubemap_index,
intensity: light_probe.intensity,
};
}
}
}
impl LightProbeInfo {
/// Given the set of light probe components, constructs and returns
/// [`LightProbeInfo`]. This is done for every light probe in the scene
/// every frame.
fn new(
(light_probe_transform, environment_map): (&GlobalTransform, &EnvironmentMapLight),
image_assets: &RenderAssets<Image>,
) -> Option<LightProbeInfo> {
if image_assets.get(&environment_map.diffuse_map).is_none()
|| image_assets.get(&environment_map.specular_map).is_none()
{
return None;
}
Some(LightProbeInfo {
affine_transform: light_probe_transform.affine(),
inverse_transform: light_probe_transform.compute_matrix().inverse(),
environment_maps: EnvironmentMapIds {
diffuse: environment_map.diffuse_map.id(),
specular: environment_map.specular_map.id(),
},
intensity: environment_map.intensity,
})
}
/// Returns true if this light probe is in the viewing frustum of the camera
/// or false if it isn't.
fn frustum_cull(&self, view_frustum: &Frustum) -> bool {
view_frustum.intersects_obb(
&Aabb {
center: Vec3A::default(),
half_extents: Vec3A::splat(0.5),
},
&self.affine_transform,
true,
false,
)
}
/// Returns the squared distance from this light probe to the camera,
/// suitable for distance sorting.
fn camera_distance_sort_key(&self, view_transform: &GlobalTransform) -> FloatOrd {
FloatOrd(
(self.affine_transform.translation - view_transform.translation_vec3a())
.length_squared(),
)
}
}

11
crates/bevy_pbr/src/material.rs
View file

@ -1,4 +1,4 @@
use crate::*;
use crate::{environment_map::RenderViewEnvironmentMaps, *};
use bevy_app::{App, Plugin};
use bevy_asset::{Asset, AssetApp, AssetEvent, AssetId, AssetServer, Assets, Handle};
use bevy_core_pipeline::{
@ -466,14 +466,12 @@ pub fn queue_material_meshes<M: Material>(
render_materials: Res<RenderMaterials<M>>,
mut render_mesh_instances: ResMut<RenderMeshInstances>,
render_material_instances: Res<RenderMaterialInstances<M>>,
images: Res<RenderAssets<Image>>,
render_lightmaps: Res<RenderLightmaps>,
mut views: Query<(
&ExtractedView,
&VisibleEntities,
Option<&Tonemapping>,
Option<&DebandDither>,
Option<&EnvironmentMapLight>,
Option<&ShadowFilteringMethod>,
Has<ScreenSpaceAmbientOcclusionSettings>,
(
@ -489,6 +487,7 @@ pub fn queue_material_meshes<M: Material>(
&mut RenderPhase<AlphaMask3d>,
&mut RenderPhase<Transmissive3d>,
&mut RenderPhase<Transparent3d>,
Has<RenderViewEnvironmentMaps>,
)>,
) where
M::Data: PartialEq + Eq + Hash + Clone,
@ -498,7 +497,6 @@ pub fn queue_material_meshes<M: Material>(
visible_entities,
tonemapping,
dither,
environment_map,
shadow_filter_method,
ssao,
(normal_prepass, depth_prepass, motion_vector_prepass, deferred_prepass),
@ -509,6 +507,7 @@ pub fn queue_material_meshes<M: Material>(
mut alpha_mask_phase,
mut transmissive_phase,
mut transparent_phase,
has_environment_maps,
) in &mut views
{
let draw_opaque_pbr = opaque_draw_functions.read().id::<DrawMaterial<M>>();
@ -539,9 +538,7 @@ pub fn queue_material_meshes<M: Material>(
view_key |= MeshPipelineKey::TEMPORAL_JITTER;
}
let environment_map_loaded = environment_map.is_some_and(|map| map.is_loaded(&images));
if environment_map_loaded {
if has_environment_maps {
view_key |= MeshPipelineKey::ENVIRONMENT_MAP;
}

19
crates/bevy_pbr/src/render/light.rs
View file

@ -196,8 +196,6 @@ pub struct GpuLights {
n_directional_lights: u32,
// offset from spot light's light index to spot light's shadow map index
spot_light_shadowmap_offset: i32,
environment_map_smallest_specular_mip_level: u32,
environment_map_intensity: f32,
}
// NOTE: this must be kept in sync with the same constants in pbr.frag
@ -645,18 +643,12 @@ pub(crate) fn spot_light_projection_matrix(angle: f32) -> Mat4 {
pub fn prepare_lights(
mut commands: Commands,
mut texture_cache: ResMut<TextureCache>,
images: Res<RenderAssets<Image>>,
render_device: Res<RenderDevice>,
render_queue: Res<RenderQueue>,
mut global_light_meta: ResMut<GlobalLightMeta>,
mut light_meta: ResMut<LightMeta>,
views: Query<
(
Entity,
&ExtractedView,
&ExtractedClusterConfig,
Option<&EnvironmentMapLight>,
),
(Entity, &ExtractedView, &ExtractedClusterConfig),
With<RenderPhase<Transparent3d>>,
>,
ambient_light: Res<AmbientLight>,
@ -890,7 +882,7 @@ pub fn prepare_lights(
.write_buffer(&render_device, &render_queue);
// set up light data for each view
for (entity, extracted_view, clusters, environment_map) in &views {
for (entity, extracted_view, clusters) in &views {
let point_light_depth_texture = texture_cache.get(
&render_device,
TextureDescriptor {
@ -957,13 +949,6 @@ pub fn prepare_lights(
// index to shadow map index, we need to subtract point light count and add directional shadowmap count.
spot_light_shadowmap_offset: num_directional_cascades_enabled as i32
- point_light_count as i32,
environment_map_smallest_specular_mip_level: environment_map
.and_then(|env_map| images.get(&env_map.specular_map))
.map(|specular_map| specular_map.mip_level_count - 1)
.unwrap_or(0),
environment_map_intensity: environment_map
.map(|env_map| env_map.intensity)
.unwrap_or(1.0),
};
// TODO: this should select lights based on relevance to the view instead of the first ones that show up in a query

32
crates/bevy_pbr/src/render/mesh.rs
View file

@ -1,3 +1,8 @@
use crate::{
MaterialBindGroupId, NotShadowCaster, NotShadowReceiver, PreviousGlobalTransform, Shadow,
ViewFogUniformOffset, ViewLightProbesUniformOffset, ViewLightsUniformOffset,
CLUSTERED_FORWARD_STORAGE_BUFFER_COUNT, MAX_CASCADES_PER_LIGHT, MAX_DIRECTIONAL_LIGHTS,
};
use bevy_app::{Plugin, PostUpdate};
use bevy_asset::{load_internal_asset, AssetId, Handle};
use bevy_core_pipeline::{
@ -21,7 +26,9 @@ use bevy_render::{
render_phase::{PhaseItem, RenderCommand, RenderCommandResult, TrackedRenderPass},
render_resource::*,
renderer::{RenderDevice, RenderQueue},
texture::*,
texture::{
BevyDefault, DefaultImageSampler, GpuImage, Image, ImageSampler, TextureFormatPixelInfo,
},
view::{ViewTarget, ViewUniformOffset, ViewVisibility},
Extract, ExtractSchedule, Render, RenderApp, RenderSet,
};
@ -48,6 +55,8 @@ use crate::render::{
};
use crate::*;
use self::environment_map::binding_arrays_are_usable;
use super::skin::SkinIndices;
#[derive(Default)]
@ -358,6 +367,12 @@ pub struct MeshPipeline {
/// ```
pub per_object_buffer_batch_size: Option<u32>,
/// Whether binding arrays (a.k.a. bindless textures) are usable on the
/// current render device.
///
/// This affects whether reflection probes can be used.
pub binding_arrays_are_usable: bool,
#[cfg(debug_assertions)]
pub did_warn_about_too_many_textures: Arc<AtomicBool>,
}
@ -416,6 +431,7 @@ impl FromWorld for MeshPipeline {
dummy_white_gpu_image,
mesh_layouts: MeshLayouts::new(&render_device),
per_object_buffer_batch_size: GpuArrayBuffer::<MeshUniform>::batch_size(&render_device),
binding_arrays_are_usable: binding_arrays_are_usable(&render_device),
#[cfg(debug_assertions)]
did_warn_about_too_many_textures: Arc::new(AtomicBool::new(false)),
}
@ -867,6 +883,10 @@ impl SpecializedMeshPipeline for MeshPipeline {
},
));
if self.binding_arrays_are_usable {
shader_defs.push("MULTIPLE_LIGHT_PROBES_IN_ARRAY".into());
}
let format = if key.contains(MeshPipelineKey::HDR) {
ViewTarget::TEXTURE_FORMAT_HDR
} else {
@ -1032,6 +1052,7 @@ impl<P: PhaseItem, const I: usize> RenderCommand<P> for SetMeshViewBindGroup<I>
Read<ViewUniformOffset>,
Read<ViewLightsUniformOffset>,
Read<ViewFogUniformOffset>,
Read<ViewLightProbesUniformOffset>,
Read<MeshViewBindGroup>,
);
type ItemData = ();
@ -1039,7 +1060,7 @@ impl<P: PhaseItem, const I: usize> RenderCommand<P> for SetMeshViewBindGroup<I>
#[inline]
fn render<'w>(
_item: &P,
(view_uniform, view_lights, view_fog, mesh_view_bind_group): ROQueryItem<
(view_uniform, view_lights, view_fog, view_light_probes, mesh_view_bind_group): ROQueryItem<
'w,
Self::ViewData,
>,
@ -1050,7 +1071,12 @@ impl<P: PhaseItem, const I: usize> RenderCommand<P> for SetMeshViewBindGroup<I>
pass.set_bind_group(
I,
&mesh_view_bind_group.value,
&[view_uniform.offset, view_lights.offset, view_fog.offset],
&[
view_uniform.offset,
view_lights.offset,
view_fog.offset,
**view_light_probes,
],
);
RenderCommandResult::Success

92
crates/bevy_pbr/src/render/mesh_view_bindings.rs
View file

@ -17,7 +17,7 @@ use bevy_render::{
render_asset::RenderAssets,
render_resource::{binding_types::*, *},
renderer::RenderDevice,
texture::{BevyDefault, FallbackImageCubemap, FallbackImageMsaa, FallbackImageZero, Image},
texture::{BevyDefault, FallbackImage, FallbackImageMsaa, FallbackImageZero, Image},
view::{Msaa, ViewUniform, ViewUniforms},
};
@ -27,9 +27,10 @@ use bevy_render::render_resource::binding_types::texture_cube;
use bevy_render::render_resource::binding_types::{texture_2d_array, texture_cube_array};
use crate::{
environment_map, prepass, EnvironmentMapLight, FogMeta, GlobalLightMeta, GpuFog, GpuLights,
GpuPointLights, LightMeta, MeshPipeline, MeshPipelineKey, ScreenSpaceAmbientOcclusionTextures,
ShadowSamplers, ViewClusterBindings, ViewShadowBindings,
environment_map::{self, RenderViewBindGroupEntries, RenderViewEnvironmentMaps},
prepass, FogMeta, GlobalLightMeta, GpuFog, GpuLights, GpuPointLights, LightMeta,
LightProbesBuffer, LightProbesUniform, MeshPipeline, MeshPipelineKey,
ScreenSpaceAmbientOcclusionTextures, ShadowSamplers, ViewClusterBindings, ViewShadowBindings,
};
#[derive(Clone)]
@ -166,6 +167,7 @@ fn buffer_layout(
fn layout_entries(
clustered_forward_buffer_binding_type: BufferBindingType,
layout_key: MeshPipelineViewLayoutKey,
render_device: &RenderDevice,
) -> Vec<BindGroupLayoutEntry> {
let mut entries = DynamicBindGroupLayoutEntries::new_with_indices(
ShaderStages::FRAGMENT,
@ -234,27 +236,29 @@ fn layout_entries(
(9, uniform_buffer::<GlobalsUniform>(false)),
// Fog
(10, uniform_buffer::<GpuFog>(true)),
// Light probes
(11, uniform_buffer::<LightProbesUniform>(true)),
// Screen space ambient occlusion texture
(
11,
12,
texture_2d(TextureSampleType::Float { filterable: false }),
),
),
);
// EnvironmentMapLight
let environment_map_entries = environment_map::get_bind_group_layout_entries();
let environment_map_entries = environment_map::get_bind_group_layout_entries(render_device);
entries = entries.extend_with_indices((
(12, environment_map_entries[0]),
(13, environment_map_entries[1]),
(14, environment_map_entries[2]),
(13, environment_map_entries[0]),
(14, environment_map_entries[1]),
(15, environment_map_entries[2]),
));
// Tonemapping
let tonemapping_lut_entries = get_lut_bind_group_layout_entries();
entries = entries.extend_with_indices((
(15, tonemapping_lut_entries[0]),
(16, tonemapping_lut_entries[1]),
(16, tonemapping_lut_entries[0]),
(17, tonemapping_lut_entries[1]),
));
// Prepass
@ -264,7 +268,7 @@ fn layout_entries(
{
for (entry, binding) in prepass::get_bind_group_layout_entries(layout_key)
.iter()
.zip([17, 18, 19, 20])
.zip([18, 19, 20, 21])
{
if let Some(entry) = entry {
entries = entries.extend_with_indices(((binding as u32, *entry),));
@ -275,10 +279,10 @@ fn layout_entries(
// View Transmission Texture
entries = entries.extend_with_indices((
(
21,
22,
texture_2d(TextureSampleType::Float { filterable: true }),
),
(22, sampler(SamplerBindingType::Filtering)),
(23, sampler(SamplerBindingType::Filtering)),
));
entries.to_vec()
@ -292,7 +296,7 @@ pub fn generate_view_layouts(
) -> [MeshPipelineViewLayout; MeshPipelineViewLayoutKey::COUNT] {
array::from_fn(|i| {
let key = MeshPipelineViewLayoutKey::from_bits_truncate(i as u32);
let entries = layout_entries(clustered_forward_buffer_binding_type, key);
let entries = layout_entries(clustered_forward_buffer_binding_type, key, render_device);
#[cfg(debug_assertions)]
let texture_count: usize = entries
@ -331,18 +335,19 @@ pub fn prepare_mesh_view_bind_groups(
Option<&ScreenSpaceAmbientOcclusionTextures>,
Option<&ViewPrepassTextures>,
Option<&ViewTransmissionTexture>,
Option<&EnvironmentMapLight>,
&Tonemapping,
Option<&RenderViewEnvironmentMaps>,
)>,
(images, mut fallback_images, fallback_cubemap, fallback_image_zero): (
(images, mut fallback_images, fallback_image, fallback_image_zero): (
Res<RenderAssets<Image>>,
FallbackImageMsaa,
Res<FallbackImageCubemap>,
Res<FallbackImage>,
Res<FallbackImageZero>,
),
msaa: Res<Msaa>,
globals_buffer: Res<GlobalsBuffer>,
tonemapping_luts: Res<TonemappingLuts>,
light_probes_buffer: Res<LightProbesBuffer>,
) {
if let (
Some(view_binding),
@ -350,12 +355,14 @@ pub fn prepare_mesh_view_bind_groups(
Some(point_light_binding),
Some(globals),
Some(fog_binding),
Some(light_probes_binding),
) = (
view_uniforms.uniforms.binding(),
light_meta.view_gpu_lights.binding(),
global_light_meta.gpu_point_lights.binding(),
globals_buffer.buffer.binding(),
fog_meta.gpu_fogs.binding(),
light_probes_buffer.binding(),
) {
for (
entity,
@ -364,8 +371,8 @@ pub fn prepare_mesh_view_bind_groups(
ssao_textures,
prepass_textures,
transmission_texture,
environment_map,
tonemapping,
render_view_environment_maps,
) in &views
{
let fallback_ssao = fallback_images
@ -393,19 +400,44 @@ pub fn prepare_mesh_view_bind_groups(
(8, cluster_bindings.offsets_and_counts_binding().unwrap()),
(9, globals.clone()),
(10, fog_binding.clone()),
(11, ssao_view),
(11, light_probes_binding.clone()),
(12, ssao_view),
));
let env_map_bindings =
environment_map::get_bindings(environment_map, &images, &fallback_cubemap);
entries = entries.extend_with_indices((
(12, env_map_bindings.0),
(13, env_map_bindings.1),
(14, env_map_bindings.2),
));
let bind_group_entries = RenderViewBindGroupEntries::get(
render_view_environment_maps,
&images,
&fallback_image,
&render_device,
);
match bind_group_entries {
RenderViewBindGroupEntries::Single {
diffuse_texture_view,
specular_texture_view,
sampler,
} => {
entries = entries.extend_with_indices((
(13, diffuse_texture_view),
(14, specular_texture_view),
(15, sampler),
));
}
RenderViewBindGroupEntries::Multiple {
ref diffuse_texture_views,
ref specular_texture_views,
sampler,
} => {
entries = entries.extend_with_indices((
(13, diffuse_texture_views.as_slice()),
(14, specular_texture_views.as_slice()),
(15, sampler),
));
}
}
let lut_bindings = get_lut_bindings(&images, &tonemapping_luts, tonemapping);
entries = entries.extend_with_indices(((15, lut_bindings.0), (16, lut_bindings.1)));
entries = entries.extend_with_indices(((16, lut_bindings.0), (17, lut_bindings.1)));
// When using WebGL, we can't have a depth texture with multisampling
let prepass_bindings;
@ -415,7 +447,7 @@ pub fn prepare_mesh_view_bind_groups(
for (binding, index) in prepass_bindings
.iter()
.map(Option::as_ref)
.zip([17, 18, 19, 20])
.zip([18, 19, 20, 21])
.flat_map(|(b, i)| b.map(|b| (b, i)))
{
entries = entries.extend_with_indices(((index, binding),));
@ -431,7 +463,7 @@ pub fn prepare_mesh_view_bind_groups(
.unwrap_or(&fallback_image_zero.sampler);
entries =
entries.extend_with_indices(((21, transmission_view), (22, transmission_sampler)));
entries.extend_with_indices(((22, transmission_view), (23, transmission_sampler)));
commands.entity(entity).insert(MeshViewBindGroup {
value: render_device.create_bind_group("mesh_view_bind_group", layout, &entries),

36
crates/bevy_pbr/src/render/mesh_view_bindings.wgsl
View file

@ -33,44 +33,50 @@
@group(0) @binding(9) var<uniform> globals: Globals;
@group(0) @binding(10) var<uniform> fog: types::Fog;
@group(0) @binding(11) var<uniform> light_probes: types::LightProbes;
@group(0) @binding(11) var screen_space_ambient_occlusion_texture: texture_2d<f32>;
@group(0) @binding(12) var screen_space_ambient_occlusion_texture: texture_2d<f32>;
@group(0) @binding(12) var environment_map_diffuse: texture_cube<f32>;
@group(0) @binding(13) var environment_map_specular: texture_cube<f32>;
@group(0) @binding(14) var environment_map_sampler: sampler;
#ifdef MULTIPLE_LIGHT_PROBES_IN_ARRAY
@group(0) @binding(13) var diffuse_environment_maps: binding_array<texture_cube<f32>, 8u>;
@group(0) @binding(14) var specular_environment_maps: binding_array<texture_cube<f32>, 8u>;
#else
@group(0) @binding(13) var diffuse_environment_map: texture_cube<f32>;
@group(0) @binding(14) var specular_environment_map: texture_cube<f32>;
#endif
@group(0) @binding(15) var environment_map_sampler: sampler;
@group(0) @binding(15) var dt_lut_texture: texture_3d<f32>;
@group(0) @binding(16) var dt_lut_sampler: sampler;
@group(0) @binding(16) var dt_lut_texture: texture_3d<f32>;
@group(0) @binding(17) var dt_lut_sampler: sampler;
#ifdef MULTISAMPLED
#ifdef DEPTH_PREPASS
@group(0) @binding(17) var depth_prepass_texture: texture_depth_multisampled_2d;
@group(0) @binding(18) var depth_prepass_texture: texture_depth_multisampled_2d;
#endif // DEPTH_PREPASS
#ifdef NORMAL_PREPASS
@group(0) @binding(18) var normal_prepass_texture: texture_multisampled_2d<f32>;
@group(0) @binding(19) var normal_prepass_texture: texture_multisampled_2d<f32>;
#endif // NORMAL_PREPASS
#ifdef MOTION_VECTOR_PREPASS
@group(0) @binding(19) var motion_vector_prepass_texture: texture_multisampled_2d<f32>;
@group(0) @binding(20) var motion_vector_prepass_texture: texture_multisampled_2d<f32>;
#endif // MOTION_VECTOR_PREPASS
#else // MULTISAMPLED
#ifdef DEPTH_PREPASS
@group(0) @binding(17) var depth_prepass_texture: texture_depth_2d;
@group(0) @binding(18) var depth_prepass_texture: texture_depth_2d;
#endif // DEPTH_PREPASS
#ifdef NORMAL_PREPASS
@group(0) @binding(18) var normal_prepass_texture: texture_2d<f32>;
@group(0) @binding(19) var normal_prepass_texture: texture_2d<f32>;
#endif // NORMAL_PREPASS
#ifdef MOTION_VECTOR_PREPASS
@group(0) @binding(19) var motion_vector_prepass_texture: texture_2d<f32>;
@group(0) @binding(20) var motion_vector_prepass_texture: texture_2d<f32>;
#endif // MOTION_VECTOR_PREPASS
#endif // MULTISAMPLED
#ifdef DEFERRED_PREPASS
@group(0) @binding(20) var deferred_prepass_texture: texture_2d<u32>;
@group(0) @binding(21) var deferred_prepass_texture: texture_2d<u32>;
#endif // DEFERRED_PREPASS
@group(0) @binding(21) var view_transmission_texture: texture_2d<f32>;
@group(0) @binding(22) var view_transmission_sampler: sampler;
@group(0) @binding(22) var view_transmission_texture: texture_2d<f32>;
@group(0) @binding(23) var view_transmission_sampler: sampler;

22
crates/bevy_pbr/src/render/mesh_view_types.wgsl
View file

@ -110,3 +110,25 @@ struct ClusterOffsetsAndCounts {
data: array<vec4<u32>, 1024u>,
};
#endif
struct ReflectionProbe {
// This is stored as the transpose in order to save space in this structure.
// It'll be transposed in the `environment_map_light` function.
inverse_transpose_transform: mat3x4<f32>,
cubemap_index: i32,
intensity: f32,
};
struct LightProbes {
// This must match `MAX_VIEW_REFLECTION_PROBES` on the Rust side.
reflection_probes: array<ReflectionProbe, 8u>,
reflection_probe_count: i32,
// The index of the view environment map cubemap binding, or -1 if there's
// no such cubemap.
view_cubemap_index: i32,
// The smallest valid mipmap level for the specular environment cubemap
// associated with the view.
smallest_specular_mip_level_for_view: u32,
// The intensity of the environment map associated with the view.
intensity_for_view: f32,
};

22
crates/bevy_pbr/src/render/pbr_functions.wgsl
View file

@ -323,7 +323,16 @@ fn apply_pbr_lighting(
// Environment map light (indirect)
#ifdef ENVIRONMENT_MAP
let environment_light = environment_map::environment_map_light(perceptual_roughness, roughness, diffuse_color, NdotV, f_ab, in.N, R, F0);
let environment_light = environment_map::environment_map_light(
perceptual_roughness,
roughness,
diffuse_color,
NdotV,
f_ab,
in.N,
R,
F0,
in.world_position.xyz);
indirect_light += (environment_light.diffuse * diffuse_occlusion) + (environment_light.specular * specular_occlusion);
// we'll use the specular component of the transmitted environment
@ -349,7 +358,16 @@ fn apply_pbr_lighting(
refract(in.V, -in.N, 1.0 / ior) * thickness // add refracted vector scaled by thickness, towards exit point
); // normalize to find exit point view vector
let transmitted_environment_light = bevy_pbr::environment_map::environment_map_light(perceptual_roughness, roughness, vec3<f32>(1.0), 1.0, f_ab, -in.N, T, vec3<f32>(1.0));
let transmitted_environment_light = bevy_pbr::environment_map::environment_map_light(
perceptual_roughness,
roughness,
vec3<f32>(1.0),
1.0,
f_ab,
-in.N,
T,
vec3<f32>(1.0),
in.world_position.xyz);
transmitted_light += transmitted_environment_light.diffuse * diffuse_transmissive_color;
specular_transmitted_environment_light = transmitted_environment_light.specular * specular_transmissive_color;
}

6
crates/bevy_render/Cargo.toml
View file

@ -20,7 +20,7 @@ dds = ["ddsfile"]
pnm = ["image/pnm"]
bevy_ci_testing = ["bevy_app/bevy_ci_testing"]
shader_format_glsl = ["naga/glsl-in", "naga/wgsl-out"]
shader_format_glsl = ["naga/glsl-in", "naga/wgsl-out", "naga_oil/glsl"]
shader_format_spirv = ["wgpu/spirv", "naga/spv-in", "naga/spv-out"]
# For ktx2 supercompression
@ -67,7 +67,9 @@ wgpu = { version = "0.18", features = [
"fragile-send-sync-non-atomic-wasm",
] }
naga = { version = "0.14.2", features = ["wgsl-in"] }
naga_oil = "0.11"
naga_oil = { version = "0.11", default-features = false, features = [
"test_shader",
] }
serde = { version = "1", features = ["derive"] }
bitflags = "2.3"
bytemuck = { version = "1.5", features = ["derive"] }

22
crates/bevy_render/src/render_resource/shader.rs
View file

@ -213,7 +213,12 @@ impl From<&Source> for naga_oil::compose::ShaderLanguage {
fn from(value: &Source) -> Self {
match value {
Source::Wgsl(_) => naga_oil::compose::ShaderLanguage::Wgsl,
#[cfg(any(feature = "shader_format_glsl", target_arch = "wasm32"))]
Source::Glsl(_, _) => naga_oil::compose::ShaderLanguage::Glsl,
#[cfg(all(not(feature = "shader_format_glsl"), not(target_arch = "wasm32")))]
Source::Glsl(_, _) => panic!(
"GLSL is not supported in this configuration; use the feature `shader_format_glsl`"
),
Source::SpirV(_) => panic!("spirv not yet implemented"),
}
}
@ -223,13 +228,16 @@ impl From<&Source> for naga_oil::compose::ShaderType {
fn from(value: &Source) -> Self {
match value {
Source::Wgsl(_) => naga_oil::compose::ShaderType::Wgsl,
Source::Glsl(_, naga::ShaderStage::Vertex) => naga_oil::compose::ShaderType::GlslVertex,
Source::Glsl(_, naga::ShaderStage::Fragment) => {
naga_oil::compose::ShaderType::GlslFragment
}
Source::Glsl(_, naga::ShaderStage::Compute) => {
panic!("glsl compute not yet implemented")
}
#[cfg(any(feature = "shader_format_glsl", target_arch = "wasm32"))]
Source::Glsl(_, shader_stage) => match shader_stage {
naga::ShaderStage::Vertex => naga_oil::compose::ShaderType::GlslVertex,
naga::ShaderStage::Fragment => naga_oil::compose::ShaderType::GlslFragment,
naga::ShaderStage::Compute => panic!("glsl compute not yet implemented"),
},
#[cfg(all(not(feature = "shader_format_glsl"), not(target_arch = "wasm32")))]
Source::Glsl(_, _) => panic!(
"GLSL is not supported in this configuration; use the feature `shader_format_glsl`"
),
Source::SpirV(_) => panic!("spirv not yet implemented"),
}
}

363
examples/3d/reflection_probes.rs Normal file
View file

@ -0,0 +1,363 @@
//! This example shows how to place reflection probes in the scene.
//!
//! Press Space to switch between no reflections, environment map reflections
//! (i.e. the skybox only, not the cubes), and a full reflection probe that
//! reflects the skybox and the cubes. Press Enter to pause rotation.
//!
//! Reflection probes don't work on WebGL 2 or WebGPU.
use bevy::core_pipeline::Skybox;
use bevy::prelude::*;
use std::fmt::{Display, Formatter, Result as FmtResult};
// Rotation speed in radians per frame.
const ROTATION_SPEED: f32 = 0.005;
static STOP_ROTATION_HELP_TEXT: &str = "Press Enter to stop rotation";
static START_ROTATION_HELP_TEXT: &str = "Press Enter to start rotation";
static REFLECTION_MODE_HELP_TEXT: &str = "Press Space to switch reflection mode";
// The mode the application is in.
#[derive(Resource)]
struct AppStatus {
// Which environment maps the user has requested to display.
reflection_mode: ReflectionMode,
// Whether the user has requested the scene to rotate.
rotating: bool,
}
// Which environment maps the user has requested to display.
#[derive(Clone, Copy)]
enum ReflectionMode {
// No environment maps are shown.
None = 0,
// Only a world environment map is shown.
EnvironmentMap = 1,
// Both a world environment map and a reflection probe are present. The
// reflection probe is shown in the sphere.
ReflectionProbe = 2,
}
// The various reflection maps.
#[derive(Resource)]
struct Cubemaps {
// The blurry diffuse cubemap. This is used for both the world environment
// map and the reflection probe. (In reality you wouldn't do this, but this
// reduces complexity of this example a bit.)
diffuse: Handle<Image>,
// The specular cubemap that reflects the world, but not the cubes.
specular_environment_map: Handle<Image>,
// The specular cubemap that reflects both the world and the cubes.
specular_reflection_probe: Handle<Image>,
// The skybox cubemap image. This is almost the same as
// `specular_environment_map`.
skybox: Handle<Image>,
}
fn main() {
// Create the app.
App::new()
.add_plugins(DefaultPlugins)
.init_resource::<AppStatus>()
.init_resource::<Cubemaps>()
.add_systems(Startup, setup)
.add_systems(PreUpdate, add_environment_map_to_camera)
.add_systems(Update, change_reflection_type)
.add_systems(Update, toggle_rotation)
.add_systems(
Update,
rotate_camera
.after(toggle_rotation)
.after(change_reflection_type),
)
.add_systems(Update, update_text.after(rotate_camera))
.run();
}
// Spawns all the scene objects.
fn setup(
mut commands: Commands,
mut meshes: ResMut<Assets<Mesh>>,
mut materials: ResMut<Assets<StandardMaterial>>,
asset_server: Res<AssetServer>,
app_status: Res<AppStatus>,
cubemaps: Res<Cubemaps>,
) {
spawn_scene(&mut commands, &asset_server);
spawn_camera(&mut commands);
spawn_sphere(&mut commands, &mut meshes, &mut materials);
spawn_reflection_probe(&mut commands, &cubemaps);
spawn_text(&mut commands, &asset_server, &app_status);
}
// Spawns the cubes, light, and camera.
fn spawn_scene(commands: &mut Commands, asset_server: &AssetServer) {
commands.spawn(SceneBundle {
scene: asset_server.load("models/cubes/Cubes.glb#Scene0"),
..SceneBundle::default()
});
}
// Spawns the camera.
fn spawn_camera(commands: &mut Commands) {
commands.spawn(Camera3dBundle {
camera: Camera {
hdr: true,
..default()
},
transform: Transform::from_xyz(-6.483, 0.325, 4.381).looking_at(Vec3::ZERO, Vec3::Y),
..default()
});
}
// Creates the sphere mesh and spawns it.
fn spawn_sphere(
commands: &mut Commands,
meshes: &mut Assets<Mesh>,
materials: &mut Assets<StandardMaterial>,
) {
// Create a sphere mesh.
let sphere_mesh = meshes.add(
Mesh::try_from(shape::Icosphere {
radius: 1.0,
subdivisions: 7,
})
.unwrap(),
);
// Create a sphere.
commands.spawn(PbrBundle {
mesh: sphere_mesh.clone(),
material: materials.add(StandardMaterial {
base_color: Color::hex("#ffd891").unwrap(),
metallic: 1.0,
perceptual_roughness: 0.0,
..StandardMaterial::default()
}),
transform: Transform::default(),
..PbrBundle::default()
});
}
// Spawns the reflection probe.
fn spawn_reflection_probe(commands: &mut Commands, cubemaps: &Cubemaps) {
commands.spawn(ReflectionProbeBundle {
spatial: SpatialBundle {
// 2.0 because the sphere's radius is 1.0 and we want to fully enclose it.
transform: Transform::from_scale(Vec3::splat(2.0)),
..SpatialBundle::default()
},
light_probe: LightProbe,
environment_map: EnvironmentMapLight {
diffuse_map: cubemaps.diffuse.clone(),
specular_map: cubemaps.specular_reflection_probe.clone(),
intensity: 150.0,
},
});
}
// Spawns the help text.
fn spawn_text(commands: &mut Commands, asset_server: &AssetServer, app_status: &AppStatus) {
// Create the text.
commands.spawn(
TextBundle {
text: app_status.create_text(asset_server),
..TextBundle::default()
}
.with_style(Style {
position_type: PositionType::Absolute,
bottom: Val::Px(10.0),
left: Val::Px(10.0),
..default()
}),
);
}
// Adds a world environment map to the camera. This separate system is needed because the camera is
// managed by the scene spawner, as it's part of the glTF file with the cubes, so we have to add
// the environment map after the fact.
fn add_environment_map_to_camera(
mut commands: Commands,
query: Query<Entity, Added<Camera3d>>,
cubemaps: Res<Cubemaps>,
) {
for camera_entity in query.iter() {
commands
.entity(camera_entity)
.insert(create_camera_environment_map_light(&cubemaps))
.insert(Skybox {
image: cubemaps.skybox.clone(),
brightness: 150.0,
});
}
}
// A system that handles switching between different reflection modes.
fn change_reflection_type(
mut commands: Commands,
light_probe_query: Query<Entity, With<LightProbe>>,
camera_query: Query<Entity, With<Camera3d>>,
keyboard: Res<ButtonInput<KeyCode>>,
mut app_status: ResMut<AppStatus>,
cubemaps: Res<Cubemaps>,
) {
// Only do anything if space was pressed.
if !keyboard.just_pressed(KeyCode::Space) {
return;
}
// Switch reflection mode.
app_status.reflection_mode =
ReflectionMode::try_from((app_status.reflection_mode as u32 + 1) % 3).unwrap();
// Add or remove the light probe.
for light_probe in light_probe_query.iter() {
commands.entity(light_probe).despawn();
}
match app_status.reflection_mode {
ReflectionMode::None | ReflectionMode::EnvironmentMap => {}
ReflectionMode::ReflectionProbe => spawn_reflection_probe(&mut commands, &cubemaps),
}
// Add or remove the environment map from the camera.
for camera in camera_query.iter() {
match app_status.reflection_mode {
ReflectionMode::None => {
commands.entity(camera).remove::<EnvironmentMapLight>();
}
ReflectionMode::EnvironmentMap | ReflectionMode::ReflectionProbe => {
commands
.entity(camera)
.insert(create_camera_environment_map_light(&cubemaps));
}
}
}
}
// A system that handles enabling and disabling rotation.
fn toggle_rotation(keyboard: Res<ButtonInput<KeyCode>>, mut app_status: ResMut<AppStatus>) {
if keyboard.just_pressed(KeyCode::Enter) {
app_status.rotating = !app_status.rotating;
}
}
// A system that updates the help text.
fn update_text(
mut text_query: Query<&mut Text>,
app_status: Res<AppStatus>,
asset_server: Res<AssetServer>,
) {
for mut text in text_query.iter_mut() {
*text = app_status.create_text(&asset_server);
}
}
impl TryFrom<u32> for ReflectionMode {
type Error = ();
fn try_from(value: u32) -> Result<Self, Self::Error> {
match value {
0 => Ok(ReflectionMode::None),
1 => Ok(ReflectionMode::EnvironmentMap),
2 => Ok(ReflectionMode::ReflectionProbe),
_ => Err(()),
}
}
}
impl Display for ReflectionMode {
fn fmt(&self, formatter: &mut Formatter<'_>) -> FmtResult {
let text = match *self {
ReflectionMode::None => "No reflections",
ReflectionMode::EnvironmentMap => "Environment map",
ReflectionMode::ReflectionProbe => "Reflection probe",
};
formatter.write_str(text)
}
}
impl AppStatus {
// Constructs the help text at the bottom of the screen based on the
// application status.
fn create_text(&self, asset_server: &AssetServer) -> Text {
let rotation_help_text = if self.rotating {
STOP_ROTATION_HELP_TEXT
} else {
START_ROTATION_HELP_TEXT
};
Text::from_section(
format!(
"{}\n{}\n{}",
self.reflection_mode, rotation_help_text, REFLECTION_MODE_HELP_TEXT
),
TextStyle {
font: asset_server.load("fonts/FiraMono-Medium.ttf"),
font_size: 24.0,
color: Color::ANTIQUE_WHITE,
},
)
}
}
// Creates the world environment map light, used as a fallback if no reflection
// probe is applicable to a mesh.
fn create_camera_environment_map_light(cubemaps: &Cubemaps) -> EnvironmentMapLight {
EnvironmentMapLight {
diffuse_map: cubemaps.diffuse.clone(),
specular_map: cubemaps.specular_environment_map.clone(),
intensity: 150.0,
}
}
// Rotates the camera a bit every frame.
fn rotate_camera(
mut camera_query: Query<&mut Transform, With<Camera3d>>,
app_status: Res<AppStatus>,
) {
if !app_status.rotating {
return;
}
for mut transform in camera_query.iter_mut() {
transform.translation = Vec2::from_angle(ROTATION_SPEED)
.rotate(transform.translation.xz())
.extend(transform.translation.y)
.xzy();
transform.look_at(Vec3::ZERO, Vec3::Y);
}
}
// Loads the cubemaps from the assets directory.
impl FromWorld for Cubemaps {
fn from_world(world: &mut World) -> Self {
let asset_server = world.resource::<AssetServer>();
// Just use the specular map for the skybox since it's not too blurry.
// In reality you wouldn't do this--you'd use a real skybox texture--but
// reusing the textures like this saves space in the Bevy repository.
let specular_map = asset_server.load("environment_maps/pisa_specular_rgb9e5_zstd.ktx2");
Cubemaps {
diffuse: asset_server.load("environment_maps/pisa_diffuse_rgb9e5_zstd.ktx2"),
specular_reflection_probe: asset_server
.load("environment_maps/cubes_reflection_probe_specular_rgb9e5_zstd.ktx2"),
specular_environment_map: specular_map.clone(),
skybox: specular_map,
}
}
}
impl Default for AppStatus {
fn default() -> Self {
Self {
reflection_mode: ReflectionMode::ReflectionProbe,
rotating: true,
}
}
}

1
examples/README.md
View file

@ -136,6 +136,7 @@ Example | Description
[Parallax Mapping](../examples/3d/parallax_mapping.rs) | Demonstrates use of a normal map and depth map for parallax mapping
[Parenting](../examples/3d/parenting.rs) | Demonstrates parent->child relationships and relative transformations
[Physically Based Rendering](../examples/3d/pbr.rs) | Demonstrates use of Physically Based Rendering (PBR) properties
[Reflection Probes](../examples/3d/reflection_probes.rs) | Demonstrates reflection probes
[Render to Texture](../examples/3d/render_to_texture.rs) | Shows how to render to a texture, useful for mirrors, UI, or exporting images
[Screen Space Ambient Occlusion](../examples/3d/ssao.rs) | A scene showcasing screen space ambient occlusion
[Shadow Biases](../examples/3d/shadow_biases.rs) | Demonstrates how shadow biases affect shadows in a 3d scene