Mirrors/bevy
2
0
Fork 0
mirror of https://github.com/bevyengine/bevy synced 2024-11-21 12:13:25 +00:00
Code Issues Projects Releases Packages Wiki Activity

Implement irradiance volumes. (#10268)

Browse source 
# Objective

Bevy could benefit from *irradiance volumes*, also known as *voxel
global illumination* or simply as light probes (though this term is not
preferred, as multiple techniques can be called light probes).
Irradiance volumes are a form of baked global illumination; they work by
sampling the light at the centers of each voxel within a cuboid. At
runtime, the voxels surrounding the fragment center are sampled and
interpolated to produce indirect diffuse illumination.

## Solution

This is divided into two sections. The first is copied and pasted from
the irradiance volume module documentation and describes the technique.
The second part consists of notes on the implementation.

### Overview

An *irradiance volume* is a cuboid voxel region consisting of
regularly-spaced precomputed samples of diffuse indirect light. They're
ideal if you have a dynamic object such as a character that can move
about
static non-moving geometry such as a level in a game, and you want that
dynamic object to be affected by the light bouncing off that static
geometry.

To use irradiance volumes, you need to precompute, or *bake*, the
indirect
light in your scene. Bevy doesn't currently come with a way to do this.
Fortunately, [Blender] provides a [baking tool] as part of the Eevee
renderer, and its irradiance volumes are compatible with those used by
Bevy.
The [`bevy-baked-gi`] project provides a tool, `export-blender-gi`, that
can
extract the baked irradiance volumes from the Blender `.blend` file and
package them up into a `.ktx2` texture for use by the engine. See the
documentation in the `bevy-baked-gi` project for more details as to this
workflow.

Like all light probes in Bevy, irradiance volumes are 1×1×1 cubes that
can
be arbitrarily scaled, rotated, and positioned in a scene with the
[`bevy_transform::components::Transform`] component. The 3D voxel grid
will
be stretched to fill the interior of the cube, and the illumination from
the
irradiance volume will apply to all fragments within that bounding
region.

Bevy's irradiance volumes are based on Valve's [*ambient cubes*] as used
in
*Half-Life 2* ([Mitchell 2006], slide 27). These encode a single color
of
light from the six 3D cardinal directions and blend the sides together
according to the surface normal.

The primary reason for choosing ambient cubes is to match Blender, so
that
its Eevee renderer can be used for baking. However, they also have some
advantages over the common second-order spherical harmonics approach:
ambient cubes don't suffer from ringing artifacts, they are smaller (6
colors for ambient cubes as opposed to 9 for spherical harmonics), and
evaluation is faster. A smaller basis allows for a denser grid of voxels
with the same storage requirements.

If you wish to use a tool other than `export-blender-gi` to produce the
irradiance volumes, you'll need to pack the irradiance volumes in the
following format. The irradiance volume of resolution *(Rx, Ry, Rz)* is
expected to be a 3D texture of dimensions *(Rx, 2Ry, 3Rz)*. The
unnormalized
texture coordinate *(s, t, p)* of the voxel at coordinate *(x, y, z)*
with
side *S* ∈ *{-X, +X, -Y, +Y, -Z, +Z}* is as follows:

```text
s = x

t = y + ⎰  0 if S ∈ {-X, -Y, -Z}
        ⎱ Ry if S ∈ {+X, +Y, +Z}

        ⎧   0 if S ∈ {-X, +X}
p = z + ⎨  Rz if S ∈ {-Y, +Y}
        ⎩ 2Rz if S ∈ {-Z, +Z}
```

Visually, in a left-handed coordinate system with Y up, viewed from the
right, the 3D texture looks like a stacked series of voxel grids, one
for
each cube side, in this order:

| **+X** | **+Y** | **+Z** |
| ------ | ------ | ------ |
| **-X** | **-Y** | **-Z** |

A terminology note: Other engines may refer to irradiance volumes as
*voxel
global illumination*, *VXGI*, or simply as *light probes*. Sometimes
*light
probe* refers to what Bevy calls a reflection probe. In Bevy, *light
probe*
is a generic term that encompasses all cuboid bounding regions that
capture
indirect illumination, whether based on voxels or not.

Note that, if binding arrays aren't supported (e.g. on WebGPU or WebGL
2),
then only the closest irradiance volume to the view will be taken into
account during rendering.

[*ambient cubes*]:
https://advances.realtimerendering.com/s2006/Mitchell-ShadingInValvesSourceEngine.pdf

[Mitchell 2006]:
https://advances.realtimerendering.com/s2006/Mitchell-ShadingInValvesSourceEngine.pdf

[Blender]: http://blender.org/

[baking tool]:
https://docs.blender.org/manual/en/latest/render/eevee/render_settings/indirect_lighting.html

[`bevy-baked-gi`]: https://github.com/pcwalton/bevy-baked-gi

### Implementation notes

This patch generalizes light probes so as to reuse as much code as
possible between irradiance volumes and the existing reflection probes.
This approach was chosen because both techniques share numerous
similarities:

1. Both irradiance volumes and reflection probes are cuboid bounding
regions.
2. Both are responsible for providing baked indirect light.
3. Both techniques involve presenting a variable number of textures to
the shader from which indirect light is sampled. (In the current
implementation, this uses binding arrays.)
4. Both irradiance volumes and reflection probes require gathering and
sorting probes by distance on CPU.
5. Both techniques require the GPU to search through a list of bounding
regions.
6. Both will eventually want to have falloff so that we can smoothly
blend as objects enter and exit the probes' influence ranges. (This is
not implemented yet to keep this patch relatively small and reviewable.)

To do this, we generalize most of the methods in the reflection probes
patch #11366 to be generic over a trait, `LightProbeComponent`. This
trait is implemented by both `EnvironmentMapLight` (for reflection
probes) and `IrradianceVolume` (for irradiance volumes). Using a trait
will allow us to add more types of light probes in the future. In
particular, I highly suspect we will want real-time reflection planes
for mirrors in the future, which can be easily slotted into this
framework.

## Changelog

> This section is optional. If this was a trivial fix, or has no
externally-visible impact, you can delete this section.

### Added
* A new `IrradianceVolume` asset type is available for baked voxelized
light probes. You can bake the global illumination using Blender or
another tool of your choice and use it in Bevy to apply indirect
illumination to dynamic objects.
This commit is contained in:
Patrick Walton 2024-02-06 15:23:20 -08:00 committed by GitHub
parent cf15e6bba3
commit 4c15dd0fc5
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
20 changed files with 1923 additions and 441 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
Cargo.toml
View file

@ -935,6 +935,17 @@ description = "Showcases wireframe rendering"
category = "3D Rendering"
wasm = false
[[example]]
name = "irradiance_volumes"
path = "examples/3d/irradiance_volumes.rs"
doc-scrape-examples = true
[package.metadata.example.irradiance_volumes]
name = "Irradiance Volumes"
description = "Demonstrates irradiance volumes"
category = "3D Rendering"
wasm = false
[[example]]
name = "lightmaps"
path = "examples/3d/lightmaps.rs"

BIN
assets/irradiance_volumes/Example.vxgi.ktx2 Normal file
View file

Binary file not shown.

BIN
assets/models/IrradianceVolumeExample/IrradianceVolumeExample.glb Normal file
View file

Binary file not shown.

35
assets/shaders/irradiance_volume_voxel_visualization.wgsl Normal file
View file

@ -0,0 +1,35 @@
#import bevy_pbr::forward_io::VertexOutput
#import bevy_pbr::irradiance_volume
#import bevy_pbr::mesh_view_bindings
struct VoxelVisualizationIrradianceVolumeInfo {
transform: mat4x4<f32>,
inverse_transform: mat4x4<f32>,
resolution: vec3<u32>,
// A scale factor that's applied to the diffuse and specular light from the
// light probe. This is in units of cd/m² (candela per square meter).
intensity: f32,
}
@group(2) @binding(100)
var<uniform> irradiance_volume_info: VoxelVisualizationIrradianceVolumeInfo;
@fragment
fn fragment(mesh: VertexOutput) -> @location(0) vec4<f32> {
// Snap the world position we provide to `irradiance_volume_light()` to the
// middle of the nearest texel.
var unit_pos = (irradiance_volume_info.inverse_transform *
vec4(mesh.world_position.xyz, 1.0f)).xyz;
let resolution = vec3<f32>(irradiance_volume_info.resolution);
let stp = clamp((unit_pos + 0.5) * resolution, vec3(0.5f), resolution - vec3(0.5f));
let stp_rounded = round(stp - 0.5f) + 0.5f;
let rounded_world_pos = (irradiance_volume_info.transform * vec4(stp_rounded, 1.0f)).xyz;
// `irradiance_volume_light()` multiplies by intensity, so cancel it out.
// If we take intensity into account, the cubes will be way too bright.
let rgb = irradiance_volume::irradiance_volume_light(
mesh.world_position.xyz,
mesh.world_normal) / irradiance_volume_info.intensity;
return vec4<f32>(rgb, 1.0f);
}

4
crates/bevy_core_pipeline/src/tonemapping/tonemapping_shared.wgsl
View file

@ -7,8 +7,8 @@
@group(0) @binding(3) var dt_lut_texture: texture_3d<f32>;
@group(0) @binding(4) var dt_lut_sampler: sampler;
#else
@group(0) @binding(16) var dt_lut_texture: texture_3d<f32>;
@group(0) @binding(17) var dt_lut_sampler: sampler;
@group(0) @binding(18) var dt_lut_texture: texture_3d<f32>;
@group(0) @binding(19) var dt_lut_sampler: sampler;
#endif
fn sample_current_lut(p: vec3<f32>) -> vec3<f32> {

17
crates/bevy_pbr/src/deferred/mod.rs
View file

@ -1,6 +1,7 @@
use crate::{
environment_map::RenderViewEnvironmentMaps, graph::LabelsPbr, MeshPipeline, MeshViewBindGroup,
ScreenSpaceAmbientOcclusionSettings, ViewLightProbesUniformOffset,
graph::LabelsPbr, irradiance_volume::IrradianceVolume, prelude::EnvironmentMapLight,
MeshPipeline, MeshViewBindGroup, RenderViewLightProbes, ScreenSpaceAmbientOcclusionSettings,
ViewLightProbesUniformOffset,
};
use bevy_app::prelude::*;
use bevy_asset::{load_internal_asset, Handle};
@ -284,6 +285,10 @@ impl SpecializedRenderPipeline for DeferredLightingLayout {
shader_defs.push("ENVIRONMENT_MAP".into());
}
if key.contains(MeshPipelineKey::IRRADIANCE_VOLUME) {
shader_defs.push("IRRADIANCE_VOLUME".into());
}
if key.contains(MeshPipelineKey::NORMAL_PREPASS) {
shader_defs.push("NORMAL_PREPASS".into());
}
@ -407,7 +412,8 @@ pub fn prepare_deferred_lighting_pipelines(
Has<DepthPrepass>,
Has<MotionVectorPrepass>,
),
Has<RenderViewEnvironmentMaps>,
Has<RenderViewLightProbes<EnvironmentMapLight>>,
Has<RenderViewLightProbes<IrradianceVolume>>,
),
With<DeferredPrepass>,
>,
@ -421,6 +427,7 @@ pub fn prepare_deferred_lighting_pipelines(
ssao,
(normal_prepass, depth_prepass, motion_vector_prepass),
has_environment_maps,
has_irradiance_volumes,
) in &views
{
let mut view_key = MeshPipelineKey::from_hdr(view.hdr);
@ -474,6 +481,10 @@ pub fn prepare_deferred_lighting_pipelines(
view_key |= MeshPipelineKey::ENVIRONMENT_MAP;
}
if has_irradiance_volumes {
view_key |= MeshPipelineKey::IRRADIANCE_VOLUME;
}
match shadow_filter_method.unwrap_or(&ShadowFilteringMethod::default()) {
ShadowFilteringMethod::Hardware2x2 => {
view_key |= MeshPipelineKey::SHADOW_FILTER_METHOD_HARDWARE_2X2;

220
crates/bevy_pbr/src/light_probe/environment_map.rs
View file

@ -60,23 +60,22 @@ use bevy_render::{
TextureSampleType, TextureView,
},
renderer::RenderDevice,
settings::WgpuFeatures,
texture::{FallbackImage, Image},
};
use bevy_utils::HashMap;
use std::num::NonZeroU32;
use std::ops::Deref;
use crate::{LightProbe, MAX_VIEW_REFLECTION_PROBES};
use crate::{
add_cubemap_texture_view, binding_arrays_are_usable, LightProbe, MAX_VIEW_LIGHT_PROBES,
};
use super::{LightProbeComponent, RenderViewLightProbes};
/// A handle to the environment map helper shader.
pub const ENVIRONMENT_MAP_SHADER_HANDLE: Handle<Shader> =
Handle::weak_from_u128(154476556247605696);
/// How many texture bindings are used in the fragment shader, *not* counting
/// environment maps.
const STANDARD_MATERIAL_FRAGMENT_SHADER_MIN_TEXTURE_BINDINGS: usize = 16;
/// A pair of cubemap textures that represent the surroundings of a specific
/// area in space.
///
@ -103,7 +102,7 @@ pub struct EnvironmentMapLight {
///
/// This is for use in the render app.
#[derive(Clone, Copy, PartialEq, Eq, Hash)]
pub(crate) struct EnvironmentMapIds {
pub struct EnvironmentMapIds {
/// The blurry image that represents diffuse radiance surrounding a region.
pub(crate) diffuse: AssetId<Image>,
/// The typically-sharper, mipmapped image that represents specular radiance
@ -127,29 +126,9 @@ pub struct ReflectionProbeBundle {
pub environment_map: EnvironmentMapLight,
}
/// A component, part of the render world, that stores the mapping from
/// environment map ID to texture index in the diffuse and specular binding
/// arrays.
///
/// Cubemap textures belonging to environment maps are collected into binding
/// arrays, and the index of each texture is presented to the shader for runtime
/// lookup.
///
/// This component is attached to each view in the render world, because each
/// view may have a different set of cubemaps that it considers and therefore
/// cubemap indices are per-view.
#[derive(Component, Default)]
pub struct RenderViewEnvironmentMaps {
/// The list of environment maps presented to the shader, in order.
binding_index_to_cubemap: Vec<EnvironmentMapIds>,
/// The reverse of `binding_index_to_cubemap`: a map from the environment
/// map IDs to the index in `binding_index_to_cubemap`.
cubemap_to_binding_index: HashMap<EnvironmentMapIds, u32>,
}
/// All the bind group entries necessary for PBR shaders to access the
/// environment maps exposed to a view.
pub(crate) enum RenderViewBindGroupEntries<'a> {
pub(crate) enum RenderViewEnvironmentMapBindGroupEntries<'a> {
/// The version used when binding arrays aren't available on the current
/// platform.
Single {
@ -164,12 +143,12 @@ pub(crate) enum RenderViewBindGroupEntries<'a> {
sampler: &'a Sampler,
},
/// The version used when binding arrays aren't available on the current
/// The version used when binding arrays are available on the current
/// platform.
Multiple {
/// A texture view of each diffuse cubemap, in the same order that they are
/// supplied to the view (i.e. in the same order as
/// `binding_index_to_cubemap` in [`RenderViewEnvironmentMaps`]).
/// `binding_index_to_cubemap` in [`RenderViewLightProbes`]).
///
/// This is a vector of `wgpu::TextureView`s. But we don't want to import
/// `wgpu` in this crate, so we refer to it indirectly like this.
@ -184,6 +163,19 @@ pub(crate) enum RenderViewBindGroupEntries<'a> {
},
}
/// Information about the environment map attached to the view, if any. This is
/// a global environment map that lights everything visible in the view, as
/// opposed to a light probe which affects only a specific area.
pub struct EnvironmentMapViewLightProbeInfo {
/// The index of the diffuse and specular cubemaps in the binding arrays.
pub(crate) cubemap_index: i32,
/// The smallest mip level of the specular cubemap.
pub(crate) smallest_specular_mip_level: u32,
/// The scale factor applied to the diffuse and specular light in the
/// cubemap. This is in units of cd/m² (candela per square meter).
pub(crate) intensity: f32,
}
impl ExtractInstance for EnvironmentMapIds {
type QueryData = Read<EnvironmentMapLight>;
@ -197,32 +189,6 @@ impl ExtractInstance for EnvironmentMapIds {
}
}
impl RenderViewEnvironmentMaps {
pub(crate) fn new() -> Self {
Self::default()
}
}
impl RenderViewEnvironmentMaps {
/// Whether there are no environment maps associated with the view.
pub(crate) fn is_empty(&self) -> bool {
self.binding_index_to_cubemap.is_empty()
}
/// Adds a cubemap to the list of bindings, if it wasn't there already, and
/// returns its index within that list.
pub(crate) fn get_or_insert_cubemap(&mut self, cubemap_id: &EnvironmentMapIds) -> u32 {
*self
.cubemap_to_binding_index
.entry(*cubemap_id)
.or_insert_with(|| {
let index = self.binding_index_to_cubemap.len() as u32;
self.binding_index_to_cubemap.push(*cubemap_id);
index
})
}
}
/// Returns the bind group layout entries for the environment map diffuse and
/// specular binding arrays respectively, in addition to the sampler.
pub(crate) fn get_bind_group_layout_entries(
@ -232,7 +198,7 @@ pub(crate) fn get_bind_group_layout_entries(
binding_types::texture_cube(TextureSampleType::Float { filterable: true });
if binding_arrays_are_usable(render_device) {
texture_cube_binding =
texture_cube_binding.count(NonZeroU32::new(MAX_VIEW_REFLECTION_PROBES as _).unwrap());
texture_cube_binding.count(NonZeroU32::new(MAX_VIEW_LIGHT_PROBES as _).unwrap());
}
[
@ -242,30 +208,30 @@ pub(crate) fn get_bind_group_layout_entries(
]
}
impl<'a> RenderViewBindGroupEntries<'a> {
impl<'a> RenderViewEnvironmentMapBindGroupEntries<'a> {
/// Looks up and returns the bindings for the environment map diffuse and
/// specular binding arrays respectively, as well as the sampler.
pub(crate) fn get(
render_view_environment_maps: Option<&RenderViewEnvironmentMaps>,
render_view_environment_maps: Option<&RenderViewLightProbes<EnvironmentMapLight>>,
images: &'a RenderAssets<Image>,
fallback_image: &'a FallbackImage,
render_device: &RenderDevice,
) -> RenderViewBindGroupEntries<'a> {
) -> RenderViewEnvironmentMapBindGroupEntries<'a> {
if binding_arrays_are_usable(render_device) {
let mut diffuse_texture_views = vec![];
let mut specular_texture_views = vec![];
let mut sampler = None;
if let Some(environment_maps) = render_view_environment_maps {
for &cubemap_id in &environment_maps.binding_index_to_cubemap {
add_texture_view(
for &cubemap_id in &environment_maps.binding_index_to_textures {
add_cubemap_texture_view(
&mut diffuse_texture_views,
&mut sampler,
cubemap_id.diffuse,
images,
fallback_image,
);
add_texture_view(
add_cubemap_texture_view(
&mut specular_texture_views,
&mut sampler,
cubemap_id.specular,
@ -277,16 +243,11 @@ impl<'a> RenderViewBindGroupEntries<'a> {
// Pad out the bindings to the size of the binding array using fallback
// textures. This is necessary on D3D12 and Metal.
diffuse_texture_views.resize(
MAX_VIEW_REFLECTION_PROBES,
&*fallback_image.cube.texture_view,
);
specular_texture_views.resize(
MAX_VIEW_REFLECTION_PROBES,
&*fallback_image.cube.texture_view,
);
diffuse_texture_views.resize(MAX_VIEW_LIGHT_PROBES, &*fallback_image.cube.texture_view);
specular_texture_views
.resize(MAX_VIEW_LIGHT_PROBES, &*fallback_image.cube.texture_view);
return RenderViewBindGroupEntries::Multiple {
return RenderViewEnvironmentMapBindGroupEntries::Multiple {
diffuse_texture_views,
specular_texture_views,
sampler: sampler.unwrap_or(&fallback_image.cube.sampler),
@ -294,11 +255,11 @@ impl<'a> RenderViewBindGroupEntries<'a> {
}
if let Some(environment_maps) = render_view_environment_maps {
if let Some(cubemap) = environment_maps.binding_index_to_cubemap.first() {
if let Some(cubemap) = environment_maps.binding_index_to_textures.first() {
if let (Some(diffuse_image), Some(specular_image)) =
(images.get(cubemap.diffuse), images.get(cubemap.specular))
{
return RenderViewBindGroupEntries::Single {
return RenderViewEnvironmentMapBindGroupEntries::Single {
diffuse_texture_view: &diffuse_image.texture_view,
specular_texture_view: &specular_image.texture_view,
sampler: &diffuse_image.sampler,
@ -307,7 +268,7 @@ impl<'a> RenderViewBindGroupEntries<'a> {
}
}
RenderViewBindGroupEntries::Single {
RenderViewEnvironmentMapBindGroupEntries::Single {
diffuse_texture_view: &fallback_image.cube.texture_view,
specular_texture_view: &fallback_image.cube.texture_view,
sampler: &fallback_image.cube.sampler,
@ -315,56 +276,71 @@ impl<'a> RenderViewBindGroupEntries<'a> {
}
}
/// Adds a diffuse or specular texture view to the `texture_views` list, and
/// populates `sampler` if this is the first such view.
fn add_texture_view<'a>(
texture_views: &mut Vec<&'a <TextureView as Deref>::Target>,
sampler: &mut Option<&'a Sampler>,
image_id: AssetId<Image>,
images: &'a RenderAssets<Image>,
fallback_image: &'a FallbackImage,
impl LightProbeComponent for EnvironmentMapLight {
type AssetId = EnvironmentMapIds;
// Information needed to render with the environment map attached to the
// view.
type ViewLightProbeInfo = EnvironmentMapViewLightProbeInfo;
fn id(&self, image_assets: &RenderAssets<Image>) -> Option<Self::AssetId> {
if image_assets.get(&self.diffuse_map).is_none()
|| image_assets.get(&self.specular_map).is_none()
{
None
} else {
Some(EnvironmentMapIds {
diffuse: self.diffuse_map.id(),
specular: self.specular_map.id(),
})
}
}
fn intensity(&self) -> f32 {
self.intensity
}
fn create_render_view_light_probes(
view_component: Option<&EnvironmentMapLight>,
image_assets: &RenderAssets<Image>,
) -> RenderViewLightProbes<Self> {
let mut render_view_light_probes = RenderViewLightProbes::new();
// Find the index of the cubemap associated with the view, and determine
// its smallest mip level.
if let Some(EnvironmentMapLight {
diffuse_map: diffuse_map_handle,
specular_map: specular_map_handle,
intensity,
}) = view_component
{
if let (Some(_), Some(specular_map)) = (
image_assets.get(diffuse_map_handle),
image_assets.get(specular_map_handle),
) {
match images.get(image_id) {
None => {
// Use the fallback image if the cubemap isn't loaded yet.
texture_views.push(&*fallback_image.cube.texture_view);
}
Some(image) => {
// If this is the first texture view, populate `sampler`.
if sampler.is_none() {
*sampler = Some(&image.sampler);
render_view_light_probes.view_light_probe_info = EnvironmentMapViewLightProbeInfo {
cubemap_index: render_view_light_probes.get_or_insert_cubemap(
&EnvironmentMapIds {
diffuse: diffuse_map_handle.id(),
specular: specular_map_handle.id(),
},
) as i32,
smallest_specular_mip_level: specular_map.mip_level_count - 1,
intensity: *intensity,
};
}
};
texture_views.push(&*image.texture_view);
}
render_view_light_probes
}
}
/// Many things can go wrong when attempting to use texture binding arrays
/// (a.k.a. bindless textures). This function checks for these pitfalls:
///
/// 1. If GLSL support is enabled at the feature level, then in debug mode
/// `naga_oil` will attempt to compile all shader modules under GLSL to check
/// validity of names, even if GLSL isn't actually used. This will cause a crash
/// if binding arrays are enabled, because binding arrays are currently
/// unimplemented in the GLSL backend of Naga. Therefore, we disable binding
/// arrays if the `shader_format_glsl` feature is present.
///
/// 2. If there aren't enough texture bindings available to accommodate all the
/// binding arrays, the driver will panic. So we also bail out if there aren't
/// enough texture bindings available in the fragment shader.
///
/// 3. If binding arrays aren't supported on the hardware, then we obviously
/// can't use them.
///
/// If binding arrays aren't usable, we disable reflection probes, as they rely
/// on them.
pub(crate) fn binding_arrays_are_usable(render_device: &RenderDevice) -> bool {
!cfg!(feature = "shader_format_glsl")
&& render_device.limits().max_storage_textures_per_shader_stage
>= (STANDARD_MATERIAL_FRAGMENT_SHADER_MIN_TEXTURE_BINDINGS + MAX_VIEW_REFLECTION_PROBES)
as u32
&& render_device
.features()
.contains(WgpuFeatures::TEXTURE_BINDING_ARRAY)
impl Default for EnvironmentMapViewLightProbeInfo {
fn default() -> Self {
Self {
cubemap_index: -1,
smallest_specular_mip_level: 0,
intensity: 1.0,
}
}
}

91
crates/bevy_pbr/src/light_probe/environment_map.wgsl
View file

@ -1,5 +1,6 @@
#define_import_path bevy_pbr::environment_map
#import bevy_pbr::light_probe::query_light_probe
#import bevy_pbr::mesh_view_bindings as bindings
#import bevy_pbr::mesh_view_bindings::light_probes
@ -24,71 +25,46 @@ fn compute_radiances(
N: vec3<f32>,
R: vec3<f32>,
world_position: vec3<f32>,
found_diffuse_indirect: bool,
) -> EnvironmentMapRadiances {
var radiances: EnvironmentMapRadiances;
// Search for a reflection probe that contains the fragment.
//
// TODO: Interpolate between multiple reflection probes.
var cubemap_index: i32 = -1;
var intensity: f32 = 1.0;
for (var reflection_probe_index: i32 = 0;
reflection_probe_index < light_probes.reflection_probe_count;
reflection_probe_index += 1) {
let reflection_probe = light_probes.reflection_probes[reflection_probe_index];
// Unpack the inverse transform.
let inverse_transpose_transform = mat4x4<f32>(
reflection_probe.inverse_transpose_transform[0],
reflection_probe.inverse_transpose_transform[1],
reflection_probe.inverse_transpose_transform[2],
vec4<f32>(0.0, 0.0, 0.0, 1.0));
let inverse_transform = transpose(inverse_transpose_transform);
// Check to see if the transformed point is inside the unit cube
// centered at the origin.
let probe_space_pos = (inverse_transform * vec4<f32>(world_position, 1.0)).xyz;
if (all(abs(probe_space_pos) <= vec3(0.5))) {
cubemap_index = reflection_probe.cubemap_index;
intensity = reflection_probe.intensity;
// TODO: Workaround for ICE in DXC https://github.com/microsoft/DirectXShaderCompiler/issues/6183
// This works because it's the last thing that happens in the for loop, and will break as soon as it
// goes back to the top of the loop.
// We can't use `break` here because of the ICE.
// break;
reflection_probe_index = light_probes.reflection_probe_count;
}
}
var query_result = query_light_probe(
light_probes.reflection_probes,
light_probes.reflection_probe_count,
world_position);
// If we didn't find a reflection probe, use the view environment map if applicable.
if (cubemap_index < 0) {
cubemap_index = light_probes.view_cubemap_index;
intensity = light_probes.intensity_for_view;
if (query_result.texture_index < 0) {
query_result.texture_index = light_probes.view_cubemap_index;
query_result.intensity = light_probes.intensity_for_view;
}
// If there's no cubemap, bail out.
if (cubemap_index < 0) {
if (query_result.texture_index < 0) {
radiances.irradiance = vec3(0.0);
radiances.radiance = vec3(0.0);
return radiances;
}
// Split-sum approximation for image based lighting: https://cdn2.unrealengine.com/Resources/files/2013SiggraphPresentationsNotes-26915738.pdf
let radiance_level = perceptual_roughness * f32(textureNumLevels(bindings::specular_environment_maps[cubemap_index]) - 1u);
let radiance_level = perceptual_roughness * f32(textureNumLevels(
bindings::specular_environment_maps[query_result.texture_index]) - 1u);
#ifndef LIGHTMAP
if (!found_diffuse_indirect) {
radiances.irradiance = textureSampleLevel(
bindings::diffuse_environment_maps[cubemap_index],
bindings::diffuse_environment_maps[query_result.texture_index],
bindings::environment_map_sampler,
vec3(N.xy, -N.z),
0.0).rgb * intensity;
#endif // LIGHTMAP
0.0).rgb * query_result.intensity;
}
radiances.radiance = textureSampleLevel(
bindings::specular_environment_maps[cubemap_index],
bindings::specular_environment_maps[query_result.texture_index],
bindings::environment_map_sampler,
vec3(R.xy, -R.z),
radiance_level).rgb * intensity;
radiance_level).rgb * query_result.intensity;
return radiances;
}
@ -100,6 +76,7 @@ fn compute_radiances(
N: vec3<f32>,
R: vec3<f32>,
world_position: vec3<f32>,
found_diffuse_indirect: bool,
) -> EnvironmentMapRadiances {
var radiances: EnvironmentMapRadiances;
@ -116,13 +93,13 @@ fn compute_radiances(
let intensity = light_probes.intensity_for_view;
#ifndef LIGHTMAP
if (!found_diffuse_indirect) {
radiances.irradiance = textureSampleLevel(
bindings::diffuse_environment_map,
bindings::environment_map_sampler,
vec3(N.xy, -N.z),
0.0).rgb * intensity;
#endif // LIGHTMAP
}
radiances.radiance = textureSampleLevel(
bindings::specular_environment_map,
@ -145,8 +122,21 @@ fn environment_map_light(
R: vec3<f32>,
F0: vec3<f32>,
world_position: vec3<f32>,
found_diffuse_indirect: bool,
) -> EnvironmentMapLight {
let radiances = compute_radiances(perceptual_roughness, N, R, world_position);
var out: EnvironmentMapLight;
let radiances = compute_radiances(
perceptual_roughness,
N,
R,
world_position,
found_diffuse_indirect);
if (all(radiances.irradiance == vec3(0.0)) && all(radiances.radiance == vec3(0.0))) {
out.diffuse = vec3(0.0);
out.specular = vec3(0.0);
return out;
}
// No real world material has specular values under 0.02, so we use this range as a
// "pre-baked specular occlusion" that extinguishes the fresnel term, for artistic control.
@ -166,15 +156,12 @@ fn environment_map_light(
let Edss = 1.0 - (FssEss + FmsEms);
let kD = diffuse_color * Edss;
var out: EnvironmentMapLight;
// If there's a lightmap, ignore the diffuse component of the reflection
// probe, so we don't double-count light.
#ifdef LIGHTMAP
out.diffuse = vec3(0.0);
#else
if (!found_diffuse_indirect) {
out.diffuse = (FmsEms + kD) * radiances.irradiance;
#endif
} else {
out.diffuse = vec3(0.0);
}
out.specular = FssEss * radiances.radiance;
return out;

338
crates/bevy_pbr/src/light_probe/irradiance_volume.rs Normal file
View file

@ -0,0 +1,338 @@
//! Irradiance volumes, also known as voxel global illumination.
//!
//! An *irradiance volume* is a cuboid voxel region consisting of
//! regularly-spaced precomputed samples of diffuse indirect light. They're
//! ideal if you have a dynamic object such as a character that can move about
//! static non-moving geometry such as a level in a game, and you want that
//! dynamic object to be affected by the light bouncing off that static
//! geometry.
//!
//! To use irradiance volumes, you need to precompute, or *bake*, the indirect
//! light in your scene. Bevy doesn't currently come with a way to do this.
//! Fortunately, [Blender] provides a [baking tool] as part of the Eevee
//! renderer, and its irradiance volumes are compatible with those used by Bevy.
//! The [`bevy-baked-gi`] project provides a tool, `export-blender-gi`, that can
//! extract the baked irradiance volumes from the Blender `.blend` file and
//! package them up into a `.ktx2` texture for use by the engine. See the
//! documentation in the `bevy-baked-gi` project for more details on this
//! workflow.
//!
//! Like all light probes in Bevy, irradiance volumes are 1×1×1 cubes that can
//! be arbitrarily scaled, rotated, and positioned in a scene with the
//! [`bevy_transform::components::Transform`] component. The 3D voxel grid will
//! be stretched to fill the interior of the cube, and the illumination from the
//! irradiance volume will apply to all fragments within that bounding region.
//!
//! Bevy's irradiance volumes are based on Valve's [*ambient cubes*] as used in
//! *Half-Life 2* ([Mitchell 2006, slide 27]). These encode a single color of
//! light from the six 3D cardinal directions and blend the sides together
//! according to the surface normal. For an explanation of why ambient cubes
//! were chosen over spherical harmonics, see [Why ambient cubes?] below.
//!
//! If you wish to use a tool other than `export-blender-gi` to produce the
//! irradiance volumes, you'll need to pack the irradiance volumes in the
//! following format. The irradiance volume of resolution *(Rx, Ry, Rz)* is
//! expected to be a 3D texture of dimensions *(Rx, 2Ry, 3Rz)*. The unnormalized
//! texture coordinate *(s, t, p)* of the voxel at coordinate *(x, y, z)* with
//! side *S* ∈ *{-X, +X, -Y, +Y, -Z, +Z}* is as follows:
//!
//! ```text
//! s = x
//!
//! t = y + ⎰ 0 if S ∈ {-X, -Y, -Z}
//! ⎱ Ry if S ∈ {+X, +Y, +Z}
//!
//! ⎧ 0 if S ∈ {-X, +X}
//! p = z + ⎨ Rz if S ∈ {-Y, +Y}
//! ⎩ 2Rz if S ∈ {-Z, +Z}
//! ```
//!
//! Visually, in a left-handed coordinate system with Y up, viewed from the
//! right, the 3D texture looks like a stacked series of voxel grids, one for
//! each cube side, in this order:
//!
//! | **+X** | **+Y** | **+Z** |
//! | ------ | ------ | ------ |
//! | **-X** | **-Y** | **-Z** |
//!
//! A terminology note: Other engines may refer to irradiance volumes as *voxel
//! global illumination*, *VXGI*, or simply as *light probes*. Sometimes *light
//! probe* refers to what Bevy calls a reflection probe. In Bevy, *light probe*
//! is a generic term that encompasses all cuboid bounding regions that capture
//! indirect illumination, whether based on voxels or not.
//!
//! Note that, if binding arrays aren't supported (e.g. on WebGPU or WebGL 2),
//! then only the closest irradiance volume to the view will be taken into
//! account during rendering. The required `wgpu` features are
//! [`bevy_render::settings::WgpuFeatures::TEXTURE_BINDING_ARRAY`] and
//! [`bevy_render::settings::WgpuFeatures::SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING`].
//!
//! ## Why ambient cubes?
//!
//! This section describes the motivation behind the decision to use ambient
//! cubes in Bevy. It's not needed to use the feature; feel free to skip it
//! unless you're interested in its internal design.
//!
//! Bevy uses *Half-Life 2*-style ambient cubes (usually abbreviated as *HL2*)
//! as the representation of irradiance for light probes instead of the
//! more-popular spherical harmonics (*SH*). This might seem to be a surprising
//! choice, but it turns out to work well for the specific case of voxel
//! sampling on the GPU. Spherical harmonics have two problems that make them
//! less ideal for this use case:
//!
//! 1. The level 1 spherical harmonic coefficients can be negative. That
//! prevents the use of the efficient [RGB9E5 texture format], which only
//! encodes unsigned floating point numbers, and forces the use of the
//! less-efficient [RGBA16F format] if hardware interpolation is desired.
//!
//! 2. As an alternative to RGBA16F, level 1 spherical harmonics can be
//! normalized and scaled to the SH0 base color, as [Frostbite] does. This
//! allows them to be packed in standard LDR RGBA8 textures. However, this
//! prevents the use of hardware trilinear filtering, as the nonuniform scale
//! factor means that hardware interpolation no longer produces correct results.
//! The 8 texture fetches needed to interpolate between voxels can be upwards of
//! twice as slow as the hardware interpolation.
//!
//! The following chart summarizes the costs and benefits of ambient cubes,
//! level 1 spherical harmonics, and level 2 spherical harmonics:
//!
//! | Technique | HW-interpolated samples | Texel fetches | Bytes per voxel | Quality |
//! | ------------------------ | ----------------------- | ------------- | --------------- | ------- |
//! | Ambient cubes | 3 | 0 | 24 | Medium |
//! | Level 1 SH, compressed | 0 | 36 | 16 | Low |
//! | Level 1 SH, uncompressed | 4 | 0 | 24 | Low |
//! | Level 2 SH, compressed | 0 | 72 | 28 | High |
//! | Level 2 SH, uncompressed | 9 | 0 | 54 | High |
//!
//! (Note that the number of bytes per voxel can be reduced using various
//! texture compression methods, but the overall ratios remain similar.)
//!
//! From these data, we can see that ambient cubes balance fast lookups (from
//! leveraging hardware interpolation) with relatively-small storage
//! requirements and acceptable quality. Hence, they were chosen for irradiance
//! volumes in Bevy.
//!
//! [*ambient cubes*]: https://advances.realtimerendering.com/s2006/Mitchell-ShadingInValvesSourceEngine.pdf
//!
//! [spherical harmonics]: https://en.wikipedia.org/wiki/Spherical_harmonic_lighting
//!
//! [RGB9E5 texture format]: https://www.khronos.org/opengl/wiki/Small_Float_Formats#RGB9_E5
//!
//! [RGBA16F format]: https://www.khronos.org/opengl/wiki/Small_Float_Formats#Low-bitdepth_floats
//!
//! [Frostbite]: https://media.contentapi.ea.com/content/dam/eacom/frostbite/files/gdc2018-precomputedgiobalilluminationinfrostbite.pdf#page=53
//!
//! [Mitchell 2006, slide 27]: https://advances.realtimerendering.com/s2006/Mitchell-ShadingInValvesSourceEngine.pdf#page=27
//!
//! [Blender]: http://blender.org/
//!
//! [baking tool]: https://docs.blender.org/manual/en/latest/render/eevee/render_settings/indirect_lighting.html
//!
//! [`bevy-baked-gi`]: https://github.com/pcwalton/bevy-baked-gi
//!
//! [Why ambient cubes?]: #why-ambient-cubes
use bevy_ecs::component::Component;
use bevy_render::{
render_asset::RenderAssets,
render_resource::{
binding_types, BindGroupLayoutEntryBuilder, Sampler, SamplerBindingType, Shader,
TextureSampleType, TextureView,
},
renderer::RenderDevice,
texture::{FallbackImage, Image},
};
use std::{num::NonZeroU32, ops::Deref};
use bevy_asset::{AssetId, Handle};
use bevy_reflect::Reflect;
use crate::{
add_cubemap_texture_view, binding_arrays_are_usable, RenderViewLightProbes,
MAX_VIEW_LIGHT_PROBES,
};
use super::LightProbeComponent;
pub const IRRADIANCE_VOLUME_SHADER_HANDLE: Handle<Shader> =
Handle::weak_from_u128(160299515939076705258408299184317675488);
/// The component that defines an irradiance volume.
///
/// See [`crate::irradiance_volume`] for detailed information.
#[derive(Clone, Default, Reflect, Component, Debug)]
pub struct IrradianceVolume {
/// The 3D texture that represents the ambient cubes, encoded in the format
/// described in [`crate::irradiance_volume`].
pub voxels: Handle<Image>,
/// Scale factor applied to the diffuse and specular light generated by this component.
///
/// After applying this multiplier, the resulting values should
/// be in units of [cd/m^2](https://en.wikipedia.org/wiki/Candela_per_square_metre).
///
/// See also <https://google.github.io/filament/Filament.html#lighting/imagebasedlights/iblunit>.
pub intensity: f32,
}
/// All the bind group entries necessary for PBR shaders to access the
/// irradiance volumes exposed to a view.
pub(crate) enum RenderViewIrradianceVolumeBindGroupEntries<'a> {
/// The version used when binding arrays aren't available on the current platform.
Single {
/// The texture view of the closest light probe.
texture_view: &'a TextureView,
/// A sampler used to sample voxels of the irradiance volume.
sampler: &'a Sampler,
},
/// The version used when binding arrays are available on the current
/// platform.
Multiple {
/// A texture view of the voxels of each irradiance volume, in the same
/// order that they are supplied to the view (i.e. in the same order as
/// `binding_index_to_cubemap` in [`RenderViewLightProbes`]).
///
/// This is a vector of `wgpu::TextureView`s. But we don't want to import
/// `wgpu` in this crate, so we refer to it indirectly like this.
texture_views: Vec<&'a <TextureView as Deref>::Target>,
/// A sampler used to sample voxels of the irradiance volumes.
sampler: &'a Sampler,
},
}
impl<'a> RenderViewIrradianceVolumeBindGroupEntries<'a> {
/// Looks up and returns the bindings for any irradiance volumes visible in
/// the view, as well as the sampler.
pub(crate) fn get(
render_view_irradiance_volumes: Option<&RenderViewLightProbes<IrradianceVolume>>,
images: &'a RenderAssets<Image>,
fallback_image: &'a FallbackImage,
render_device: &RenderDevice,
) -> RenderViewIrradianceVolumeBindGroupEntries<'a> {
if binding_arrays_are_usable(render_device) {
RenderViewIrradianceVolumeBindGroupEntries::get_multiple(
render_view_irradiance_volumes,
images,
fallback_image,
)
} else {
RenderViewIrradianceVolumeBindGroupEntries::get_single(
render_view_irradiance_volumes,
images,
fallback_image,
)
}
}
/// Looks up and returns the bindings for any irradiance volumes visible in
/// the view, as well as the sampler. This is the version used when binding
/// arrays are available on the current platform.
fn get_multiple(
render_view_irradiance_volumes: Option<&RenderViewLightProbes<IrradianceVolume>>,
images: &'a RenderAssets<Image>,
fallback_image: &'a FallbackImage,
) -> RenderViewIrradianceVolumeBindGroupEntries<'a> {
let mut texture_views = vec![];
let mut sampler = None;
if let Some(irradiance_volumes) = render_view_irradiance_volumes {
for &cubemap_id in &irradiance_volumes.binding_index_to_textures {
add_cubemap_texture_view(
&mut texture_views,
&mut sampler,
cubemap_id,
images,
fallback_image,
);
}
}
// Pad out the bindings to the size of the binding array using fallback
// textures. This is necessary on D3D12 and Metal.
texture_views.resize(MAX_VIEW_LIGHT_PROBES, &*fallback_image.d3.texture_view);
RenderViewIrradianceVolumeBindGroupEntries::Multiple {
texture_views,
sampler: sampler.unwrap_or(&fallback_image.d3.sampler),
}
}
/// Looks up and returns the bindings for any irradiance volumes visible in
/// the view, as well as the sampler. This is the version used when binding
/// arrays aren't available on the current platform.
fn get_single(
render_view_irradiance_volumes: Option<&RenderViewLightProbes<IrradianceVolume>>,
images: &'a RenderAssets<Image>,
fallback_image: &'a FallbackImage,
) -> RenderViewIrradianceVolumeBindGroupEntries<'a> {
if let Some(irradiance_volumes) = render_view_irradiance_volumes {
if let Some(irradiance_volume) = irradiance_volumes.render_light_probes.first() {
if irradiance_volume.texture_index >= 0 {
if let Some(image_id) = irradiance_volumes
.binding_index_to_textures
.get(irradiance_volume.texture_index as usize)
{
if let Some(image) = images.get(*image_id) {
return RenderViewIrradianceVolumeBindGroupEntries::Single {
texture_view: &image.texture_view,
sampler: &image.sampler,
};
}
}
}
}
}
RenderViewIrradianceVolumeBindGroupEntries::Single {
texture_view: &fallback_image.d3.texture_view,
sampler: &fallback_image.d3.sampler,
}
}
}
/// Returns the bind group layout entries for the voxel texture and sampler
/// respectively.
pub(crate) fn get_bind_group_layout_entries(
render_device: &RenderDevice,
) -> [BindGroupLayoutEntryBuilder; 2] {
let mut texture_3d_binding =
binding_types::texture_3d(TextureSampleType::Float { filterable: true });
if binding_arrays_are_usable(render_device) {
texture_3d_binding =
texture_3d_binding.count(NonZeroU32::new(MAX_VIEW_LIGHT_PROBES as _).unwrap());
}
[
texture_3d_binding,
binding_types::sampler(SamplerBindingType::Filtering),
]
}
impl LightProbeComponent for IrradianceVolume {
type AssetId = AssetId<Image>;
// Irradiance volumes can't be attached to the view, so we store nothing
// here.
type ViewLightProbeInfo = ();
fn id(&self, image_assets: &RenderAssets<Image>) -> Option<Self::AssetId> {
if image_assets.get(&self.voxels).is_none() {
None
} else {
Some(self.voxels.id())
}
}
fn intensity(&self) -> f32 {
self.intensity
}
fn create_render_view_light_probes(
_: Option<&Self>,
_: &RenderAssets<Image>,
) -> RenderViewLightProbes<Self> {
RenderViewLightProbes::new()
}
}

55
crates/bevy_pbr/src/light_probe/irradiance_volume.wgsl Normal file
View file

@ -0,0 +1,55 @@
#define_import_path bevy_pbr::irradiance_volume
#import bevy_pbr::light_probe::query_light_probe
#import bevy_pbr::mesh_view_bindings::{
irradiance_volumes,
irradiance_volume,
irradiance_volume_sampler,
light_probes,
};
// See:
// https://advances.realtimerendering.com/s2006/Mitchell-ShadingInValvesSourceEngine.pdf
// Slide 28, "Ambient Cube Basis"
fn irradiance_volume_light(world_position: vec3<f32>, N: vec3<f32>) -> vec3<f32> {
// Search for an irradiance volume that contains the fragment.
let query_result = query_light_probe(
light_probes.irradiance_volumes,
light_probes.irradiance_volume_count,
world_position);
// If there was no irradiance volume found, bail out.
if (query_result.texture_index < 0) {
return vec3(0.0f);
}
#ifdef MULTIPLE_LIGHT_PROBES_IN_ARRAY
let irradiance_volume_texture = irradiance_volumes[query_result.texture_index];
#else
let irradiance_volume_texture = irradiance_volume;
#endif
let atlas_resolution = vec3<f32>(textureDimensions(irradiance_volume_texture));
let resolution = vec3<f32>(textureDimensions(irradiance_volume_texture) / vec3(1u, 2u, 3u));
// Make sure to clamp to the edges to avoid texture bleed.
var unit_pos = (query_result.inverse_transform * vec4(world_position, 1.0f)).xyz;
let stp = clamp((unit_pos + 0.5) * resolution, vec3(0.5f), resolution - vec3(0.5f));
let uvw = stp / atlas_resolution;
// The bottom half of each cube slice is the negative part, so choose it if applicable on each
// slice.
let neg_offset = select(vec3(0.0f), vec3(0.5f), N < vec3(0.0f));
let uvw_x = uvw + vec3(0.0f, neg_offset.x, 0.0f);
let uvw_y = uvw + vec3(0.0f, neg_offset.y, 1.0f / 3.0f);
let uvw_z = uvw + vec3(0.0f, neg_offset.z, 2.0f / 3.0f);
let rgb_x = textureSample(irradiance_volume_texture, irradiance_volume_sampler, uvw_x).rgb;
let rgb_y = textureSample(irradiance_volume_texture, irradiance_volume_sampler, uvw_y).rgb;
let rgb_z = textureSample(irradiance_volume_texture, irradiance_volume_sampler, uvw_z).rgb;
// Use Valve's formula to sample.
let NN = N * N;
return (rgb_x * NN.x + rgb_y * NN.y + rgb_z * NN.z) * query_result.intensity;
}

69
crates/bevy_pbr/src/light_probe/light_probe.wgsl Normal file
View file

@ -0,0 +1,69 @@
#define_import_path bevy_pbr::light_probe
#import bevy_pbr::mesh_view_types::LightProbe
// The result of searching for a light probe.
struct LightProbeQueryResult {
// The index of the light probe texture or textures in the binding array or
// arrays.
texture_index: i32,
// A scale factor that's applied to the diffuse and specular light from the
// light probe. This is in units of cd/m² (candela per square meter).
intensity: f32,
// Transform from world space to the light probe model space. In light probe
// model space, the light probe is a 1×1×1 cube centered on the origin.
inverse_transform: mat4x4<f32>,
};
fn transpose_affine_matrix(matrix: mat3x4<f32>) -> mat4x4<f32> {
let matrix4x4 = mat4x4<f32>(
matrix[0],
matrix[1],
matrix[2],
vec4<f32>(0.0, 0.0, 0.0, 1.0));
return transpose(matrix4x4);
}
// Searches for a light probe that contains the fragment.
//
// TODO: Interpolate between multiple light probes.
fn query_light_probe(
in_light_probes: array<LightProbe, 8u>,
light_probe_count: i32,
world_position: vec3<f32>,
) -> LightProbeQueryResult {
// This is needed to index into the array with a non-constant expression.
var light_probes = in_light_probes;
var result: LightProbeQueryResult;
result.texture_index = -1;
for (var light_probe_index: i32 = 0;
light_probe_index < light_probe_count && result.texture_index < 0;
light_probe_index += 1) {
let light_probe = light_probes[light_probe_index];
// Unpack the inverse transform.
let inverse_transform =
transpose_affine_matrix(light_probe.inverse_transpose_transform);
// Check to see if the transformed point is inside the unit cube
// centered at the origin.
let probe_space_pos = (inverse_transform * vec4<f32>(world_position, 1.0f)).xyz;
if (all(abs(probe_space_pos) <= vec3(0.5f))) {
result.texture_index = light_probe.cubemap_index;
result.intensity = light_probe.intensity;
result.inverse_transform = inverse_transform;
// TODO: Workaround for ICE in DXC https://github.com/microsoft/DirectXShaderCompiler/issues/6183
// We can't use `break` here because of the ICE.
// So instead we rely on the fact that we set `result.texture_index`
// above and check its value in the `for` loop header before
// looping.
// break;
}
}
return result;
}

659
crates/bevy_pbr/src/light_probe/mod.rs
View file

@ -1,7 +1,7 @@
//! Light probes for baked global illumination.
use bevy_app::{App, Plugin};
use bevy_asset::load_internal_asset;
use bevy_asset::{load_internal_asset, AssetId, Handle};
use bevy_core_pipeline::core_3d::Camera3d;
use bevy_derive::{Deref, DerefMut};
use bevy_ecs::{
@ -18,26 +18,42 @@ use bevy_render::{
extract_instances::ExtractInstancesPlugin,
primitives::{Aabb, Frustum},
render_asset::RenderAssets,
render_resource::{DynamicUniformBuffer, Shader, ShaderType},
render_resource::{DynamicUniformBuffer, Sampler, Shader, ShaderType, TextureView},
renderer::{RenderDevice, RenderQueue},
texture::Image,
settings::WgpuFeatures,
texture::{FallbackImage, Image},
view::ExtractedView,
Extract, ExtractSchedule, Render, RenderApp, RenderSet,
};
use bevy_transform::prelude::GlobalTransform;
use bevy_utils::{EntityHashMap, FloatOrd};
use bevy_utils::{tracing::error, FloatOrd, HashMap};
use crate::light_probe::environment_map::{
binding_arrays_are_usable, EnvironmentMapIds, EnvironmentMapLight, RenderViewEnvironmentMaps,
ENVIRONMENT_MAP_SHADER_HANDLE,
use std::hash::Hash;
use std::ops::Deref;
use crate::{
irradiance_volume::IRRADIANCE_VOLUME_SHADER_HANDLE,
light_probe::environment_map::{
EnvironmentMapIds, EnvironmentMapLight, ENVIRONMENT_MAP_SHADER_HANDLE,
},
};
pub mod environment_map;
use self::irradiance_volume::IrradianceVolume;
/// The maximum number of reflection probes that each view will consider.
pub const LIGHT_PROBE_SHADER_HANDLE: Handle<Shader> = Handle::weak_from_u128(8954249792581071582);
pub mod environment_map;
pub mod irradiance_volume;
/// The maximum number of each type of light probe that each view will consider.
///
/// Because the fragment shader does a linear search through the list for each
/// fragment, this number needs to be relatively small.
pub const MAX_VIEW_REFLECTION_PROBES: usize = 8;
pub const MAX_VIEW_LIGHT_PROBES: usize = 8;
/// How many texture bindings are used in the fragment shader, *not* counting
/// environment maps or irradiance volumes.
const STANDARD_MATERIAL_FRAGMENT_SHADER_MIN_TEXTURE_BINDINGS: usize = 16;
/// Adds support for light probes: cuboid bounding regions that apply global
/// illumination to objects within them.
@ -55,25 +71,56 @@ pub struct LightProbePlugin;
/// that should take this light probe into account.
///
/// Note that a light probe will have no effect unless the entity contains some
/// kind of illumination. At present, the only supported type of illumination is
/// the [`EnvironmentMapLight`].
/// kind of illumination, which can either be an [`EnvironmentMapLight`] or an
/// [`IrradianceVolume`].
///
/// When multiple sources of indirect illumination can be applied to a fragment,
/// the highest-quality one is chosen. Diffuse and specular illumination are
/// considered separately, so, for example, Bevy may decide to sample the
/// diffuse illumination from an irradiance volume and the specular illumination
/// from a reflection probe. From highest priority to lowest priority, the
/// ranking is as follows:
///
/// | Rank | Diffuse | Specular |
/// | ---- | -------------------- | -------------------- |
/// | 1 | Lightmap | Lightmap |
/// | 2 | Irradiance volume | Reflection probe |
/// | 3 | Reflection probe | View environment map |
/// | 4 | View environment map | |
///
/// Note that ambient light is always added to the diffuse component and does
/// not participate in the ranking. That is, ambient light is applied in
/// addition to, not instead of, the light sources above.
///
/// A terminology note: Unfortunately, there is little agreement across game and
/// graphics engines as to what to call the various techniques that Bevy groups
/// under the term *light probe*. In Bevy, a *light probe* is the generic term
/// that encompasses both *reflection probes* and *irradiance volumes*. In
/// object-oriented terms, *light probe* is the superclass, and *reflection
/// probe* and *irradiance volume* are subclasses. In other engines, you may see
/// the term *light probe* refer to an irradiance volume with a single voxel, or
/// perhaps some other technique, while in Bevy *light probe* refers not to a
/// specific technique but rather to a class of techniques. Developers familiar
/// with other engines should be aware of this terminology difference.
#[derive(Component, Debug, Clone, Copy, Default, Reflect)]
#[reflect(Component, Default)]
pub struct LightProbe;
/// A GPU type that stores information about a reflection probe.
/// A GPU type that stores information about a light probe.
#[derive(Clone, Copy, ShaderType, Default)]
struct RenderReflectionProbe {
struct RenderLightProbe {
/// The transform from the world space to the model space. This is used to
/// efficiently check for bounding box intersection.
inverse_transpose_transform: [Vec4; 3],
/// The index of the environment map in the diffuse and specular cubemap
/// binding arrays.
cubemap_index: i32,
/// The index of the texture or textures in the appropriate binding array or
/// arrays.
///
/// For example, for reflection probes this is the index of the cubemap in
/// the diffuse and specular texture arrays.
texture_index: i32,
/// Scale factor applied to the diffuse and specular light generated by this
/// reflection probe.
/// Scale factor applied to the light generated by this light probe.
///
/// See the comment in [`EnvironmentMapLight`] for details.
intensity: f32,
@ -85,11 +132,18 @@ struct RenderReflectionProbe {
pub struct LightProbesUniform {
/// The list of applicable reflection probes, sorted from nearest to the
/// camera to the farthest away from the camera.
reflection_probes: [RenderReflectionProbe; MAX_VIEW_REFLECTION_PROBES],
reflection_probes: [RenderLightProbe; MAX_VIEW_LIGHT_PROBES],
/// The list of applicable irradiance volumes, sorted from nearest to the
/// camera to the farthest away from the camera.
irradiance_volumes: [RenderLightProbe; MAX_VIEW_LIGHT_PROBES],
/// The number of reflection probes in the list.
reflection_probe_count: i32,
/// The number of irradiance volumes in the list.
irradiance_volume_count: i32,
/// The index of the diffuse and specular environment maps associated with
/// the view itself. This is used as a fallback if no reflection probe in
/// the list contains the fragment.
@ -105,11 +159,6 @@ pub struct LightProbesUniform {
intensity_for_view: f32,
}
/// A map from each camera to the light probe uniform associated with it.
#[derive(Resource, Default, Deref, DerefMut)]
struct RenderLightProbes(EntityHashMap<Entity, LightProbesUniform>);
/// A GPU buffer that stores information about all light probes.
#[derive(Resource, Default, Deref, DerefMut)]
pub struct LightProbesBuffer(DynamicUniformBuffer<LightProbesUniform>);
@ -119,24 +168,123 @@ pub struct LightProbesBuffer(DynamicUniformBuffer<LightProbesUniform>);
pub struct ViewLightProbesUniformOffset(u32);
/// Information that [`gather_light_probes`] keeps about each light probe.
#[derive(Clone, Copy)]
///
/// This information is parameterized by the [`LightProbeComponent`] type. This
/// will either be [`EnvironmentMapLight`] for reflection probes or
/// [`IrradianceVolume`] for irradiance volumes.
#[allow(dead_code)]
struct LightProbeInfo {
struct LightProbeInfo<C>
where
C: LightProbeComponent,
{
// The transform from world space to light probe space.
inverse_transform: Mat4,
// The transform from light probe space to world space.
affine_transform: Affine3A,
// The diffuse and specular environment maps associated with this light
// probe.
environment_maps: EnvironmentMapIds,
// Scale factor applied to the diffuse and specular light generated by this
// reflection probe.
//
// See the comment in [`EnvironmentMapLight`] for details.
intensity: f32,
// The IDs of all assets associated with this light probe.
//
// Because each type of light probe component may reference different types
// of assets (e.g. a reflection probe references two cubemap assets while an
// irradiance volume references a single 3D texture asset), this is generic.
asset_id: C::AssetId,
}
/// A component, part of the render world, that stores the mapping from asset ID
/// or IDs to the texture index in the appropriate binding arrays.
///
/// Cubemap textures belonging to environment maps are collected into binding
/// arrays, and the index of each texture is presented to the shader for runtime
/// lookup. 3D textures belonging to reflection probes are likewise collected
/// into binding arrays, and the shader accesses the 3D texture by index.
///
/// This component is attached to each view in the render world, because each
/// view may have a different set of light probes that it considers and therefore
/// the texture indices are per-view.
#[derive(Component, Default)]
pub struct RenderViewLightProbes<C>
where
C: LightProbeComponent,
{
/// The list of environment maps presented to the shader, in order.
binding_index_to_textures: Vec<C::AssetId>,
/// The reverse of `binding_index_to_cubemap`: a map from the texture ID to
/// the index in `binding_index_to_cubemap`.
cubemap_to_binding_index: HashMap<C::AssetId, u32>,
/// Information about each light probe, ready for upload to the GPU, sorted
/// in order from closest to the camera to farthest.
///
/// Note that this is not necessarily ordered by binding index. So don't
/// write code like
/// `render_light_probes[cubemap_to_binding_index[asset_id]]`; instead
/// search for the light probe with the appropriate binding index in this
/// array.
render_light_probes: Vec<RenderLightProbe>,
/// Information needed to render the light probe attached directly to the
/// view, if applicable.
///
/// A light probe attached directly to a view represents a "global" light
/// probe that affects all objects not in the bounding region of any light
/// probe. Currently, the only light probe type that supports this is the
/// [`EnvironmentMapLight`].
view_light_probe_info: C::ViewLightProbeInfo,
}
/// A trait implemented by all components that represent light probes.
///
/// Currently, the two light probe types are [`EnvironmentMapLight`] and
/// [`IrradianceVolume`], for reflection probes and irradiance volumes
/// respectively.
///
/// Most light probe systems are written to be generic over the type of light
/// probe. This allows much of the code to be shared and enables easy addition
/// of more light probe types (e.g. real-time reflection planes) in the future.
pub trait LightProbeComponent: Send + Sync + Component + Sized {
/// Holds [`AssetId`]s of the texture or textures that this light probe
/// references.
///
/// This can just be [`AssetId`] if the light probe only references one
/// texture. If it references multiple textures, it will be a structure
/// containing those asset IDs.
type AssetId: Send + Sync + Clone + Eq + Hash;
/// If the light probe can be attached to the view itself (as opposed to a
/// cuboid region within the scene), this contains the information that will
/// be passed to the GPU in order to render it. Otherwise, this will be
/// `()`.
///
/// Currently, only reflection probes (i.e. [`EnvironmentMapLight`]) can be
/// attached directly to views.
type ViewLightProbeInfo: Send + Sync + Default;
/// Returns the asset ID or asset IDs of the texture or textures referenced
/// by this light probe.
fn id(&self, image_assets: &RenderAssets<Image>) -> Option<Self::AssetId>;
/// Returns the intensity of this light probe.
///
/// This is a scaling factor that will be multiplied by the value or values
/// sampled from the texture.
fn intensity(&self) -> f32;
/// Creates an instance of [`RenderViewLightProbes`] containing all the
/// information needed to render this light probe.
///
/// This is called for every light probe in view every frame.
fn create_render_view_light_probes(
view_component: Option<&Self>,
image_assets: &RenderAssets<Image>,
) -> RenderViewLightProbes<Self>;
}
impl LightProbe {
@ -149,15 +297,28 @@ impl LightProbe {
impl Plugin for LightProbePlugin {
fn build(&self, app: &mut App) {
load_internal_asset!(
app,
LIGHT_PROBE_SHADER_HANDLE,
"light_probe.wgsl",
Shader::from_wgsl
);
load_internal_asset!(
app,
ENVIRONMENT_MAP_SHADER_HANDLE,
"environment_map.wgsl",
Shader::from_wgsl
);
load_internal_asset!(
app,
IRRADIANCE_VOLUME_SHADER_HANDLE,
"irradiance_volume.wgsl",
Shader::from_wgsl
);
app.register_type::<LightProbe>()
.register_type::<EnvironmentMapLight>();
.register_type::<EnvironmentMapLight>()
.register_type::<IrradianceVolume>();
}
fn finish(&self, app: &mut App) {
@ -168,8 +329,8 @@ impl Plugin for LightProbePlugin {
render_app
.add_plugins(ExtractInstancesPlugin::<EnvironmentMapIds>::new())
.init_resource::<LightProbesBuffer>()
.init_resource::<RenderLightProbes>()
.add_systems(ExtractSchedule, gather_light_probes)
.add_systems(ExtractSchedule, gather_light_probes::<EnvironmentMapLight>)
.add_systems(ExtractSchedule, gather_light_probes::<IrradianceVolume>)
.add_systems(
Render,
upload_light_probes.in_set(RenderSet::PrepareResources),
@ -177,107 +338,151 @@ impl Plugin for LightProbePlugin {
}
}
/// Gathers up all light probes in the scene and assigns them to views,
/// performing frustum culling and distance sorting in the process.
///
/// This populates the [`RenderLightProbes`] resource.
#[allow(clippy::too_many_arguments)]
fn gather_light_probes(
mut render_light_probes: ResMut<RenderLightProbes>,
/// Gathers up all light probes of a single type in the scene and assigns them
/// to views, performing frustum culling and distance sorting in the process.
fn gather_light_probes<C>(
image_assets: Res<RenderAssets<Image>>,
render_device: Res<RenderDevice>,
light_probe_query: Extract<Query<(&GlobalTransform, &EnvironmentMapLight), With<LightProbe>>>,
view_query: Extract<
Query<
(
Entity,
&GlobalTransform,
&Frustum,
Option<&EnvironmentMapLight>,
),
With<Camera3d>,
>,
>,
mut light_probes: Local<Vec<LightProbeInfo>>,
mut view_light_probes: Local<Vec<LightProbeInfo>>,
light_probe_query: Extract<Query<(&GlobalTransform, &C), With<LightProbe>>>,
view_query: Extract<Query<(Entity, &GlobalTransform, &Frustum, Option<&C>), With<Camera3d>>>,
mut reflection_probes: Local<Vec<LightProbeInfo<C>>>,
mut view_reflection_probes: Local<Vec<LightProbeInfo<C>>>,
mut commands: Commands,
) {
) where
C: LightProbeComponent,
{
// Create [`LightProbeInfo`] for every light probe in the scene.
light_probes.clear();
light_probes.extend(
reflection_probes.clear();
reflection_probes.extend(
light_probe_query
.iter()
.filter_map(|query_row| LightProbeInfo::new(query_row, &image_assets)),
);
// Build up the light probes uniform and the key table.
render_light_probes.clear();
for (view_entity, view_transform, view_frustum, view_environment_maps) in view_query.iter() {
for (view_entity, view_transform, view_frustum, view_component) in view_query.iter() {
// Cull light probes outside the view frustum.
view_light_probes.clear();
view_light_probes.extend(
light_probes
view_reflection_probes.clear();
view_reflection_probes.extend(
reflection_probes
.iter()
.filter(|light_probe_info| light_probe_info.frustum_cull(view_frustum))
.cloned(),
);
// Sort by distance to camera.
view_light_probes.sort_by_cached_key(|light_probe_info| {
view_reflection_probes.sort_by_cached_key(|light_probe_info| {
light_probe_info.camera_distance_sort_key(view_transform)
});
// Create the light probes uniform.
let (light_probes_uniform, render_view_environment_maps) = LightProbesUniform::build(
view_environment_maps,
&view_light_probes,
&image_assets,
&render_device,
);
// Create the light probes list.
let mut render_view_light_probes =
C::create_render_view_light_probes(view_component, &image_assets);
// Record the uniforms.
render_light_probes.insert(view_entity, light_probes_uniform);
// Gather up the light probes in the list.
render_view_light_probes.maybe_gather_light_probes(&view_reflection_probes);
// Record the per-view environment maps.
let mut commands = commands.get_or_spawn(view_entity);
if render_view_environment_maps.is_empty() {
commands.remove::<RenderViewEnvironmentMaps>();
// Record the per-view light probes.
if render_view_light_probes.is_empty() {
commands
.get_or_spawn(view_entity)
.remove::<RenderViewLightProbes<C>>();
} else {
commands.insert(render_view_environment_maps);
commands
.get_or_spawn(view_entity)
.insert(render_view_light_probes);
}
}
}
/// Uploads the result of [`gather_light_probes`] to the GPU.
// A system that runs after [`gather_light_probes`] and populates the GPU
// uniforms with the results.
//
// Note that, unlike [`gather_light_probes`], this system is not generic over
// the type of light probe. It collects light probes of all types together into
// a single structure, ready to be passed to the shader.
fn upload_light_probes(
mut commands: Commands,
light_probes_uniforms: Res<RenderLightProbes>,
views: Query<Entity, With<ExtractedView>>,
mut light_probes_buffer: ResMut<LightProbesBuffer>,
mut view_light_probes_query: Query<(
Option<&RenderViewLightProbes<EnvironmentMapLight>>,
Option<&RenderViewLightProbes<IrradianceVolume>>,
)>,
render_device: Res<RenderDevice>,
render_queue: Res<RenderQueue>,
) {
// Get the uniform buffer writer.
let Some(mut writer) =
light_probes_buffer.get_writer(light_probes_uniforms.len(), &render_device, &render_queue)
// Initialize the uniform buffer writer.
let mut writer = light_probes_buffer
.get_writer(views.iter().len(), &render_device, &render_queue)
.unwrap();
// Process each view.
for view_entity in views.iter() {
let Ok((render_view_environment_maps, render_view_irradiance_volumes)) =
view_light_probes_query.get_mut(view_entity)
else {
return;
error!("Failed to find `RenderViewLightProbes` for the view!");
continue;
};
// Send each view's uniforms to the GPU.
for (&view_entity, light_probes_uniform) in light_probes_uniforms.iter() {
// Initialize the uniform with only the view environment map, if there
// is one.
let mut light_probes_uniform = LightProbesUniform {
reflection_probes: [RenderLightProbe::default(); MAX_VIEW_LIGHT_PROBES],
irradiance_volumes: [RenderLightProbe::default(); MAX_VIEW_LIGHT_PROBES],
reflection_probe_count: render_view_environment_maps
.map(|maps| maps.len())
.unwrap_or_default()
.min(MAX_VIEW_LIGHT_PROBES) as i32,
irradiance_volume_count: render_view_irradiance_volumes
.map(|maps| maps.len())
.unwrap_or_default()
.min(MAX_VIEW_LIGHT_PROBES) as i32,
view_cubemap_index: render_view_environment_maps
.map(|maps| maps.view_light_probe_info.cubemap_index)
.unwrap_or(-1),
smallest_specular_mip_level_for_view: render_view_environment_maps
.map(|maps| maps.view_light_probe_info.smallest_specular_mip_level)
.unwrap_or(0),
intensity_for_view: render_view_environment_maps
.map(|maps| maps.view_light_probe_info.intensity)
.unwrap_or(1.0),
};
// Add any environment maps that [`gather_light_probes`] found to the
// uniform.
if let Some(render_view_environment_maps) = render_view_environment_maps {
render_view_environment_maps.add_to_uniform(
&mut light_probes_uniform.reflection_probes,
&mut light_probes_uniform.reflection_probe_count,
);
}
// Add any irradiance volumes that [`gather_light_probes`] found to the
// uniform.
if let Some(render_view_irradiance_volumes) = render_view_irradiance_volumes {
render_view_irradiance_volumes.add_to_uniform(
&mut light_probes_uniform.irradiance_volumes,
&mut light_probes_uniform.irradiance_volume_count,
);
}
// Queue the view's uniforms to be written to the GPU.
let uniform_offset = writer.write(&light_probes_uniform);
commands
.entity(view_entity)
.insert(ViewLightProbesUniformOffset(
writer.write(light_probes_uniform),
));
.insert(ViewLightProbesUniformOffset(uniform_offset));
}
}
impl Default for LightProbesUniform {
fn default() -> Self {
Self {
reflection_probes: [RenderReflectionProbe::default(); MAX_VIEW_REFLECTION_PROBES],
reflection_probes: [RenderLightProbe::default(); MAX_VIEW_LIGHT_PROBES],
irradiance_volumes: [RenderLightProbe::default(); MAX_VIEW_LIGHT_PROBES],
reflection_probe_count: 0,
irradiance_volume_count: 0,
view_cubemap_index: -1,
smallest_specular_mip_level_for_view: 0,
intensity_for_view: 1.0,
@ -285,131 +490,22 @@ impl Default for LightProbesUniform {
}
}
impl LightProbesUniform {
/// Constructs a [`LightProbesUniform`] containing all the environment maps
/// that fragments rendered by a single view need to consider.
///
/// The `view_environment_maps` parameter describes the environment maps
/// attached to the view. The `light_probes` parameter is expected to be the
/// list of light probes in the scene, sorted by increasing view distance
/// from the camera.
fn build(
view_environment_maps: Option<&EnvironmentMapLight>,
light_probes: &[LightProbeInfo],
image_assets: &RenderAssets<Image>,
render_device: &RenderDevice,
) -> (LightProbesUniform, RenderViewEnvironmentMaps) {
let mut render_view_environment_maps = RenderViewEnvironmentMaps::new();
// Find the index of the cubemap associated with the view, and determine
// its smallest mip level.
let mut view_cubemap_index = -1;
let mut smallest_specular_mip_level_for_view = 0;
let mut intensity_for_view = 1.0;
if let Some(EnvironmentMapLight {
diffuse_map: diffuse_map_handle,
specular_map: specular_map_handle,
intensity,
}) = view_environment_maps
impl<C> LightProbeInfo<C>
where
C: LightProbeComponent,
{
if let (Some(_), Some(specular_map)) = (
image_assets.get(diffuse_map_handle),
image_assets.get(specular_map_handle),
) {
view_cubemap_index =
render_view_environment_maps.get_or_insert_cubemap(&EnvironmentMapIds {
diffuse: diffuse_map_handle.id(),
specular: specular_map_handle.id(),
}) as i32;
smallest_specular_mip_level_for_view = specular_map.mip_level_count - 1;
intensity_for_view = *intensity;
}
};
// Initialize the uniform to only contain the view environment map, if
// applicable.
let mut uniform = LightProbesUniform {
reflection_probes: [RenderReflectionProbe::default(); MAX_VIEW_REFLECTION_PROBES],
reflection_probe_count: light_probes.len().min(MAX_VIEW_REFLECTION_PROBES) as i32,
view_cubemap_index,
smallest_specular_mip_level_for_view,
intensity_for_view,
};
// Add reflection probes from the scene, if supported by the current
// platform.
uniform.maybe_gather_reflection_probes(
&mut render_view_environment_maps,
light_probes,
render_device,
);
(uniform, render_view_environment_maps)
}
/// Gathers up all reflection probes in the scene and writes them into this
/// uniform and `render_view_environment_maps`.
fn maybe_gather_reflection_probes(
&mut self,
render_view_environment_maps: &mut RenderViewEnvironmentMaps,
light_probes: &[LightProbeInfo],
render_device: &RenderDevice,
) {
if !binding_arrays_are_usable(render_device) {
return;
}
for (reflection_probe, light_probe) in self
.reflection_probes
.iter_mut()
.zip(light_probes.iter().take(MAX_VIEW_REFLECTION_PROBES))
{
// Determine the index of the cubemap in the binding array.
let cubemap_index = render_view_environment_maps
.get_or_insert_cubemap(&light_probe.environment_maps)
as i32;
// Transpose the inverse transform to compress the structure on the
// GPU (from 4 `Vec4`s to 3 `Vec4`s). The shader will transpose it
// to recover the original inverse transform.
let inverse_transpose_transform = light_probe.inverse_transform.transpose();
// Write in the reflection probe data.
*reflection_probe = RenderReflectionProbe {
inverse_transpose_transform: [
inverse_transpose_transform.x_axis,
inverse_transpose_transform.y_axis,
inverse_transpose_transform.z_axis,
],
cubemap_index,
intensity: light_probe.intensity,
};
}
}
}
impl LightProbeInfo {
/// Given the set of light probe components, constructs and returns
/// [`LightProbeInfo`]. This is done for every light probe in the scene
/// every frame.
fn new(
(light_probe_transform, environment_map): (&GlobalTransform, &EnvironmentMapLight),
(light_probe_transform, environment_map): (&GlobalTransform, &C),
image_assets: &RenderAssets<Image>,
) -> Option<LightProbeInfo> {
if image_assets.get(&environment_map.diffuse_map).is_none()
|| image_assets.get(&environment_map.specular_map).is_none()
{
return None;
}
Some(LightProbeInfo {
) -> Option<LightProbeInfo<C>> {
environment_map.id(image_assets).map(|id| LightProbeInfo {
affine_transform: light_probe_transform.affine(),
inverse_transform: light_probe_transform.compute_matrix().inverse(),
environment_maps: EnvironmentMapIds {
diffuse: environment_map.diffuse_map.id(),
specular: environment_map.specular_map.id(),
},
intensity: environment_map.intensity,
asset_id: id,
intensity: environment_map.intensity(),
})
}
@ -436,3 +532,150 @@ impl LightProbeInfo {
)
}
}
impl<C> RenderViewLightProbes<C>
where
C: LightProbeComponent,
{
/// Creates a new empty list of light probes.
fn new() -> RenderViewLightProbes<C> {
RenderViewLightProbes {
binding_index_to_textures: vec![],
cubemap_to_binding_index: HashMap::new(),
render_light_probes: vec![],
view_light_probe_info: C::ViewLightProbeInfo::default(),
}
}
/// Returns true if there are no light probes in the list.
pub(crate) fn is_empty(&self) -> bool {
self.binding_index_to_textures.is_empty()
}
/// Returns the number of light probes in the list.
pub(crate) fn len(&self) -> usize {
self.binding_index_to_textures.len()
}
/// Adds a cubemap to the list of bindings, if it wasn't there already, and
/// returns its index within that list.
pub(crate) fn get_or_insert_cubemap(&mut self, cubemap_id: &C::AssetId) -> u32 {
*self
.cubemap_to_binding_index
.entry((*cubemap_id).clone())
.or_insert_with(|| {
let index = self.binding_index_to_textures.len() as u32;
self.binding_index_to_textures.push((*cubemap_id).clone());
index
})
}
/// Adds all the light probes in this structure to the supplied array, which
/// is expected to be shipped to the GPU.
fn add_to_uniform(
&self,
render_light_probes: &mut [RenderLightProbe; MAX_VIEW_LIGHT_PROBES],
render_light_probe_count: &mut i32,
) {
render_light_probes[0..self.render_light_probes.len()]
.copy_from_slice(&self.render_light_probes[..]);
*render_light_probe_count = self.render_light_probes.len() as i32;
}
/// Gathers up all light probes of the given type in the scene and records
/// them in this structure.
fn maybe_gather_light_probes(&mut self, light_probes: &[LightProbeInfo<C>]) {
for light_probe in light_probes.iter().take(MAX_VIEW_LIGHT_PROBES) {
// Determine the index of the cubemap in the binding array.
let cubemap_index = self.get_or_insert_cubemap(&light_probe.asset_id);
// Transpose the inverse transform to compress the structure on the
// GPU (from 4 `Vec4`s to 3 `Vec4`s). The shader will transpose it
// to recover the original inverse transform.
let inverse_transpose_transform = light_probe.inverse_transform.transpose();
// Write in the light probe data.
self.render_light_probes.push(RenderLightProbe {
inverse_transpose_transform: [
inverse_transpose_transform.x_axis,
inverse_transpose_transform.y_axis,
inverse_transpose_transform.z_axis,
],
texture_index: cubemap_index as i32,
intensity: light_probe.intensity,
});
}
}
}
impl<C> Clone for LightProbeInfo<C>
where
C: LightProbeComponent,
{
fn clone(&self) -> Self {
Self {
inverse_transform: self.inverse_transform,
affine_transform: self.affine_transform,
intensity: self.intensity,
asset_id: self.asset_id.clone(),
}
}
}
/// Adds a diffuse or specular texture view to the `texture_views` list, and
/// populates `sampler` if this is the first such view.
pub(crate) fn add_cubemap_texture_view<'a>(
texture_views: &mut Vec<&'a <TextureView as Deref>::Target>,
sampler: &mut Option<&'a Sampler>,
image_id: AssetId<Image>,
images: &'a RenderAssets<Image>,
fallback_image: &'a FallbackImage,
) {
match images.get(image_id) {
None => {
// Use the fallback image if the cubemap isn't loaded yet.
texture_views.push(&*fallback_image.cube.texture_view);
}
Some(image) => {
// If this is the first texture view, populate `sampler`.
if sampler.is_none() {
*sampler = Some(&image.sampler);
}
texture_views.push(&*image.texture_view);
}
}
}
/// Many things can go wrong when attempting to use texture binding arrays
/// (a.k.a. bindless textures). This function checks for these pitfalls:
///
/// 1. If GLSL support is enabled at the feature level, then in debug mode
/// `naga_oil` will attempt to compile all shader modules under GLSL to check
/// validity of names, even if GLSL isn't actually used. This will cause a crash
/// if binding arrays are enabled, because binding arrays are currently
/// unimplemented in the GLSL backend of Naga. Therefore, we disable binding
/// arrays if the `shader_format_glsl` feature is present.
///
/// 2. If there aren't enough texture bindings available to accommodate all the
/// binding arrays, the driver will panic. So we also bail out if there aren't
/// enough texture bindings available in the fragment shader.
///
/// 3. If binding arrays aren't supported on the hardware, then we obviously
/// can't use them.
///
/// 4. If binding arrays are supported on the hardware, but they can only be
/// accessed by uniform indices, that's not good enough, and we bail out.
///
/// If binding arrays aren't usable, we disable reflection probes and limit the
/// number of irradiance volumes in the scene to 1.
pub(crate) fn binding_arrays_are_usable(render_device: &RenderDevice) -> bool {
!cfg!(feature = "shader_format_glsl")
&& render_device.limits().max_storage_textures_per_shader_stage
>= (STANDARD_MATERIAL_FRAGMENT_SHADER_MIN_TEXTURE_BINDINGS + MAX_VIEW_LIGHT_PROBES)
as u32
&& render_device.features().contains(
WgpuFeatures::TEXTURE_BINDING_ARRAY
| WgpuFeatures::SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING,
)
}

15
crates/bevy_pbr/src/material.rs
View file

@ -1,4 +1,4 @@
use crate::{environment_map::RenderViewEnvironmentMaps, *};
use crate::*;
use bevy_app::{App, Plugin};
use bevy_asset::{Asset, AssetApp, AssetEvent, AssetId, AssetServer, Assets, Handle};
use bevy_core_pipeline::{
@ -34,6 +34,8 @@ use bevy_utils::{tracing::error, HashMap, HashSet};
use std::hash::Hash;
use std::marker::PhantomData;
use self::{irradiance_volume::IrradianceVolume, prelude::EnvironmentMapLight};
/// Materials are used alongside [`MaterialPlugin`] and [`MaterialMeshBundle`]
/// to spawn entities that are rendered with a specific [`Material`] type. They serve as an easy to use high level
/// way to render [`Mesh`] entities with custom shader logic.
@ -487,7 +489,10 @@ pub fn queue_material_meshes<M: Material>(
&mut RenderPhase<AlphaMask3d>,
&mut RenderPhase<Transmissive3d>,
&mut RenderPhase<Transparent3d>,
Has<RenderViewEnvironmentMaps>,
(
Has<RenderViewLightProbes<EnvironmentMapLight>>,
Has<RenderViewLightProbes<IrradianceVolume>>,
),
)>,
) where
M::Data: PartialEq + Eq + Hash + Clone,
@ -507,7 +512,7 @@ pub fn queue_material_meshes<M: Material>(
mut alpha_mask_phase,
mut transmissive_phase,
mut transparent_phase,
has_environment_maps,
(has_environment_maps, has_irradiance_volumes),
) in &mut views
{
let draw_opaque_pbr = opaque_draw_functions.read().id::<DrawMaterial<M>>();
@ -542,6 +547,10 @@ pub fn queue_material_meshes<M: Material>(
view_key |= MeshPipelineKey::ENVIRONMENT_MAP;
}
if has_irradiance_volumes {
view_key |= MeshPipelineKey::IRRADIANCE_VOLUME;
}
if let Some(projection) = projection {
view_key |= match projection {
Projection::Perspective(_) => MeshPipelineKey::VIEW_PROJECTION_PERSPECTIVE,

7
crates/bevy_pbr/src/render/mesh.rs
View file

@ -49,8 +49,6 @@ use crate::render::{
};
use crate::*;
use self::environment_map::binding_arrays_are_usable;
use super::skin::SkinIndices;
#[derive(Default)]
@ -510,6 +508,7 @@ bitflags::bitflags! {
const MORPH_TARGETS = 1 << 12;
const READS_VIEW_TRANSMISSION_TEXTURE = 1 << 13;
const LIGHTMAPPED = 1 << 14;
const IRRADIANCE_VOLUME = 1 << 15;
const BLEND_RESERVED_BITS = Self::BLEND_MASK_BITS << Self::BLEND_SHIFT_BITS; // ← Bitmask reserving bits for the blend state
const BLEND_OPAQUE = 0 << Self::BLEND_SHIFT_BITS; // ← Values are just sequential within the mask, and can range from 0 to 3
const BLEND_PREMULTIPLIED_ALPHA = 1 << Self::BLEND_SHIFT_BITS; //
@ -830,6 +829,10 @@ impl SpecializedMeshPipeline for MeshPipeline {
shader_defs.push("ENVIRONMENT_MAP".into());
}
if key.contains(MeshPipelineKey::IRRADIANCE_VOLUME) {
shader_defs.push("IRRADIANCE_VOLUME".into());
}
if key.contains(MeshPipelineKey::LIGHTMAPPED) {
shader_defs.push("LIGHTMAP".into());
}

65
crates/bevy_pbr/src/render/mesh_view_bindings.rs
View file

@ -29,11 +29,13 @@ use bevy_render::render_resource::binding_types::texture_cube;
feature = "webgpu"
))]
use bevy_render::render_resource::binding_types::{texture_2d_array, texture_cube_array};
use environment_map::EnvironmentMapLight;
use crate::{
environment_map::{self, RenderViewBindGroupEntries, RenderViewEnvironmentMaps},
environment_map::{self, RenderViewEnvironmentMapBindGroupEntries},
irradiance_volume::{self, IrradianceVolume, RenderViewIrradianceVolumeBindGroupEntries},
prepass, FogMeta, GlobalLightMeta, GpuFog, GpuLights, GpuPointLights, LightMeta,
LightProbesBuffer, LightProbesUniform, MeshPipeline, MeshPipelineKey,
LightProbesBuffer, LightProbesUniform, MeshPipeline, MeshPipelineKey, RenderViewLightProbes,
ScreenSpaceAmbientOcclusionTextures, ShadowSamplers, ViewClusterBindings, ViewShadowBindings,
};
@ -266,11 +268,18 @@ fn layout_entries(
(15, environment_map_entries[2]),
));
// Irradiance volumes
let irradiance_volume_entries = irradiance_volume::get_bind_group_layout_entries(render_device);
entries = entries.extend_with_indices((
(16, irradiance_volume_entries[0]),
(17, irradiance_volume_entries[1]),
));
// Tonemapping
let tonemapping_lut_entries = get_lut_bind_group_layout_entries();
entries = entries.extend_with_indices((
(16, tonemapping_lut_entries[0]),
(17, tonemapping_lut_entries[1]),
(18, tonemapping_lut_entries[0]),
(19, tonemapping_lut_entries[1]),
));
// Prepass
@ -280,7 +289,7 @@ fn layout_entries(
{
for (entry, binding) in prepass::get_bind_group_layout_entries(layout_key)
.iter()
.zip([18, 19, 20, 21])
.zip([20, 21, 22, 23])
{
if let Some(entry) = entry {
entries = entries.extend_with_indices(((binding as u32, *entry),));
@ -291,10 +300,10 @@ fn layout_entries(
// View Transmission Texture
entries = entries.extend_with_indices((
(
22,
24,
texture_2d(TextureSampleType::Float { filterable: true }),
),
(23, sampler(SamplerBindingType::Filtering)),
(25, sampler(SamplerBindingType::Filtering)),
));
entries.to_vec()
@ -348,7 +357,8 @@ pub fn prepare_mesh_view_bind_groups(
Option<&ViewPrepassTextures>,
Option<&ViewTransmissionTexture>,
&Tonemapping,
Option<&RenderViewEnvironmentMaps>,
Option<&RenderViewLightProbes<EnvironmentMapLight>>,
Option<&RenderViewLightProbes<IrradianceVolume>>,
)>,
(images, mut fallback_images, fallback_image, fallback_image_zero): (
Res<RenderAssets<Image>>,
@ -385,6 +395,7 @@ pub fn prepare_mesh_view_bind_groups(
transmission_texture,
tonemapping,
render_view_environment_maps,
render_view_irradiance_volumes,
) in &views
{
let fallback_ssao = fallback_images
@ -416,15 +427,15 @@ pub fn prepare_mesh_view_bind_groups(
(12, ssao_view),
));
let bind_group_entries = RenderViewBindGroupEntries::get(
let environment_map_bind_group_entries = RenderViewEnvironmentMapBindGroupEntries::get(
render_view_environment_maps,
&images,
&fallback_image,
&render_device,
);
match bind_group_entries {
RenderViewBindGroupEntries::Single {
match environment_map_bind_group_entries {
RenderViewEnvironmentMapBindGroupEntries::Single {
diffuse_texture_view,
specular_texture_view,
sampler,
@ -435,7 +446,7 @@ pub fn prepare_mesh_view_bind_groups(
(15, sampler),
));
}
RenderViewBindGroupEntries::Multiple {
RenderViewEnvironmentMapBindGroupEntries::Multiple {
ref diffuse_texture_views,
ref specular_texture_views,
sampler,
@ -448,8 +459,32 @@ pub fn prepare_mesh_view_bind_groups(
}
}
let irradiance_volume_bind_group_entries =
RenderViewIrradianceVolumeBindGroupEntries::get(
render_view_irradiance_volumes,
&images,
&fallback_image,
&render_device,
);
match irradiance_volume_bind_group_entries {
RenderViewIrradianceVolumeBindGroupEntries::Single {
texture_view,
sampler,
} => {
entries = entries.extend_with_indices(((16, texture_view), (17, sampler)));
}
RenderViewIrradianceVolumeBindGroupEntries::Multiple {
ref texture_views,
sampler,
} => {
entries = entries
.extend_with_indices(((16, texture_views.as_slice()), (17, sampler)));
}
}
let lut_bindings = get_lut_bindings(&images, &tonemapping_luts, tonemapping);
entries = entries.extend_with_indices(((16, lut_bindings.0), (17, lut_bindings.1)));
entries = entries.extend_with_indices(((18, lut_bindings.0), (19, lut_bindings.1)));
// When using WebGL, we can't have a depth texture with multisampling
let prepass_bindings;
@ -459,7 +494,7 @@ pub fn prepare_mesh_view_bind_groups(
for (binding, index) in prepass_bindings
.iter()
.map(Option::as_ref)
.zip([18, 19, 20, 21])
.zip([20, 21, 22, 23])
.flat_map(|(b, i)| b.map(|b| (b, i)))
{
entries = entries.extend_with_indices(((index, binding),));
@ -475,7 +510,7 @@ pub fn prepare_mesh_view_bind_groups(
.unwrap_or(&fallback_image_zero.sampler);
entries =
entries.extend_with_indices(((22, transmission_view), (23, transmission_sampler)));
entries.extend_with_indices(((24, transmission_view), (25, transmission_sampler)));
commands.entity(entity).insert(MeshViewBindGroup {
value: render_device.create_bind_group("mesh_view_bind_group", layout, &entries),

30
crates/bevy_pbr/src/render/mesh_view_bindings.wgsl
View file

@ -46,37 +46,45 @@
#endif
@group(0) @binding(15) var environment_map_sampler: sampler;
@group(0) @binding(16) var dt_lut_texture: texture_3d<f32>;
@group(0) @binding(17) var dt_lut_sampler: sampler;
#ifdef MULTIPLE_LIGHT_PROBES_IN_ARRAY
@group(0) @binding(16) var irradiance_volumes: binding_array<texture_3d<f32>, 8u>;
#else
@group(0) @binding(16) var irradiance_volume: texture_3d<f32>;
#endif
@group(0) @binding(17) var irradiance_volume_sampler: sampler;
// NB: If you change these, make sure to update `tonemapping_shared.wgsl` too.
@group(0) @binding(18) var dt_lut_texture: texture_3d<f32>;
@group(0) @binding(19) var dt_lut_sampler: sampler;
#ifdef MULTISAMPLED
#ifdef DEPTH_PREPASS
@group(0) @binding(18) var depth_prepass_texture: texture_depth_multisampled_2d;
@group(0) @binding(20) var depth_prepass_texture: texture_depth_multisampled_2d;
#endif // DEPTH_PREPASS
#ifdef NORMAL_PREPASS
@group(0) @binding(19) var normal_prepass_texture: texture_multisampled_2d<f32>;
@group(0) @binding(21) var normal_prepass_texture: texture_multisampled_2d<f32>;
#endif // NORMAL_PREPASS
#ifdef MOTION_VECTOR_PREPASS
@group(0) @binding(20) var motion_vector_prepass_texture: texture_multisampled_2d<f32>;
@group(0) @binding(22) var motion_vector_prepass_texture: texture_multisampled_2d<f32>;
#endif // MOTION_VECTOR_PREPASS
#else // MULTISAMPLED
#ifdef DEPTH_PREPASS
@group(0) @binding(18) var depth_prepass_texture: texture_depth_2d;
@group(0) @binding(20) var depth_prepass_texture: texture_depth_2d;
#endif // DEPTH_PREPASS
#ifdef NORMAL_PREPASS
@group(0) @binding(19) var normal_prepass_texture: texture_2d<f32>;
@group(0) @binding(21) var normal_prepass_texture: texture_2d<f32>;
#endif // NORMAL_PREPASS
#ifdef MOTION_VECTOR_PREPASS
@group(0) @binding(20) var motion_vector_prepass_texture: texture_2d<f32>;
@group(0) @binding(22) var motion_vector_prepass_texture: texture_2d<f32>;
#endif // MOTION_VECTOR_PREPASS
#endif // MULTISAMPLED
#ifdef DEFERRED_PREPASS
@group(0) @binding(21) var deferred_prepass_texture: texture_2d<u32>;
@group(0) @binding(23) var deferred_prepass_texture: texture_2d<u32>;
#endif // DEFERRED_PREPASS
@group(0) @binding(22) var view_transmission_texture: texture_2d<f32>;
@group(0) @binding(23) var view_transmission_sampler: sampler;
@group(0) @binding(24) var view_transmission_texture: texture_2d<f32>;
@group(0) @binding(25) var view_transmission_sampler: sampler;

6
crates/bevy_pbr/src/render/mesh_view_types.wgsl
View file

@ -111,7 +111,7 @@ struct ClusterOffsetsAndCounts {
};
#endif
struct ReflectionProbe {
struct LightProbe {
// This is stored as the transpose in order to save space in this structure.
// It'll be transposed in the `environment_map_light` function.
inverse_transpose_transform: mat3x4<f32>,
@ -121,8 +121,10 @@ struct ReflectionProbe {
struct LightProbes {
// This must match `MAX_VIEW_REFLECTION_PROBES` on the Rust side.
reflection_probes: array<ReflectionProbe, 8u>,
reflection_probes: array<LightProbe, 8u>,
irradiance_volumes: array<LightProbe, 8u>,
reflection_probe_count: i32,
irradiance_volume_count: i32,
// The index of the view environment map cubemap binding, or -1 if there's
// no such cubemap.
view_cubemap_index: i32,

48
crates/bevy_pbr/src/render/pbr_functions.wgsl
View file

@ -10,6 +10,7 @@
clustered_forward as clustering,
shadows,
ambient,
irradiance_volume,
mesh_types::{MESH_FLAGS_SHADOW_RECEIVER_BIT, MESH_FLAGS_TRANSMITTED_SHADOW_RECEIVER_BIT},
utils::E,
}
@ -306,8 +307,7 @@ fn apply_pbr_lighting(
#endif
}
// Ambient light (indirect)
var indirect_light = ambient::ambient_light(in.world_position, in.N, in.V, NdotV, diffuse_color, F0, perceptual_roughness, diffuse_occlusion);
var indirect_light = vec3(0.0f);
#ifdef STANDARD_MATERIAL_DIFFUSE_TRANSMISSION
// NOTE: We use the diffuse transmissive color, the second Lambertian lobe's calculated
@ -321,7 +321,36 @@ fn apply_pbr_lighting(
transmitted_light += ambient::ambient_light(diffuse_transmissive_lobe_world_position, -in.N, -in.V, 1.0, diffuse_transmissive_color, vec3<f32>(0.0), 1.0, vec3<f32>(1.0));
#endif
// Diffuse indirect lighting can come from a variety of sources. The
// priority goes like this:
//
// 1. Lightmap (highest)
// 2. Irradiance volume
// 3. Environment map (lowest)
//
// When we find a source of diffuse indirect lighting, we stop accumulating
// any more diffuse indirect light. This avoids double-counting if, for
// example, both lightmaps and irradiance volumes are present.
#ifdef LIGHTMAP
if (all(indirect_light == vec3(0.0f))) {
indirect_light += in.lightmap_light * diffuse_color;
}
#endif
#ifdef IRRADIANCE_VOLUME {
// Irradiance volume light (indirect)
if (all(indirect_light == vec3(0.0f))) {
let irradiance_volume_light = irradiance_volume::irradiance_volume_light(
in.world_position.xyz, in.N);
indirect_light += irradiance_volume_light * diffuse_color * diffuse_occlusion;
}
#endif
// Environment map light (indirect)
//
// Note that up until this point, we have only accumulated diffuse light.
// This call is the first call that can accumulate specular light.
#ifdef ENVIRONMENT_MAP
let environment_light = environment_map::environment_map_light(
perceptual_roughness,
@ -332,8 +361,11 @@ fn apply_pbr_lighting(
in.N,
R,
F0,
in.world_position.xyz);
indirect_light += (environment_light.diffuse * diffuse_occlusion) + (environment_light.specular * specular_occlusion);
in.world_position.xyz,
any(indirect_light != vec3(0.0f)));
indirect_light += environment_light.diffuse * diffuse_occlusion +
environment_light.specular * specular_occlusion;
// we'll use the specular component of the transmitted environment
// light in the call to `specular_transmissive_light()` below
@ -367,7 +399,8 @@ fn apply_pbr_lighting(
-in.N,
T,
vec3<f32>(1.0),
in.world_position.xyz);
in.world_position.xyz,
false);
#ifdef STANDARD_MATERIAL_DIFFUSE_TRANSMISSION
transmitted_light += transmitted_environment_light.diffuse * diffuse_transmissive_color;
#endif
@ -381,9 +414,8 @@ fn apply_pbr_lighting(
let specular_transmitted_environment_light = vec3<f32>(0.0);
#endif
#ifdef LIGHTMAP
indirect_light += in.lightmap_light * diffuse_color;
#endif
// Ambient light (indirect)
indirect_light += ambient::ambient_light(in.world_position, in.N, in.V, NdotV, diffuse_color, F0, perceptual_roughness, diffuse_occlusion);
let emissive_light = emissive.rgb * output_color.a;

667
examples/3d/irradiance_volumes.rs Normal file
View file

@ -0,0 +1,667 @@
//! This example shows how irradiance volumes affect the indirect lighting of
//! objects in a scene.
//!
//! The controls are as follows:
//!
//! * Space toggles the irradiance volume on and off.
//!
//! * Enter toggles the camera rotation on and off.
//!
//! * Tab switches the object between a plain sphere and a running fox.
//!
//! * Backspace shows and hides the voxel cubes.
//!
//! * Clicking anywhere moves the object.
use bevy::core_pipeline::Skybox;
use bevy::math::{uvec3, vec3};
use bevy::pbr::irradiance_volume::IrradianceVolume;
use bevy::pbr::{ExtendedMaterial, MaterialExtension, NotShadowCaster};
use bevy::prelude::shape::{Cube, UVSphere};
use bevy::prelude::*;
use bevy::render::render_resource::{AsBindGroup, ShaderRef, ShaderType};
use bevy::window::PrimaryWindow;
// Rotation speed in radians per frame.
const ROTATION_SPEED: f32 = 0.005;
const FOX_SCALE: f32 = 0.05;
const SPHERE_SCALE: f32 = 2.0;
const IRRADIANCE_VOLUME_INTENSITY: f32 = 150.0;
const AMBIENT_LIGHT_BRIGHTNESS: f32 = 0.06;
const VOXEL_CUBE_SCALE: f32 = 0.4;
static DISABLE_IRRADIANCE_VOLUME_HELP_TEXT: &str = "Space: Disable the irradiance volume";
static ENABLE_IRRADIANCE_VOLUME_HELP_TEXT: &str = "Space: Enable the irradiance volume";
static HIDE_VOXELS_HELP_TEXT: &str = "Backspace: Hide the voxels";
static SHOW_VOXELS_HELP_TEXT: &str = "Backspace: Show the voxels";
static STOP_ROTATION_HELP_TEXT: &str = "Enter: Stop rotation";
static START_ROTATION_HELP_TEXT: &str = "Enter: Start rotation";
static SWITCH_TO_FOX_HELP_TEXT: &str = "Tab: Switch to a skinned mesh";
static SWITCH_TO_SPHERE_HELP_TEXT: &str = "Tab: Switch to a plain sphere mesh";
static CLICK_TO_MOVE_HELP_TEXT: &str = "Left click: Move the object";
static GIZMO_COLOR: Color = Color::YELLOW;
static VOXEL_TRANSFORM: Mat4 = Mat4::from_cols_array_2d(&[
[-42.317566, 0.0, 0.0, 0.0],
[0.0, 0.0, 44.601563, 0.0],
[0.0, 16.73776, 0.0, 0.0],
[0.0, 6.544792, 0.0, 1.0],
]);
// The mode the application is in.
#[derive(Resource)]
struct AppStatus {
// Whether the user wants the irradiance volume to be applied.
irradiance_volume_present: bool,
// Whether the user wants the unskinned sphere mesh or the skinned fox mesh.
model: ExampleModel,
// Whether the user has requested the scene to rotate.
rotating: bool,
// Whether the user has requested the voxels to be displayed.
voxels_visible: bool,
}
// Which model the user wants to display.
#[derive(Clone, Copy, PartialEq)]
enum ExampleModel {
// The plain sphere.
Sphere,
// The fox, which is skinned.
Fox,
}
// Handles to all the assets used in this example.
#[derive(Resource)]
struct ExampleAssets {
// The glTF scene containing the colored floor.
main_scene: Handle<Scene>,
// The 3D texture containing the irradiance volume.
irradiance_volume: Handle<Image>,
// The plain sphere mesh.
main_sphere: Handle<Mesh>,
// The material used for the sphere.
main_sphere_material: Handle<StandardMaterial>,
// The glTF scene containing the animated fox.
fox: Handle<Scene>,
// The animation that the fox will play.
fox_animation: Handle<AnimationClip>,
// The voxel cube mesh.
voxel_cube: Handle<Mesh>,
// The skybox.
skybox: Handle<Image>,
}
// The sphere and fox both have this component.
#[derive(Component)]
struct MainObject;
// Marks each of the voxel cubes.
#[derive(Component)]
struct VoxelCube;
// Marks the voxel cube parent object.
#[derive(Component)]
struct VoxelCubeParent;
type VoxelVisualizationMaterial = ExtendedMaterial<StandardMaterial, VoxelVisualizationExtension>;
#[derive(Asset, TypePath, AsBindGroup, Debug, Clone)]
struct VoxelVisualizationExtension {
#[uniform(100)]
irradiance_volume_info: VoxelVisualizationIrradianceVolumeInfo,
}
#[derive(ShaderType, Debug, Clone)]
struct VoxelVisualizationIrradianceVolumeInfo {
transform: Mat4,
inverse_transform: Mat4,
resolution: UVec3,
intensity: f32,
}
fn main() {
// Create the example app.
App::new()
.add_plugins(DefaultPlugins.set(WindowPlugin {
primary_window: Some(Window {
title: "Bevy Irradiance Volumes Example".into(),
..default()
}),
..default()
}))
.add_plugins(MaterialPlugin::<VoxelVisualizationMaterial>::default())
.init_resource::<AppStatus>()
.init_resource::<ExampleAssets>()
.insert_resource(AmbientLight {
color: Color::WHITE,
brightness: 0.0,
})
.add_systems(Startup, setup)
.add_systems(PreUpdate, create_cubes)
.add_systems(Update, rotate_camera)
.add_systems(Update, play_animations)
.add_systems(
Update,
handle_mouse_clicks
.after(rotate_camera)
.after(play_animations),
)
.add_systems(
Update,
change_main_object
.after(rotate_camera)
.after(play_animations),
)
.add_systems(
Update,
toggle_irradiance_volumes
.after(rotate_camera)
.after(play_animations),
)
.add_systems(
Update,
toggle_voxel_visibility
.after(rotate_camera)
.after(play_animations),
)
.add_systems(
Update,
toggle_rotation.after(rotate_camera).after(play_animations),
)
.add_systems(
Update,
draw_gizmo
.after(handle_mouse_clicks)
.after(change_main_object)
.after(toggle_irradiance_volumes)
.after(toggle_voxel_visibility)
.after(toggle_rotation),
)
.add_systems(
Update,
update_text
.after(handle_mouse_clicks)
.after(change_main_object)
.after(toggle_irradiance_volumes)
.after(toggle_voxel_visibility)
.after(toggle_rotation),
)
.run();
}
// Spawns all the scene objects.
fn setup(
mut commands: Commands,
assets: Res<ExampleAssets>,
app_status: Res<AppStatus>,
asset_server: Res<AssetServer>,
) {
spawn_main_scene(&mut commands, &assets);
spawn_camera(&mut commands, &assets);
spawn_irradiance_volume(&mut commands, &assets);
spawn_light(&mut commands);
spawn_sphere(&mut commands, &assets);
spawn_voxel_cube_parent(&mut commands);
spawn_fox(&mut commands, &assets);
spawn_text(&mut commands, &app_status, &asset_server);
}
fn spawn_main_scene(commands: &mut Commands, assets: &ExampleAssets) {
commands.spawn(SceneBundle {
scene: assets.main_scene.clone(),
..SceneBundle::default()
});
}
fn spawn_camera(commands: &mut Commands, assets: &ExampleAssets) {
commands
.spawn(Camera3dBundle {
transform: Transform::from_xyz(-10.012, 4.8605, 13.281).looking_at(Vec3::ZERO, Vec3::Y),
..default()
})
.insert(Skybox {
image: assets.skybox.clone(),
brightness: 150.0,
});
}
fn spawn_irradiance_volume(commands: &mut Commands, assets: &ExampleAssets) {
commands
.spawn(SpatialBundle {
transform: Transform::from_matrix(VOXEL_TRANSFORM),
..SpatialBundle::default()
})
.insert(IrradianceVolume {
voxels: assets.irradiance_volume.clone(),
intensity: IRRADIANCE_VOLUME_INTENSITY,
})
.insert(LightProbe);
}
fn spawn_light(commands: &mut Commands) {
commands.spawn(PointLightBundle {
point_light: PointLight {
intensity: 250000.0,
shadows_enabled: true,
..default()
},
transform: Transform::from_xyz(4.0762, 5.9039, 1.0055),
..default()
});
}
fn spawn_sphere(commands: &mut Commands, assets: &ExampleAssets) {
commands
.spawn(PbrBundle {
mesh: assets.main_sphere.clone(),
material: assets.main_sphere_material.clone(),
transform: Transform::from_xyz(0.0, SPHERE_SCALE, 0.0)
.with_scale(Vec3::splat(SPHERE_SCALE)),
..default()
})
.insert(MainObject);
}
fn spawn_voxel_cube_parent(commands: &mut Commands) {
commands
.spawn(SpatialBundle {
visibility: Visibility::Hidden,
..default()
})
.insert(VoxelCubeParent);
}
fn spawn_fox(commands: &mut Commands, assets: &ExampleAssets) {
commands
.spawn(SceneBundle {
scene: assets.fox.clone(),
visibility: Visibility::Hidden,
transform: Transform::from_scale(Vec3::splat(FOX_SCALE)),
..default()
})
.insert(MainObject);
}
fn spawn_text(commands: &mut Commands, app_status: &AppStatus, asset_server: &AssetServer) {
commands.spawn(
TextBundle {
text: app_status.create_text(asset_server),
..TextBundle::default()
}
.with_style(Style {
position_type: PositionType::Absolute,
bottom: Val::Px(10.0),
left: Val::Px(10.0),
..default()
}),
);
}
// A system that updates the help text.
fn update_text(
mut text_query: Query<&mut Text>,
app_status: Res<AppStatus>,
asset_server: Res<AssetServer>,
) {
for mut text in text_query.iter_mut() {
*text = app_status.create_text(&asset_server);
}
}
impl AppStatus {
// Constructs the help text at the bottom of the screen based on the
// application status.
fn create_text(&self, asset_server: &AssetServer) -> Text {
let irradiance_volume_help_text = if self.irradiance_volume_present {
DISABLE_IRRADIANCE_VOLUME_HELP_TEXT
} else {
ENABLE_IRRADIANCE_VOLUME_HELP_TEXT
};
let voxels_help_text = if self.voxels_visible {
HIDE_VOXELS_HELP_TEXT
} else {
SHOW_VOXELS_HELP_TEXT
};
let rotation_help_text = if self.rotating {
STOP_ROTATION_HELP_TEXT
} else {
START_ROTATION_HELP_TEXT
};
let switch_mesh_help_text = match self.model {
ExampleModel::Sphere => SWITCH_TO_FOX_HELP_TEXT,
ExampleModel::Fox => SWITCH_TO_SPHERE_HELP_TEXT,
};
Text::from_section(
format!(
"{}\n{}\n{}\n{}\n{}",
CLICK_TO_MOVE_HELP_TEXT,
voxels_help_text,
irradiance_volume_help_text,
rotation_help_text,
switch_mesh_help_text
),
TextStyle {
font: asset_server.load("fonts/FiraMono-Medium.ttf"),
font_size: 24.0,
color: Color::ANTIQUE_WHITE,
},
)
}
}
// Rotates the camera a bit every frame.
fn rotate_camera(
mut camera_query: Query<&mut Transform, With<Camera3d>>,
app_status: Res<AppStatus>,
) {
if !app_status.rotating {
return;
}
for mut transform in camera_query.iter_mut() {
transform.translation = Vec2::from_angle(ROTATION_SPEED)
.rotate(transform.translation.xz())
.extend(transform.translation.y)
.xzy();
transform.look_at(Vec3::ZERO, Vec3::Y);
}
}
// Toggles between the unskinned sphere model and the skinned fox model if the
// user requests it.
fn change_main_object(
keyboard: Res<ButtonInput<KeyCode>>,
mut app_status: ResMut<AppStatus>,
mut sphere_query: Query<
&mut Visibility,
(With<MainObject>, With<Handle<Mesh>>, Without<Handle<Scene>>),
>,
mut fox_query: Query<&mut Visibility, (With<MainObject>, With<Handle<Scene>>)>,
) {
if !keyboard.just_pressed(KeyCode::Tab) {
return;
}
let Some(mut sphere_visibility) = sphere_query.iter_mut().next() else {
return;
};
let Some(mut fox_visibility) = fox_query.iter_mut().next() else {
return;
};
match app_status.model {
ExampleModel::Sphere => {
*sphere_visibility = Visibility::Hidden;
*fox_visibility = Visibility::Visible;
app_status.model = ExampleModel::Fox;
}
ExampleModel::Fox => {
*sphere_visibility = Visibility::Visible;
*fox_visibility = Visibility::Hidden;
app_status.model = ExampleModel::Sphere;
}
}
}
impl Default for AppStatus {
fn default() -> Self {
Self {
irradiance_volume_present: true,
rotating: true,
model: ExampleModel::Sphere,
voxels_visible: false,
}
}
}
// Turns on and off the irradiance volume as requested by the user.
fn toggle_irradiance_volumes(
mut commands: Commands,
keyboard: Res<ButtonInput<KeyCode>>,
light_probe_query: Query<Entity, With<LightProbe>>,
mut app_status: ResMut<AppStatus>,
assets: Res<ExampleAssets>,
mut ambient_light: ResMut<AmbientLight>,
) {
if !keyboard.just_pressed(KeyCode::Space) {
return;
};
let Some(light_probe) = light_probe_query.iter().next() else {
return;
};
if app_status.irradiance_volume_present {
commands.entity(light_probe).remove::<IrradianceVolume>();
ambient_light.brightness = AMBIENT_LIGHT_BRIGHTNESS * IRRADIANCE_VOLUME_INTENSITY;
app_status.irradiance_volume_present = false;
} else {
commands.entity(light_probe).insert(IrradianceVolume {
voxels: assets.irradiance_volume.clone(),
intensity: IRRADIANCE_VOLUME_INTENSITY,
});
ambient_light.brightness = 0.0;
app_status.irradiance_volume_present = true;
}
}
fn toggle_rotation(keyboard: Res<ButtonInput<KeyCode>>, mut app_status: ResMut<AppStatus>) {
if keyboard.just_pressed(KeyCode::Enter) {
app_status.rotating = !app_status.rotating;
}
}
// Handles clicks on the plane that reposition the object.
fn handle_mouse_clicks(
buttons: Res<ButtonInput<MouseButton>>,
windows: Query<&Window, With<PrimaryWindow>>,
cameras: Query<(&Camera, &GlobalTransform)>,
mut main_objects: Query<&mut Transform, With<MainObject>>,
) {
if !buttons.pressed(MouseButton::Left) {
return;
}
let Some(mouse_position) = windows
.iter()
.next()
.and_then(|window| window.cursor_position())
else {
return;
};
let Some((camera, camera_transform)) = cameras.iter().next() else {
return;
};
// Figure out where the user clicked on the plane.
let Some(ray) = camera.viewport_to_world(camera_transform, mouse_position) else {
return;
};
let Some(ray_distance) = ray.intersect_plane(Vec3::ZERO, Plane3d::new(Vec3::Y)) else {
return;
};
let plane_intersection = ray.origin + ray.direction.normalize() * ray_distance;
// Move all the main objeccts.
for mut transform in main_objects.iter_mut() {
transform.translation = vec3(
plane_intersection.x,
transform.translation.y,
plane_intersection.z,
);
}
}
impl FromWorld for ExampleAssets {
fn from_world(world: &mut World) -> Self {
// Load all the assets.
let asset_server = world.resource::<AssetServer>();
let fox = asset_server.load("models/animated/Fox.glb#Scene0");
let main_scene =
asset_server.load("models/IrradianceVolumeExample/IrradianceVolumeExample.glb#Scene0");
let irradiance_volume = asset_server.load::<Image>("irradiance_volumes/Example.vxgi.ktx2");
let fox_animation =
asset_server.load::<AnimationClip>("models/animated/Fox.glb#Animation1");
// Just use a specular map for the skybox since it's not too blurry.
// In reality you wouldn't do this--you'd use a real skybox texture--but
// reusing the textures like this saves space in the Bevy repository.
let skybox = asset_server.load::<Image>("environment_maps/pisa_specular_rgb9e5_zstd.ktx2");
let mut mesh_assets = world.resource_mut::<Assets<Mesh>>();
let main_sphere = mesh_assets.add(UVSphere::default());
let voxel_cube = mesh_assets.add(Cube::default());
let mut standard_material_assets = world.resource_mut::<Assets<StandardMaterial>>();
let main_material = standard_material_assets.add(Color::SILVER);
ExampleAssets {
main_sphere,
fox,
main_sphere_material: main_material,
main_scene,
irradiance_volume,
fox_animation,
voxel_cube,
skybox,
}
}
}
// Plays the animation on the fox.
fn play_animations(assets: Res<ExampleAssets>, mut players: Query<&mut AnimationPlayer>) {
for mut player in players.iter_mut() {
// This will safely do nothing if the animation is already playing.
player.play(assets.fox_animation.clone()).repeat();
}
}
fn create_cubes(
image_assets: Res<Assets<Image>>,
mut commands: Commands,
irradiance_volumes: Query<(&IrradianceVolume, &GlobalTransform)>,
voxel_cube_parents: Query<Entity, With<VoxelCubeParent>>,
voxel_cubes: Query<Entity, With<VoxelCube>>,
example_assets: Res<ExampleAssets>,
mut voxel_visualization_material_assets: ResMut<Assets<VoxelVisualizationMaterial>>,
) {
// If voxel cubes have already been spawned, don't do anything.
if !voxel_cubes.is_empty() {
return;
}
let Some(voxel_cube_parent) = voxel_cube_parents.iter().next() else {
return;
};
for (irradiance_volume, global_transform) in irradiance_volumes.iter() {
let Some(image) = image_assets.get(&irradiance_volume.voxels) else {
continue;
};
let resolution = image.texture_descriptor.size;
let voxel_cube_material = voxel_visualization_material_assets.add(ExtendedMaterial {
base: StandardMaterial::from(Color::RED),
extension: VoxelVisualizationExtension {
irradiance_volume_info: VoxelVisualizationIrradianceVolumeInfo {
transform: VOXEL_TRANSFORM.inverse(),
inverse_transform: VOXEL_TRANSFORM,
resolution: uvec3(
resolution.width,
resolution.height,
resolution.depth_or_array_layers,
),
intensity: IRRADIANCE_VOLUME_INTENSITY,
},
},
});
let scale = vec3(
1.0 / resolution.width as f32,
1.0 / resolution.height as f32,
1.0 / resolution.depth_or_array_layers as f32,
);
// Spawn a cube for each voxel.
for z in 0..resolution.depth_or_array_layers {
for y in 0..resolution.height {
for x in 0..resolution.width {
let uvw = (uvec3(x, y, z).as_vec3() + 0.5) * scale - 0.5;
let pos = global_transform.transform_point(uvw);
let voxel_cube = commands
.spawn(MaterialMeshBundle {
mesh: example_assets.voxel_cube.clone(),
material: voxel_cube_material.clone(),
transform: Transform::from_scale(Vec3::splat(VOXEL_CUBE_SCALE))
.with_translation(pos),
..default()
})
.insert(VoxelCube)
.insert(NotShadowCaster)
.id();
commands.entity(voxel_cube_parent).add_child(voxel_cube);
}
}
}
}
}
// Draws a gizmo showing the bounds of the irradiance volume.
fn draw_gizmo(
mut gizmos: Gizmos,
irradiance_volume_query: Query<&GlobalTransform, With<IrradianceVolume>>,
app_status: Res<AppStatus>,
) {
if app_status.voxels_visible {
for transform in irradiance_volume_query.iter() {
gizmos.cuboid(*transform, GIZMO_COLOR);
}
}
}
// Handles a request from the user to toggle the voxel visibility on and off.
fn toggle_voxel_visibility(
keyboard: Res<ButtonInput<KeyCode>>,
mut app_status: ResMut<AppStatus>,
mut voxel_cube_parent_query: Query<&mut Visibility, With<VoxelCubeParent>>,
) {
if !keyboard.just_pressed(KeyCode::Backspace) {
return;
}
app_status.voxels_visible = !app_status.voxels_visible;
for mut visibility in voxel_cube_parent_query.iter_mut() {
*visibility = if app_status.voxels_visible {
Visibility::Visible
} else {
Visibility::Hidden
};
}
}
impl MaterialExtension for VoxelVisualizationExtension {
fn fragment_shader() -> ShaderRef {
"shaders/irradiance_volume_voxel_visualization.wgsl".into()
}
}

1
examples/README.md
View file

@ -130,6 +130,7 @@ Example | Description
[Deterministic rendering](../examples/3d/deterministic.rs) | Stop flickering from z-fighting at a performance cost
[Fog](../examples/3d/fog.rs) | A scene showcasing the distance fog effect
[Generate Custom Mesh](../examples/3d/generate_custom_mesh.rs) | Simple showcase of how to generate a custom mesh with a custom texture
[Irradiance Volumes](../examples/3d/irradiance_volumes.rs) | Demonstrates irradiance volumes
[Lighting](../examples/3d/lighting.rs) | Illustrates various lighting options in a simple scene
[Lightmaps](../examples/3d/lightmaps.rs) | Rendering a scene with baked lightmaps
[Lines](../examples/3d/lines.rs) | Create a custom material to draw 3d lines