EnvironmentMapLight, BRDF Improvements (#7051)

(Before)
![image](https://user-images.githubusercontent.com/47158642/213946111-15ec758f-1f1d-443c-b196-1fdcd4ae49da.png)
(After)
![image](https://user-images.githubusercontent.com/47158642/217051179-67381e73-dd44-461b-a2c7-87b0440ef8de.png)
![image](https://user-images.githubusercontent.com/47158642/212492404-524e4ad3-7837-4ed4-8b20-2abc276aa8e8.png)

# Objective
- Improve lighting; especially reflections.
- Closes https://github.com/bevyengine/bevy/issues/4581.

## Solution
- Implement environment maps, providing better ambient light.
- Add microfacet multibounce approximation for specular highlights from Filament.
- Occlusion is no longer incorrectly applied to direct lighting. It now only applies to diffuse indirect light. Unsure if it's also supposed to apply to specular indirect light - the glTF specification just says "indirect light". In the case of ambient occlusion, for instance, that's usually only calculated as diffuse though. For now, I'm choosing to apply this just to indirect diffuse light, and not specular.
- Modified the PBR example to use an environment map, and have labels.
- Added `FallbackImageCubemap`.

## Implementation
- IBL technique references can be found in environment_map.wgsl.
- It's more accurate to use a LUT for the scale/bias. Filament has a good reference on generating this LUT. For now, I just used an analytic approximation.
 - For now, environment maps must first be prefiltered outside of bevy using a 3rd party tool. See the `EnvironmentMap` documentation.
- Eventually, we should have our own prefiltering code, so that we can have dynamically changing environment maps, as well as let users drop in an HDR image and use asset preprocessing to create the needed textures using only bevy. 

---

## Changelog
- Added an `EnvironmentMapLight` camera component that adds additional ambient light to a scene.
- StandardMaterials will now appear brighter and more saturated at high roughness, due to internal material changes. This is more physically correct.
- Fixed StandardMaterial occlusion being incorrectly applied to direct lighting.
- Added `FallbackImageCubemap`.

Co-authored-by: IceSentry <c.giguere42@gmail.com>
Co-authored-by: James Liu <contact@jamessliu.com>
Co-authored-by: Rob Parrett <robparrett@gmail.com>
This commit is contained in:
JMS55 2023-02-09 16:46:32 +00:00
parent 1ca8755cc5
commit dd4299bcf9
20 changed files with 522 additions and 80 deletions

View file

@ -190,14 +190,14 @@ jobs:
- name: Build bevy
# this uses the same command as when running the example to ensure build is reused
run: |
TRACE_CHROME=trace-alien_cake_addict.json CI_TESTING_CONFIG=.github/example-run/alien_cake_addict.ron cargo build --example alien_cake_addict --features "bevy_ci_testing,trace,trace_chrome"
TRACE_CHROME=trace-alien_cake_addict.json CI_TESTING_CONFIG=.github/example-run/alien_cake_addict.ron cargo build --example alien_cake_addict --features "bevy_ci_testing,trace,trace_chrome,ktx2,zstd"
- name: Run examples
run: |
for example in .github/example-run/*.ron; do
example_name=`basename $example .ron`
echo -n $example_name > last_example_run
echo "running $example_name - "`date`
time TRACE_CHROME=trace-$example_name.json CI_TESTING_CONFIG=$example xvfb-run cargo run --example $example_name --features "bevy_ci_testing,trace,trace_chrome"
time TRACE_CHROME=trace-$example_name.json CI_TESTING_CONFIG=$example xvfb-run cargo run --example $example_name --features "bevy_ci_testing,trace,trace_chrome,ktx2,zstd"
sleep 10
done
zip traces.zip trace*.json

View file

@ -80,7 +80,7 @@ jobs:
shell: bash
# this uses the same command as when running the example to ensure build is reused
run: |
WGPU_BACKEND=dx12 CI_TESTING_CONFIG=.github/example-run/alien_cake_addict.ron cargo build --example alien_cake_addict --features "bevy_ci_testing"
WGPU_BACKEND=dx12 CI_TESTING_CONFIG=.github/example-run/alien_cake_addict.ron cargo build --example alien_cake_addict --features "bevy_ci_testing,ktx2,zstd"
- name: Run examples
shell: bash
@ -88,7 +88,7 @@ jobs:
for example in .github/example-run/*.ron; do
example_name=`basename $example .ron`
echo "running $example_name - "`date`
time WGPU_BACKEND=dx12 CI_TESTING_CONFIG=$example cargo run --example $example_name --features "bevy_ci_testing"
time WGPU_BACKEND=dx12 CI_TESTING_CONFIG=$example cargo run --example $example_name --features "bevy_ci_testing,ktx2,zstd"
sleep 10
done

View file

@ -372,6 +372,7 @@ wasm = false
[[example]]
name = "load_gltf"
path = "examples/3d/load_gltf.rs"
required-features = ["ktx2", "zstd"]
[package.metadata.example.load_gltf]
name = "Load glTF"
@ -422,6 +423,7 @@ wasm = true
[[example]]
name = "pbr"
path = "examples/3d/pbr.rs"
required-features = ["ktx2", "zstd"]
[package.metadata.example.pbr]
name = "Physically Based Rendering"
@ -1430,6 +1432,7 @@ wasm = true
[[example]]
name = "scene_viewer"
path = "examples/tools/scene_viewer/main.rs"
required-features = ["ktx2", "zstd"]
[package.metadata.example.scene_viewer]
name = "Scene Viewer"

View file

@ -0,0 +1,6 @@
The pisa_*.ktx2 files were generated from https://github.com/KhronosGroup/glTF-Sample-Environments/blob/master/pisa.hdr using the following tools and commands:
- IBL environment map prefiltering to cubemaps: https://github.com/KhronosGroup/glTF-IBL-Sampler
- Diffuse: ./cli -inputPath pisa.hdr -outCubeMap pisa_diffuse.ktx2 -distribution Lambertian -cubeMapResolution 32
- Specular: ./cli -inputPath pisa.hdr -outCubeMap pisa_specular.ktx2 -distribution GGX -cubeMapResolution 512
- Converting to rgb9e5 format with zstd 'supercompression': https://github.com/DGriffin91/bevy_mod_environment_map_tools
- cargo run --release -- --inputs pisa_diffuse.ktx2,pisa_specular.ktx2 --outputs pisa_diffuse_rgb9e5_zstd.ktx2,pisa_specular_rgb9e5_zstd.ktx2

Binary file not shown.

Binary file not shown.

View file

@ -0,0 +1,43 @@
#define_import_path bevy_pbr::environment_map
struct EnvironmentMapLight {
diffuse: vec3<f32>,
specular: vec3<f32>,
};
fn environment_map_light(
perceptual_roughness: f32,
roughness: f32,
diffuse_color: vec3<f32>,
NdotV: f32,
f_ab: vec2<f32>,
N: vec3<f32>,
R: vec3<f32>,
F0: vec3<f32>,
) -> EnvironmentMapLight {
// Split-sum approximation for image based lighting: https://cdn2.unrealengine.com/Resources/files/2013SiggraphPresentationsNotes-26915738.pdf
let smallest_specular_mip_level = textureNumLevels(environment_map_specular) - 1i;
let radiance_level = perceptual_roughness * f32(smallest_specular_mip_level);
let irradiance = textureSample(environment_map_diffuse, environment_map_sampler, N).rgb;
let radiance = textureSampleLevel(environment_map_specular, environment_map_sampler, R, radiance_level).rgb;
// Multiscattering approximation: https://www.jcgt.org/published/0008/01/03/paper.pdf
// Useful reference: https://bruop.github.io/ibl
let Fr = max(vec3(1.0 - roughness), F0) - F0;
let kS = F0 + Fr * pow(1.0 - NdotV, 5.0);
let FssEss = kS * f_ab.x + f_ab.y;
let Ess = f_ab.x + f_ab.y;
let Ems = 1.0 - Ess;
let Favg = F0 + (1.0 - F0) / 21.0;
let Fms = FssEss * Favg / (1.0 - Ems * Favg);
let FmsEms = Fms * Ems;
let Edss = 1.0 - (FssEss + FmsEms);
let kD = diffuse_color * Edss;
var out: EnvironmentMapLight;
out.diffuse = (FmsEms + kD) * irradiance;
out.specular = FssEss * radiance;
return out;
}

View file

@ -0,0 +1,137 @@
use bevy_app::{App, Plugin};
use bevy_asset::{load_internal_asset, Handle, HandleUntyped};
use bevy_core_pipeline::prelude::Camera3d;
use bevy_ecs::{prelude::Component, query::With};
use bevy_reflect::{Reflect, TypeUuid};
use bevy_render::{
extract_component::{ExtractComponent, ExtractComponentPlugin},
render_asset::RenderAssets,
render_resource::{
BindGroupEntry, BindGroupLayoutEntry, BindingResource, BindingType, SamplerBindingType,
Shader, ShaderStages, TextureSampleType, TextureViewDimension,
},
texture::{FallbackImageCubemap, Image},
};
pub const ENVIRONMENT_MAP_SHADER_HANDLE: HandleUntyped =
HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 154476556247605696);
pub struct EnvironmentMapPlugin;
impl Plugin for EnvironmentMapPlugin {
fn build(&self, app: &mut App) {
load_internal_asset!(
app,
ENVIRONMENT_MAP_SHADER_HANDLE,
"environment_map.wgsl",
Shader::from_wgsl
);
app.register_type::<EnvironmentMapLight>()
.add_plugin(ExtractComponentPlugin::<EnvironmentMapLight>::default());
}
}
/// Environment map based ambient lighting representing light from distant scenery.
///
/// When added to a 3D camera, this component adds indirect light
/// to every point of the scene (including inside, enclosed areas) based on
/// an environment cubemap texture. This is similiar to [`crate::AmbientLight`], but
/// higher quality, and is intended for outdoor scenes.
///
/// The environment map must be prefiltered into a diffuse and specular cubemap based on the
/// [split-sum approximation](https://cdn2.unrealengine.com/Resources/files/2013SiggraphPresentationsNotes-26915738.pdf).
///
/// To prefilter your environment map, you can use `KhronosGroup`'s [glTF-IBL-Sampler](https://github.com/KhronosGroup/glTF-IBL-Sampler).
/// The diffuse map uses the Lambertian distribution, and the specular map uses the GGX distribution.
///
/// `KhronosGroup` also has several prefiltered environment maps that can be found [here](https://github.com/KhronosGroup/glTF-Sample-Environments).
#[derive(Component, Reflect, Clone)]
pub struct EnvironmentMapLight {
pub diffuse_map: Handle<Image>,
pub specular_map: Handle<Image>,
}
impl EnvironmentMapLight {
/// Whether or not all textures neccesary to use the environment map
/// have been loaded by the asset server.
pub fn is_loaded(&self, images: &RenderAssets<Image>) -> bool {
images.get(&self.diffuse_map).is_some() && images.get(&self.specular_map).is_some()
}
}
impl ExtractComponent for EnvironmentMapLight {
type Query = &'static Self;
type Filter = With<Camera3d>;
type Out = Self;
fn extract_component(item: bevy_ecs::query::QueryItem<'_, Self::Query>) -> Option<Self::Out> {
Some(item.clone())
}
}
pub fn get_bindings<'a>(
environment_map_light: Option<&EnvironmentMapLight>,
images: &'a RenderAssets<Image>,
fallback_image_cubemap: &'a FallbackImageCubemap,
bindings: [u32; 3],
) -> [BindGroupEntry<'a>; 3] {
let (diffuse_map, specular_map) = match (
environment_map_light.and_then(|env_map| images.get(&env_map.diffuse_map)),
environment_map_light.and_then(|env_map| images.get(&env_map.specular_map)),
) {
(Some(diffuse_map), Some(specular_map)) => {
(&diffuse_map.texture_view, &specular_map.texture_view)
}
_ => (
&fallback_image_cubemap.texture_view,
&fallback_image_cubemap.texture_view,
),
};
[
BindGroupEntry {
binding: bindings[0],
resource: BindingResource::TextureView(diffuse_map),
},
BindGroupEntry {
binding: bindings[1],
resource: BindingResource::TextureView(specular_map),
},
BindGroupEntry {
binding: bindings[2],
resource: BindingResource::Sampler(&fallback_image_cubemap.sampler),
},
]
}
pub fn get_bind_group_layout_entries(bindings: [u32; 3]) -> [BindGroupLayoutEntry; 3] {
[
BindGroupLayoutEntry {
binding: bindings[0],
visibility: ShaderStages::FRAGMENT,
ty: BindingType::Texture {
sample_type: TextureSampleType::Float { filterable: true },
view_dimension: TextureViewDimension::Cube,
multisampled: false,
},
count: None,
},
BindGroupLayoutEntry {
binding: bindings[1],
visibility: ShaderStages::FRAGMENT,
ty: BindingType::Texture {
sample_type: TextureSampleType::Float { filterable: true },
view_dimension: TextureViewDimension::Cube,
multisampled: false,
},
count: None,
},
BindGroupLayoutEntry {
binding: bindings[2],
visibility: ShaderStages::FRAGMENT,
ty: BindingType::Sampler(SamplerBindingType::Filtering),
count: None,
},
]
}

View file

@ -2,6 +2,7 @@ pub mod wireframe;
mod alpha;
mod bundle;
mod environment_map;
mod fog;
mod light;
mod material;
@ -10,8 +11,8 @@ mod prepass;
mod render;
pub use alpha::*;
use bevy_transform::TransformSystem;
pub use bundle::*;
pub use environment_map::EnvironmentMapLight;
pub use fog::*;
pub use light::*;
pub use material::*;
@ -27,6 +28,7 @@ pub mod prelude {
DirectionalLightBundle, MaterialMeshBundle, PbrBundle, PointLightBundle,
SpotLightBundle,
},
environment_map::EnvironmentMapLight,
fog::{FogFalloff, FogSettings},
light::{AmbientLight, DirectionalLight, PointLight, SpotLight},
material::{Material, MaterialPlugin},
@ -55,6 +57,8 @@ use bevy_render::{
view::{ViewSet, VisibilitySystems},
ExtractSchedule, RenderApp, RenderSet,
};
use bevy_transform::TransformSystem;
use environment_map::EnvironmentMapPlugin;
pub const PBR_TYPES_SHADER_HANDLE: HandleUntyped =
HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 1708015359337029744);
@ -172,6 +176,7 @@ impl Plugin for PbrPlugin {
prepass_enabled: self.prepass_enabled,
..Default::default()
})
.add_plugin(EnvironmentMapPlugin)
.init_resource::<AmbientLight>()
.init_resource::<GlobalVisiblePointLights>()
.init_resource::<DirectionalLightShadowMap>()

View file

@ -1,6 +1,6 @@
use crate::{
AlphaMode, DrawMesh, MeshPipeline, MeshPipelineKey, MeshUniform, PrepassPlugin,
SetMeshBindGroup, SetMeshViewBindGroup,
AlphaMode, DrawMesh, EnvironmentMapLight, MeshPipeline, MeshPipelineKey, MeshUniform,
PrepassPlugin, SetMeshBindGroup, SetMeshViewBindGroup,
};
use bevy_app::{App, Plugin};
use bevy_asset::{AddAsset, AssetEvent, AssetServer, Assets, Handle};
@ -361,10 +361,12 @@ pub fn queue_material_meshes<M: Material>(
render_meshes: Res<RenderAssets<Mesh>>,
render_materials: Res<RenderMaterials<M>>,
material_meshes: Query<(&Handle<M>, &Handle<Mesh>, &MeshUniform)>,
images: Res<RenderAssets<Image>>,
mut views: Query<(
&ExtractedView,
&VisibleEntities,
Option<&Tonemapping>,
Option<&EnvironmentMapLight>,
&mut RenderPhase<Opaque3d>,
&mut RenderPhase<AlphaMask3d>,
&mut RenderPhase<Transparent3d>,
@ -376,6 +378,7 @@ pub fn queue_material_meshes<M: Material>(
view,
visible_entities,
tonemapping,
environment_map,
mut opaque_phase,
mut alpha_mask_phase,
mut transparent_phase,
@ -388,6 +391,14 @@ pub fn queue_material_meshes<M: Material>(
let mut view_key = MeshPipelineKey::from_msaa_samples(msaa.samples())
| MeshPipelineKey::from_hdr(view.hdr);
let environment_map_loaded = match environment_map {
Some(environment_map) => environment_map.is_loaded(&images),
None => false,
};
if environment_map_loaded {
view_key |= MeshPipelineKey::ENVIRONMENT_MAP;
}
if let Some(Tonemapping::Enabled { deband_dither }) = tonemapping {
if !view.hdr {
view_key |= MeshPipelineKey::TONEMAP_IN_SHADER;
@ -397,8 +408,8 @@ pub fn queue_material_meshes<M: Material>(
}
}
}
let rangefinder = view.rangefinder3d();
let rangefinder = view.rangefinder3d();
for visible_entity in &visible_entities.entities {
if let Ok((material_handle, mesh_handle, mesh_uniform)) =
material_meshes.get(*visible_entity)

View file

@ -1,8 +1,8 @@
use crate::{
FogMeta, GlobalLightMeta, GpuFog, GpuLights, GpuPointLights, LightMeta, NotShadowCaster,
NotShadowReceiver, ShadowPipeline, ViewClusterBindings, ViewFogUniformOffset,
ViewLightsUniformOffset, ViewShadowBindings, CLUSTERED_FORWARD_STORAGE_BUFFER_COUNT,
MAX_CASCADES_PER_LIGHT, MAX_DIRECTIONAL_LIGHTS,
environment_map, EnvironmentMapLight, FogMeta, GlobalLightMeta, GpuFog, GpuLights,
GpuPointLights, LightMeta, NotShadowCaster, NotShadowReceiver, ShadowPipeline,
ViewClusterBindings, ViewFogUniformOffset, ViewLightsUniformOffset, ViewShadowBindings,
CLUSTERED_FORWARD_STORAGE_BUFFER_COUNT, MAX_CASCADES_PER_LIGHT, MAX_DIRECTIONAL_LIGHTS,
};
use bevy_app::Plugin;
use bevy_asset::{load_internal_asset, Assets, Handle, HandleUntyped};
@ -27,8 +27,8 @@ use bevy_render::{
render_resource::*,
renderer::{RenderDevice, RenderQueue},
texture::{
BevyDefault, DefaultImageSampler, FallbackImagesDepth, FallbackImagesMsaa, GpuImage, Image,
ImageSampler, TextureFormatPixelInfo,
BevyDefault, DefaultImageSampler, FallbackImageCubemap, FallbackImagesDepth,
FallbackImagesMsaa, GpuImage, Image, ImageSampler, TextureFormatPixelInfo,
},
view::{ComputedVisibility, ViewTarget, ViewUniform, ViewUniformOffset, ViewUniforms},
Extract, ExtractSchedule, RenderApp, RenderSet,
@ -412,10 +412,16 @@ impl FromWorld for MeshPipeline {
count: None,
},
];
// EnvironmentMapLight
let environment_map_entries =
environment_map::get_bind_group_layout_entries([11, 12, 13]);
entries.extend_from_slice(&environment_map_entries);
if cfg!(not(feature = "webgl")) {
// Depth texture
entries.push(BindGroupLayoutEntry {
binding: 11,
binding: 14,
visibility: ShaderStages::FRAGMENT,
ty: BindingType::Texture {
multisampled,
@ -426,7 +432,7 @@ impl FromWorld for MeshPipeline {
});
// Normal texture
entries.push(BindGroupLayoutEntry {
binding: 12,
binding: 15,
visibility: ShaderStages::FRAGMENT,
ty: BindingType::Texture {
multisampled,
@ -436,6 +442,7 @@ impl FromWorld for MeshPipeline {
count: None,
});
}
entries
}
@ -574,6 +581,7 @@ bitflags::bitflags! {
const DEPTH_PREPASS = (1 << 3);
const NORMAL_PREPASS = (1 << 4);
const ALPHA_MASK = (1 << 5);
const ENVIRONMENT_MAP = (1 << 6);
const BLEND_RESERVED_BITS = Self::BLEND_MASK_BITS << Self::BLEND_SHIFT_BITS; // ← Bitmask reserving bits for the blend state
const BLEND_OPAQUE = (0 << Self::BLEND_SHIFT_BITS); // ← Values are just sequential within the mask, and can range from 0 to 3
const BLEND_PREMULTIPLIED_ALPHA = (1 << Self::BLEND_SHIFT_BITS); //
@ -741,6 +749,10 @@ impl SpecializedMeshPipeline for MeshPipeline {
}
}
if key.contains(MeshPipelineKey::ENVIRONMENT_MAP) {
shader_defs.push("ENVIRONMENT_MAP".into());
}
let format = if key.contains(MeshPipelineKey::HDR) {
ViewTarget::TEXTURE_FORMAT_HDR
} else {
@ -905,9 +917,12 @@ pub fn queue_mesh_view_bind_groups(
&ViewShadowBindings,
&ViewClusterBindings,
Option<&ViewPrepassTextures>,
Option<&EnvironmentMapLight>,
)>,
images: Res<RenderAssets<Image>>,
mut fallback_images: FallbackImagesMsaa,
mut fallback_depths: FallbackImagesDepth,
fallback_cubemap: Res<FallbackImageCubemap>,
msaa: Res<Msaa>,
globals_buffer: Res<GlobalsBuffer>,
) {
@ -924,7 +939,14 @@ pub fn queue_mesh_view_bind_groups(
globals_buffer.buffer.binding(),
fog_meta.gpu_fogs.binding(),
) {
for (entity, view_shadow_bindings, view_cluster_bindings, prepass_textures) in &views {
for (
entity,
view_shadow_bindings,
view_cluster_bindings,
prepass_textures,
environment_map,
) in &views
{
let layout = if msaa.samples() > 1 {
&mesh_pipeline.view_layout_multisampled
} else {
@ -982,6 +1004,14 @@ pub fn queue_mesh_view_bind_groups(
},
];
let env_map = environment_map::get_bindings(
environment_map,
&images,
&fallback_cubemap,
[11, 12, 13],
);
entries.extend_from_slice(&env_map);
// When using WebGL with MSAA, we can't create the fallback textures required by the prepass
// When using WebGL, and MSAA is disabled, we can't bind the textures either
if cfg!(not(feature = "webgl")) {
@ -994,7 +1024,7 @@ pub fn queue_mesh_view_bind_groups(
}
};
entries.push(BindGroupEntry {
binding: 11,
binding: 14,
resource: BindingResource::TextureView(depth_view),
});
@ -1007,7 +1037,7 @@ pub fn queue_mesh_view_bind_groups(
}
};
entries.push(BindGroupEntry {
binding: 12,
binding: 15,
resource: BindingResource::TextureView(normal_view),
});
}

View file

@ -46,14 +46,21 @@ var<uniform> globals: Globals;
@group(0) @binding(10)
var<uniform> fog: Fog;
#ifdef MULTISAMPLED
@group(0) @binding(11)
var depth_prepass_texture: texture_depth_multisampled_2d;
var environment_map_diffuse: texture_cube<f32>;
@group(0) @binding(12)
var environment_map_specular: texture_cube<f32>;
@group(0) @binding(13)
var environment_map_sampler: sampler;
#ifdef MULTISAMPLED
@group(0) @binding(14)
var depth_prepass_texture: texture_depth_multisampled_2d;
@group(0) @binding(15)
var normal_prepass_texture: texture_multisampled_2d<f32>;
#else
@group(0) @binding(11)
@group(0) @binding(14)
var depth_prepass_texture: texture_depth_2d;
@group(0) @binding(12)
@group(0) @binding(15)
var normal_prepass_texture: texture_2d<f32>;
#endif

View file

@ -12,8 +12,8 @@ fn ambient_light(
perceptual_roughness: f32,
occlusion: f32,
) -> vec3<f32> {
let diffuse_ambient = EnvBRDFApprox(diffuse_color, 1.0, NdotV);
let specular_ambient = EnvBRDFApprox(specular_color, perceptual_roughness, NdotV);
let diffuse_ambient = EnvBRDFApprox(diffuse_color, F_AB(1.0, NdotV)) * occlusion;
let specular_ambient = EnvBRDFApprox(specular_color, F_AB(perceptual_roughness, NdotV));
return (diffuse_ambient + specular_ambient) * lights.ambient_color.rgb * occlusion;
}
return (diffuse_ambient + specular_ambient) * lights.ambient_color.rgb;
}

View file

@ -4,6 +4,9 @@
#import bevy_core_pipeline::tonemapping
#endif
#ifdef ENVIRONMENT_MAP
#import bevy_pbr::environment_map
#endif
fn alpha_discard(material: StandardMaterial, output_color: vec4<f32>) -> vec4<f32> {
var color = output_color;
@ -183,8 +186,9 @@ fn pbr(
let R = reflect(-in.V, in.N);
// accumulate color
var light_accum: vec3<f32> = vec3<f32>(0.0);
let f_ab = F_AB(perceptual_roughness, NdotV);
var direct_light: vec3<f32> = vec3<f32>(0.0);
let view_z = dot(vec4<f32>(
view.inverse_view[0].z,
@ -195,7 +199,7 @@ fn pbr(
let cluster_index = fragment_cluster_index(in.frag_coord.xy, view_z, in.is_orthographic);
let offset_and_counts = unpack_offset_and_counts(cluster_index);
// point lights
// Point lights (direct)
for (var i: u32 = offset_and_counts[0]; i < offset_and_counts[0] + offset_and_counts[1]; i = i + 1u) {
let light_id = get_light_id(i);
var shadow: f32 = 1.0;
@ -203,11 +207,11 @@ fn pbr(
&& (point_lights.data[light_id].flags & POINT_LIGHT_FLAGS_SHADOWS_ENABLED_BIT) != 0u) {
shadow = fetch_point_shadow(light_id, in.world_position, in.world_normal);
}
let light_contrib = point_light(in.world_position.xyz, light_id, roughness, NdotV, in.N, in.V, R, F0, diffuse_color);
light_accum = light_accum + light_contrib * shadow;
let light_contrib = point_light(in.world_position.xyz, light_id, roughness, NdotV, in.N, in.V, R, F0, f_ab, diffuse_color);
direct_light += light_contrib * shadow;
}
// spot lights
// Spot lights (direct)
for (var i: u32 = offset_and_counts[0] + offset_and_counts[1]; i < offset_and_counts[0] + offset_and_counts[1] + offset_and_counts[2]; i = i + 1u) {
let light_id = get_light_id(i);
var shadow: f32 = 1.0;
@ -215,10 +219,11 @@ fn pbr(
&& (point_lights.data[light_id].flags & POINT_LIGHT_FLAGS_SHADOWS_ENABLED_BIT) != 0u) {
shadow = fetch_spot_shadow(light_id, in.world_position, in.world_normal);
}
let light_contrib = spot_light(in.world_position.xyz, light_id, roughness, NdotV, in.N, in.V, R, F0, diffuse_color);
light_accum = light_accum + light_contrib * shadow;
let light_contrib = spot_light(in.world_position.xyz, light_id, roughness, NdotV, in.N, in.V, R, F0, f_ab, diffuse_color);
direct_light += light_contrib * shadow;
}
// Directional lights (direct)
let n_directional_lights = lights.n_directional_lights;
for (var i: u32 = 0u; i < n_directional_lights; i = i + 1u) {
var shadow: f32 = 1.0;
@ -226,17 +231,27 @@ fn pbr(
&& (lights.directional_lights[i].flags & DIRECTIONAL_LIGHT_FLAGS_SHADOWS_ENABLED_BIT) != 0u) {
shadow = fetch_directional_shadow(i, in.world_position, in.world_normal, view_z);
}
var light_contrib = directional_light(i, roughness, NdotV, in.N, in.V, R, F0, diffuse_color);
var light_contrib = directional_light(i, roughness, NdotV, in.N, in.V, R, F0, f_ab, diffuse_color);
#ifdef DIRECTIONAL_LIGHT_SHADOW_MAP_DEBUG_CASCADES
light_contrib = cascade_debug_visualization(light_contrib, i, view_z);
#endif
light_accum = light_accum + light_contrib * shadow;
direct_light += light_contrib * shadow;
}
let ambient_contrib = ambient_light(in.world_position, in.N, in.V, NdotV, diffuse_color, F0, perceptual_roughness, occlusion);
// Ambient light (indirect)
var indirect_light = ambient_light(in.world_position, in.N, in.V, NdotV, diffuse_color, F0, perceptual_roughness, occlusion);
// Environment map light (indirect)
#ifdef ENVIRONMENT_MAP
let environment_light = environment_map_light(perceptual_roughness, roughness, diffuse_color, NdotV, f_ab, in.N, R, F0);
indirect_light += (environment_light.diffuse * occlusion) + environment_light.specular;
#endif
let emissive_light = emissive.rgb * output_color.a;
// Total light
output_color = vec4<f32>(
light_accum + ambient_contrib + emissive.rgb * output_color.a,
direct_light + indirect_light + emissive_light,
output_color.a
);
@ -280,7 +295,7 @@ fn apply_fog(input_color: vec4<f32>, fragment_world_position: vec3<f32>, view_wo
let distance = length(view_to_world);
var scattering = vec3<f32>(0.0);
if (fog.directional_light_color.a > 0.0) {
if fog.directional_light_color.a > 0.0 {
let view_to_world_normalized = view_to_world / distance;
let n_directional_lights = lights.n_directional_lights;
for (var i: u32 = 0u; i < n_directional_lights; i = i + 1u) {
@ -295,13 +310,13 @@ fn apply_fog(input_color: vec4<f32>, fragment_world_position: vec3<f32>, view_wo
}
}
if (fog.mode == FOG_MODE_LINEAR) {
if fog.mode == FOG_MODE_LINEAR {
return linear_fog(input_color, distance, scattering);
} else if (fog.mode == FOG_MODE_EXPONENTIAL) {
} else if fog.mode == FOG_MODE_EXPONENTIAL {
return exponential_fog(input_color, distance, scattering);
} else if (fog.mode == FOG_MODE_EXPONENTIAL_SQUARED) {
} else if fog.mode == FOG_MODE_EXPONENTIAL_SQUARED {
return exponential_squared_fog(input_color, distance, scattering);
} else if (fog.mode == FOG_MODE_ATMOSPHERIC) {
} else if fog.mode == FOG_MODE_ATMOSPHERIC {
return atmospheric_fog(input_color, distance, scattering);
} else {
return input_color;
@ -320,7 +335,7 @@ fn premultiply_alpha(standard_material_flags: u32, color: vec4<f32>) -> vec4<f32
//
// result = 1 * src_color + (1 - src_alpha) * dst_color
let alpha_mode = standard_material_flags & STANDARD_MATERIAL_FLAGS_ALPHA_MODE_RESERVED_BITS;
if (alpha_mode == STANDARD_MATERIAL_FLAGS_ALPHA_MODE_BLEND) {
if alpha_mode == STANDARD_MATERIAL_FLAGS_ALPHA_MODE_BLEND {
// Here, we premultiply `src_color` by `src_alpha` (ahead of time, here in the shader)
//
// src_color *= src_alpha
@ -332,7 +347,7 @@ fn premultiply_alpha(standard_material_flags: u32, color: vec4<f32>) -> vec4<f32
//
// Which is the blend operation for regular alpha blending `BlendState::ALPHA_BLENDING`
return vec4<f32>(color.rgb * color.a, color.a);
} else if (alpha_mode == STANDARD_MATERIAL_FLAGS_ALPHA_MODE_ADD) {
} else if alpha_mode == STANDARD_MATERIAL_FLAGS_ALPHA_MODE_ADD {
// Here, we premultiply `src_color` by `src_alpha`, and replace `src_alpha` with 0.0:
//
// src_color *= src_alpha

View file

@ -101,13 +101,27 @@ fn fresnel(f0: vec3<f32>, LoH: f32) -> vec3<f32> {
// Cook-Torrance approximation of the microfacet model integration using Fresnel law F to model f_m
// f_r(v,l) = { D(h,α) G(v,l,α) F(v,h,f0) } / { 4 (nv) (nl) }
fn specular(f0: vec3<f32>, roughness: f32, h: vec3<f32>, NoV: f32, NoL: f32,
NoH: f32, LoH: f32, specularIntensity: f32) -> vec3<f32> {
fn specular(
f0: vec3<f32>,
roughness: f32,
h: vec3<f32>,
NoV: f32,
NoL: f32,
NoH: f32,
LoH: f32,
specularIntensity: f32,
f_ab: vec2<f32>
) -> vec3<f32> {
let D = D_GGX(roughness, NoH, h);
let V = V_SmithGGXCorrelated(roughness, NoV, NoL);
let F = fresnel(f0, LoH);
return (specularIntensity * D * V) * F;
var Fr = (specularIntensity * D * V) * F;
// Multiscattering approximation: https://google.github.io/filament/Filament.html#listing_energycompensationimpl
Fr *= 1.0 + f0 * (1.0 / f_ab.x - 1.0);
return Fr;
}
// Diffuse BRDF
@ -131,14 +145,19 @@ fn Fd_Burley(roughness: f32, NoV: f32, NoL: f32, LoH: f32) -> f32 {
return lightScatter * viewScatter * (1.0 / PI);
}
// From https://www.unrealengine.com/en-US/blog/physically-based-shading-on-mobile
fn EnvBRDFApprox(f0: vec3<f32>, perceptual_roughness: f32, NoV: f32) -> vec3<f32> {
// Scale/bias approximation
// https://www.unrealengine.com/en-US/blog/physically-based-shading-on-mobile
// TODO: Use a LUT (more accurate)
fn F_AB(perceptual_roughness: f32, NoV: f32) -> vec2<f32> {
let c0 = vec4<f32>(-1.0, -0.0275, -0.572, 0.022);
let c1 = vec4<f32>(1.0, 0.0425, 1.04, -0.04);
let r = perceptual_roughness * c0 + c1;
let a004 = min(r.x * r.x, exp2(-9.28 * NoV)) * r.x + r.y;
let AB = vec2<f32>(-1.04, 1.04) * a004 + r.zw;
return f0 * AB.x + AB.y;
return vec2<f32>(-1.04, 1.04) * a004 + r.zw;
}
fn EnvBRDFApprox(f0: vec3<f32>, f_ab: vec2<f32>) -> vec3<f32> {
return f0 * f_ab.x + f_ab.y;
}
fn perceptualRoughnessToRoughness(perceptualRoughness: f32) -> f32 {
@ -150,14 +169,21 @@ fn perceptualRoughnessToRoughness(perceptualRoughness: f32) -> f32 {
}
fn point_light(
world_position: vec3<f32>, light_id: u32, roughness: f32, NdotV: f32, N: vec3<f32>, V: vec3<f32>,
R: vec3<f32>, F0: vec3<f32>, diffuseColor: vec3<f32>
world_position: vec3<f32>,
light_id: u32,
roughness: f32,
NdotV: f32,
N: vec3<f32>,
V: vec3<f32>,
R: vec3<f32>,
F0: vec3<f32>,
f_ab: vec2<f32>,
diffuseColor: vec3<f32>
) -> vec3<f32> {
let light = &point_lights.data[light_id];
let light_to_frag = (*light).position_radius.xyz - world_position.xyz;
let distance_square = dot(light_to_frag, light_to_frag);
let rangeAttenuation =
getDistanceAttenuation(distance_square, (*light).color_inverse_square_range.w);
let rangeAttenuation = getDistanceAttenuation(distance_square, (*light).color_inverse_square_range.w);
// Specular.
// Representative Point Area Lights.
@ -175,7 +201,7 @@ fn point_light(
var NoH: f32 = saturate(dot(N, H));
var LoH: f32 = saturate(dot(L, H));
let specular_light = specular(F0, roughness, H, NdotV, NoL, NoH, LoH, specularIntensity);
let specular_light = specular(F0, roughness, H, NdotV, NoL, NoH, LoH, specularIntensity, f_ab);
// Diffuse.
// Comes after specular since its NoL is used in the lighting equation.
@ -200,24 +226,30 @@ fn point_light(
// NOTE: (*light).color.rgb is premultiplied with (*light).intensity / 4 π (which would be the luminous intensity) on the CPU
// TODO compensate for energy loss https://google.github.io/filament/Filament.html#materialsystem/improvingthebrdfs/energylossinspecularreflectance
return ((diffuse + specular_light) * (*light).color_inverse_square_range.rgb) * (rangeAttenuation * NoL);
}
fn spot_light(
world_position: vec3<f32>, light_id: u32, roughness: f32, NdotV: f32, N: vec3<f32>, V: vec3<f32>,
R: vec3<f32>, F0: vec3<f32>, diffuseColor: vec3<f32>
world_position: vec3<f32>,
light_id: u32,
roughness: f32,
NdotV: f32,
N: vec3<f32>,
V: vec3<f32>,
R: vec3<f32>,
F0: vec3<f32>,
f_ab: vec2<f32>,
diffuseColor: vec3<f32>
) -> vec3<f32> {
// reuse the point light calculations
let point_light = point_light(world_position, light_id, roughness, NdotV, N, V, R, F0, diffuseColor);
let point_light = point_light(world_position, light_id, roughness, NdotV, N, V, R, F0, f_ab, diffuseColor);
let light = &point_lights.data[light_id];
// reconstruct spot dir from x/z and y-direction flag
var spot_dir = vec3<f32>((*light).light_custom_data.x, 0.0, (*light).light_custom_data.y);
spot_dir.y = sqrt(max(0.0, 1.0 - spot_dir.x * spot_dir.x - spot_dir.z * spot_dir.z));
if (((*light).flags & POINT_LIGHT_FLAGS_SPOT_LIGHT_Y_NEGATIVE) != 0u) {
if ((*light).flags & POINT_LIGHT_FLAGS_SPOT_LIGHT_Y_NEGATIVE) != 0u {
spot_dir.y = -spot_dir.y;
}
let light_to_frag = (*light).position_radius.xyz - world_position.xyz;
@ -232,7 +264,7 @@ fn spot_light(
return point_light * spot_attenuation;
}
fn directional_light(light_id: u32, roughness: f32, NdotV: f32, normal: vec3<f32>, view: vec3<f32>, R: vec3<f32>, F0: vec3<f32>, diffuseColor: vec3<f32>) -> vec3<f32> {
fn directional_light(light_id: u32, roughness: f32, NdotV: f32, normal: vec3<f32>, view: vec3<f32>, R: vec3<f32>, F0: vec3<f32>, f_ab: vec2<f32>, diffuseColor: vec3<f32>) -> vec3<f32> {
let light = &lights.directional_lights[light_id];
let incident_light = (*light).direction_to_light.xyz;
@ -244,7 +276,7 @@ fn directional_light(light_id: u32, roughness: f32, NdotV: f32, normal: vec3<f32
let diffuse = diffuseColor * Fd_Burley(roughness, NdotV, NoL, LoH);
let specularIntensity = 1.0;
let specular_light = specular(F0, roughness, half_vector, NdotV, NoL, NoH, LoH, specularIntensity);
let specular_light = specular(F0, roughness, half_vector, NdotV, NoL, NoH, LoH, specularIntensity, f_ab);
return (specular_light + diffuse) * (*light).color.rgb * NoL;
}

View file

@ -1,3 +1,5 @@
use std::num::NonZeroU32;
use crate::{render_resource::*, texture::DefaultImageSampler};
use bevy_derive::{Deref, DerefMut};
use bevy_ecs::{
@ -21,17 +23,33 @@ use crate::{
#[derive(Resource, Deref)]
pub struct FallbackImage(GpuImage);
/// A [`RenderApp`](crate::RenderApp) resource that contains a "cubemap fallback image",
/// which can be used in situations where an image was not explicitly defined. The most common
/// use case is [`AsBindGroup`] implementations (such as materials) that support optional textures.
#[derive(Resource, Deref)]
pub struct FallbackImageCubemap(GpuImage);
fn fallback_image_new(
render_device: &RenderDevice,
render_queue: &RenderQueue,
default_sampler: &DefaultImageSampler,
format: TextureFormat,
dimension: TextureViewDimension,
samples: u32,
) -> GpuImage {
// TODO make this configurable
let data = vec![255; format.pixel_size()];
let mut image = Image::new_fill(Extent3d::default(), TextureDimension::D2, &data, format);
let extents = Extent3d {
width: 1,
height: 1,
depth_or_array_layers: match dimension {
TextureViewDimension::Cube => 6,
_ => 1,
},
};
let mut image = Image::new_fill(extents, TextureDimension::D2, &data, format);
image.texture_descriptor.sample_count = samples;
image.texture_descriptor.usage |= TextureUsages::RENDER_ATTACHMENT;
@ -42,7 +60,11 @@ fn fallback_image_new(
render_device.create_texture_with_data(render_queue, &image.texture_descriptor, &image.data)
};
let texture_view = texture.create_view(&TextureViewDescriptor::default());
let texture_view = texture.create_view(&TextureViewDescriptor {
dimension: Some(dimension),
array_layer_count: NonZeroU32::new(extents.depth_or_array_layers),
..TextureViewDescriptor::default()
});
let sampler = match image.sampler_descriptor {
ImageSampler::Default => (**default_sampler).clone(),
ImageSampler::Descriptor(descriptor) => render_device.create_sampler(&descriptor),
@ -69,6 +91,23 @@ impl FromWorld for FallbackImage {
render_queue,
default_sampler,
TextureFormat::bevy_default(),
TextureViewDimension::D2,
1,
))
}
}
impl FromWorld for FallbackImageCubemap {
fn from_world(world: &mut bevy_ecs::prelude::World) -> Self {
let render_device = world.resource::<RenderDevice>();
let render_queue = world.resource::<RenderQueue>();
let default_sampler = world.resource::<DefaultImageSampler>();
Self(fallback_image_new(
render_device,
render_queue,
default_sampler,
TextureFormat::bevy_default(),
TextureViewDimension::Cube,
1,
))
}
@ -108,6 +147,7 @@ impl<'w> FallbackImagesMsaa<'w> {
&self.render_queue,
&self.default_sampler,
TextureFormat::bevy_default(),
TextureViewDimension::D2,
sample_count,
)
})
@ -130,6 +170,7 @@ impl<'w> FallbackImagesDepth<'w> {
&self.render_queue,
&self.default_sampler,
TextureFormat::Depth32Float,
TextureViewDimension::D2,
sample_count,
)
})

View file

@ -103,6 +103,7 @@ impl Plugin for ImagePlugin {
.insert_resource(DefaultImageSampler(default_sampler))
.init_resource::<TextureCache>()
.init_resource::<FallbackImage>()
.init_resource::<FallbackImageCubemap>()
.init_resource::<FallbackImageMsaaCache>()
.init_resource::<FallbackImageDepthCache>()
.add_system(update_texture_cache_system.in_set(RenderSet::Cleanup));

View file

@ -21,10 +21,17 @@ fn main() {
}
fn setup(mut commands: Commands, asset_server: Res<AssetServer>) {
commands.spawn(Camera3dBundle {
transform: Transform::from_xyz(0.7, 0.7, 1.0).looking_at(Vec3::new(0.0, 0.3, 0.0), Vec3::Y),
..default()
});
commands.spawn((
Camera3dBundle {
transform: Transform::from_xyz(0.7, 0.7, 1.0)
.looking_at(Vec3::new(0.0, 0.3, 0.0), Vec3::Y),
..default()
},
EnvironmentMapLight {
diffuse_map: asset_server.load("environment_maps/pisa_diffuse_rgb9e5_zstd.ktx2"),
specular_map: asset_server.load("environment_maps/pisa_specular_rgb9e5_zstd.ktx2"),
},
));
commands.spawn(DirectionalLightBundle {
directional_light: DirectionalLight {
shadows_enabled: true,

View file

@ -1,11 +1,12 @@
//! This example shows how to configure Physically Based Rendering (PBR) parameters.
use bevy::prelude::*;
use bevy::{asset::LoadState, prelude::*};
fn main() {
App::new()
.add_plugins(DefaultPlugins)
.add_startup_system(setup)
.add_system(environment_map_load_finish)
.run();
}
@ -14,6 +15,7 @@ fn setup(
mut commands: Commands,
mut meshes: ResMut<Assets<Mesh>>,
mut materials: ResMut<Assets<StandardMaterial>>,
asset_server: Res<AssetServer>,
) {
// add entities to the world
for y in -2..=2 {
@ -59,6 +61,7 @@ fn setup(
transform: Transform::from_xyz(-5.0, -2.5, 0.0),
..default()
});
// light
commands.spawn(PointLightBundle {
transform: Transform::from_xyz(50.0, 50.0, 50.0),
@ -69,14 +72,108 @@ fn setup(
},
..default()
});
// camera
commands.spawn(Camera3dBundle {
transform: Transform::from_xyz(0.0, 0.0, 8.0).looking_at(Vec3::default(), Vec3::Y),
projection: OrthographicProjection {
scale: 0.01,
// labels
commands.spawn(
TextBundle::from_section(
"Perceptual Roughness",
TextStyle {
font: asset_server.load("fonts/FiraMono-Medium.ttf"),
font_size: 36.0,
color: Color::WHITE,
},
)
.with_style(Style {
position_type: PositionType::Absolute,
position: UiRect {
top: Val::Px(20.0),
left: Val::Px(100.0),
..default()
},
..default()
}
.into(),
}),
);
commands.spawn(TextBundle {
text: Text::from_section(
"Metallic",
TextStyle {
font: asset_server.load("fonts/FiraMono-Medium.ttf"),
font_size: 36.0,
color: Color::WHITE,
},
),
style: Style {
position_type: PositionType::Absolute,
position: UiRect {
top: Val::Px(130.0),
right: Val::Px(0.0),
..default()
},
..default()
},
transform: Transform {
rotation: Quat::from_rotation_z(std::f32::consts::PI / 2.0),
..default()
},
..default()
});
commands.spawn((
TextBundle::from_section(
"Loading Environment Map...",
TextStyle {
font: asset_server.load("fonts/FiraMono-Medium.ttf"),
font_size: 36.0,
color: Color::RED,
},
)
.with_style(Style {
position_type: PositionType::Absolute,
position: UiRect {
bottom: Val::Px(20.0),
right: Val::Px(20.0),
..default()
},
..default()
}),
EnvironmentMapLabel,
));
// camera
commands.spawn((
Camera3dBundle {
transform: Transform::from_xyz(0.0, 0.0, 8.0).looking_at(Vec3::default(), Vec3::Y),
projection: OrthographicProjection {
scale: 0.01,
..default()
}
.into(),
..default()
},
EnvironmentMapLight {
diffuse_map: asset_server.load("environment_maps/pisa_diffuse_rgb9e5_zstd.ktx2"),
specular_map: asset_server.load("environment_maps/pisa_specular_rgb9e5_zstd.ktx2"),
},
));
}
fn environment_map_load_finish(
mut commands: Commands,
asset_server: Res<AssetServer>,
environment_maps: Query<&EnvironmentMapLight>,
label_query: Query<Entity, With<EnvironmentMapLabel>>,
) {
if let Ok(environment_map) = environment_maps.get_single() {
if asset_server.get_load_state(&environment_map.diffuse_map) == LoadState::Loaded
&& asset_server.get_load_state(&environment_map.specular_map) == LoadState::Loaded
{
if let Ok(label_entity) = label_query.get_single() {
commands.entity(label_entity).despawn();
}
}
}
}
#[derive(Component)]
struct EnvironmentMapLabel;

View file

@ -76,6 +76,7 @@ fn setup_scene_after_load(
mut commands: Commands,
mut setup: Local<bool>,
mut scene_handle: ResMut<SceneHandle>,
asset_server: Res<AssetServer>,
meshes: Query<(&GlobalTransform, Option<&Aabb>), With<Handle<Mesh>>>,
) {
if scene_handle.is_loaded && !*setup {
@ -127,6 +128,12 @@ fn setup_scene_after_load(
},
..default()
},
EnvironmentMapLight {
diffuse_map: asset_server
.load("assets/environment_maps/pisa_diffuse_rgb9e5_zstd.ktx2"),
specular_map: asset_server
.load("assets/environment_maps/pisa_specular_rgb9e5_zstd.ktx2"),
},
camera_controller,
));