bevy/crates/bevy_pbr/src/render/parallax_mapping.wgsl

117 lines
4.7 KiB
WebGPU Shading Language
Raw Normal View History

Add parallax mapping to bevy PBR (#5928) # Objective Add a [parallax mapping] shader to bevy. Please note that this is a 3d technique, NOT a 2d sidescroller feature. ## Solution - Add related fields to `StandardMaterial` - update the pbr shader - Add an example taking advantage of parallax mapping A pre-existing implementation exists at: https://github.com/nicopap/bevy_mod_paramap/ The implementation is derived from: https://web.archive.org/web/20150419215321/http://sunandblackcat.com/tipFullView.php?l=eng&topicid=28 Further discussion on literature is found in the `bevy_mod_paramap` README. ### Limitations - The mesh silhouette isn't affected by the depth map. - The depth of the pixel does not reflect its visual position, resulting in artifacts for depth-dependent features such as fog or SSAO - GLTF does not define a height map texture, so somehow the user will always need to work around this limitation, though [an extension is in the works][gltf] ### Future work - It's possible to update the depth in the depth buffer to follow the parallaxed texture. This would enable interop with depth-based visual effects, it also allows `discard`ing pixels of materials when computed depth is higher than the one in depth buffer - Cheap lower quality single-sample method using [offset limiting] - Add distance fading, to disable parallaxing (relatively expensive) on distant objects - GLTF extension to allow defining height maps. Or a workaround implemented through a blender plugin to the GLTF exporter that uses the `extras` field to add height map. - [Quadratic surface vertex attributes][oliveira_3] to enable parallax mapping on bending surfaces and allow clean silhouetting. - noise based sampling, to limit the pancake artifacts. - Cone mapping ([GPU gems], [Simcity (2013)][simcity]). Requires preprocessing, increase depth map size, reduces sample count greatly. - [Quadtree parallax mapping][qpm] (also requires preprocessing) - Self-shadowing of parallax-mapped surfaces by modifying the shadow map - Generate depth map from normal map [link to slides], [blender question] https://user-images.githubusercontent.com/26321040/223563792-dffcc6ab-70e8-4ff9-90d1-b36c338695ad.mp4 [blender question]: https://blender.stackexchange.com/questions/89278/how-to-get-a-smooth-curvature-map-from-a-normal-map [link to slides]: https://developer.download.nvidia.com/assets/gamedev/docs/nmap2displacement.pdf [oliveira_3]: https://www.inf.ufrgs.br/~oliveira/pubs_files/Oliveira_Policarpo_RP-351_Jan_2005.pdf [GPU gems]: https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-18-relaxed-cone-stepping-relief-mapping [simcity]: https://community.simtropolis.com/omnibus/other-games/building-and-rendering-simcity-2013-r247/ [offset limiting]: https://raw.githubusercontent.com/marcusstenbeck/tncg14-parallax-mapping/master/documents/Parallax%20Mapping%20with%20Offset%20Limiting%20-%20A%20Per-Pixel%20Approximation%20of%20Uneven%20Surfaces.pdf [gltf]: https://github.com/KhronosGroup/glTF/pull/2196 [qpm]: https://www.gamedevs.org/uploads/quadtree-displacement-mapping-with-height-blending.pdf --- ## Changelog - Add a `depth_map` field to the `StandardMaterial`, it is a grayscale image where white represents bottom and black the top. If `depth_map` is set, bevy's pbr shader will use it to do [parallax mapping] to give an increased feel of depth to the material. This is similar to a displacement map, but with infinite precision at fairly low cost. - The fields `parallax_mapping_method`, `parallax_depth_scale` and `max_parallax_layer_count` allow finer grained control over the behavior of the parallax shader. - Add the `parallax_mapping` example to show off the effect. [parallax mapping]: https://en.wikipedia.org/wiki/Parallax_mapping --------- Co-authored-by: Robert Swain <robert.swain@gmail.com>
2023-04-15 10:25:14 +00:00
#define_import_path bevy_pbr::parallax_mapping
fn sample_depth_map(uv: vec2<f32>) -> f32 {
// We use `textureSampleLevel` over `textureSample` because the wgpu DX12
// backend (Fxc) panics when using "gradient instructions" inside a loop.
// It results in the whole loop being unrolled by the shader compiler,
// which it can't do because the upper limit of the loop in steep parallax
// mapping is a variable set by the user.
// The "gradient instructions" comes from `textureSample` computing MIP level
// based on UV derivative. With `textureSampleLevel`, we provide ourselves
// the MIP level, so no gradient instructions are used, and we can use
// sample_depth_map in our loop.
// See https://stackoverflow.com/questions/56581141/direct3d11-gradient-instruction-used-in-a-loop-with-varying-iteration-forcing
return textureSampleLevel(depth_map_texture, depth_map_sampler, uv, 0.0).r;
}
// An implementation of parallax mapping, see https://en.wikipedia.org/wiki/Parallax_mapping
// Code derived from: https://web.archive.org/web/20150419215321/http://sunandblackcat.com/tipFullView.php?l=eng&topicid=28
fn parallaxed_uv(
depth_scale: f32,
max_layer_count: f32,
max_steps: u32,
// The original interpolated uv
uv: vec2<f32>,
// The vector from the camera to the fragment at the surface in tangent space
Vt: vec3<f32>,
) -> vec2<f32> {
if max_layer_count < 1.0 {
return uv;
}
var uv = uv;
// Steep Parallax Mapping
// ======================
// Split the depth map into `layer_count` layers.
// When Vt hits the surface of the mesh (excluding depth displacement),
// if the depth is not below or on surface including depth displacement (textureSample), then
// look forward (+= delta_uv) on depth texture according to
// Vt and distance between hit surface and depth map surface,
// repeat until below the surface.
//
// Where `layer_count` is interpolated between `1.0` and
// `max_layer_count` according to the steepness of Vt.
let view_steepness = abs(Vt.z);
// We mix with minimum value 1.0 because otherwise,
// with 0.0, we get a division by zero in surfaces parallel to viewport,
// resulting in a singularity.
let layer_count = mix(max_layer_count, 1.0, view_steepness);
let layer_depth = 1.0 / layer_count;
var delta_uv = depth_scale * layer_depth * Vt.xy * vec2(1.0, -1.0) / view_steepness;
var current_layer_depth = 0.0;
var texture_depth = sample_depth_map(uv);
// texture_depth > current_layer_depth means the depth map depth is deeper
// than the depth the ray would be at at this UV offset so the ray has not
// intersected the surface
for (var i: i32 = 0; texture_depth > current_layer_depth && i <= i32(layer_count); i++) {
current_layer_depth += layer_depth;
uv += delta_uv;
texture_depth = sample_depth_map(uv);
}
#ifdef RELIEF_MAPPING
// Relief Mapping
// ==============
// "Refine" the rough result from Steep Parallax Mapping
// with a **binary search** between the layer selected by steep parallax
// and the next one to find a point closer to the depth map surface.
// This reduces the jaggy step artifacts from steep parallax mapping.
delta_uv *= 0.5;
var delta_depth = 0.5 * layer_depth;
uv -= delta_uv;
current_layer_depth -= delta_depth;
for (var i: u32 = 0u; i < max_steps; i++) {
texture_depth = sample_depth_map(uv);
// Halve the deltas for the next step
delta_uv *= 0.5;
delta_depth *= 0.5;
// Step based on whether the current depth is above or below the depth map
if (texture_depth > current_layer_depth) {
uv += delta_uv;
current_layer_depth += delta_depth;
} else {
uv -= delta_uv;
current_layer_depth -= delta_depth;
}
}
#else
// Parallax Occlusion mapping
// ==========================
// "Refine" Steep Parallax Mapping by interpolating between the
// previous layer's depth and the computed layer depth.
// Only requires a single lookup, unlike Relief Mapping, but
// may skip small details and result in writhing material artifacts.
let previous_uv = uv - delta_uv;
let next_depth = texture_depth - current_layer_depth;
let previous_depth = sample_depth_map(previous_uv) - current_layer_depth + layer_depth;
let weight = next_depth / (next_depth - previous_depth);
uv = mix(uv, previous_uv, weight);
current_layer_depth += mix(next_depth, previous_depth, weight);
#endif
// Note: `current_layer_depth` is not returned, but may be useful
// for light computation later on in future improvements of the pbr shader.
return uv;
}