Add parallax mapping to bevy PBR (#5928)
# Objective
Add a [parallax mapping] shader to bevy. Please note that
this is a 3d technique, NOT a 2d sidescroller feature.
## Solution
- Add related fields to `StandardMaterial`
- update the pbr shader
- Add an example taking advantage of parallax mapping
A pre-existing implementation exists at:
https://github.com/nicopap/bevy_mod_paramap/
The implementation is derived from:
https://web.archive.org/web/20150419215321/http://sunandblackcat.com/tipFullView.php?l=eng&topicid=28
Further discussion on literature is found in the `bevy_mod_paramap`
README.
### Limitations
- The mesh silhouette isn't affected by the depth map.
- The depth of the pixel does not reflect its visual position, resulting
in artifacts for depth-dependent features such as fog or SSAO
- GLTF does not define a height map texture, so somehow the user will
always need to work around this limitation, though [an extension is in
the works][gltf]
### Future work
- It's possible to update the depth in the depth buffer to follow the
parallaxed texture. This would enable interop with depth-based
visual effects, it also allows `discard`ing pixels of materials when
computed depth is higher than the one in depth buffer
- Cheap lower quality single-sample method using [offset limiting]
- Add distance fading, to disable parallaxing (relatively expensive)
on distant objects
- GLTF extension to allow defining height maps. Or a workaround
implemented through a blender plugin to the GLTF exporter that
uses the `extras` field to add height map.
- [Quadratic surface vertex attributes][oliveira_3] to enable parallax
mapping on bending surfaces and allow clean silhouetting.
- noise based sampling, to limit the pancake artifacts.
- Cone mapping ([GPU gems], [Simcity (2013)][simcity]). Requires
preprocessing, increase depth map size, reduces sample count greatly.
- [Quadtree parallax mapping][qpm] (also requires preprocessing)
- Self-shadowing of parallax-mapped surfaces by modifying the shadow map
- Generate depth map from normal map [link to slides], [blender
question]
https://user-images.githubusercontent.com/26321040/223563792-dffcc6ab-70e8-4ff9-90d1-b36c338695ad.mp4
[blender question]:
https://blender.stackexchange.com/questions/89278/how-to-get-a-smooth-curvature-map-from-a-normal-map
[link to slides]:
https://developer.download.nvidia.com/assets/gamedev/docs/nmap2displacement.pdf
[oliveira_3]:
https://www.inf.ufrgs.br/~oliveira/pubs_files/Oliveira_Policarpo_RP-351_Jan_2005.pdf
[GPU gems]:
https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-18-relaxed-cone-stepping-relief-mapping
[simcity]:
https://community.simtropolis.com/omnibus/other-games/building-and-rendering-simcity-2013-r247/
[offset limiting]:
https://raw.githubusercontent.com/marcusstenbeck/tncg14-parallax-mapping/master/documents/Parallax%20Mapping%20with%20Offset%20Limiting%20-%20A%20Per-Pixel%20Approximation%20of%20Uneven%20Surfaces.pdf
[gltf]: https://github.com/KhronosGroup/glTF/pull/2196
[qpm]:
https://www.gamedevs.org/uploads/quadtree-displacement-mapping-with-height-blending.pdf
---
## Changelog
- Add a `depth_map` field to the `StandardMaterial`, it is a grayscale
image where white represents bottom and black the top. If `depth_map`
is set, bevy's pbr shader will use it to do [parallax mapping] to
give an increased feel of depth to the material. This is similar to a
displacement map, but with infinite precision at fairly low cost.
- The fields `parallax_mapping_method`, `parallax_depth_scale` and
`max_parallax_layer_count` allow finer grained control over the
behavior of the parallax shader.
- Add the `parallax_mapping` example to show off the effect.
[parallax mapping]: https://en.wikipedia.org/wiki/Parallax_mapping
---------
Co-authored-by: Robert Swain <robert.swain@gmail.com>
2023-04-15 10:25:14 +00:00
|
|
|
#define_import_path bevy_pbr::parallax_mapping
|
|
|
|
|
2023-10-21 11:51:58 +00:00
|
|
|
#import bevy_pbr::pbr_bindings::{depth_map_texture, depth_map_sampler}
|
improve shader import model (#5703)
# Objective
operate on naga IR directly to improve handling of shader modules.
- give codespan reporting into imported modules
- allow glsl to be used from wgsl and vice-versa
the ultimate objective is to make it possible to
- provide user hooks for core shader functions (to modify light
behaviour within the standard pbr pipeline, for example)
- make automatic binding slot allocation possible
but ... since this is already big, adds some value and (i think) is at
feature parity with the existing code, i wanted to push this now.
## Solution
i made a crate called naga_oil (https://github.com/robtfm/naga_oil -
unpublished for now, could be part of bevy) which manages modules by
- building each module independantly to naga IR
- creating "header" files for each supported language, which are used to
build dependent modules/shaders
- make final shaders by combining the shader IR with the IR for imported
modules
then integrated this into bevy, replacing some of the existing shader
processing stuff. also reworked examples to reflect this.
## Migration Guide
shaders that don't use `#import` directives should work without changes.
the most notable user-facing difference is that imported
functions/variables/etc need to be qualified at point of use, and
there's no "leakage" of visible stuff into your shader scope from the
imports of your imports, so if you used things imported by your imports,
you now need to import them directly and qualify them.
the current strategy of including/'spreading' `mesh_vertex_output`
directly into a struct doesn't work any more, so these need to be
modified as per the examples (e.g. color_material.wgsl, or many others).
mesh data is assumed to be in bindgroup 2 by default, if mesh data is
bound into bindgroup 1 instead then the shader def `MESH_BINDGROUP_1`
needs to be added to the pipeline shader_defs.
2023-06-27 00:29:22 +00:00
|
|
|
|
Add parallax mapping to bevy PBR (#5928)
# Objective
Add a [parallax mapping] shader to bevy. Please note that
this is a 3d technique, NOT a 2d sidescroller feature.
## Solution
- Add related fields to `StandardMaterial`
- update the pbr shader
- Add an example taking advantage of parallax mapping
A pre-existing implementation exists at:
https://github.com/nicopap/bevy_mod_paramap/
The implementation is derived from:
https://web.archive.org/web/20150419215321/http://sunandblackcat.com/tipFullView.php?l=eng&topicid=28
Further discussion on literature is found in the `bevy_mod_paramap`
README.
### Limitations
- The mesh silhouette isn't affected by the depth map.
- The depth of the pixel does not reflect its visual position, resulting
in artifacts for depth-dependent features such as fog or SSAO
- GLTF does not define a height map texture, so somehow the user will
always need to work around this limitation, though [an extension is in
the works][gltf]
### Future work
- It's possible to update the depth in the depth buffer to follow the
parallaxed texture. This would enable interop with depth-based
visual effects, it also allows `discard`ing pixels of materials when
computed depth is higher than the one in depth buffer
- Cheap lower quality single-sample method using [offset limiting]
- Add distance fading, to disable parallaxing (relatively expensive)
on distant objects
- GLTF extension to allow defining height maps. Or a workaround
implemented through a blender plugin to the GLTF exporter that
uses the `extras` field to add height map.
- [Quadratic surface vertex attributes][oliveira_3] to enable parallax
mapping on bending surfaces and allow clean silhouetting.
- noise based sampling, to limit the pancake artifacts.
- Cone mapping ([GPU gems], [Simcity (2013)][simcity]). Requires
preprocessing, increase depth map size, reduces sample count greatly.
- [Quadtree parallax mapping][qpm] (also requires preprocessing)
- Self-shadowing of parallax-mapped surfaces by modifying the shadow map
- Generate depth map from normal map [link to slides], [blender
question]
https://user-images.githubusercontent.com/26321040/223563792-dffcc6ab-70e8-4ff9-90d1-b36c338695ad.mp4
[blender question]:
https://blender.stackexchange.com/questions/89278/how-to-get-a-smooth-curvature-map-from-a-normal-map
[link to slides]:
https://developer.download.nvidia.com/assets/gamedev/docs/nmap2displacement.pdf
[oliveira_3]:
https://www.inf.ufrgs.br/~oliveira/pubs_files/Oliveira_Policarpo_RP-351_Jan_2005.pdf
[GPU gems]:
https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-18-relaxed-cone-stepping-relief-mapping
[simcity]:
https://community.simtropolis.com/omnibus/other-games/building-and-rendering-simcity-2013-r247/
[offset limiting]:
https://raw.githubusercontent.com/marcusstenbeck/tncg14-parallax-mapping/master/documents/Parallax%20Mapping%20with%20Offset%20Limiting%20-%20A%20Per-Pixel%20Approximation%20of%20Uneven%20Surfaces.pdf
[gltf]: https://github.com/KhronosGroup/glTF/pull/2196
[qpm]:
https://www.gamedevs.org/uploads/quadtree-displacement-mapping-with-height-blending.pdf
---
## Changelog
- Add a `depth_map` field to the `StandardMaterial`, it is a grayscale
image where white represents bottom and black the top. If `depth_map`
is set, bevy's pbr shader will use it to do [parallax mapping] to
give an increased feel of depth to the material. This is similar to a
displacement map, but with infinite precision at fairly low cost.
- The fields `parallax_mapping_method`, `parallax_depth_scale` and
`max_parallax_layer_count` allow finer grained control over the
behavior of the parallax shader.
- Add the `parallax_mapping` example to show off the effect.
[parallax mapping]: https://en.wikipedia.org/wiki/Parallax_mapping
---------
Co-authored-by: Robert Swain <robert.swain@gmail.com>
2023-04-15 10:25:14 +00:00
|
|
|
fn sample_depth_map(uv: vec2<f32>) -> f32 {
|
|
|
|
// We use `textureSampleLevel` over `textureSample` because the wgpu DX12
|
|
|
|
// backend (Fxc) panics when using "gradient instructions" inside a loop.
|
|
|
|
// It results in the whole loop being unrolled by the shader compiler,
|
|
|
|
// which it can't do because the upper limit of the loop in steep parallax
|
|
|
|
// mapping is a variable set by the user.
|
|
|
|
// The "gradient instructions" comes from `textureSample` computing MIP level
|
|
|
|
// based on UV derivative. With `textureSampleLevel`, we provide ourselves
|
|
|
|
// the MIP level, so no gradient instructions are used, and we can use
|
|
|
|
// sample_depth_map in our loop.
|
|
|
|
// See https://stackoverflow.com/questions/56581141/direct3d11-gradient-instruction-used-in-a-loop-with-varying-iteration-forcing
|
|
|
|
return textureSampleLevel(depth_map_texture, depth_map_sampler, uv, 0.0).r;
|
|
|
|
}
|
|
|
|
|
|
|
|
// An implementation of parallax mapping, see https://en.wikipedia.org/wiki/Parallax_mapping
|
|
|
|
// Code derived from: https://web.archive.org/web/20150419215321/http://sunandblackcat.com/tipFullView.php?l=eng&topicid=28
|
|
|
|
fn parallaxed_uv(
|
|
|
|
depth_scale: f32,
|
|
|
|
max_layer_count: f32,
|
|
|
|
max_steps: u32,
|
|
|
|
// The original interpolated uv
|
Webgpu support (#8336)
# Objective
- Support WebGPU
- alternative to #5027 that doesn't need any async / await
- fixes #8315
- Surprise fix #7318
## Solution
### For async renderer initialisation
- Update the plugin lifecycle:
- app builds the plugin
- calls `plugin.build`
- registers the plugin
- app starts the event loop
- event loop waits for `ready` of all registered plugins in the same
order
- returns `true` by default
- then call all `finish` then all `cleanup` in the same order as
registered
- then execute the schedule
In the case of the renderer, to avoid anything async:
- building the renderer plugin creates a detached task that will send
back the initialised renderer through a mutex in a resource
- `ready` will wait for the renderer to be present in the resource
- `finish` will take that renderer and place it in the expected
resources by other plugins
- other plugins (that expect the renderer to be available) `finish` are
called and they are able to set up their pipelines
- `cleanup` is called, only custom one is still for pipeline rendering
### For WebGPU support
- update the `build-wasm-example` script to support passing `--api
webgpu` that will build the example with WebGPU support
- feature for webgl2 was always enabled when building for wasm. it's now
in the default feature list and enabled on all platforms, so check for
this feature must also check that the target_arch is `wasm32`
---
## Migration Guide
- `Plugin::setup` has been renamed `Plugin::cleanup`
- `Plugin::finish` has been added, and plugins adding pipelines should
do it in this function instead of `Plugin::build`
```rust
// Before
impl Plugin for MyPlugin {
fn build(&self, app: &mut App) {
app.insert_resource::<MyResource>
.add_systems(Update, my_system);
let render_app = match app.get_sub_app_mut(RenderApp) {
Ok(render_app) => render_app,
Err(_) => return,
};
render_app
.init_resource::<RenderResourceNeedingDevice>()
.init_resource::<OtherRenderResource>();
}
}
// After
impl Plugin for MyPlugin {
fn build(&self, app: &mut App) {
app.insert_resource::<MyResource>
.add_systems(Update, my_system);
let render_app = match app.get_sub_app_mut(RenderApp) {
Ok(render_app) => render_app,
Err(_) => return,
};
render_app
.init_resource::<OtherRenderResource>();
}
fn finish(&self, app: &mut App) {
let render_app = match app.get_sub_app_mut(RenderApp) {
Ok(render_app) => render_app,
Err(_) => return,
};
render_app
.init_resource::<RenderResourceNeedingDevice>();
}
}
```
2023-05-04 22:07:57 +00:00
|
|
|
original_uv: vec2<f32>,
|
Add parallax mapping to bevy PBR (#5928)
# Objective
Add a [parallax mapping] shader to bevy. Please note that
this is a 3d technique, NOT a 2d sidescroller feature.
## Solution
- Add related fields to `StandardMaterial`
- update the pbr shader
- Add an example taking advantage of parallax mapping
A pre-existing implementation exists at:
https://github.com/nicopap/bevy_mod_paramap/
The implementation is derived from:
https://web.archive.org/web/20150419215321/http://sunandblackcat.com/tipFullView.php?l=eng&topicid=28
Further discussion on literature is found in the `bevy_mod_paramap`
README.
### Limitations
- The mesh silhouette isn't affected by the depth map.
- The depth of the pixel does not reflect its visual position, resulting
in artifacts for depth-dependent features such as fog or SSAO
- GLTF does not define a height map texture, so somehow the user will
always need to work around this limitation, though [an extension is in
the works][gltf]
### Future work
- It's possible to update the depth in the depth buffer to follow the
parallaxed texture. This would enable interop with depth-based
visual effects, it also allows `discard`ing pixels of materials when
computed depth is higher than the one in depth buffer
- Cheap lower quality single-sample method using [offset limiting]
- Add distance fading, to disable parallaxing (relatively expensive)
on distant objects
- GLTF extension to allow defining height maps. Or a workaround
implemented through a blender plugin to the GLTF exporter that
uses the `extras` field to add height map.
- [Quadratic surface vertex attributes][oliveira_3] to enable parallax
mapping on bending surfaces and allow clean silhouetting.
- noise based sampling, to limit the pancake artifacts.
- Cone mapping ([GPU gems], [Simcity (2013)][simcity]). Requires
preprocessing, increase depth map size, reduces sample count greatly.
- [Quadtree parallax mapping][qpm] (also requires preprocessing)
- Self-shadowing of parallax-mapped surfaces by modifying the shadow map
- Generate depth map from normal map [link to slides], [blender
question]
https://user-images.githubusercontent.com/26321040/223563792-dffcc6ab-70e8-4ff9-90d1-b36c338695ad.mp4
[blender question]:
https://blender.stackexchange.com/questions/89278/how-to-get-a-smooth-curvature-map-from-a-normal-map
[link to slides]:
https://developer.download.nvidia.com/assets/gamedev/docs/nmap2displacement.pdf
[oliveira_3]:
https://www.inf.ufrgs.br/~oliveira/pubs_files/Oliveira_Policarpo_RP-351_Jan_2005.pdf
[GPU gems]:
https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-18-relaxed-cone-stepping-relief-mapping
[simcity]:
https://community.simtropolis.com/omnibus/other-games/building-and-rendering-simcity-2013-r247/
[offset limiting]:
https://raw.githubusercontent.com/marcusstenbeck/tncg14-parallax-mapping/master/documents/Parallax%20Mapping%20with%20Offset%20Limiting%20-%20A%20Per-Pixel%20Approximation%20of%20Uneven%20Surfaces.pdf
[gltf]: https://github.com/KhronosGroup/glTF/pull/2196
[qpm]:
https://www.gamedevs.org/uploads/quadtree-displacement-mapping-with-height-blending.pdf
---
## Changelog
- Add a `depth_map` field to the `StandardMaterial`, it is a grayscale
image where white represents bottom and black the top. If `depth_map`
is set, bevy's pbr shader will use it to do [parallax mapping] to
give an increased feel of depth to the material. This is similar to a
displacement map, but with infinite precision at fairly low cost.
- The fields `parallax_mapping_method`, `parallax_depth_scale` and
`max_parallax_layer_count` allow finer grained control over the
behavior of the parallax shader.
- Add the `parallax_mapping` example to show off the effect.
[parallax mapping]: https://en.wikipedia.org/wiki/Parallax_mapping
---------
Co-authored-by: Robert Swain <robert.swain@gmail.com>
2023-04-15 10:25:14 +00:00
|
|
|
// The vector from the camera to the fragment at the surface in tangent space
|
|
|
|
Vt: vec3<f32>,
|
|
|
|
) -> vec2<f32> {
|
|
|
|
if max_layer_count < 1.0 {
|
Webgpu support (#8336)
# Objective
- Support WebGPU
- alternative to #5027 that doesn't need any async / await
- fixes #8315
- Surprise fix #7318
## Solution
### For async renderer initialisation
- Update the plugin lifecycle:
- app builds the plugin
- calls `plugin.build`
- registers the plugin
- app starts the event loop
- event loop waits for `ready` of all registered plugins in the same
order
- returns `true` by default
- then call all `finish` then all `cleanup` in the same order as
registered
- then execute the schedule
In the case of the renderer, to avoid anything async:
- building the renderer plugin creates a detached task that will send
back the initialised renderer through a mutex in a resource
- `ready` will wait for the renderer to be present in the resource
- `finish` will take that renderer and place it in the expected
resources by other plugins
- other plugins (that expect the renderer to be available) `finish` are
called and they are able to set up their pipelines
- `cleanup` is called, only custom one is still for pipeline rendering
### For WebGPU support
- update the `build-wasm-example` script to support passing `--api
webgpu` that will build the example with WebGPU support
- feature for webgl2 was always enabled when building for wasm. it's now
in the default feature list and enabled on all platforms, so check for
this feature must also check that the target_arch is `wasm32`
---
## Migration Guide
- `Plugin::setup` has been renamed `Plugin::cleanup`
- `Plugin::finish` has been added, and plugins adding pipelines should
do it in this function instead of `Plugin::build`
```rust
// Before
impl Plugin for MyPlugin {
fn build(&self, app: &mut App) {
app.insert_resource::<MyResource>
.add_systems(Update, my_system);
let render_app = match app.get_sub_app_mut(RenderApp) {
Ok(render_app) => render_app,
Err(_) => return,
};
render_app
.init_resource::<RenderResourceNeedingDevice>()
.init_resource::<OtherRenderResource>();
}
}
// After
impl Plugin for MyPlugin {
fn build(&self, app: &mut App) {
app.insert_resource::<MyResource>
.add_systems(Update, my_system);
let render_app = match app.get_sub_app_mut(RenderApp) {
Ok(render_app) => render_app,
Err(_) => return,
};
render_app
.init_resource::<OtherRenderResource>();
}
fn finish(&self, app: &mut App) {
let render_app = match app.get_sub_app_mut(RenderApp) {
Ok(render_app) => render_app,
Err(_) => return,
};
render_app
.init_resource::<RenderResourceNeedingDevice>();
}
}
```
2023-05-04 22:07:57 +00:00
|
|
|
return original_uv;
|
Add parallax mapping to bevy PBR (#5928)
# Objective
Add a [parallax mapping] shader to bevy. Please note that
this is a 3d technique, NOT a 2d sidescroller feature.
## Solution
- Add related fields to `StandardMaterial`
- update the pbr shader
- Add an example taking advantage of parallax mapping
A pre-existing implementation exists at:
https://github.com/nicopap/bevy_mod_paramap/
The implementation is derived from:
https://web.archive.org/web/20150419215321/http://sunandblackcat.com/tipFullView.php?l=eng&topicid=28
Further discussion on literature is found in the `bevy_mod_paramap`
README.
### Limitations
- The mesh silhouette isn't affected by the depth map.
- The depth of the pixel does not reflect its visual position, resulting
in artifacts for depth-dependent features such as fog or SSAO
- GLTF does not define a height map texture, so somehow the user will
always need to work around this limitation, though [an extension is in
the works][gltf]
### Future work
- It's possible to update the depth in the depth buffer to follow the
parallaxed texture. This would enable interop with depth-based
visual effects, it also allows `discard`ing pixels of materials when
computed depth is higher than the one in depth buffer
- Cheap lower quality single-sample method using [offset limiting]
- Add distance fading, to disable parallaxing (relatively expensive)
on distant objects
- GLTF extension to allow defining height maps. Or a workaround
implemented through a blender plugin to the GLTF exporter that
uses the `extras` field to add height map.
- [Quadratic surface vertex attributes][oliveira_3] to enable parallax
mapping on bending surfaces and allow clean silhouetting.
- noise based sampling, to limit the pancake artifacts.
- Cone mapping ([GPU gems], [Simcity (2013)][simcity]). Requires
preprocessing, increase depth map size, reduces sample count greatly.
- [Quadtree parallax mapping][qpm] (also requires preprocessing)
- Self-shadowing of parallax-mapped surfaces by modifying the shadow map
- Generate depth map from normal map [link to slides], [blender
question]
https://user-images.githubusercontent.com/26321040/223563792-dffcc6ab-70e8-4ff9-90d1-b36c338695ad.mp4
[blender question]:
https://blender.stackexchange.com/questions/89278/how-to-get-a-smooth-curvature-map-from-a-normal-map
[link to slides]:
https://developer.download.nvidia.com/assets/gamedev/docs/nmap2displacement.pdf
[oliveira_3]:
https://www.inf.ufrgs.br/~oliveira/pubs_files/Oliveira_Policarpo_RP-351_Jan_2005.pdf
[GPU gems]:
https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-18-relaxed-cone-stepping-relief-mapping
[simcity]:
https://community.simtropolis.com/omnibus/other-games/building-and-rendering-simcity-2013-r247/
[offset limiting]:
https://raw.githubusercontent.com/marcusstenbeck/tncg14-parallax-mapping/master/documents/Parallax%20Mapping%20with%20Offset%20Limiting%20-%20A%20Per-Pixel%20Approximation%20of%20Uneven%20Surfaces.pdf
[gltf]: https://github.com/KhronosGroup/glTF/pull/2196
[qpm]:
https://www.gamedevs.org/uploads/quadtree-displacement-mapping-with-height-blending.pdf
---
## Changelog
- Add a `depth_map` field to the `StandardMaterial`, it is a grayscale
image where white represents bottom and black the top. If `depth_map`
is set, bevy's pbr shader will use it to do [parallax mapping] to
give an increased feel of depth to the material. This is similar to a
displacement map, but with infinite precision at fairly low cost.
- The fields `parallax_mapping_method`, `parallax_depth_scale` and
`max_parallax_layer_count` allow finer grained control over the
behavior of the parallax shader.
- Add the `parallax_mapping` example to show off the effect.
[parallax mapping]: https://en.wikipedia.org/wiki/Parallax_mapping
---------
Co-authored-by: Robert Swain <robert.swain@gmail.com>
2023-04-15 10:25:14 +00:00
|
|
|
}
|
Webgpu support (#8336)
# Objective
- Support WebGPU
- alternative to #5027 that doesn't need any async / await
- fixes #8315
- Surprise fix #7318
## Solution
### For async renderer initialisation
- Update the plugin lifecycle:
- app builds the plugin
- calls `plugin.build`
- registers the plugin
- app starts the event loop
- event loop waits for `ready` of all registered plugins in the same
order
- returns `true` by default
- then call all `finish` then all `cleanup` in the same order as
registered
- then execute the schedule
In the case of the renderer, to avoid anything async:
- building the renderer plugin creates a detached task that will send
back the initialised renderer through a mutex in a resource
- `ready` will wait for the renderer to be present in the resource
- `finish` will take that renderer and place it in the expected
resources by other plugins
- other plugins (that expect the renderer to be available) `finish` are
called and they are able to set up their pipelines
- `cleanup` is called, only custom one is still for pipeline rendering
### For WebGPU support
- update the `build-wasm-example` script to support passing `--api
webgpu` that will build the example with WebGPU support
- feature for webgl2 was always enabled when building for wasm. it's now
in the default feature list and enabled on all platforms, so check for
this feature must also check that the target_arch is `wasm32`
---
## Migration Guide
- `Plugin::setup` has been renamed `Plugin::cleanup`
- `Plugin::finish` has been added, and plugins adding pipelines should
do it in this function instead of `Plugin::build`
```rust
// Before
impl Plugin for MyPlugin {
fn build(&self, app: &mut App) {
app.insert_resource::<MyResource>
.add_systems(Update, my_system);
let render_app = match app.get_sub_app_mut(RenderApp) {
Ok(render_app) => render_app,
Err(_) => return,
};
render_app
.init_resource::<RenderResourceNeedingDevice>()
.init_resource::<OtherRenderResource>();
}
}
// After
impl Plugin for MyPlugin {
fn build(&self, app: &mut App) {
app.insert_resource::<MyResource>
.add_systems(Update, my_system);
let render_app = match app.get_sub_app_mut(RenderApp) {
Ok(render_app) => render_app,
Err(_) => return,
};
render_app
.init_resource::<OtherRenderResource>();
}
fn finish(&self, app: &mut App) {
let render_app = match app.get_sub_app_mut(RenderApp) {
Ok(render_app) => render_app,
Err(_) => return,
};
render_app
.init_resource::<RenderResourceNeedingDevice>();
}
}
```
2023-05-04 22:07:57 +00:00
|
|
|
var uv = original_uv;
|
Add parallax mapping to bevy PBR (#5928)
# Objective
Add a [parallax mapping] shader to bevy. Please note that
this is a 3d technique, NOT a 2d sidescroller feature.
## Solution
- Add related fields to `StandardMaterial`
- update the pbr shader
- Add an example taking advantage of parallax mapping
A pre-existing implementation exists at:
https://github.com/nicopap/bevy_mod_paramap/
The implementation is derived from:
https://web.archive.org/web/20150419215321/http://sunandblackcat.com/tipFullView.php?l=eng&topicid=28
Further discussion on literature is found in the `bevy_mod_paramap`
README.
### Limitations
- The mesh silhouette isn't affected by the depth map.
- The depth of the pixel does not reflect its visual position, resulting
in artifacts for depth-dependent features such as fog or SSAO
- GLTF does not define a height map texture, so somehow the user will
always need to work around this limitation, though [an extension is in
the works][gltf]
### Future work
- It's possible to update the depth in the depth buffer to follow the
parallaxed texture. This would enable interop with depth-based
visual effects, it also allows `discard`ing pixels of materials when
computed depth is higher than the one in depth buffer
- Cheap lower quality single-sample method using [offset limiting]
- Add distance fading, to disable parallaxing (relatively expensive)
on distant objects
- GLTF extension to allow defining height maps. Or a workaround
implemented through a blender plugin to the GLTF exporter that
uses the `extras` field to add height map.
- [Quadratic surface vertex attributes][oliveira_3] to enable parallax
mapping on bending surfaces and allow clean silhouetting.
- noise based sampling, to limit the pancake artifacts.
- Cone mapping ([GPU gems], [Simcity (2013)][simcity]). Requires
preprocessing, increase depth map size, reduces sample count greatly.
- [Quadtree parallax mapping][qpm] (also requires preprocessing)
- Self-shadowing of parallax-mapped surfaces by modifying the shadow map
- Generate depth map from normal map [link to slides], [blender
question]
https://user-images.githubusercontent.com/26321040/223563792-dffcc6ab-70e8-4ff9-90d1-b36c338695ad.mp4
[blender question]:
https://blender.stackexchange.com/questions/89278/how-to-get-a-smooth-curvature-map-from-a-normal-map
[link to slides]:
https://developer.download.nvidia.com/assets/gamedev/docs/nmap2displacement.pdf
[oliveira_3]:
https://www.inf.ufrgs.br/~oliveira/pubs_files/Oliveira_Policarpo_RP-351_Jan_2005.pdf
[GPU gems]:
https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-18-relaxed-cone-stepping-relief-mapping
[simcity]:
https://community.simtropolis.com/omnibus/other-games/building-and-rendering-simcity-2013-r247/
[offset limiting]:
https://raw.githubusercontent.com/marcusstenbeck/tncg14-parallax-mapping/master/documents/Parallax%20Mapping%20with%20Offset%20Limiting%20-%20A%20Per-Pixel%20Approximation%20of%20Uneven%20Surfaces.pdf
[gltf]: https://github.com/KhronosGroup/glTF/pull/2196
[qpm]:
https://www.gamedevs.org/uploads/quadtree-displacement-mapping-with-height-blending.pdf
---
## Changelog
- Add a `depth_map` field to the `StandardMaterial`, it is a grayscale
image where white represents bottom and black the top. If `depth_map`
is set, bevy's pbr shader will use it to do [parallax mapping] to
give an increased feel of depth to the material. This is similar to a
displacement map, but with infinite precision at fairly low cost.
- The fields `parallax_mapping_method`, `parallax_depth_scale` and
`max_parallax_layer_count` allow finer grained control over the
behavior of the parallax shader.
- Add the `parallax_mapping` example to show off the effect.
[parallax mapping]: https://en.wikipedia.org/wiki/Parallax_mapping
---------
Co-authored-by: Robert Swain <robert.swain@gmail.com>
2023-04-15 10:25:14 +00:00
|
|
|
|
|
|
|
// Steep Parallax Mapping
|
|
|
|
// ======================
|
|
|
|
// Split the depth map into `layer_count` layers.
|
|
|
|
// When Vt hits the surface of the mesh (excluding depth displacement),
|
|
|
|
// if the depth is not below or on surface including depth displacement (textureSample), then
|
|
|
|
// look forward (+= delta_uv) on depth texture according to
|
|
|
|
// Vt and distance between hit surface and depth map surface,
|
|
|
|
// repeat until below the surface.
|
|
|
|
//
|
|
|
|
// Where `layer_count` is interpolated between `1.0` and
|
|
|
|
// `max_layer_count` according to the steepness of Vt.
|
|
|
|
|
|
|
|
let view_steepness = abs(Vt.z);
|
|
|
|
// We mix with minimum value 1.0 because otherwise,
|
|
|
|
// with 0.0, we get a division by zero in surfaces parallel to viewport,
|
|
|
|
// resulting in a singularity.
|
|
|
|
let layer_count = mix(max_layer_count, 1.0, view_steepness);
|
|
|
|
let layer_depth = 1.0 / layer_count;
|
|
|
|
var delta_uv = depth_scale * layer_depth * Vt.xy * vec2(1.0, -1.0) / view_steepness;
|
|
|
|
|
|
|
|
var current_layer_depth = 0.0;
|
|
|
|
var texture_depth = sample_depth_map(uv);
|
|
|
|
|
|
|
|
// texture_depth > current_layer_depth means the depth map depth is deeper
|
|
|
|
// than the depth the ray would be at at this UV offset so the ray has not
|
|
|
|
// intersected the surface
|
|
|
|
for (var i: i32 = 0; texture_depth > current_layer_depth && i <= i32(layer_count); i++) {
|
|
|
|
current_layer_depth += layer_depth;
|
|
|
|
uv += delta_uv;
|
|
|
|
texture_depth = sample_depth_map(uv);
|
|
|
|
}
|
|
|
|
|
|
|
|
#ifdef RELIEF_MAPPING
|
|
|
|
// Relief Mapping
|
|
|
|
// ==============
|
|
|
|
// "Refine" the rough result from Steep Parallax Mapping
|
|
|
|
// with a **binary search** between the layer selected by steep parallax
|
|
|
|
// and the next one to find a point closer to the depth map surface.
|
|
|
|
// This reduces the jaggy step artifacts from steep parallax mapping.
|
|
|
|
|
|
|
|
delta_uv *= 0.5;
|
|
|
|
var delta_depth = 0.5 * layer_depth;
|
|
|
|
|
|
|
|
uv -= delta_uv;
|
|
|
|
current_layer_depth -= delta_depth;
|
|
|
|
|
|
|
|
for (var i: u32 = 0u; i < max_steps; i++) {
|
|
|
|
texture_depth = sample_depth_map(uv);
|
|
|
|
|
|
|
|
// Halve the deltas for the next step
|
|
|
|
delta_uv *= 0.5;
|
|
|
|
delta_depth *= 0.5;
|
|
|
|
|
|
|
|
// Step based on whether the current depth is above or below the depth map
|
|
|
|
if (texture_depth > current_layer_depth) {
|
|
|
|
uv += delta_uv;
|
|
|
|
current_layer_depth += delta_depth;
|
|
|
|
} else {
|
|
|
|
uv -= delta_uv;
|
|
|
|
current_layer_depth -= delta_depth;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
#else
|
|
|
|
// Parallax Occlusion mapping
|
|
|
|
// ==========================
|
|
|
|
// "Refine" Steep Parallax Mapping by interpolating between the
|
|
|
|
// previous layer's depth and the computed layer depth.
|
|
|
|
// Only requires a single lookup, unlike Relief Mapping, but
|
|
|
|
// may skip small details and result in writhing material artifacts.
|
|
|
|
let previous_uv = uv - delta_uv;
|
|
|
|
let next_depth = texture_depth - current_layer_depth;
|
|
|
|
let previous_depth = sample_depth_map(previous_uv) - current_layer_depth + layer_depth;
|
|
|
|
|
|
|
|
let weight = next_depth / (next_depth - previous_depth);
|
|
|
|
|
|
|
|
uv = mix(uv, previous_uv, weight);
|
|
|
|
|
|
|
|
current_layer_depth += mix(next_depth, previous_depth, weight);
|
|
|
|
#endif
|
|
|
|
|
|
|
|
// Note: `current_layer_depth` is not returned, but may be useful
|
|
|
|
// for light computation later on in future improvements of the pbr shader.
|
|
|
|
return uv;
|
|
|
|
}
|