bevy/crates
Nicola Papale 8df014fbaf
Add parallax mapping to bevy PBR (#5928)
# Objective

Add a [parallax mapping] shader to bevy. Please note that
this is a 3d technique, NOT a 2d sidescroller feature.

## Solution

- Add related fields to `StandardMaterial`
- update the pbr shader
- Add an example taking advantage of parallax mapping

A pre-existing implementation exists at:
https://github.com/nicopap/bevy_mod_paramap/

The implementation is derived from:

https://web.archive.org/web/20150419215321/http://sunandblackcat.com/tipFullView.php?l=eng&topicid=28

Further discussion on literature is found in the `bevy_mod_paramap`
README.

### Limitations

- The mesh silhouette isn't affected by the depth map.
- The depth of the pixel does not reflect its visual position, resulting
  in artifacts for depth-dependent features such as fog or SSAO
- GLTF does not define a height map texture, so somehow the user will
  always need to work around this limitation, though [an extension is in
  the works][gltf]

### Future work

- It's possible to update the depth in the depth buffer to follow the
  parallaxed texture. This would enable interop with depth-based
  visual effects, it also allows `discard`ing pixels of materials when
  computed depth is higher than the one in depth buffer
- Cheap lower quality single-sample method using [offset limiting]
- Add distance fading, to disable parallaxing (relatively expensive)
  on distant objects
- GLTF extension to allow defining height maps. Or a workaround
  implemented through a blender plugin to the GLTF exporter that
  uses the `extras` field to add height map.
- [Quadratic surface vertex attributes][oliveira_3] to enable parallax
  mapping on bending surfaces and allow clean silhouetting.
- noise based sampling, to limit the pancake artifacts.
- Cone mapping ([GPU gems], [Simcity (2013)][simcity]). Requires
  preprocessing, increase depth map size, reduces sample count greatly.
- [Quadtree parallax mapping][qpm] (also requires preprocessing)
- Self-shadowing of parallax-mapped surfaces by modifying the shadow map
- Generate depth map from normal map [link to slides], [blender
question]


https://user-images.githubusercontent.com/26321040/223563792-dffcc6ab-70e8-4ff9-90d1-b36c338695ad.mp4

[blender question]:
https://blender.stackexchange.com/questions/89278/how-to-get-a-smooth-curvature-map-from-a-normal-map
[link to slides]:
https://developer.download.nvidia.com/assets/gamedev/docs/nmap2displacement.pdf
[oliveira_3]:
https://www.inf.ufrgs.br/~oliveira/pubs_files/Oliveira_Policarpo_RP-351_Jan_2005.pdf
[GPU gems]:
https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-18-relaxed-cone-stepping-relief-mapping
[simcity]:
https://community.simtropolis.com/omnibus/other-games/building-and-rendering-simcity-2013-r247/
[offset limiting]:
https://raw.githubusercontent.com/marcusstenbeck/tncg14-parallax-mapping/master/documents/Parallax%20Mapping%20with%20Offset%20Limiting%20-%20A%20Per-Pixel%20Approximation%20of%20Uneven%20Surfaces.pdf
[gltf]: https://github.com/KhronosGroup/glTF/pull/2196
[qpm]:
https://www.gamedevs.org/uploads/quadtree-displacement-mapping-with-height-blending.pdf

---

## Changelog

- Add a `depth_map` field to the `StandardMaterial`, it is a grayscale
  image where white represents bottom and black the top. If `depth_map`
  is set, bevy's pbr shader will use it to do [parallax mapping] to
  give an increased feel of depth to the material. This is similar to a
  displacement map, but with infinite precision at fairly low cost.
- The fields `parallax_mapping_method`, `parallax_depth_scale` and
  `max_parallax_layer_count` allow finer grained control over the
  behavior of the parallax shader.
- Add the `parallax_mapping` example to show off the effect.

[parallax mapping]: https://en.wikipedia.org/wiki/Parallax_mapping

---------

Co-authored-by: Robert Swain <robert.swain@gmail.com>
2023-04-15 10:25:14 +00:00
..
bevy_a11y Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_animation Improved AnimationPlugin declaration. (#8361) 2023-04-12 19:27:38 +00:00
bevy_app Non-breaking change* from UK spellings to US (#8291) 2023-04-08 16:22:46 +00:00
bevy_asset Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_audio Ability to set a Global Volume (#7706) 2023-04-10 14:08:43 +00:00
bevy_core Non-breaking change* from UK spellings to US (#8291) 2023-04-08 16:22:46 +00:00
bevy_core_pipeline Non-breaking change* from UK spellings to US (#8291) 2023-04-08 16:22:46 +00:00
bevy_derive Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_diagnostic Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_dylib Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_dynamic_plugin Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_ecs Add a regression test for change detection between piped systems (#8368) 2023-04-13 17:59:29 +00:00
bevy_ecs_compile_fail_tests Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_encase_derive Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_gilrs Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_gizmos Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_gltf Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_hierarchy Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_input Non-breaking change* from UK spellings to US (#8291) 2023-04-08 16:22:46 +00:00
bevy_internal Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_log Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_macro_utils Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_math Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_mikktspace Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_pbr Add parallax mapping to bevy PBR (#5928) 2023-04-15 10:25:14 +00:00
bevy_ptr Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_reflect Non-breaking change* from UK spellings to US (#8291) 2023-04-08 16:22:46 +00:00
bevy_reflect_compile_fail_tests bevy_reflect: Fix trailing comma breaking derives (#8014) 2023-03-27 21:47:33 +00:00
bevy_render added multi-line string formatting (#8350) 2023-04-13 18:00:17 +00:00
bevy_scene Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_sprite Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_tasks Cleaned up panic messages (#8219) 2023-04-12 18:27:28 +00:00
bevy_text Non-breaking change* from UK spellings to US (#8291) 2023-04-08 16:22:46 +00:00
bevy_time Fix docs for fixed timestep (#8363) 2023-04-14 19:41:31 +00:00
bevy_transform Fix transform propagation of orphaned entities (#7264) 2023-04-09 20:53:33 +00:00
bevy_ui UiRect axes constructor (#7656) 2023-04-13 20:52:21 +00:00
bevy_utils Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00
bevy_window Non-breaking change* from UK spellings to US (#8291) 2023-04-08 16:22:46 +00:00
bevy_winit Suppress the clippy::type_complexity lint (#8313) 2023-04-06 21:27:36 +00:00