Add mesh picking backend and MeshRayCast system parameter (#15800)

# Objective

Closes #15545.

`bevy_picking` supports UI and sprite picking, but not mesh picking.
Being able to pick meshes would be extremely useful for various games,
tools, and our own examples, as well as scene editors and inspectors.
So, we need a mesh picking backend!

Luckily,
[`bevy_mod_picking`](https://github.com/aevyrie/bevy_mod_picking) (which
`bevy_picking` is based on) by @aevyrie already has a [backend for
it](74f0c3c0fb/backends/bevy_picking_raycast/src/lib.rs)
using [`bevy_mod_raycast`](https://github.com/aevyrie/bevy_mod_raycast).
As a side product of adding mesh picking, we also get support for
performing ray casts on meshes!

## Solution

Upstream a large chunk of the immediate-mode ray casting functionality
from `bevy_mod_raycast`, and add a mesh picking backend based on
`bevy_mod_picking`. Huge thanks to @aevyrie who did all the hard work on
these incredible crates!

All meshes are pickable by default. Picking can be disabled for
individual entities by adding `PickingBehavior::IGNORE`, like normal.
Or, if you want mesh picking to be entirely opt-in, you can set
`MeshPickingBackendSettings::require_markers` to `true` and add a
`RayCastPickable` component to the desired camera and target entities.

You can also use the new `MeshRayCast` system parameter to cast rays
into the world manually:

```rust
fn ray_cast_system(mut ray_cast: MeshRayCast, foo_query: Query<(), With<Foo>>) {
    let ray = Ray3d::new(Vec3::ZERO, Dir3::X);

    // Only ray cast against entities with the `Foo` component.
    let filter = |entity| foo_query.contains(entity);

    // Never early-exit. Note that you can change behavior per-entity.
    let early_exit_test = |_entity| false;

    // Ignore the visibility of entities. This allows ray casting hidden entities.
    let visibility = RayCastVisibility::Any;

    let settings = RayCastSettings::default()
        .with_filter(&filter)
        .with_early_exit_test(&early_exit_test)
        .with_visibility(visibility);

    // Cast the ray with the settings, returning a list of intersections.
    let hits = ray_cast.cast_ray(ray, &settings);
}
```

This is largely a direct port, but I did make several changes to match
our APIs better, remove things we don't need or that I think are
unnecessary, and do some general improvements to code quality and
documentation.

### Changes Relative to `bevy_mod_raycast` and `bevy_mod_picking`

- Every `Raycast` and "raycast" has been renamed to `RayCast` and "ray
cast" (similar reasoning as the "Naming" section in #15724)
- `Raycast` system param has been renamed to `MeshRayCast` to avoid
naming conflicts and to be explicit that it is not for colliders
- `RaycastBackend` has been renamed to `MeshPickingBackend`
- `RayCastVisibility` variants are now `Any`, `Visible`, and
`VisibleInView` instead of `Ignore`, `MustBeVisible`, and
`MustBeVisibleAndInView`
- `NoBackfaceCulling` has been renamed to `RayCastBackfaces`, to avoid
implying that it affects the rendering of backfaces for meshes (it
doesn't)
- `SimplifiedMesh` and `RayCastBackfaces` live near other ray casting
API types, not in their own 10 LoC module
- All intersection logic and types are in the same `intersections`
module, not split across several modules
- Some intersection types have been renamed to be clearer and more
consistent
	- `IntersectionData` -> `RayMeshHit` 
	- `RayHit` -> `RayTriangleHit`
- General documentation and code quality improvements

### Removed / Not Ported

- Removed unused ray helpers and types, like `PrimitiveIntersection`
- Removed getters on intersection types, and made their properties
public
- There is no `2d` feature, and `Raycast::mesh_query` and
`Raycast::mesh2d_query` have been merged into `MeshRayCast::mesh_query`,
which handles both 2D and 3D
- I assume this existed previously because `Mesh2dHandle` used to be in
`bevy_sprite`. Now both the 2D and 3D mesh are in `bevy_render`.
- There is no `debug` feature or ray debug rendering
- There is no deferred API (`RaycastSource`)
- There is no `CursorRayPlugin` (the picking backend handles this)

### Note for Reviewers

In case it's helpful, the [first
commit](281638ef10)
here is essentially a one-to-one port. The rest of the commits are
primarily refactoring and cleaning things up in the ways listed earlier,
as well as changes to the module structure.

It may also be useful to compare the original [picking
backend](74f0c3c0fb/backends/bevy_picking_raycast/src/lib.rs)
and [`bevy_mod_raycast`](https://github.com/aevyrie/bevy_mod_raycast) to
this PR. Feel free to mention if there are any changes that I should
revert or something I should not include in this PR.

## Testing

I tested mesh picking and relevant components in some examples, for both
2D and 3D meshes, and added a new `mesh_picking` example. I also
~~stole~~ ported over the [ray-mesh intersection
benchmark](dbc5ef32fe/benches/ray_mesh_intersection.rs)
from `bevy_mod_raycast`.

---

## Showcase

Below is a version of the `2d_shapes` example modified to demonstrate 2D
mesh picking. This is not included in this PR.


https://github.com/user-attachments/assets/7742528c-8630-4c00-bacd-81576ac432bf

And below is the new `mesh_picking` example:


https://github.com/user-attachments/assets/b65c7a5a-fa3a-4c2d-8bbd-e7a2c772986e

There is also a really cool new `mesh_ray_cast` example ported over from
`bevy_mod_raycast`:


https://github.com/user-attachments/assets/3c5eb6c0-bd94-4fb0-bec6-8a85668a06c9

---------

Co-authored-by: Aevyrie <aevyrie@gmail.com>
Co-authored-by: Trent <2771466+tbillington@users.noreply.github.com>
Co-authored-by: François Mockers <mockersf@gmail.com>
This commit is contained in:
Joona Aalto 2024-10-13 20:24:19 +03:00 committed by GitHub
parent 6f7d0e5725
commit 0e30b68b20
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
13 changed files with 1350 additions and 1 deletions

View file

@ -111,6 +111,7 @@ default = [
"bevy_gilrs",
"bevy_gizmos",
"bevy_gltf",
"bevy_mesh_picking_backend",
"bevy_pbr",
"bevy_picking",
"bevy_remote",
@ -136,6 +137,9 @@ default = [
"x11",
]
# Provides an implementation for picking meshes
bevy_mesh_picking_backend = ["bevy_picking"]
# Provides an implementation for picking sprites
bevy_sprite_picking_backend = ["bevy_picking"]
@ -1213,6 +1217,17 @@ setup = [
],
]
[[example]]
name = "mesh_ray_cast"
path = "examples/3d/mesh_ray_cast.rs"
doc-scrape-examples = true
[package.metadata.example.mesh_ray_cast]
name = "Mesh Ray Cast"
description = "Demonstrates ray casting with the `MeshRayCast` system parameter"
category = "3D Rendering"
wasm = true
[[example]]
name = "lightmaps"
path = "examples/3d/lightmaps.rs"
@ -3695,6 +3710,18 @@ description = "Demonstrates how to rotate the skybox and the environment map sim
category = "3D Rendering"
wasm = false
[[example]]
name = "mesh_picking"
path = "examples/picking/mesh_picking.rs"
doc-scrape-examples = true
required-features = ["bevy_mesh_picking_backend"]
[package.metadata.example.mesh_picking]
name = "Mesh Picking"
description = "Demonstrates picking meshes"
category = "Picking"
wasm = true
[[example]]
name = "simple_picking"
path = "examples/picking/simple_picking.rs"

View file

@ -14,6 +14,7 @@ bevy_app = { path = "../crates/bevy_app" }
bevy_ecs = { path = "../crates/bevy_ecs", features = ["multi_threaded"] }
bevy_hierarchy = { path = "../crates/bevy_hierarchy" }
bevy_math = { path = "../crates/bevy_math" }
bevy_picking = { path = "../crates/bevy_picking", features = ["bevy_mesh"] }
bevy_reflect = { path = "../crates/bevy_reflect", features = ["functions"] }
bevy_render = { path = "../crates/bevy_render" }
bevy_tasks = { path = "../crates/bevy_tasks" }
@ -37,6 +38,11 @@ name = "ecs"
path = "benches/bevy_ecs/benches.rs"
harness = false
[[bench]]
name = "ray_mesh_intersection"
path = "benches/bevy_picking/ray_mesh_intersection.rs"
harness = false
[[bench]]
name = "reflect_function"
path = "benches/bevy_reflect/function.rs"

View file

@ -0,0 +1,120 @@
use bevy_math::{Dir3, Mat4, Ray3d, Vec3};
use bevy_picking::{mesh_picking::ray_cast, prelude::*};
use criterion::{black_box, criterion_group, criterion_main, Criterion};
fn ptoxznorm(p: u32, size: u32) -> (f32, f32) {
let ij = (p / (size), p % (size));
(ij.0 as f32 / size as f32, ij.1 as f32 / size as f32)
}
struct SimpleMesh {
positions: Vec<[f32; 3]>,
normals: Vec<[f32; 3]>,
indices: Vec<u32>,
}
fn mesh_creation(vertices_per_side: u32) -> SimpleMesh {
let mut positions = Vec::new();
let mut normals = Vec::new();
for p in 0..vertices_per_side.pow(2) {
let xz = ptoxznorm(p, vertices_per_side);
positions.push([xz.0 - 0.5, 0.0, xz.1 - 0.5]);
normals.push([0.0, 1.0, 0.0]);
}
let mut indices = vec![];
for p in 0..vertices_per_side.pow(2) {
if p % (vertices_per_side) != vertices_per_side - 1
&& p / (vertices_per_side) != vertices_per_side - 1
{
indices.extend_from_slice(&[p, p + 1, p + vertices_per_side]);
indices.extend_from_slice(&[p + vertices_per_side, p + 1, p + vertices_per_side + 1]);
}
}
SimpleMesh {
positions,
normals,
indices,
}
}
fn ray_mesh_intersection(c: &mut Criterion) {
let mut group = c.benchmark_group("ray_mesh_intersection");
group.warm_up_time(std::time::Duration::from_millis(500));
for vertices_per_side in [10_u32, 100, 1000] {
group.bench_function(format!("{}_vertices", vertices_per_side.pow(2)), |b| {
let ray = Ray3d::new(Vec3::new(0.0, 1.0, 0.0), Dir3::NEG_Y);
let mesh_to_world = Mat4::IDENTITY;
let mesh = mesh_creation(vertices_per_side);
b.iter(|| {
black_box(ray_cast::ray_mesh_intersection(
ray,
&mesh_to_world,
&mesh.positions,
Some(&mesh.normals),
Some(&mesh.indices),
ray_cast::Backfaces::Cull,
));
});
});
}
}
fn ray_mesh_intersection_no_cull(c: &mut Criterion) {
let mut group = c.benchmark_group("ray_mesh_intersection_no_cull");
group.warm_up_time(std::time::Duration::from_millis(500));
for vertices_per_side in [10_u32, 100, 1000] {
group.bench_function(format!("{}_vertices", vertices_per_side.pow(2)), |b| {
let ray = Ray3d::new(Vec3::new(0.0, 1.0, 0.0), Dir3::NEG_Y);
let mesh_to_world = Mat4::IDENTITY;
let mesh = mesh_creation(vertices_per_side);
b.iter(|| {
black_box(ray_cast::ray_mesh_intersection(
ray,
&mesh_to_world,
&mesh.positions,
Some(&mesh.normals),
Some(&mesh.indices),
ray_cast::Backfaces::Include,
));
});
});
}
}
fn ray_mesh_intersection_no_intersection(c: &mut Criterion) {
let mut group = c.benchmark_group("ray_mesh_intersection_no_intersection");
group.warm_up_time(std::time::Duration::from_millis(500));
for vertices_per_side in [10_u32, 100, 1000] {
group.bench_function(format!("{}_vertices", (vertices_per_side).pow(2)), |b| {
let ray = Ray3d::new(Vec3::new(0.0, 1.0, 0.0), Dir3::X);
let mesh_to_world = Mat4::IDENTITY;
let mesh = mesh_creation(vertices_per_side);
b.iter(|| {
black_box(ray_cast::ray_mesh_intersection(
ray,
&mesh_to_world,
&mesh.positions,
Some(&mesh.normals),
Some(&mesh.indices),
ray_cast::Backfaces::Cull,
));
});
});
}
}
criterion_group!(
benches,
ray_mesh_intersection,
ray_mesh_intersection_no_cull,
ray_mesh_intersection_no_intersection
);
criterion_main!(benches);

View file

@ -214,9 +214,10 @@ bevy_dev_tools = ["dep:bevy_dev_tools"]
# Enable support for the Bevy Remote Protocol
bevy_remote = ["dep:bevy_remote"]
# Provides a picking functionality
# Provides picking functionality
bevy_picking = [
"dep:bevy_picking",
"bevy_picking/bevy_mesh",
"bevy_ui?/bevy_picking",
"bevy_sprite?/bevy_picking",
]

View file

@ -7,6 +7,10 @@ homepage = "https://bevyengine.org"
repository = "https://github.com/bevyengine/bevy"
license = "MIT OR Apache-2.0"
[features]
# Provides a mesh picking backend
bevy_mesh = ["dep:bevy_mesh", "dep:crossbeam-channel"]
[dependencies]
bevy_app = { path = "../bevy_app", version = "0.15.0-dev" }
bevy_asset = { path = "../bevy_asset", version = "0.15.0-dev" }
@ -15,6 +19,7 @@ bevy_ecs = { path = "../bevy_ecs", version = "0.15.0-dev" }
bevy_hierarchy = { path = "../bevy_hierarchy", version = "0.15.0-dev" }
bevy_input = { path = "../bevy_input", version = "0.15.0-dev" }
bevy_math = { path = "../bevy_math", version = "0.15.0-dev" }
bevy_mesh = { path = "../bevy_mesh", version = "0.15.0-dev", optional = true }
bevy_reflect = { path = "../bevy_reflect", version = "0.15.0-dev" }
bevy_render = { path = "../bevy_render", version = "0.15.0-dev" }
bevy_time = { path = "../bevy_time", version = "0.15.0-dev" }
@ -22,6 +27,7 @@ bevy_transform = { path = "../bevy_transform", version = "0.15.0-dev" }
bevy_utils = { path = "../bevy_utils", version = "0.15.0-dev" }
bevy_window = { path = "../bevy_window", version = "0.15.0-dev" }
crossbeam-channel = { version = "0.5", optional = true }
uuid = { version = "1.1", features = ["v4"] }
[lints]

View file

@ -156,6 +156,8 @@ pub mod backend;
pub mod events;
pub mod focus;
pub mod input;
#[cfg(feature = "bevy_mesh")]
pub mod mesh_picking;
pub mod pointer;
use bevy_app::prelude::*;
@ -166,6 +168,12 @@ use bevy_reflect::prelude::*;
///
/// This includes the most common types in this crate, re-exported for your convenience.
pub mod prelude {
#[cfg(feature = "bevy_mesh")]
#[doc(hidden)]
pub use crate::mesh_picking::{
ray_cast::{MeshRayCast, RayCastBackfaces, RayCastSettings, RayCastVisibility},
MeshPickingBackend, MeshPickingBackendSettings, RayCastPickable,
};
#[doc(hidden)]
pub use crate::{
events::*, input::PointerInputPlugin, pointer::PointerButton, DefaultPickingPlugins,
@ -274,6 +282,8 @@ impl Plugin for DefaultPickingPlugins {
PickingPlugin::default(),
InteractionPlugin,
));
#[cfg(feature = "bevy_mesh")]
app.add_plugins(mesh_picking::MeshPickingBackend);
}
}

View file

@ -0,0 +1,133 @@
//! A [mesh ray casting](ray_cast) backend for [`bevy_picking`](crate).
//!
//! By default, all meshes are pickable. Picking can be disabled for individual entities
//! by adding [`PickingBehavior::IGNORE`].
//!
//! To make mesh picking entirely opt-in, set [`MeshPickingBackendSettings::require_markers`]
//! to `true` and add a [`RayCastPickable`] component to the desired camera and target entities.
//!
//! To manually perform mesh ray casts independent of picking, use the [`MeshRayCast`] system parameter.
pub mod ray_cast;
use crate::{
backend::{ray::RayMap, HitData, PointerHits},
prelude::*,
PickSet,
};
use bevy_app::prelude::*;
use bevy_ecs::prelude::*;
use bevy_reflect::prelude::*;
use bevy_render::{prelude::*, view::RenderLayers};
use ray_cast::{MeshRayCast, RayCastSettings, RayCastVisibility, SimplifiedMesh};
/// Runtime settings for the [`MeshPickingBackend`].
#[derive(Resource, Reflect)]
#[reflect(Resource, Default)]
pub struct MeshPickingBackendSettings {
/// When set to `true` ray casting will only happen between cameras and entities marked with
/// [`RayCastPickable`]. `false` by default.
///
/// This setting is provided to give you fine-grained control over which cameras and entities
/// should be used by the mesh picking backend at runtime.
pub require_markers: bool,
/// Determines how mesh picking should consider [`Visibility`]. When set to [`RayCastVisibility::Any`],
/// ray casts can be performed against both visible and hidden entities.
///
/// Defaults to [`RayCastVisibility::VisibleInView`], only performing picking against visible entities
/// that are in the view of a camera.
pub ray_cast_visibility: RayCastVisibility,
}
impl Default for MeshPickingBackendSettings {
fn default() -> Self {
Self {
require_markers: false,
ray_cast_visibility: RayCastVisibility::VisibleInView,
}
}
}
/// An optional component that marks cameras and target entities that should be used in the [`MeshPickingBackend`].
/// Only needed if [`MeshPickingBackendSettings::require_markers`] is set to `true`, and ignored otherwise.
#[derive(Debug, Clone, Default, Component, Reflect)]
#[reflect(Component, Default)]
pub struct RayCastPickable;
/// Adds the mesh picking backend to your app.
#[derive(Clone, Default)]
pub struct MeshPickingBackend;
impl Plugin for MeshPickingBackend {
fn build(&self, app: &mut App) {
app.init_resource::<MeshPickingBackendSettings>()
.register_type::<(RayCastPickable, MeshPickingBackendSettings, SimplifiedMesh)>()
.add_systems(PreUpdate, update_hits.in_set(PickSet::Backend));
}
}
/// Casts rays into the scene using [`MeshPickingBackendSettings`] and sends [`PointerHits`] events.
#[allow(clippy::too_many_arguments)]
pub fn update_hits(
backend_settings: Res<MeshPickingBackendSettings>,
ray_map: Res<RayMap>,
picking_cameras: Query<(&Camera, Option<&RayCastPickable>, Option<&RenderLayers>)>,
pickables: Query<&PickingBehavior>,
marked_targets: Query<&RayCastPickable>,
layers: Query<&RenderLayers>,
mut ray_cast: MeshRayCast,
mut output: EventWriter<PointerHits>,
) {
for (&ray_id, &ray) in ray_map.map().iter() {
let Ok((camera, cam_pickable, cam_layers)) = picking_cameras.get(ray_id.camera) else {
continue;
};
if backend_settings.require_markers && cam_pickable.is_none() {
continue;
}
let cam_layers = cam_layers.to_owned().unwrap_or_default();
let settings = RayCastSettings {
visibility: backend_settings.ray_cast_visibility,
filter: &|entity| {
let marker_requirement =
!backend_settings.require_markers || marked_targets.get(entity).is_ok();
// Other entities missing render layers are on the default layer 0
let entity_layers = layers.get(entity).cloned().unwrap_or_default();
let render_layers_match = cam_layers.intersects(&entity_layers);
let is_pickable = pickables
.get(entity)
.map(|p| p.is_hoverable)
.unwrap_or(true);
marker_requirement && render_layers_match && is_pickable
},
early_exit_test: &|entity_hit| {
pickables
.get(entity_hit)
.is_ok_and(|pickable| pickable.should_block_lower)
},
};
let picks = ray_cast
.cast_ray(ray, &settings)
.iter()
.map(|(entity, hit)| {
let hit_data = HitData::new(
ray_id.camera,
hit.distance,
Some(hit.point),
Some(hit.normal),
);
(*entity, hit_data)
})
.collect::<Vec<_>>();
let order = camera.order as f32;
if !picks.is_empty() {
output.send(PointerHits::new(ray_id.pointer, picks, order));
}
}
}

View file

@ -0,0 +1,387 @@
use bevy_math::{bounding::Aabb3d, Dir3, Mat4, Ray3d, Vec3, Vec3A};
use bevy_reflect::Reflect;
use bevy_render::mesh::{Indices, Mesh, PrimitiveTopology, VertexAttributeValues};
use bevy_utils::tracing::{error, warn};
use super::Backfaces;
/// Hit data for an intersection between a ray and a mesh.
#[derive(Debug, Clone, Reflect)]
pub struct RayMeshHit {
/// The point of intersection in world space.
pub point: Vec3,
/// The normal vector of the triangle at the point of intersection. Not guaranteed to be normalized for scaled meshes.
pub normal: Vec3,
/// The barycentric coordinates of the intersection.
pub barycentric_coords: Vec3,
/// The distance from the ray origin to the intersection point.
pub distance: f32,
/// The vertices of the triangle that was hit.
pub triangle: Option<[Vec3A; 3]>,
/// The index of the triangle that was hit.
pub triangle_index: Option<usize>,
}
/// Hit data for an intersection between a ray and a triangle.
#[derive(Default, Debug)]
pub struct RayTriangleHit {
pub distance: f32,
pub barycentric_coords: (f32, f32),
}
/// Casts a ray on a mesh, and returns the intersection.
pub(super) fn ray_intersection_over_mesh(
mesh: &Mesh,
mesh_transform: &Mat4,
ray: Ray3d,
backface_culling: Backfaces,
) -> Option<RayMeshHit> {
if mesh.primitive_topology() != PrimitiveTopology::TriangleList {
error!(
"Invalid intersection check: `TriangleList` is the only supported `PrimitiveTopology`"
);
return None;
}
// Get the vertex positions and normals from the mesh.
let vertex_positions: &Vec<[f32; 3]> = match mesh.attribute(Mesh::ATTRIBUTE_POSITION) {
None => {
error!("Mesh does not contain vertex positions");
return None;
}
Some(vertex_values) => match &vertex_values {
VertexAttributeValues::Float32x3(positions) => positions,
_ => {
error!("Unexpected types in {:?}", Mesh::ATTRIBUTE_POSITION);
return None;
}
},
};
let vertex_normals: Option<&[[f32; 3]]> =
if let Some(normal_values) = mesh.attribute(Mesh::ATTRIBUTE_NORMAL) {
match &normal_values {
VertexAttributeValues::Float32x3(normals) => Some(normals),
_ => None,
}
} else {
None
};
if let Some(indices) = &mesh.indices() {
match indices {
Indices::U16(vertex_indices) => ray_mesh_intersection(
ray,
mesh_transform,
vertex_positions,
vertex_normals,
Some(vertex_indices),
backface_culling,
),
Indices::U32(vertex_indices) => ray_mesh_intersection(
ray,
mesh_transform,
vertex_positions,
vertex_normals,
Some(vertex_indices),
backface_culling,
),
}
} else {
ray_mesh_intersection(
ray,
mesh_transform,
vertex_positions,
vertex_normals,
None::<&[usize]>,
backface_culling,
)
}
}
/// Checks if a ray intersects a mesh, and returns the nearest intersection if one exists.
pub fn ray_mesh_intersection<Index: Clone + Copy>(
ray: Ray3d,
mesh_transform: &Mat4,
vertex_positions: &[[f32; 3]],
vertex_normals: Option<&[[f32; 3]]>,
indices: Option<&[Index]>,
backface_culling: Backfaces,
) -> Option<RayMeshHit>
where
usize: TryFrom<Index>,
{
// The ray cast can hit the same mesh many times, so we need to track which hit is
// closest to the camera, and record that.
let mut closest_hit_distance = f32::MAX;
let mut closest_hit = None;
let world_to_mesh = mesh_transform.inverse();
let mesh_space_ray = Ray3d::new(
world_to_mesh.transform_point3(ray.origin),
Dir3::new(world_to_mesh.transform_vector3(*ray.direction)).ok()?,
);
if let Some(indices) = indices {
// Make sure this chunk has 3 vertices to avoid a panic.
if indices.len() % 3 != 0 {
warn!("Index list not a multiple of 3");
return None;
}
// Now that we're in the vector of vertex indices, we want to look at the vertex
// positions for each triangle, so we'll take indices in chunks of three, where each
// chunk of three indices are references to the three vertices of a triangle.
for index_chunk in indices.chunks_exact(3) {
let [index1, index2, index3] = [
usize::try_from(index_chunk[0]).ok()?,
usize::try_from(index_chunk[1]).ok()?,
usize::try_from(index_chunk[2]).ok()?,
];
let triangle_index = Some(index1);
let tri_vertex_positions = [
Vec3A::from(vertex_positions[index1]),
Vec3A::from(vertex_positions[index2]),
Vec3A::from(vertex_positions[index3]),
];
let tri_normals = vertex_normals.map(|normals| {
[
Vec3A::from(normals[index1]),
Vec3A::from(normals[index2]),
Vec3A::from(normals[index3]),
]
});
let Some(hit) = triangle_intersection(
tri_vertex_positions,
tri_normals,
closest_hit_distance,
&mesh_space_ray,
backface_culling,
) else {
continue;
};
closest_hit = Some(RayMeshHit {
point: mesh_transform.transform_point3(hit.point),
normal: mesh_transform.transform_vector3(hit.normal),
barycentric_coords: hit.barycentric_coords,
distance: mesh_transform
.transform_vector3(mesh_space_ray.direction * hit.distance)
.length(),
triangle: hit.triangle.map(|tri| {
[
mesh_transform.transform_point3a(tri[0]),
mesh_transform.transform_point3a(tri[1]),
mesh_transform.transform_point3a(tri[2]),
]
}),
triangle_index,
});
closest_hit_distance = hit.distance;
}
} else {
for (i, chunk) in vertex_positions.chunks_exact(3).enumerate() {
let &[a, b, c] = chunk else {
continue;
};
let triangle_index = Some(i);
let tri_vertex_positions = [Vec3A::from(a), Vec3A::from(b), Vec3A::from(c)];
let tri_normals = vertex_normals.map(|normals| {
[
Vec3A::from(normals[i]),
Vec3A::from(normals[i + 1]),
Vec3A::from(normals[i + 2]),
]
});
let Some(hit) = triangle_intersection(
tri_vertex_positions,
tri_normals,
closest_hit_distance,
&mesh_space_ray,
backface_culling,
) else {
continue;
};
closest_hit = Some(RayMeshHit {
point: mesh_transform.transform_point3(hit.point),
normal: mesh_transform.transform_vector3(hit.normal),
barycentric_coords: hit.barycentric_coords,
distance: mesh_transform
.transform_vector3(mesh_space_ray.direction * hit.distance)
.length(),
triangle: hit.triangle.map(|tri| {
[
mesh_transform.transform_point3a(tri[0]),
mesh_transform.transform_point3a(tri[1]),
mesh_transform.transform_point3a(tri[2]),
]
}),
triangle_index,
});
closest_hit_distance = hit.distance;
}
}
closest_hit
}
#[inline(always)]
fn triangle_intersection(
tri_vertices: [Vec3A; 3],
tri_normals: Option<[Vec3A; 3]>,
max_distance: f32,
ray: &Ray3d,
backface_culling: Backfaces,
) -> Option<RayMeshHit> {
let hit = ray_triangle_intersection(ray, &tri_vertices, backface_culling)?;
if hit.distance < 0.0 || hit.distance > max_distance {
return None;
};
let point = ray.get_point(hit.distance);
let u = hit.barycentric_coords.0;
let v = hit.barycentric_coords.1;
let w = 1.0 - u - v;
let barycentric = Vec3::new(u, v, w);
let normal = if let Some(normals) = tri_normals {
normals[1] * u + normals[2] * v + normals[0] * w
} else {
(tri_vertices[1] - tri_vertices[0])
.cross(tri_vertices[2] - tri_vertices[0])
.normalize()
};
Some(RayMeshHit {
point,
normal: normal.into(),
barycentric_coords: barycentric,
distance: hit.distance,
triangle: Some(tri_vertices),
triangle_index: None,
})
}
/// Takes a ray and triangle and computes the intersection.
#[inline(always)]
fn ray_triangle_intersection(
ray: &Ray3d,
triangle: &[Vec3A; 3],
backface_culling: Backfaces,
) -> Option<RayTriangleHit> {
// Source: https://www.scratchapixel.com/lessons/3d-basic-rendering/ray-tracing-rendering-a-triangle/moller-trumbore-ray-triangle-intersection
let vector_v0_to_v1: Vec3A = triangle[1] - triangle[0];
let vector_v0_to_v2: Vec3A = triangle[2] - triangle[0];
let p_vec: Vec3A = (Vec3A::from(*ray.direction)).cross(vector_v0_to_v2);
let determinant: f32 = vector_v0_to_v1.dot(p_vec);
match backface_culling {
Backfaces::Cull => {
// if the determinant is negative the triangle is back facing
// if the determinant is close to 0, the ray misses the triangle
// This test checks both cases
if determinant < f32::EPSILON {
return None;
}
}
Backfaces::Include => {
// ray and triangle are parallel if det is close to 0
if determinant.abs() < f32::EPSILON {
return None;
}
}
}
let determinant_inverse = 1.0 / determinant;
let t_vec = Vec3A::from(ray.origin) - triangle[0];
let u = t_vec.dot(p_vec) * determinant_inverse;
if !(0.0..=1.0).contains(&u) {
return None;
}
let q_vec = t_vec.cross(vector_v0_to_v1);
let v = Vec3A::from(*ray.direction).dot(q_vec) * determinant_inverse;
if v < 0.0 || u + v > 1.0 {
return None;
}
// The distance between ray origin and intersection is t.
let t: f32 = vector_v0_to_v2.dot(q_vec) * determinant_inverse;
Some(RayTriangleHit {
distance: t,
barycentric_coords: (u, v),
})
}
// TODO: It'd be nice to reuse `RayCast3d::aabb_intersection_at`, but it assumes a normalized ray.
// In our case, the ray is transformed to model space, which could involve scaling.
/// Checks if the ray intersects with the AABB of a mesh, returning the distance to the point of intersection.
/// The distance is zero if the ray starts inside the AABB.
pub fn ray_aabb_intersection_3d(ray: Ray3d, aabb: &Aabb3d, model_to_world: &Mat4) -> Option<f32> {
// Transform the ray to model space
let world_to_model = model_to_world.inverse();
let ray_direction: Vec3A = world_to_model.transform_vector3a((*ray.direction).into());
let ray_direction_recip = ray_direction.recip();
let ray_origin: Vec3A = world_to_model.transform_point3a(ray.origin.into());
// Check if the ray intersects the mesh's AABB. It's useful to work in model space
// because we can do an AABB intersection test, instead of an OBB intersection test.
// NOTE: This is largely copied from `RayCast3d::aabb_intersection_at`.
let positive = ray_direction.signum().cmpgt(Vec3A::ZERO);
let min = Vec3A::select(positive, aabb.min, aabb.max);
let max = Vec3A::select(positive, aabb.max, aabb.min);
// Calculate the minimum/maximum time for each axis based on how much the direction goes that
// way. These values can get arbitrarily large, or even become NaN, which is handled by the
// min/max operations below
let tmin = (min - ray_origin) * ray_direction_recip;
let tmax = (max - ray_origin) * ray_direction_recip;
// An axis that is not relevant to the ray direction will be NaN. When one of the arguments
// to min/max is NaN, the other argument is used.
// An axis for which the direction is the wrong way will return an arbitrarily large
// negative value.
let tmin = tmin.max_element().max(0.0);
let tmax = tmax.min_element();
if tmin <= tmax {
Some(tmin)
} else {
None
}
}
#[cfg(test)]
mod tests {
use bevy_math::Vec3;
use super::*;
// Triangle vertices to be used in a left-hand coordinate system
const V0: [f32; 3] = [1.0, -1.0, 2.0];
const V1: [f32; 3] = [1.0, 2.0, -1.0];
const V2: [f32; 3] = [1.0, -1.0, -1.0];
#[test]
fn ray_cast_triangle_mt() {
let triangle = [V0.into(), V1.into(), V2.into()];
let ray = Ray3d::new(Vec3::ZERO, Dir3::X);
let result = ray_triangle_intersection(&ray, &triangle, Backfaces::Include);
assert!(result.unwrap().distance - 1.0 <= f32::EPSILON);
}
#[test]
fn ray_cast_triangle_mt_culling() {
let triangle = [V2.into(), V1.into(), V0.into()];
let ray = Ray3d::new(Vec3::ZERO, Dir3::X);
let result = ray_triangle_intersection(&ray, &triangle, Backfaces::Cull);
assert!(result.is_none());
}
}

View file

@ -0,0 +1,307 @@
//! Ray casting for meshes.
//!
//! See the [`MeshRayCast`] system parameter for more information.
mod intersections;
use bevy_derive::{Deref, DerefMut};
use bevy_math::{bounding::Aabb3d, Ray3d};
use bevy_reflect::{std_traits::ReflectDefault, Reflect};
use bevy_render::mesh::Mesh;
use intersections::*;
pub use intersections::{ray_aabb_intersection_3d, ray_mesh_intersection, RayMeshHit};
use bevy_asset::{Assets, Handle};
use bevy_ecs::{prelude::*, system::lifetimeless::Read, system::SystemParam};
use bevy_math::FloatOrd;
use bevy_render::{prelude::*, primitives::Aabb};
use bevy_transform::components::GlobalTransform;
use bevy_utils::tracing::*;
/// How a ray cast should handle [`Visibility`].
#[derive(Clone, Copy, Reflect)]
pub enum RayCastVisibility {
/// Completely ignore visibility checks. Hidden items can still be ray casted against.
Any,
/// Only cast rays against entities that are visible in the hierarchy. See [`Visibility`].
Visible,
/// Only cast rays against entities that are visible in the hierarchy and visible to a camera or
/// light. See [`Visibility`].
VisibleInView,
}
/// Settings for a ray cast.
#[derive(Clone)]
pub struct RayCastSettings<'a> {
/// Determines how ray casting should consider [`Visibility`].
pub visibility: RayCastVisibility,
/// A predicate that is applied for every entity that ray casts are performed against.
/// Only entities that return `true` will be considered.
pub filter: &'a dyn Fn(Entity) -> bool,
/// A function that is run every time a hit is found. Ray casting will continue to check for hits
/// along the ray as long as this returns `false`.
pub early_exit_test: &'a dyn Fn(Entity) -> bool,
}
impl<'a> RayCastSettings<'a> {
/// Set the filter to apply to the ray cast.
pub fn with_filter(mut self, filter: &'a impl Fn(Entity) -> bool) -> Self {
self.filter = filter;
self
}
/// Set the early exit test to apply to the ray cast.
pub fn with_early_exit_test(mut self, early_exit_test: &'a impl Fn(Entity) -> bool) -> Self {
self.early_exit_test = early_exit_test;
self
}
/// Set the [`RayCastVisibility`] setting to apply to the ray cast.
pub fn with_visibility(mut self, visibility: RayCastVisibility) -> Self {
self.visibility = visibility;
self
}
/// This ray cast should exit as soon as the nearest hit is found.
pub fn always_early_exit(self) -> Self {
self.with_early_exit_test(&|_| true)
}
/// This ray cast should check all entities whose AABB intersects the ray and return all hits.
pub fn never_early_exit(self) -> Self {
self.with_early_exit_test(&|_| false)
}
}
impl<'a> Default for RayCastSettings<'a> {
fn default() -> Self {
Self {
visibility: RayCastVisibility::VisibleInView,
filter: &|_| true,
early_exit_test: &|_| true,
}
}
}
/// Determines whether backfaces should be culled or included in ray intersection tests.
///
/// By default, backfaces are culled.
#[derive(Copy, Clone, Default, Reflect)]
#[reflect(Default)]
pub enum Backfaces {
/// Cull backfaces.
#[default]
Cull,
/// Include backfaces.
Include,
}
/// Disables backface culling for [ray casts](MeshRayCast) on this entity.
#[derive(Component, Copy, Clone, Default, Reflect)]
#[reflect(Component, Default)]
pub struct RayCastBackfaces;
/// A simplified mesh component that can be used for [ray casting](super::MeshRayCast).
///
/// Consider using this component for complex meshes that don't need perfectly accurate ray casting.
#[derive(Component, Clone, Debug, Deref, DerefMut, Reflect)]
#[reflect(Component, Debug)]
pub struct SimplifiedMesh(pub Handle<Mesh>);
type MeshFilter = Or<(With<Mesh3d>, With<Mesh2d>, With<SimplifiedMesh>)>;
/// Add this ray casting [`SystemParam`] to your system to cast rays into the world with an
/// immediate-mode API. Call `cast_ray` to immediately perform a ray cast and get a result.
///
/// Under the hood, this is a collection of regular bevy queries, resources, and local parameters
/// that are added to your system.
///
/// ## Usage
///
/// The following system casts a ray into the world with the ray positioned at the origin, pointing in
/// the X-direction, and returns a list of intersections:
///
/// ```
/// # use bevy_math::prelude::*;
/// # use bevy_picking::prelude::*;
/// fn ray_cast_system(mut ray_cast: MeshRayCast) {
/// let ray = Ray3d::new(Vec3::ZERO, Dir3::X);
/// let hits = ray_cast.cast_ray(ray, &RayCastSettings::default());
/// }
/// ```
///
/// ## Configuration
///
/// You can specify the behavior of the ray cast using [`RayCastSettings`]. This allows you to filter out
/// entities, configure early-out behavior, and set whether the [`Visibility`] of an entity should be
/// considered.
///
/// ```
/// # use bevy_ecs::prelude::*;
/// # use bevy_math::prelude::*;
/// # use bevy_picking::prelude::*;
/// # #[derive(Component)]
/// # struct Foo;
/// fn ray_cast_system(mut ray_cast: MeshRayCast, foo_query: Query<(), With<Foo>>) {
/// let ray = Ray3d::new(Vec3::ZERO, Dir3::X);
///
/// // Only ray cast against entities with the `Foo` component.
/// let filter = |entity| foo_query.contains(entity);
///
/// // Never early-exit. Note that you can change behavior per-entity.
/// let early_exit_test = |_entity| false;
///
/// // Ignore the visibility of entities. This allows ray casting hidden entities.
/// let visibility = RayCastVisibility::Any;
///
/// let settings = RayCastSettings::default()
/// .with_filter(&filter)
/// .with_early_exit_test(&early_exit_test)
/// .with_visibility(visibility);
///
/// // Cast the ray with the settings, returning a list of intersections.
/// let hits = ray_cast.cast_ray(ray, &settings);
/// }
/// ```
#[derive(SystemParam)]
pub struct MeshRayCast<'w, 's> {
#[doc(hidden)]
pub meshes: Res<'w, Assets<Mesh>>,
#[doc(hidden)]
pub hits: Local<'s, Vec<(FloatOrd, (Entity, RayMeshHit))>>,
#[doc(hidden)]
pub output: Local<'s, Vec<(Entity, RayMeshHit)>>,
#[doc(hidden)]
pub culled_list: Local<'s, Vec<(FloatOrd, Entity)>>,
#[doc(hidden)]
pub culling_query: Query<
'w,
's,
(
Read<InheritedVisibility>,
Read<ViewVisibility>,
Read<Aabb>,
Read<GlobalTransform>,
Entity,
),
MeshFilter,
>,
#[doc(hidden)]
pub mesh_query: Query<
'w,
's,
(
Option<Read<Mesh2d>>,
Option<Read<Mesh3d>>,
Option<Read<SimplifiedMesh>>,
Has<RayCastBackfaces>,
Read<GlobalTransform>,
),
MeshFilter,
>,
}
impl<'w, 's> MeshRayCast<'w, 's> {
/// Casts the `ray` into the world and returns a sorted list of intersections, nearest first.
pub fn cast_ray(&mut self, ray: Ray3d, settings: &RayCastSettings) -> &[(Entity, RayMeshHit)] {
let ray_cull = info_span!("ray culling");
let ray_cull_guard = ray_cull.enter();
self.hits.clear();
self.culled_list.clear();
self.output.clear();
// Check all entities to see if the ray intersects the AABB. Use this to build a short list
// of entities that are in the path of the ray.
let (aabb_hits_tx, aabb_hits_rx) = crossbeam_channel::unbounded::<(FloatOrd, Entity)>();
let visibility_setting = settings.visibility;
self.culling_query.par_iter().for_each(
|(inherited_visibility, view_visibility, aabb, transform, entity)| {
let should_ray_cast = match visibility_setting {
RayCastVisibility::Any => true,
RayCastVisibility::Visible => inherited_visibility.get(),
RayCastVisibility::VisibleInView => view_visibility.get(),
};
if should_ray_cast {
if let Some(distance) = ray_aabb_intersection_3d(
ray,
&Aabb3d::new(aabb.center, aabb.half_extents),
&transform.compute_matrix(),
) {
aabb_hits_tx.send((FloatOrd(distance), entity)).ok();
}
}
},
);
*self.culled_list = aabb_hits_rx.try_iter().collect();
// Sort by the distance along the ray.
self.culled_list.sort_by_key(|(aabb_near, _)| *aabb_near);
drop(ray_cull_guard);
// Perform ray casts against the culled entities.
let mut nearest_blocking_hit = FloatOrd(f32::INFINITY);
let ray_cast_guard = debug_span!("ray_cast");
self.culled_list
.iter()
.filter(|(_, entity)| (settings.filter)(*entity))
.for_each(|(aabb_near, entity)| {
// Get the mesh components and transform.
let Ok((mesh2d, mesh3d, simplified_mesh, has_backfaces, transform)) =
self.mesh_query.get(*entity)
else {
return;
};
// Get the underlying mesh handle. One of these will always be `Some` because of the query filters.
let Some(mesh_handle) = simplified_mesh
.map(|m| &m.0)
.or(mesh3d.map(|m| &m.0).or(mesh2d.map(|m| &m.0)))
else {
return;
};
// Is it even possible the mesh could be closer than the current best?
if *aabb_near > nearest_blocking_hit {
return;
}
// Does the mesh handle resolve?
let Some(mesh) = self.meshes.get(mesh_handle) else {
return;
};
let backfaces = match has_backfaces {
true => Backfaces::Include,
false => Backfaces::Cull,
};
// Perform the actual ray cast.
let _ray_cast_guard = ray_cast_guard.enter();
let transform = transform.compute_matrix();
let intersection = ray_intersection_over_mesh(mesh, &transform, ray, backfaces);
if let Some(intersection) = intersection {
let distance = FloatOrd(intersection.distance);
if (settings.early_exit_test)(*entity) && distance < nearest_blocking_hit {
// The reason we don't just return here is because right now we are
// going through the AABBs in order, but that doesn't mean that an
// AABB that starts further away can't end up with a closer hit than
// an AABB that starts closer. We need to keep checking AABBs that
// could possibly contain a nearer hit.
nearest_blocking_hit = distance.min(nearest_blocking_hit);
}
self.hits.push((distance, (*entity, intersection)));
};
});
self.hits.retain(|(dist, _)| *dist <= nearest_blocking_hit);
self.hits.sort_by_key(|(k, _)| *k);
let hits = self.hits.iter().map(|(_, (e, i))| (*e, i.to_owned()));
self.output.extend(hits);
self.output.as_ref()
}
}

View file

@ -22,6 +22,7 @@ The default feature set enables most of the expected features of a game engine,
|bevy_gilrs|Adds gamepad support|
|bevy_gizmos|Adds support for rendering gizmos|
|bevy_gltf|[glTF](https://www.khronos.org/gltf/) support|
|bevy_mesh_picking_backend|Provides an implementation for picking meshes|
|bevy_pbr|Adds PBR rendering|
|bevy_picking|Provides picking functionality|
|bevy_remote|Enable the Bevy Remote Protocol|

View file

@ -0,0 +1,112 @@
//! Demonstrates how to use the [`MeshRayCast`] system parameter to chain multiple ray casts
//! and bounce off of surfaces.
use std::f32::consts::{FRAC_PI_2, PI};
use bevy::{
color::palettes::css,
core_pipeline::{bloom::Bloom, tonemapping::Tonemapping},
math::vec3,
picking::backend::ray::RayMap,
prelude::*,
};
fn main() {
App::new()
.add_plugins(DefaultPlugins)
.add_systems(Startup, setup)
.add_systems(Update, bouncing_raycast)
.insert_resource(ClearColor(Color::BLACK))
.run();
}
const MAX_BOUNCES: usize = 64;
const LASER_SPEED: f32 = 0.03;
fn bouncing_raycast(
mut ray_cast: MeshRayCast,
mut gizmos: Gizmos,
time: Res<Time>,
// The ray map stores rays cast by the cursor
ray_map: Res<RayMap>,
) {
// Cast an automatically moving ray and bounce it off of surfaces
let t = ops::cos((time.elapsed_seconds() - 4.0).max(0.0) * LASER_SPEED) * PI;
let ray_pos = Vec3::new(ops::sin(t), ops::cos(3.0 * t) * 0.5, ops::cos(t)) * 0.5;
let ray_dir = Dir3::new(-ray_pos).unwrap();
let ray = Ray3d::new(ray_pos, ray_dir);
gizmos.sphere(ray_pos, 0.1, Color::WHITE);
bounce_ray(ray, &mut ray_cast, &mut gizmos, Color::from(css::RED));
// Cast a ray from the cursor and bounce it off of surfaces
for (_, ray) in ray_map.iter() {
bounce_ray(*ray, &mut ray_cast, &mut gizmos, Color::from(css::GREEN));
}
}
// Bounces a ray off of surfaces `MAX_BOUNCES` times.
fn bounce_ray(mut ray: Ray3d, ray_cast: &mut MeshRayCast, gizmos: &mut Gizmos, color: Color) {
let mut intersections = Vec::with_capacity(MAX_BOUNCES + 1);
intersections.push((ray.origin, Color::srgb(30.0, 0.0, 0.0)));
for i in 0..MAX_BOUNCES {
// Cast the ray and get the first hit
let Some((_, hit)) = ray_cast.cast_ray(ray, &RayCastSettings::default()).first() else {
break;
};
// Draw the point of intersection and add it to the list
let brightness = 1.0 + 10.0 * (1.0 - i as f32 / MAX_BOUNCES as f32);
intersections.push((hit.point, Color::BLACK.mix(&color, brightness)));
gizmos.sphere(hit.point, 0.005, Color::BLACK.mix(&color, brightness * 2.0));
// Reflect the ray off of the surface
ray.direction = Dir3::new(ray.direction.reflect(hit.normal)).unwrap();
ray.origin = hit.point + ray.direction * 1e-6;
}
gizmos.linestrip_gradient(intersections);
}
// Set up a simple 3D scene
fn setup(
mut commands: Commands,
mut meshes: ResMut<Assets<Mesh>>,
mut materials: ResMut<Assets<StandardMaterial>>,
) {
// Make a box of planes facing inward so the laser gets trapped inside
let plane_mesh = meshes.add(Plane3d::default());
let plane_material = materials.add(Color::from(css::GRAY).with_alpha(0.01));
let create_plane = move |translation, rotation| {
(
Transform::from_translation(translation)
.with_rotation(Quat::from_scaled_axis(rotation)),
Mesh3d(plane_mesh.clone()),
MeshMaterial3d(plane_material.clone()),
)
};
commands.spawn(create_plane(vec3(0.0, 0.5, 0.0), Vec3::X * PI));
commands.spawn(create_plane(vec3(0.0, -0.5, 0.0), Vec3::ZERO));
commands.spawn(create_plane(vec3(0.5, 0.0, 0.0), Vec3::Z * FRAC_PI_2));
commands.spawn(create_plane(vec3(-0.5, 0.0, 0.0), Vec3::Z * -FRAC_PI_2));
commands.spawn(create_plane(vec3(0.0, 0.0, 0.5), Vec3::X * -FRAC_PI_2));
commands.spawn(create_plane(vec3(0.0, 0.0, -0.5), Vec3::X * FRAC_PI_2));
// Light
commands.spawn((
DirectionalLight::default(),
Transform::from_rotation(Quat::from_euler(EulerRot::XYZ, -0.1, 0.2, 0.0)),
));
// Camera
commands.spawn((
Camera3d::default(),
Camera {
hdr: true,
..default()
},
Transform::from_xyz(1.5, 1.5, 1.5).looking_at(Vec3::ZERO, Vec3::Y),
Tonemapping::TonyMcMapface,
Bloom::default(),
));
}

View file

@ -156,6 +156,7 @@ Example | Description
[Lines](../examples/3d/lines.rs) | Create a custom material to draw 3d lines
[Load glTF](../examples/3d/load_gltf.rs) | Loads and renders a glTF file as a scene
[Load glTF extras](../examples/3d/load_gltf_extras.rs) | Loads and renders a glTF file as a scene, including the gltf extras
[Mesh Ray Cast](../examples/3d/mesh_ray_cast.rs) | Demonstrates ray casting with the `MeshRayCast` system parameter
[Meshlet](../examples/3d/meshlet.rs) | Meshlet rendering for dense high-poly scenes (experimental)
[Motion Blur](../examples/3d/motion_blur.rs) | Demonstrates per-pixel motion blur
[Order Independent Transparency](../examples/3d/order_independent_transparency.rs) | Demonstrates how to use OIT
@ -375,6 +376,7 @@ Example | Description
Example | Description
--- | ---
[Mesh Picking](../examples/picking/mesh_picking.rs) | Demonstrates picking meshes
[Showcases simple picking events and usage](../examples/picking/simple_picking.rs) | Demonstrates how to use picking events to spawn simple objects
[Sprite Picking](../examples/picking/sprite_picking.rs) | Demonstrates picking sprites and sprite atlases

View file

@ -0,0 +1,237 @@
//! A simple 3D scene to demonstrate mesh picking.
//!
//! By default, all meshes are pickable. Picking can be disabled for individual entities
//! by adding [`PickingBehavior::IGNORE`].
//!
//! If you want mesh picking to be entirely opt-in, you can set [`MeshPickingBackendSettings::require_markers`]
//! to `true` and add a [`RayCastPickable`] component to the desired camera and target entities.
use std::f32::consts::PI;
use bevy::{
color::palettes::{
css::{PINK, RED, SILVER},
tailwind::{CYAN_300, YELLOW_300},
},
picking::backend::PointerHits,
prelude::*,
};
fn main() {
App::new()
.add_plugins(DefaultPlugins)
.init_resource::<SceneMaterials>()
.add_systems(Startup, setup)
.add_systems(Update, (on_mesh_hover, rotate))
.run();
}
/// Materials for the scene
#[derive(Resource, Default)]
struct SceneMaterials {
pub white: Handle<StandardMaterial>,
pub ground: Handle<StandardMaterial>,
pub hover: Handle<StandardMaterial>,
pub pressed: Handle<StandardMaterial>,
}
/// A marker component for our shapes so we can query them separately from the ground plane.
#[derive(Component)]
struct Shape;
const SHAPES_X_EXTENT: f32 = 14.0;
const EXTRUSION_X_EXTENT: f32 = 16.0;
const Z_EXTENT: f32 = 5.0;
fn setup(
mut commands: Commands,
mut meshes: ResMut<Assets<Mesh>>,
mut materials: ResMut<Assets<StandardMaterial>>,
mut scene_materials: ResMut<SceneMaterials>,
) {
// Set up the materials.
scene_materials.white = materials.add(Color::WHITE);
scene_materials.ground = materials.add(Color::from(SILVER));
scene_materials.hover = materials.add(Color::from(CYAN_300));
scene_materials.pressed = materials.add(Color::from(YELLOW_300));
let shapes = [
meshes.add(Cuboid::default()),
meshes.add(Tetrahedron::default()),
meshes.add(Capsule3d::default()),
meshes.add(Torus::default()),
meshes.add(Cylinder::default()),
meshes.add(Cone::default()),
meshes.add(ConicalFrustum::default()),
meshes.add(Sphere::default().mesh().ico(5).unwrap()),
meshes.add(Sphere::default().mesh().uv(32, 18)),
];
let extrusions = [
meshes.add(Extrusion::new(Rectangle::default(), 1.)),
meshes.add(Extrusion::new(Capsule2d::default(), 1.)),
meshes.add(Extrusion::new(Annulus::default(), 1.)),
meshes.add(Extrusion::new(Circle::default(), 1.)),
meshes.add(Extrusion::new(Ellipse::default(), 1.)),
meshes.add(Extrusion::new(RegularPolygon::default(), 1.)),
meshes.add(Extrusion::new(Triangle2d::default(), 1.)),
];
let num_shapes = shapes.len();
// Spawn the shapes. The meshes will be pickable by default.
for (i, shape) in shapes.into_iter().enumerate() {
commands
.spawn((
Mesh3d(shape),
MeshMaterial3d(scene_materials.white.clone()),
Transform::from_xyz(
-SHAPES_X_EXTENT / 2. + i as f32 / (num_shapes - 1) as f32 * SHAPES_X_EXTENT,
2.0,
Z_EXTENT / 2.,
)
.with_rotation(Quat::from_rotation_x(-PI / 4.)),
Shape,
))
.observe(on_pointer_over)
.observe(on_pointer_out)
.observe(on_pointer_down)
.observe(on_pointer_up);
}
let num_extrusions = extrusions.len();
for (i, shape) in extrusions.into_iter().enumerate() {
commands
.spawn((
Mesh3d(shape),
MeshMaterial3d(scene_materials.white.clone()),
Transform::from_xyz(
-EXTRUSION_X_EXTENT / 2.
+ i as f32 / (num_extrusions - 1) as f32 * EXTRUSION_X_EXTENT,
2.0,
-Z_EXTENT / 2.,
)
.with_rotation(Quat::from_rotation_x(-PI / 4.)),
Shape,
))
.observe(on_pointer_over)
.observe(on_pointer_out)
.observe(on_pointer_down)
.observe(on_pointer_up);
}
// Disable picking for the ground plane.
commands.spawn((
Mesh3d(meshes.add(Plane3d::default().mesh().size(50.0, 50.0).subdivisions(10))),
MeshMaterial3d(scene_materials.ground.clone()),
PickingBehavior::IGNORE,
));
// Light
commands.spawn((
PointLight {
shadows_enabled: true,
intensity: 10_000_000.,
range: 100.0,
shadow_depth_bias: 0.2,
..default()
},
Transform::from_xyz(8.0, 16.0, 8.0),
));
// Camera
commands.spawn((
Camera3d::default(),
Transform::from_xyz(0.0, 7., 14.0).looking_at(Vec3::new(0., 1., 0.), Vec3::Y),
));
// Instructions
commands.spawn((
Text::new("Hover over the shapes to pick them"),
Style {
position_type: PositionType::Absolute,
top: Val::Px(12.0),
left: Val::Px(12.0),
..default()
},
));
}
/// Changes the material when the pointer is over the mesh.
fn on_pointer_over(
trigger: Trigger<Pointer<Over>>,
scene_materials: Res<SceneMaterials>,
mut query: Query<&mut MeshMaterial3d<StandardMaterial>>,
) {
if let Ok(mut material) = query.get_mut(trigger.entity()) {
material.0 = scene_materials.hover.clone();
}
}
/// Resets the material when the pointer leaves the mesh.
fn on_pointer_out(
trigger: Trigger<Pointer<Out>>,
scene_materials: Res<SceneMaterials>,
mut query: Query<&mut MeshMaterial3d<StandardMaterial>>,
) {
if let Ok(mut material) = query.get_mut(trigger.entity()) {
material.0 = scene_materials.white.clone();
}
}
/// Changes the material when the pointer is pressed.
fn on_pointer_down(
trigger: Trigger<Pointer<Down>>,
scene_materials: Res<SceneMaterials>,
mut query: Query<&mut MeshMaterial3d<StandardMaterial>>,
) {
if let Ok(mut material) = query.get_mut(trigger.entity()) {
material.0 = scene_materials.pressed.clone();
}
}
/// Resets the material when the pointer is released.
fn on_pointer_up(
trigger: Trigger<Pointer<Up>>,
scene_materials: Res<SceneMaterials>,
mut query: Query<&mut MeshMaterial3d<StandardMaterial>>,
) {
if let Ok(mut material) = query.get_mut(trigger.entity()) {
material.0 = scene_materials.hover.clone();
}
}
/// Draws the closest point of intersection for pointer hits.
fn on_mesh_hover(
mut pointer_hits: EventReader<PointerHits>,
meshes: Query<Entity, With<Mesh3d>>,
mut gizmos: Gizmos,
) {
for hit in pointer_hits.read() {
// Get the first mesh hit.
// The hits are sorted by distance from the camera, so this is the closest hit.
let Some(closest_hit) = hit
.picks
.iter()
.filter_map(|(entity, hit)| meshes.get(*entity).map(|_| hit).ok())
.next()
else {
continue;
};
let (Some(point), Some(normal)) = (closest_hit.position, closest_hit.normal) else {
return;
};
gizmos.sphere(point, 0.05, RED);
gizmos.arrow(point, point + normal * 0.5, PINK);
}
}
/// Rotates the shapes.
fn rotate(mut query: Query<&mut Transform, With<Shape>>, time: Res<Time>) {
for mut transform in &mut query {
transform.rotate_y(time.delta_seconds() / 2.);
}
}