Mirrors/bevy
2
0
Fork 0
mirror of https://github.com/bevyengine/bevy synced 2024-11-10 07:04:33 +00:00
Code Issues Projects Releases Packages Wiki Activity

Use lifetimed, type erased pointers in bevy_ecs (#3001)

Browse source 
# Objective

`bevy_ecs` has large amounts of unsafe code which is hard to get right and makes it difficult to audit for soundness.

## Solution

Introduce lifetimed, type-erased pointers: `Ptr<'a>` `PtrMut<'a>` `OwningPtr<'a>'` and `ThinSlicePtr<'a, T>` which are newtypes around a raw pointer with a lifetime and conceptually representing strong invariants about the pointee and validity of the pointer.

The process of converting bevy_ecs to use these has already caught multiple cases of unsound behavior.

## Changelog

TL;DR for release notes: `bevy_ecs` now uses lifetimed, type-erased pointers internally, significantly improving safety and legibility without sacrificing performance. This should have approximately no end user impact, unless you were meddling with the (unfortunately public) internals of `bevy_ecs`.

- `Fetch`, `FilterFetch` and `ReadOnlyFetch` trait no longer have a `'state` lifetime
    - this was unneeded
- `ReadOnly/Fetch` associated types on `WorldQuery` are now on a new `WorldQueryGats<'world>` trait
    - was required to work around lack of Generic Associated Types (we wish to express `type Fetch<'a>: Fetch<'a>`)
- `derive(WorldQuery)` no longer requires `'w` lifetime on struct
    - this was unneeded, and improves the end user experience
- `EntityMut::get_unchecked_mut` returns `&'_ mut T` not `&'w mut T`
    - allows easier use of unsafe API with less footguns, and can be worked around via lifetime transmutery as a user
- `Bundle::from_components` now takes a `ctx` parameter to pass to the `FnMut` closure
    - required because closure return types can't borrow from captures
- `Fetch::init` takes `&'world World`, `Fetch::set_archetype` takes `&'world Archetype` and `&'world Tables`, `Fetch::set_table` takes `&'world Table`
    - allows types implementing `Fetch` to store borrows into world
- `WorldQuery` trait now has a `shrink` fn to shorten the lifetime in `Fetch::<'a>::Item`
    - this works around lack of subtyping of assoc types, rust doesnt allow you to turn `<T as Fetch<'static>>::Item'` into `<T as Fetch<'a>>::Item'`
    - `QueryCombinationsIter` requires this
- Most types implementing `Fetch` now have a lifetime `'w`
    - allows the fetches to store borrows of world data instead of using raw pointers

## Migration guide

- `EntityMut::get_unchecked_mut` returns a more restricted lifetime, there is no general way to migrate this as it depends on your code
- `Bundle::from_components` implementations must pass the `ctx` arg to `func`
- `Bundle::from_components` callers have to use a fn arg instead of closure captures for borrowing from world
- Remove lifetime args on `derive(WorldQuery)` structs as it is nonsensical
- `<Q as WorldQuery>::ReadOnly/Fetch` should be changed to either `RO/QueryFetch<'world>` or `<Q as WorldQueryGats<'world>>::ReadOnly/Fetch`
- `<F as Fetch<'w, 's>>` should be changed to `<F as Fetch<'w>>`
- Change the fn sigs of `Fetch::init/set_archetype/set_table` to match respective trait fn sigs
- Implement the required `fn shrink` on any `WorldQuery` implementations
- Move assoc types `Fetch` and `ReadOnlyFetch` on `WorldQuery` impls to `WorldQueryGats` impls
- Pass an appropriate `'world` lifetime to whatever fetch struct you are for some reason using

### Type inference regression

in some cases rustc may give spurrious errors when attempting to infer the `F` parameter on a query/querystate this can be fixed by manually specifying the type, i.e. `QueryState:🆕:<_, ()>(world)`. The error is rather confusing:

```rust=
error[E0271]: type mismatch resolving `<() as Fetch<'_>>::Item == bool`
    --> crates/bevy_pbr/src/render/light.rs:1413:30
     |
1413 |             main_view_query: QueryState::new(world),
     |                              ^^^^^^^^^^^^^^^ expected `bool`, found `()`
     |
     = note: required because of the requirements on the impl of `for<'x> FilterFetch<'x>` for `<() as WorldQueryGats<'x>>::Fetch`
note: required by a bound in `bevy_ecs::query::QueryState::<Q, F>::new`
    --> crates/bevy_ecs/src/query/state.rs:49:32
     |
49   |     for<'x> QueryFetch<'x, F>: FilterFetch<'x>,
     |                                ^^^^^^^^^^^^^^^ required by this bound in `bevy_ecs::query::QueryState::<Q, F>::new`
```

---

Made with help from @BoxyUwU and @alice-i-cecile 

Co-authored-by: Boxy <supbscripter@gmail.com>
This commit is contained in:
TheRawMeatball 2022-04-27 23:44:06 +00:00
parent ddce22b614
commit 73c78c3667
21 changed files with 1533 additions and 1291 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

511
crates/bevy_ecs/macros/src/fetch.rs
View file

@ -3,21 +3,19 @@ use proc_macro2::{Ident, Span};
use quote::{quote, ToTokens};
use syn::{
parse::{Parse, ParseStream},
parse_quote,
punctuated::Punctuated,
Attribute, Data, DataStruct, DeriveInput, Field, Fields, GenericArgument, GenericParam,
Lifetime, LifetimeDef, Path, PathArguments, ReturnType, Token, Type, TypePath,
Attribute, Data, DataStruct, DeriveInput, Field, Fields,
};
use crate::bevy_ecs_path;
#[derive(Default)]
struct FetchStructAttributes {
pub is_filter: bool,
pub is_mutable: bool,
pub derive_args: Punctuated<syn::NestedMeta, syn::token::Comma>,
}
static FILTER_ATTRIBUTE_NAME: &str = "filter";
static MUTABLE_ATTRIBUTE_NAME: &str = "mutable";
static DERIVE_ATTRIBUTE_NAME: &str = "derive";
@ -69,15 +67,6 @@ pub fn derive_world_query_impl(ast: DeriveInput) -> TokenStream {
DERIVE_ATTRIBUTE_NAME
);
}
} else if ident == FILTER_ATTRIBUTE_NAME {
if let syn::Meta::Path(_) = meta {
fetch_struct_attributes.is_filter = true;
} else {
panic!(
"The `{}` attribute is expected to have no value or arguments",
FILTER_ATTRIBUTE_NAME
);
}
} else {
panic!(
"Unrecognized attribute: `{}`",
@ -90,54 +79,38 @@ pub fn derive_world_query_impl(ast: DeriveInput) -> TokenStream {
.unwrap_or_else(|_| panic!("Invalid `{}` attribute format", WORLD_QUERY_ATTRIBUTE_NAME));
}
if fetch_struct_attributes.is_filter && fetch_struct_attributes.is_mutable {
panic!(
"The `{}` attribute is not expected to be used in conjunction with the `{}` attribute",
FILTER_ATTRIBUTE_NAME, MUTABLE_ATTRIBUTE_NAME
);
}
let world_lifetime = ast.generics.params.first().and_then(|param| match param {
lt @ GenericParam::Lifetime(_) => Some(lt.clone()),
_ => None,
});
// Fetch's HRTBs require substituting world lifetime with an additional one to make the
// implementation compile. I don't fully understand why this works though. If anyone's curious
// enough to try to find a better work around, I'll leave playground links here:
// - https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=da5e260a5c2f3e774142d60a199e854a (this fails)
// - https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=802517bb3d8f83c45ee8c0be360bb250 (this compiles)
let fetch_lifetime_param =
GenericParam::Lifetime(LifetimeDef::new(Lifetime::new("'fetch", Span::call_site())));
let has_world_lifetime = world_lifetime.is_some();
let world_lifetime_param = world_lifetime.unwrap_or_else(|| {
GenericParam::Lifetime(LifetimeDef::new(Lifetime::new("'world", Span::call_site())))
});
let state_lifetime_param =
GenericParam::Lifetime(LifetimeDef::new(Lifetime::new("'state", Span::call_site())));
let mut fetch_trait_punctuated_lifetimes = Punctuated::<_, Token![,]>::new();
fetch_trait_punctuated_lifetimes.push(world_lifetime_param.clone());
fetch_trait_punctuated_lifetimes.push(state_lifetime_param.clone());
let (impl_generics, ty_generics, where_clause) = ast.generics.split_for_impl();
let user_generics = ast.generics.clone();
let (user_impl_generics, user_ty_generics, user_where_clauses) = user_generics.split_for_impl();
let user_generics_with_world = {
let mut generics = ast.generics.clone();
generics.params.insert(0, parse_quote!('__w));
generics
};
let (user_impl_generics_with_world, user_ty_generics_with_world, user_where_clauses_with_world) =
user_generics_with_world.split_for_impl();
let struct_name = ast.ident.clone();
let item_struct_name = Ident::new(&format!("{}Item", struct_name), Span::call_site());
let read_only_item_struct_name = if fetch_struct_attributes.is_mutable {
Ident::new(&format!("{}ReadOnlyItem", struct_name), Span::call_site())
} else {
item_struct_name.clone()
};
let fetch_struct_name = Ident::new(&format!("{}Fetch", struct_name), Span::call_site());
let state_struct_name = Ident::new(&format!("{}State", struct_name), Span::call_site());
let read_only_fetch_struct_name = if fetch_struct_attributes.is_mutable {
Ident::new(&format!("{}ReadOnlyFetch", struct_name), Span::call_site())
} else {
fetch_struct_name.clone()
};
let fetch_associated_type = Ident::new("Fetch", Span::call_site());
let read_only_fetch_associated_type = Ident::new("ReadOnlyFetch", Span::call_site());
let state_struct_name = Ident::new(&format!("{}State", struct_name), Span::call_site());
let fetch_type_alias = Ident::new("QueryFetch", Span::call_site());
let read_only_fetch_type_alias = Ident::new("ROQueryFetch", Span::call_site());
let item_type_alias = Ident::new("QueryItem", Span::call_site());
let read_only_item_type_alias = Ident::new("ROQueryItem", Span::call_site());
let fields = match &ast.data {
Data::Struct(DataStruct {
@ -155,22 +128,9 @@ pub fn derive_world_query_impl(ast: DeriveInput) -> TokenStream {
let mut field_visibilities = Vec::new();
let mut field_idents = Vec::new();
let mut field_types = Vec::new();
let mut fetch_init_types = Vec::new();
let (world_lifetime, fetch_lifetime) = match (&world_lifetime_param, &fetch_lifetime_param) {
(GenericParam::Lifetime(world), GenericParam::Lifetime(fetch)) => {
(&world.lifetime, &fetch.lifetime)
}
_ => unreachable!(),
};
for field in fields.iter() {
let WorldQueryFieldInfo {
field_type,
fetch_init_type: init_type,
is_ignored,
attrs,
} = read_world_query_field_info(field, world_lifetime, fetch_lifetime);
let WorldQueryFieldInfo { is_ignored, attrs } = read_world_query_field_info(field);
let field_ident = field.ident.as_ref().unwrap().clone();
if is_ignored {
@ -182,183 +142,128 @@ pub fn derive_world_query_impl(ast: DeriveInput) -> TokenStream {
field_attrs.push(attrs);
field_visibilities.push(field.vis.clone());
field_idents.push(field_ident.clone());
field_types.push(field_type);
fetch_init_types.push(init_type);
field_types.push(field.ty.clone());
}
}
// We expect that only regular query declarations have a lifetime.
if fetch_struct_attributes.is_filter {
if has_world_lifetime {
panic!("Expected a struct without a lifetime");
}
} else if !has_world_lifetime {
panic!("Expected a struct with a lifetime");
}
let derive_macro_call = if fetch_struct_attributes.derive_args.is_empty() {
quote! {}
} else {
let derive_args = &fetch_struct_attributes.derive_args;
quote! { #[derive(#derive_args)] }
};
// Add `'state` and `'fetch` lifetimes that will be used in `Fetch` implementation.
let mut fetch_generics = ast.generics.clone();
if !has_world_lifetime {
fetch_generics
.params
.insert(0, world_lifetime_param.clone());
}
fetch_generics.params.insert(1, state_lifetime_param);
fetch_generics
.params
.insert(2, fetch_lifetime_param.clone());
let (fetch_impl_generics, _, _) = fetch_generics.split_for_impl();
// Replace lifetime `'world` with `'fetch`. See `replace_lifetime_for_type` for more details.
let mut fetch_generics = ast.generics.clone();
*fetch_generics.params.first_mut().unwrap() = fetch_lifetime_param;
let fetch_ty_generics = if fetch_struct_attributes.is_filter {
ty_generics.clone()
} else {
let (_, fetch_ty_generics, _) = fetch_generics.split_for_impl();
fetch_ty_generics
};
let derive_args = &fetch_struct_attributes.derive_args;
// `#[derive()]` is valid syntax
let derive_macro_call = quote! { #[derive(#derive_args)] };
let path = bevy_ecs_path();
let impl_fetch = |is_filter: bool,
fetch_associated_type: Ident,
fetch_struct_name: Ident,
item_struct_name: Ident| {
if is_filter {
quote! {
#[doc(hidden)]
#visibility struct #fetch_struct_name #impl_generics #where_clause {
#(#field_idents: <#field_types as #path::query::WorldQuery>::#fetch_associated_type,)*
#(#ignored_field_idents: #ignored_field_types,)*
}
impl #fetch_impl_generics #path::query::Fetch<#fetch_trait_punctuated_lifetimes> for #fetch_struct_name #ty_generics #where_clause {
type Item = bool;
type State = #state_struct_name #ty_generics;
unsafe fn init(_world: &#path::world::World, state: &Self::State, _last_change_tick: u32, _change_tick: u32) -> Self {
#fetch_struct_name {
#(#field_idents: <#field_types as #path::query::WorldQuery>::ReadOnlyFetch::init(_world, &state.#field_idents, _last_change_tick, _change_tick),)*
#(#ignored_field_idents: Default::default(),)*
}
}
const IS_ARCHETYPAL: bool = true #(&& <#field_types as #path::query::WorldQuery>::ReadOnlyFetch::IS_ARCHETYPAL)*;
const IS_DENSE: bool = true #(&& <#field_types as #path::query::WorldQuery>::ReadOnlyFetch::IS_DENSE)*;
#[inline]
unsafe fn set_archetype(&mut self, _state: &Self::State, _archetype: &#path::archetype::Archetype, _tables: &#path::storage::Tables) {
#(self.#field_idents.set_archetype(&_state.#field_idents, _archetype, _tables);)*
}
#[inline]
unsafe fn set_table(&mut self, _state: &Self::State, _table: &#path::storage::Table) {
#(self.#field_idents.set_table(&_state.#field_idents, _table);)*
}
#[inline]
unsafe fn table_fetch(&mut self, _table_row: usize) -> Self::Item {
use #path::query::FilterFetch;
true #(&& self.#field_idents.table_filter_fetch(_table_row))*
}
#[inline]
unsafe fn archetype_fetch(&mut self, _archetype_index: usize) -> Self::Item {
use #path::query::FilterFetch;
true #(&& self.#field_idents.archetype_filter_fetch(_archetype_index))*
}
}
}
let impl_fetch = |is_readonly: bool, fetch_struct_name: Ident, item_struct_name: Ident| {
let fetch_type_alias = if is_readonly {
&read_only_fetch_type_alias
} else {
quote! {
#derive_macro_call
#[automatically_derived]
#visibility struct #item_struct_name #impl_generics #where_clause {
#(#(#field_attrs)* #field_visibilities #field_idents: <<#field_types as #path::query::WorldQuery>::#fetch_associated_type as #path::query::Fetch<#world_lifetime, #world_lifetime>>::Item,)*
#(#(#ignored_field_attrs)* #ignored_field_visibilities #ignored_field_idents: #ignored_field_types,)*
&fetch_type_alias
};
let item_type_alias = if is_readonly {
&read_only_item_type_alias
} else {
&item_type_alias
};
quote! {
#derive_macro_call
#[automatically_derived]
#visibility struct #item_struct_name #user_impl_generics_with_world #user_where_clauses_with_world {
#(#(#field_attrs)* #field_visibilities #field_idents: #path::query::#item_type_alias<'__w, #field_types>,)*
#(#(#ignored_field_attrs)* #ignored_field_visibilities #ignored_field_idents: #ignored_field_types,)*
}
#[doc(hidden)]
#visibility struct #fetch_struct_name #user_impl_generics_with_world #user_where_clauses_with_world {
#(#field_idents: #path::query::#fetch_type_alias::<'__w, #field_types>,)*
#(#ignored_field_idents: #ignored_field_types,)*
}
impl #user_impl_generics_with_world #path::query::Fetch<'__w>
for #fetch_struct_name #user_ty_generics_with_world #user_where_clauses_with_world {
type Item = #item_struct_name #user_ty_generics_with_world;
type State = #state_struct_name #user_ty_generics;
unsafe fn init(_world: &'__w #path::world::World, state: &Self::State, _last_change_tick: u32, _change_tick: u32) -> Self {
Self {
#(#field_idents:
#path::query::#fetch_type_alias::<'__w, #field_types>::init(
_world,
&state.#field_idents,
_last_change_tick,
_change_tick
),
)*
#(#ignored_field_idents: Default::default(),)*
}
}
#[doc(hidden)]
#visibility struct #fetch_struct_name #impl_generics #where_clause {
#(#field_idents: <#field_types as #path::query::WorldQuery>::#fetch_associated_type,)*
#(#ignored_field_idents: #ignored_field_types,)*
const IS_DENSE: bool = true #(&& #path::query::#fetch_type_alias::<'__w, #field_types>::IS_DENSE)*;
const IS_ARCHETYPAL: bool = true #(&& #path::query::#fetch_type_alias::<'__w, #field_types>::IS_ARCHETYPAL)*;
/// SAFETY: we call `set_archetype` for each member that implements `Fetch`
#[inline]
unsafe fn set_archetype(
&mut self,
_state: &Self::State,
_archetype: &'__w #path::archetype::Archetype,
_tables: &'__w #path::storage::Tables
) {
#(self.#field_idents.set_archetype(&_state.#field_idents, _archetype, _tables);)*
}
impl #fetch_impl_generics #path::query::Fetch<#fetch_trait_punctuated_lifetimes> for #fetch_struct_name #fetch_ty_generics #where_clause {
type Item = #item_struct_name #ty_generics;
type State = #state_struct_name #fetch_ty_generics;
/// SAFETY: we call `set_table` for each member that implements `Fetch`
#[inline]
unsafe fn set_table(&mut self, _state: &Self::State, _table: &'__w #path::storage::Table) {
#(self.#field_idents.set_table(&_state.#field_idents, _table);)*
}
unsafe fn init(_world: &#path::world::World, state: &Self::State, _last_change_tick: u32, _change_tick: u32) -> Self {
Self {
#(#field_idents: <#fetch_init_types as #path::query::WorldQuery>::#fetch_associated_type::init(_world, &state.#field_idents, _last_change_tick, _change_tick),)*
#(#ignored_field_idents: Default::default(),)*
}
/// SAFETY: we call `table_fetch` for each member that implements `Fetch`.
#[inline]
unsafe fn table_fetch(&mut self, _table_row: usize) -> Self::Item {
Self::Item {
#(#field_idents: self.#field_idents.table_fetch(_table_row),)*
#(#ignored_field_idents: Default::default(),)*
}
}
const IS_ARCHETYPAL: bool = true #(&& <#field_types as #path::query::WorldQuery>::#fetch_associated_type::IS_ARCHETYPAL)*;
const IS_DENSE: bool = true #(&& <#field_types as #path::query::WorldQuery>::#fetch_associated_type::IS_DENSE)*;
/// SAFETY: we call `set_archetype` for each member that implements `Fetch`
#[inline]
unsafe fn set_archetype(&mut self, _state: &Self::State, _archetype: &#path::archetype::Archetype, _tables: &#path::storage::Tables) {
#(self.#field_idents.set_archetype(&_state.#field_idents, _archetype, _tables);)*
/// SAFETY: we call `archetype_fetch` for each member that implements `Fetch`.
#[inline]
unsafe fn archetype_fetch(&mut self, _archetype_index: usize) -> Self::Item {
Self::Item {
#(#field_idents: self.#field_idents.archetype_fetch(_archetype_index),)*
#(#ignored_field_idents: Default::default(),)*
}
}
/// SAFETY: we call `set_table` for each member that implements `Fetch`
#[inline]
unsafe fn set_table(&mut self, _state: &Self::State, _table: &#path::storage::Table) {
#(self.#field_idents.set_table(&_state.#field_idents, _table);)*
}
#[allow(unused_variables)]
#[inline]
unsafe fn table_filter_fetch(&mut self, _table_row: usize) -> bool {
true #(&& self.#field_idents.table_filter_fetch(_table_row))*
}
/// SAFETY: we call `table_fetch` for each member that implements `Fetch`.
#[inline]
unsafe fn table_fetch(&mut self, _table_row: usize) -> Self::Item {
Self::Item {
#(#field_idents: self.#field_idents.table_fetch(_table_row),)*
#(#ignored_field_idents: Default::default(),)*
}
}
/// SAFETY: we call `archetype_fetch` for each member that implements `Fetch`.
#[inline]
unsafe fn archetype_fetch(&mut self, _archetype_index: usize) -> Self::Item {
Self::Item {
#(#field_idents: self.#field_idents.archetype_fetch(_archetype_index),)*
#(#ignored_field_idents: Default::default(),)*
}
}
#[allow(unused_variables)]
#[inline]
unsafe fn archetype_filter_fetch(&mut self, _archetype_index: usize) -> bool {
true #(&& self.#field_idents.archetype_filter_fetch(_archetype_index))*
}
}
}
};
let fetch_impl = impl_fetch(
fetch_struct_attributes.is_filter,
fetch_associated_type,
fetch_struct_name.clone(),
item_struct_name,
);
let fetch_impl = impl_fetch(false, fetch_struct_name.clone(), item_struct_name.clone());
let state_impl = quote! {
#[doc(hidden)]
#visibility struct #state_struct_name #impl_generics #where_clause {
#visibility struct #state_struct_name #user_impl_generics #user_where_clauses {
#(#field_idents: <#field_types as #path::query::WorldQuery>::State,)*
#(#ignored_field_idents: #ignored_field_types,)*
}
// SAFETY: `update_component_access` and `update_archetype_component_access` are called for each item in the struct
unsafe impl #impl_generics #path::query::FetchState for #state_struct_name #ty_generics #where_clause {
unsafe impl #user_impl_generics #path::query::FetchState for #state_struct_name #user_ty_generics #user_where_clauses {
fn init(world: &mut #path::world::World) -> Self {
#state_struct_name {
#(#field_idents: <<#field_types as #path::query::WorldQuery>::State as #path::query::FetchState>::init(world),)*
@ -384,39 +289,33 @@ pub fn derive_world_query_impl(ast: DeriveInput) -> TokenStream {
}
};
let read_only_impl = if fetch_struct_attributes.is_filter {
quote! {}
} else if fetch_struct_attributes.is_mutable {
let fetch_impl = impl_fetch(
false,
read_only_fetch_associated_type,
let read_only_fetch_impl = if fetch_struct_attributes.is_mutable {
impl_fetch(
true,
read_only_fetch_struct_name.clone(),
read_only_item_struct_name.clone(),
);
read_only_item_struct_name,
)
} else {
quote! {}
};
let read_only_asserts = if fetch_struct_attributes.is_mutable {
quote! {
#fetch_impl
impl #impl_generics #path::query::WorldQuery for #read_only_item_struct_name #ty_generics #where_clause {
type Fetch = #read_only_fetch_struct_name #ty_generics;
type State = #state_struct_name #ty_generics;
type ReadOnlyFetch = #read_only_fetch_struct_name #ty_generics;
}
// Double-check that the data fetched by `ROQueryFetch` is read-only.
// This is technically unnecessary as `<_ as WorldQueryGats<'world>>::ReadOnlyFetch: ReadOnlyFetch`
// but to protect against future mistakes we assert the assoc type implements `ReadOnlyFetch` anyway
#( assert_readonly::<#path::query::ROQueryFetch<'__w, #field_types>>(); )*
}
} else {
quote! {
// Statically checks that the safety guarantee actually holds true. We need this to make
// sure that we don't compile `ReadOnlyFetch` if our struct contains nested `WorldQuery`
// members that don't implement it.
#[allow(dead_code)]
const _: () = {
fn assert_readonly<T: #path::query::ReadOnlyFetch>() {}
// We generate a readonly assertion for every struct member.
fn assert_all #impl_generics () #where_clause {
#(assert_readonly::<<#field_types as #path::query::WorldQuery>::Fetch>();)*
}
};
// Statically checks that the safety guarantee of `ReadOnlyFetch` for `$fetch_struct_name` actually holds true.
// We need this to make sure that we don't compile `ReadOnlyFetch` if our struct contains nested `WorldQuery`
// members that don't implement it. I.e.:
// ```
// #[derive(WorldQuery)]
// pub struct Foo { a: &'static mut MyComponent }
// ```
#( assert_readonly::<#path::query::QueryFetch<'__w, #field_types>>(); )*
}
};
@ -425,23 +324,53 @@ pub fn derive_world_query_impl(ast: DeriveInput) -> TokenStream {
#state_impl
#read_only_impl
#read_only_fetch_impl
impl #impl_generics #path::query::WorldQuery for #struct_name #ty_generics #where_clause {
type Fetch = #fetch_struct_name #ty_generics;
type State = #state_struct_name #ty_generics;
type ReadOnlyFetch = #read_only_fetch_struct_name #ty_generics;
impl #user_impl_generics #path::query::WorldQuery for #struct_name #user_ty_generics #user_where_clauses {
type State = #state_struct_name #user_ty_generics;
fn shrink<'__wlong: '__wshort, '__wshort>(item: #path::query::#item_type_alias<'__wlong, Self>)
-> #path::query::#item_type_alias<'__wshort, Self> {
#item_struct_name {
#(
#field_idents : < #field_types as #path::query::WorldQuery> :: shrink( item.#field_idents ),
)*
#(
#ignored_field_idents: item.#ignored_field_idents,
)*
}
}
}
impl #user_impl_generics_with_world #path::query::WorldQueryGats<'__w> for #struct_name #user_ty_generics #user_where_clauses {
type Fetch = #fetch_struct_name #user_ty_generics_with_world;
type ReadOnlyFetch = #read_only_fetch_struct_name #user_ty_generics_with_world;
type _State = #state_struct_name #user_ty_generics;
}
/// SAFETY: each item in the struct is read only
unsafe impl #impl_generics #path::query::ReadOnlyFetch for #read_only_fetch_struct_name #ty_generics #where_clause {}
unsafe impl #user_impl_generics_with_world #path::query::ReadOnlyFetch
for #read_only_fetch_struct_name #user_ty_generics_with_world #user_where_clauses_with_world {}
#[allow(dead_code)]
const _: () = {
fn assert_readonly<T>()
where
T: #path::query::ReadOnlyFetch,
{
}
// We generate a readonly assertion for every struct member.
fn assert_all #user_impl_generics_with_world () #user_where_clauses_with_world {
#read_only_asserts
}
};
// The original struct will most likely be left unused. As we don't want our users having
// to specify `#[allow(dead_code)]` for their custom queries, we are using this cursed
// workaround.
#[allow(dead_code)]
const _: () = {
fn dead_code_workaround #impl_generics (q: #struct_name #ty_generics) #where_clause {
fn dead_code_workaround #user_impl_generics (q: #struct_name #user_ty_generics) #user_where_clauses {
#(q.#field_idents;)*
#(q.#ignored_field_idents;)*
}
@ -451,21 +380,13 @@ pub fn derive_world_query_impl(ast: DeriveInput) -> TokenStream {
}
struct WorldQueryFieldInfo {
/// The original field type.
field_type: Type,
/// The same as `query_type` but with `'fetch` lifetime.
fetch_init_type: Type,
/// Has `#[fetch(ignore)]` or `#[filter_fetch(ignore)]` attribute.
is_ignored: bool,
/// All field attributes except for `world_query` ones.
attrs: Vec<Attribute>,
}
fn read_world_query_field_info(
field: &Field,
world_lifetime: &Lifetime,
fetch_lifetime: &Lifetime,
) -> WorldQueryFieldInfo {
fn read_world_query_field_info(field: &Field) -> WorldQueryFieldInfo {
let is_ignored = field
.attrs
.iter()
@ -503,91 +424,5 @@ fn read_world_query_field_info(
.cloned()
.collect();
let field_type = field.ty.clone();
let mut fetch_init_type: Type = field_type.clone();
replace_lifetime_for_type(&mut fetch_init_type, world_lifetime, fetch_lifetime);
WorldQueryFieldInfo {
field_type,
fetch_init_type,
is_ignored,
attrs,
}
}
// Fetch's HRTBs require substituting world lifetime with an additional one to make the
// implementation compile. I don't fully understand why this works though. If anyone's curious
// enough to try to find a better work around, I'll leave playground links here:
// - https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=da5e260a5c2f3e774142d60a199e854a (this fails)
// - https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=802517bb3d8f83c45ee8c0be360bb250 (this compiles)
fn replace_lifetime_for_type(ty: &mut Type, world_lifetime: &Lifetime, fetch_lifetime: &Lifetime) {
match ty {
Type::Path(ref mut path) => {
replace_world_lifetime_for_type_path(path, world_lifetime, fetch_lifetime)
}
Type::Reference(ref mut reference) => {
if let Some(lifetime) = reference.lifetime.as_mut() {
replace_lifetime(lifetime, world_lifetime, fetch_lifetime);
}
replace_lifetime_for_type(reference.elem.as_mut(), world_lifetime, fetch_lifetime);
}
Type::Tuple(tuple) => {
for ty in tuple.elems.iter_mut() {
replace_lifetime_for_type(ty, world_lifetime, fetch_lifetime);
}
}
ty => panic!("Unsupported type: {}", ty.to_token_stream()),
}
}
fn replace_world_lifetime_for_type_path(
path: &mut TypePath,
world_lifetime: &Lifetime,
fetch_lifetime: &Lifetime,
) {
if let Some(qself) = path.qself.as_mut() {
replace_lifetime_for_type(qself.ty.as_mut(), world_lifetime, fetch_lifetime);
}
replace_world_lifetime_for_path(&mut path.path, world_lifetime, fetch_lifetime);
}
fn replace_world_lifetime_for_path(
path: &mut Path,
world_lifetime: &Lifetime,
fetch_lifetime: &Lifetime,
) {
for segment in path.segments.iter_mut() {
match segment.arguments {
PathArguments::None => {}
PathArguments::AngleBracketed(ref mut args) => {
for arg in args.args.iter_mut() {
match arg {
GenericArgument::Lifetime(lifetime) => {
replace_lifetime(lifetime, world_lifetime, fetch_lifetime);
}
GenericArgument::Type(ty) => {
replace_lifetime_for_type(ty, world_lifetime, fetch_lifetime)
}
_ => {}
}
}
}
PathArguments::Parenthesized(ref mut args) => {
for input in args.inputs.iter_mut() {
replace_lifetime_for_type(input, world_lifetime, fetch_lifetime);
}
if let ReturnType::Type(_, _) = args.output {
panic!("Function types aren't supported");
}
}
}
}
}
fn replace_lifetime(lifetime: &mut Lifetime, world_lifetime: &Lifetime, fetch_lifetime: &Lifetime) {
if lifetime.ident == world_lifetime.ident {
lifetime.ident = fetch_lifetime.ident.clone();
}
WorldQueryFieldInfo { is_ignored, attrs }
}

14
crates/bevy_ecs/macros/src/lib.rs
View file

@ -124,18 +124,17 @@ pub fn derive_bundle(input: TokenStream) -> TokenStream {
self.#field.get_components(&mut func);
});
field_from_components.push(quote! {
#field: <#field_type as #ecs_path::bundle::Bundle>::from_components(&mut func),
#field: <#field_type as #ecs_path::bundle::Bundle>::from_components(ctx, &mut func),
});
} else {
field_component_ids.push(quote! {
component_ids.push(components.init_component::<#field_type>(storages));
});
field_get_components.push(quote! {
func((&mut self.#field as *mut #field_type).cast::<u8>());
std::mem::forget(self.#field);
#ecs_path::ptr::OwningPtr::make(self.#field, &mut func);
});
field_from_components.push(quote! {
#field: func().cast::<#field_type>().read(),
#field: func(ctx).inner().as_ptr().cast::<#field_type>().read(),
});
}
}
@ -157,14 +156,17 @@ pub fn derive_bundle(input: TokenStream) -> TokenStream {
}
#[allow(unused_variables, unused_mut, non_snake_case)]
unsafe fn from_components(mut func: impl FnMut() -> *mut u8) -> Self {
unsafe fn from_components<T, F>(ctx: &mut T, mut func: F) -> Self
where
F: FnMut(&mut T) -> #ecs_path::ptr::OwningPtr<'_>
{
Self {
#(#field_from_components)*
}
}
#[allow(unused_variables, unused_mut, forget_copy, forget_ref)]
fn get_components(mut self, mut func: impl FnMut(*mut u8)) {
fn get_components(self, mut func: impl FnMut(#ecs_path::ptr::OwningPtr<'_>)) {
#(#field_get_components)*
}
}

20
crates/bevy_ecs/src/bundle.rs
View file

@ -8,6 +8,7 @@ use crate::{
archetype::{AddBundle, Archetype, ArchetypeId, Archetypes, ComponentStatus},
component::{Component, ComponentId, ComponentTicks, Components, StorageType},
entity::{Entities, Entity, EntityLocation},
ptr::OwningPtr,
storage::{SparseSetIndex, SparseSets, Storages, Table},
};
use bevy_ecs_macros::all_tuples;
@ -85,14 +86,15 @@ pub unsafe trait Bundle: Send + Sync + 'static {
/// # Safety
/// Caller must return data for each component in the bundle, in the order of this bundle's
/// [`Component`]s
unsafe fn from_components(func: impl FnMut() -> *mut u8) -> Self
unsafe fn from_components<T, F>(ctx: &mut T, func: F) -> Self
where
F: FnMut(&mut T) -> OwningPtr<'_>,
Self: Sized;
/// Calls `func` on each value, in the order of this bundle's [`Component`]s. This will
/// [`std::mem::forget`] the bundle fields, so callers are responsible for dropping the fields
/// if that is desirable.
fn get_components(self, func: impl FnMut(*mut u8));
fn get_components(self, func: impl FnMut(OwningPtr<'_>));
}
macro_rules! tuple_impl {
@ -105,21 +107,23 @@ macro_rules! tuple_impl {
#[allow(unused_variables, unused_mut)]
#[allow(clippy::unused_unit)]
unsafe fn from_components(mut func: impl FnMut() -> *mut u8) -> Self {
unsafe fn from_components<T, F>(ctx: &mut T, mut func: F) -> Self
where
F: FnMut(&mut T) -> OwningPtr<'_>
{
#[allow(non_snake_case)]
let ($(mut $name,)*) = (
$(func().cast::<$name>(),)*
$(func(ctx).inner().cast::<$name>(),)*
);
($($name.read(),)*)
($($name.as_ptr().read(),)*)
}
#[allow(unused_variables, unused_mut)]
fn get_components(self, mut func: impl FnMut(*mut u8)) {
fn get_components(self, mut func: impl FnMut(OwningPtr<'_>)) {
#[allow(non_snake_case)]
let ($(mut $name,)*) = self;
$(
func((&mut $name as *mut $name).cast::<u8>());
std::mem::forget($name);
OwningPtr::make($name, &mut func);
)*
}
}

27
crates/bevy_ecs/src/component.rs
View file

@ -1,6 +1,7 @@
//! Types for declaring and storing [`Component`]s.
use crate::{
ptr::OwningPtr,
storage::{SparseSetIndex, Storages},
system::Resource,
};
@ -109,7 +110,7 @@ impl ComponentInfo {
}
#[inline]
pub fn drop(&self) -> unsafe fn(*mut u8) {
pub fn drop(&self) -> unsafe fn(OwningPtr<'_>) {
self.descriptor.drop
}
@ -154,7 +155,6 @@ impl SparseSetIndex for ComponentId {
}
}
#[derive(Debug)]
pub struct ComponentDescriptor {
name: String,
// SAFETY: This must remain private. It must match the statically known StorageType of the
@ -165,13 +165,28 @@ pub struct ComponentDescriptor {
is_send_and_sync: bool,
type_id: Option<TypeId>,
layout: Layout,
drop: unsafe fn(*mut u8),
// SAFETY: this function must be safe to call with pointers pointing to items of the type
// this descriptor describes.
drop: for<'a> unsafe fn(OwningPtr<'a>),
}
// We need to ignore the `drop` field in our `Debug` impl
impl std::fmt::Debug for ComponentDescriptor {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("ComponentDescriptor")
.field("name", &self.name)
.field("storage_type", &self.storage_type)
.field("is_send_and_sync", &self.is_send_and_sync)
.field("type_id", &self.type_id)
.field("layout", &self.layout)
.finish()
}
}
impl ComponentDescriptor {
// SAFETY: The pointer points to a valid value of type `T` and it is safe to drop this value.
unsafe fn drop_ptr<T>(x: *mut u8) {
x.cast::<T>().drop_in_place();
unsafe fn drop_ptr<T>(x: OwningPtr<'_>) {
x.inner().cast::<T>().as_ptr().drop_in_place()
}
pub fn new<T: Component>() -> Self {
@ -330,7 +345,7 @@ impl Components {
}
}
#[derive(Clone, Debug)]
#[derive(Copy, Clone, Debug)]
pub struct ComponentTicks {
pub(crate) added: u32,
pub(crate) changed: u32,

12
crates/bevy_ecs/src/lib.rs
View file

@ -6,6 +6,7 @@ pub mod change_detection;
pub mod component;
pub mod entity;
pub mod event;
pub mod ptr;
pub mod query;
#[cfg(feature = "bevy_reflect")]
pub mod reflect;
@ -46,14 +47,12 @@ pub use bevy_ecs_macros::all_tuples;
#[cfg(test)]
mod tests {
use crate as bevy_ecs;
use crate::prelude::Or;
use crate::{
bundle::Bundle,
component::{Component, ComponentId},
entity::Entity,
query::{
Added, ChangeTrackers, Changed, FilterFetch, FilteredAccess, Or, With, Without,
WorldQuery,
},
query::{Added, ChangeTrackers, Changed, FilteredAccess, With, Without, WorldQuery},
world::{Mut, World},
};
use bevy_tasks::TaskPool;
@ -899,10 +898,7 @@ mod tests {
}
}
fn get_filtered<F: WorldQuery>(world: &mut World) -> Vec<Entity>
where
F::Fetch: FilterFetch,
{
fn get_filtered<F: WorldQuery>(world: &mut World) -> Vec<Entity> {
world
.query_filtered::<Entity, F>()
.iter(world)

210
crates/bevy_ecs/src/ptr.rs Normal file
View file

@ -0,0 +1,210 @@
use std::{cell::UnsafeCell, marker::PhantomData, mem::MaybeUninit, ptr::NonNull};
/// Type-erased borrow of some unknown type chosen when constructing this type.
///
/// This type tries to act "borrow-like" which means that:
/// - It should be considered immutable: its target must not be changed while this pointer is alive.
/// - It must always points to a valid value of whatever the pointee type is.
/// - The lifetime `'a` accurately represents how long the pointer is valid for.
///
/// It may be helpful to think of this type as similar to `&'a dyn Any` but without
/// the metadata and able to point to data that does not correspond to a Rust type.
#[derive(Copy, Clone)]
pub struct Ptr<'a>(NonNull<u8>, PhantomData<&'a u8>);
/// Type-erased mutable borrow of some unknown type chosen when constructing this type.
///
/// This type tries to act "borrow-like" which means that:
/// - Pointer is considered exclusive and mutable. It cannot be cloned as this would lead to
/// aliased mutability.
/// - It must always points to a valid value of whatever the pointee type is.
/// - The lifetime `'a` accurately represents how long the pointer is valid for.
///
/// It may be helpful to think of this type as similar to `&'a mut dyn Any` but without
/// the metadata and able to point to data that does not correspond to a Rust type.
pub struct PtrMut<'a>(NonNull<u8>, PhantomData<&'a mut u8>);
/// Type-erased Box-like pointer to some unknown type chosen when constructing this type.
/// Conceptually represents ownership of whatever data is being pointed to and so is
/// responsible for calling its `Drop` impl. This pointer is _not_ responsible for freeing
/// the memory pointed to by this pointer as it may be pointing to an element in a `Vec` or
/// to a local in a function etc.
///
/// This type tries to act "borrow-like" like which means that:
/// - Pointer should be considered exclusive and mutable. It cannot be cloned as this would lead
/// to aliased mutability and potentially use after free bugs.
/// - It must always points to a valid value of whatever the pointee type is.
/// - The lifetime `'a` accurately represents how long the pointer is valid for.
///
/// It may be helpful to think of this type as similar to `&'a mut ManuallyDrop<dyn Any>` but
/// without the metadata and able to point to data that does not correspond to a Rust type.
pub struct OwningPtr<'a>(NonNull<u8>, PhantomData<&'a mut u8>);
macro_rules! impl_ptr {
($ptr:ident) => {
impl $ptr<'_> {
/// # Safety
/// the offset cannot make the existing ptr null, or take it out of bounds for its allocation.
#[inline]
pub unsafe fn offset(self, count: isize) -> Self {
Self(
NonNull::new_unchecked(self.0.as_ptr().offset(count)),
PhantomData,
)
}
/// # Safety
/// the offset cannot make the existing ptr null, or take it out of bounds for its allocation.
#[inline]
pub unsafe fn add(self, count: usize) -> Self {
Self(
NonNull::new_unchecked(self.0.as_ptr().add(count)),
PhantomData,
)
}
/// # Safety
///
/// The lifetime for the returned item must not exceed the lifetime `inner` is valid for
#[inline]
pub unsafe fn new(inner: NonNull<u8>) -> Self {
Self(inner, PhantomData)
}
#[inline]
pub fn inner(&self) -> NonNull<u8> {
self.0
}
}
};
}
impl_ptr!(Ptr);
impl<'a> Ptr<'a> {
/// # Safety
///
/// Another [`PtrMut`] for the same [`Ptr`] must not be created until the first is dropped.
#[inline]
pub unsafe fn assert_unique(self) -> PtrMut<'a> {
PtrMut(self.0, PhantomData)
}
/// # Safety
/// Must point to a valid `T`
#[inline]
pub unsafe fn deref<T>(self) -> &'a T {
&*self.0.as_ptr().cast()
}
}
impl_ptr!(PtrMut);
impl<'a> PtrMut<'a> {
/// Transforms this [`PtrMut`] into an [`OwningPtr`]
///
/// # Safety
/// Must have right to drop or move out of [`PtrMut`].
#[inline]
pub unsafe fn promote(self) -> OwningPtr<'a> {
OwningPtr(self.0, PhantomData)
}
/// Transforms this [`PtrMut<T>`] into a `&mut T` with the same lifetime
///
/// # Safety
/// Must point to a valid `T`
#[inline]
pub unsafe fn deref_mut<T>(self) -> &'a mut T {
&mut *self.inner().as_ptr().cast()
}
}
impl_ptr!(OwningPtr);
impl<'a> OwningPtr<'a> {
/// Consumes a value and creates an [`OwningPtr`] to it while ensuring a double drop does not happen.
#[inline]
pub fn make<T, F: FnOnce(OwningPtr<'_>) -> R, R>(val: T, f: F) -> R {
let mut temp = MaybeUninit::new(val);
let ptr = unsafe { NonNull::new_unchecked(temp.as_mut_ptr().cast::<u8>()) };
f(Self(ptr, PhantomData))
}
//// Consumes the [`OwningPtr`] to obtain ownership of the underlying data of type `T`.
///
/// # Safety
/// Must point to a valid `T`.
#[inline]
pub unsafe fn read<T>(self) -> T {
self.inner().as_ptr().cast::<T>().read()
}
}
/// Conceptually equilavent to `&'a [T]` but with length information cut out for performance reasons
pub struct ThinSlicePtr<'a, T> {
ptr: NonNull<T>,
#[cfg(debug_assertions)]
len: usize,
_marker: PhantomData<&'a [T]>,
}
impl<'a, T> ThinSlicePtr<'a, T> {
#[inline]
/// Indexes the slice without doing bounds checks
///
/// # Safety
/// `index` must be inbounds.
pub unsafe fn get(self, index: usize) -> &'a T {
#[cfg(debug_assertions)]
debug_assert!(index < self.len);
&*self.ptr.as_ptr().add(index)
}
}
impl<'a, T> Clone for ThinSlicePtr<'a, T> {
fn clone(&self) -> Self {
Self {
ptr: self.ptr,
#[cfg(debug_assertions)]
len: self.len,
_marker: PhantomData,
}
}
}
impl<'a, T> Copy for ThinSlicePtr<'a, T> {}
impl<'a, T> From<&'a [T]> for ThinSlicePtr<'a, T> {
#[inline]
fn from(slice: &'a [T]) -> Self {
Self {
ptr: unsafe { NonNull::new_unchecked(slice.as_ptr() as *mut T) },
#[cfg(debug_assertions)]
len: slice.len(),
_marker: PhantomData,
}
}
}
pub(crate) trait UnsafeCellDeref<'a, T> {
unsafe fn deref_mut(self) -> &'a mut T;
unsafe fn deref(self) -> &'a T;
unsafe fn read(self) -> T
where
T: Copy;
}
impl<'a, T> UnsafeCellDeref<'a, T> for &'a UnsafeCell<T> {
#[inline]
unsafe fn deref_mut(self) -> &'a mut T {
&mut *self.get()
}
#[inline]
unsafe fn deref(self) -> &'a T {
&*self.get()
}
#[inline]
unsafe fn read(self) -> T
where
T: Copy,
{
self.get().read()
}
}

703
crates/bevy_ecs/src/query/fetch.rs
View file

File diff suppressed because it is too large Load diff

274
crates/bevy_ecs/src/query/filter.rs
View file

@ -2,46 +2,18 @@ use crate::{
archetype::{Archetype, ArchetypeComponentId},
component::{Component, ComponentId, ComponentStorage, ComponentTicks, StorageType},
entity::Entity,
query::{Access, Fetch, FetchState, FilteredAccess, ReadOnlyFetch, WorldQuery},
ptr::{ThinSlicePtr, UnsafeCellDeref},
query::{
debug_checked_unreachable, Access, Fetch, FetchState, FilteredAccess, QueryFetch,
ROQueryFetch, WorldQuery, WorldQueryGats,
},
storage::{ComponentSparseSet, Table, Tables},
world::World,
};
use bevy_ecs_macros::all_tuples;
use std::{cell::UnsafeCell, marker::PhantomData, ptr};
use std::{cell::UnsafeCell, marker::PhantomData};
/// Extension trait for [`Fetch`] containing methods used by query filters.
/// This trait exists to allow "short circuit" behaviors for relevant query filter fetches.
///
/// This trait is automatically implemented for every type that implements [`Fetch`] trait and
/// specifies `bool` as the associated type for [`Fetch::Item`].
pub trait FilterFetch: for<'w, 's> Fetch<'w, 's> {
/// # Safety
///
/// Must always be called _after_ [`Fetch::set_archetype`]. `archetype_index` must be in the range
/// of the current archetype.
unsafe fn archetype_filter_fetch(&mut self, archetype_index: usize) -> bool;
/// # Safety
///
/// Must always be called _after_ [`Fetch::set_table`]. `table_row` must be in the range of the
/// current table.
unsafe fn table_filter_fetch(&mut self, table_row: usize) -> bool;
}
impl<T> FilterFetch for T
where
T: for<'w, 's> Fetch<'w, 's, Item = bool>,
{
#[inline]
unsafe fn archetype_filter_fetch(&mut self, archetype_index: usize) -> bool {
self.archetype_fetch(archetype_index)
}
#[inline]
unsafe fn table_filter_fetch(&mut self, table_row: usize) -> bool {
self.table_fetch(table_row)
}
}
use super::ReadOnlyFetch;
/// Filter that selects entities with a component `T`.
///
@ -73,9 +45,13 @@ where
pub struct With<T>(PhantomData<T>);
impl<T: Component> WorldQuery for With<T> {
type Fetch = WithFetch<T>;
type State = WithState<T>;
type ReadOnlyFetch = WithFetch<T>;
fn shrink<'wlong: 'wshort, 'wshort>(
item: super::QueryItem<'wlong, Self>,
) -> super::QueryItem<'wshort, Self> {
item
}
}
/// The [`Fetch`] of [`With`].
@ -123,13 +99,19 @@ unsafe impl<T: Component> FetchState for WithState<T> {
}
}
impl<'w, 's, T: Component> Fetch<'w, 's> for WithFetch<T> {
type Item = bool;
impl<T: Component> WorldQueryGats<'_> for With<T> {
type Fetch = WithFetch<T>;
type ReadOnlyFetch = WithFetch<T>;
type _State = WithState<T>;
}
impl<'w, T: Component> Fetch<'w> for WithFetch<T> {
type Item = ();
type State = WithState<T>;
unsafe fn init(
_world: &World,
_state: &Self::State,
_state: &WithState<T>,
_last_change_tick: u32,
_change_tick: u32,
) -> Self {
@ -160,18 +142,14 @@ impl<'w, 's, T: Component> Fetch<'w, 's> for WithFetch<T> {
}
#[inline]
unsafe fn archetype_fetch(&mut self, _archetype_index: usize) -> Self::Item {
true
}
unsafe fn archetype_fetch(&mut self, _archetype_index: usize) {}
#[inline]
unsafe fn table_fetch(&mut self, _table_row: usize) -> bool {
true
}
unsafe fn table_fetch(&mut self, _table_row: usize) {}
}
// SAFETY: no component access or archetype component access
unsafe impl<T> ReadOnlyFetch for WithFetch<T> {}
unsafe impl<T: Component> ReadOnlyFetch for WithFetch<T> {}
impl<T> Clone for WithFetch<T> {
fn clone(&self) -> Self {
@ -210,9 +188,13 @@ impl<T> Copy for WithFetch<T> {}
pub struct Without<T>(PhantomData<T>);
impl<T: Component> WorldQuery for Without<T> {
type Fetch = WithoutFetch<T>;
type State = WithoutState<T>;
type ReadOnlyFetch = WithoutFetch<T>;
fn shrink<'wlong: 'wshort, 'wshort>(
item: super::QueryItem<'wlong, Self>,
) -> super::QueryItem<'wshort, Self> {
item
}
}
/// The [`Fetch`] of [`Without`].
@ -260,17 +242,23 @@ unsafe impl<T: Component> FetchState for WithoutState<T> {
}
}
impl<'w, 's, T: Component> Fetch<'w, 's> for WithoutFetch<T> {
type Item = bool;
impl<T: Component> WorldQueryGats<'_> for Without<T> {
type Fetch = WithoutFetch<T>;
type ReadOnlyFetch = WithoutFetch<T>;
type _State = WithoutState<T>;
}
impl<'w, T: Component> Fetch<'w> for WithoutFetch<T> {
type Item = ();
type State = WithoutState<T>;
unsafe fn init(
_world: &World,
_state: &Self::State,
_state: &WithoutState<T>,
_last_change_tick: u32,
_change_tick: u32,
) -> Self {
Self {
WithoutFetch {
marker: PhantomData,
}
}
@ -297,18 +285,14 @@ impl<'w, 's, T: Component> Fetch<'w, 's> for WithoutFetch<T> {
}
#[inline]
unsafe fn archetype_fetch(&mut self, _archetype_index: usize) -> bool {
true
}
unsafe fn archetype_fetch(&mut self, _archetype_index: usize) {}
#[inline]
unsafe fn table_fetch(&mut self, _table_row: usize) -> bool {
true
}
unsafe fn table_fetch(&mut self, _table_row: usize) {}
}
// SAFETY: no component access or archetype component access
unsafe impl<T> ReadOnlyFetch for WithoutFetch<T> {}
unsafe impl<T: Component> ReadOnlyFetch for WithoutFetch<T> {}
impl<T> Clone for WithoutFetch<T> {
fn clone(&self) -> Self {
@ -356,60 +340,53 @@ pub struct Or<T>(pub T);
/// The [`Fetch`] of [`Or`].
#[derive(Clone, Copy)]
#[doc(hidden)]
pub struct OrFetch<T: FilterFetch> {
pub struct OrFetch<'w, T: Fetch<'w>> {
fetch: T,
matches: bool,
_marker: PhantomData<&'w ()>,
}
macro_rules! impl_query_filter_tuple {
($(($filter: ident, $state: ident)),*) => {
#[allow(unused_variables)]
#[allow(non_snake_case)]
impl<'a, $($filter: FilterFetch),*> FilterFetch for ($($filter,)*) {
#[inline]
unsafe fn table_filter_fetch(&mut self, table_row: usize) -> bool {
let ($($filter,)*) = self;
true $(&& $filter.table_filter_fetch(table_row))*
}
#[inline]
unsafe fn archetype_filter_fetch(&mut self, archetype_index: usize) -> bool {
let ($($filter,)*) = self;
true $(&& $filter.archetype_filter_fetch(archetype_index))*
}
}
impl<$($filter: WorldQuery),*> WorldQuery for Or<($($filter,)*)>
where $($filter::Fetch: FilterFetch, $filter::ReadOnlyFetch: FilterFetch),*
{
type Fetch = Or<($(OrFetch<$filter::Fetch>,)*)>;
impl<$($filter: WorldQuery),*> WorldQuery for Or<($($filter,)*)> {
type State = Or<($($filter::State,)*)>;
type ReadOnlyFetch = Or<($(OrFetch<$filter::ReadOnlyFetch>,)*)>;
}
/// SAFETY: this only works using the filter which doesn't write
unsafe impl<$($filter: FilterFetch + ReadOnlyFetch),*> ReadOnlyFetch for Or<($(OrFetch<$filter>,)*)> {}
fn shrink<'wlong: 'wshort, 'wshort>(item: super::QueryItem<'wlong, Self>) -> super::QueryItem<'wshort, Self> {
item
}
}
#[allow(unused_variables)]
#[allow(non_snake_case)]
impl<'w, 's, $($filter: FilterFetch),*> Fetch<'w, 's> for Or<($(OrFetch<$filter>,)*)> {
type State = Or<($(<$filter as Fetch<'w, 's>>::State,)*)>;
type Item = bool;
impl<'w, $($filter: WorldQueryGats<'w>),*> WorldQueryGats<'w> for Or<($($filter,)*)> {
type Fetch = Or<($(OrFetch<'w, QueryFetch<'w, $filter>>,)*)>;
type ReadOnlyFetch = Or<($(OrFetch<'w, ROQueryFetch<'w, $filter>>,)*)>;
type _State = Or<($($filter::_State,)*)>;
}
unsafe fn init(world: &World, state: &Self::State, last_change_tick: u32, change_tick: u32) -> Self {
let ($($filter,)*) = &state.0;
Or(($(OrFetch {
fetch: $filter::init(world, $filter, last_change_tick, change_tick),
matches: false,
},)*))
}
#[allow(unused_variables)]
#[allow(non_snake_case)]
impl<'w, $($filter: Fetch<'w>),*> Fetch<'w> for Or<($(OrFetch<'w, $filter>,)*)> {
type State = Or<($(<$filter as Fetch<'w>>::State,)*)>;
type Item = bool;
const IS_DENSE: bool = true $(&& $filter::IS_DENSE)*;
const IS_ARCHETYPAL: bool = true $(&& $filter::IS_ARCHETYPAL)*;
unsafe fn init(world: &'w World, state: & Or<($(<$filter as Fetch<'w>>::State,)*)>, last_change_tick: u32, change_tick: u32) -> Self {
let ($($filter,)*) = &state.0;
Or(($(OrFetch {
fetch: <$filter as Fetch<'w>>::init(world, $filter, last_change_tick, change_tick),
matches: false,
_marker: PhantomData,
},)*))
}
#[inline]
unsafe fn set_table(&mut self, state: &Self::State, table: &Table) {
unsafe fn set_table(&mut self, state: &Self::State, table: &'w Table) {
let ($($filter,)*) = &mut self.0;
let ($($state,)*) = &state.0;
$(
@ -421,7 +398,7 @@ macro_rules! impl_query_filter_tuple {
}
#[inline]
unsafe fn set_archetype(&mut self, state: &Self::State, archetype: &Archetype, tables: &Tables) {
unsafe fn set_archetype(&mut self, state: & Self::State, archetype: &'w Archetype, tables: &'w Tables) {
let ($($filter,)*) = &mut self.0;
let ($($state,)*) = &state.0;
$(
@ -443,6 +420,16 @@ macro_rules! impl_query_filter_tuple {
let ($($filter,)*) = &mut self.0;
false $(|| ($filter.matches && $filter.fetch.archetype_filter_fetch(archetype_index)))*
}
#[inline]
unsafe fn table_filter_fetch(&mut self, table_row: usize) -> bool {
self.table_fetch(table_row)
}
#[inline]
unsafe fn archetype_filter_fetch(&mut self, archetype_index: usize) -> bool {
self.archetype_fetch(archetype_index)
}
}
// SAFETY: update_component_access and update_archetype_component_access are called for each item in the tuple
@ -473,6 +460,9 @@ macro_rules! impl_query_filter_tuple {
false $(|| $filter.matches_table(table))*
}
}
// SAFE: filters are read only
unsafe impl<'w, $($filter: Fetch<'w> + ReadOnlyFetch),*> ReadOnlyFetch for Or<($(OrFetch<'w, $filter>,)*)> {}
};
}
@ -493,12 +483,12 @@ macro_rules! impl_tick_filter {
#[doc(hidden)]
$(#[$fetch_meta])*
pub struct $fetch_name<T> {
table_ticks: *const UnsafeCell<ComponentTicks>,
entity_table_rows: *const usize,
pub struct $fetch_name<'w, T> {
table_ticks: Option<ThinSlicePtr<'w, UnsafeCell<ComponentTicks>>>,
entity_table_rows: Option<ThinSlicePtr<'w, usize>>,
marker: PhantomData<T>,
entities: *const Entity,
sparse_set: *const ComponentSparseSet,
entities: Option<ThinSlicePtr<'w, Entity>>,
sparse_set: Option<&'w ComponentSparseSet>,
last_change_tick: u32,
change_tick: u32,
}
@ -511,9 +501,11 @@ macro_rules! impl_tick_filter {
}
impl<T: Component> WorldQuery for $name<T> {
type Fetch = $fetch_name<T>;
type State = $state_name<T>;
type ReadOnlyFetch = $fetch_name<T>;
fn shrink<'wlong: 'wshort, 'wshort>(item: super::QueryItem<'wlong, Self>) -> super::QueryItem<'wshort, Self> {
item
}
}
// SAFETY: this reads the T component. archetype component access and component access are updated to reflect that
@ -554,27 +546,27 @@ macro_rules! impl_tick_filter {
}
}
impl<'w, 's, T: Component> Fetch<'w, 's> for $fetch_name<T> {
impl<'w, T: Component> WorldQueryGats<'w> for $name<T> {
type Fetch = $fetch_name<'w, T>;
type ReadOnlyFetch = $fetch_name<'w, T>;
type _State = $state_name<T>;
}
impl<'w, T: Component> Fetch<'w> for $fetch_name<'w, T> {
type State = $state_name<T>;
type Item = bool;
unsafe fn init(world: &World, state: &Self::State, last_change_tick: u32, change_tick: u32) -> Self {
let mut value = Self {
table_ticks: ptr::null::<UnsafeCell<ComponentTicks>>(),
entities: ptr::null::<Entity>(),
entity_table_rows: ptr::null::<usize>(),
sparse_set: ptr::null::<ComponentSparseSet>(),
unsafe fn init(world: &'w World, state: & $state_name<T>, last_change_tick: u32, change_tick: u32) -> Self {
Self {
table_ticks: None,
entities: None,
entity_table_rows: None,
sparse_set: (T::Storage::STORAGE_TYPE == StorageType::SparseSet)
.then(|| world.storages().sparse_sets.get(state.component_id).unwrap()),
marker: PhantomData,
last_change_tick,
change_tick,
};
if T::Storage::STORAGE_TYPE == StorageType::SparseSet {
value.sparse_set = world
.storages()
.sparse_sets
.get(state.component_id).unwrap();
}
value
}
const IS_DENSE: bool = {
@ -584,50 +576,62 @@ macro_rules! impl_tick_filter {
}
};
const IS_ARCHETYPAL: bool = false;
const IS_ARCHETYPAL: bool = false;
unsafe fn set_table(&mut self, state: &Self::State, table: &Table) {
self.table_ticks = table
.get_column(state.component_id).unwrap()
.get_ticks_ptr();
unsafe fn set_table(&mut self, state: &Self::State, table: &'w Table) {
self.table_ticks = Some(table.get_column(state.component_id).unwrap().get_ticks_slice().into());
}
unsafe fn set_archetype(&mut self, state: &Self::State, archetype: &Archetype, tables: &Tables) {
unsafe fn set_archetype(&mut self, state: &Self::State, archetype: &'w Archetype, tables: &'w Tables) {
match T::Storage::STORAGE_TYPE {
StorageType::Table => {
self.entity_table_rows = archetype.entity_table_rows().as_ptr();
self.entity_table_rows = Some(archetype.entity_table_rows().into());
let table = &tables[archetype.table_id()];
self.table_ticks = table
.get_column(state.component_id).unwrap()
.get_ticks_ptr();
self.table_ticks = Some(table.get_column(state.component_id).unwrap().get_ticks_slice().into());
}
StorageType::SparseSet => self.entities = archetype.entities().as_ptr(),
StorageType::SparseSet => self.entities = Some(archetype.entities().into()),
}
}
unsafe fn table_fetch(&mut self, table_row: usize) -> bool {
$is_detected(&*(&*self.table_ticks.add(table_row)).get(), self.last_change_tick, self.change_tick)
$is_detected(&*(self.table_ticks.unwrap_or_else(|| debug_checked_unreachable()).get(table_row)).deref(), self.last_change_tick, self.change_tick)
}
unsafe fn archetype_fetch(&mut self, archetype_index: usize) -> bool {
match T::Storage::STORAGE_TYPE {
StorageType::Table => {
let table_row = *self.entity_table_rows.add(archetype_index);
$is_detected(&*(&*self.table_ticks.add(table_row)).get(), self.last_change_tick, self.change_tick)
let table_row = *self.entity_table_rows.unwrap_or_else(|| debug_checked_unreachable()).get(archetype_index);
$is_detected(&*(self.table_ticks.unwrap_or_else(|| debug_checked_unreachable()).get(table_row)).deref(), self.last_change_tick, self.change_tick)
}
StorageType::SparseSet => {
let entity = *self.entities.add(archetype_index);
let ticks = (&*self.sparse_set).get_ticks(entity).cloned().unwrap();
let entity = *self.entities.unwrap_or_else(|| debug_checked_unreachable()).get(archetype_index);
let ticks = self
.sparse_set
.unwrap_or_else(|| debug_checked_unreachable())
.get_ticks(entity)
.map(|ticks| &*ticks.get())
.cloned()
.unwrap();
$is_detected(&ticks, self.last_change_tick, self.change_tick)
}
}
}
#[inline]
unsafe fn table_filter_fetch(&mut self, table_row: usize) -> bool {
self.table_fetch(table_row)
}
#[inline]
unsafe fn archetype_filter_fetch(&mut self, archetype_index: usize) -> bool {
self.archetype_fetch(archetype_index)
}
}
/// SAFETY: read-only access
unsafe impl<T: Component> ReadOnlyFetch for $fetch_name<T> {}
unsafe impl<'w, T: Component> ReadOnlyFetch for $fetch_name<'w, T> {}
impl<T> Clone for $fetch_name<T> {
impl<T> Clone for $fetch_name<'_, T> {
fn clone(&self) -> Self {
Self {
table_ticks: self.table_ticks.clone(),
@ -641,7 +645,7 @@ macro_rules! impl_tick_filter {
}
}
impl<T> Copy for $fetch_name<T> {}
impl<T> Copy for $fetch_name<'_, T> {}
};
}

151
crates/bevy_ecs/src/query/iter.rs
View file

@ -1,19 +1,18 @@
use crate::{
archetype::{ArchetypeId, Archetypes},
query::{Fetch, FilterFetch, QueryState, ReadOnlyFetch, WorldQuery},
query::{Fetch, QueryState, WorldQuery},
storage::{TableId, Tables},
world::World,
};
use std::{marker::PhantomData, mem::MaybeUninit};
use super::{QueryFetch, QueryItem, ReadOnlyFetch};
/// An [`Iterator`] over query results of a [`Query`](crate::system::Query).
///
/// This struct is created by the [`Query::iter`](crate::system::Query::iter) and
/// [`Query::iter_mut`](crate::system::Query::iter_mut) methods.
pub struct QueryIter<'w, 's, Q: WorldQuery, QF: Fetch<'w, 's, State = Q::State>, F: WorldQuery>
where
F::Fetch: FilterFetch,
{
pub struct QueryIter<'w, 's, Q: WorldQuery, QF: Fetch<'w, State = Q::State>, F: WorldQuery> {
tables: &'w Tables,
archetypes: &'w Archetypes,
query_state: &'s QueryState<Q, F>,
@ -21,15 +20,14 @@ where
table_id_iter: std::slice::Iter<'s, TableId>,
archetype_id_iter: std::slice::Iter<'s, ArchetypeId>,
fetch: QF,
filter: F::Fetch,
filter: QueryFetch<'w, F>,
current_len: usize,
current_index: usize,
}
impl<'w, 's, Q: WorldQuery, QF, F: WorldQuery> QueryIter<'w, 's, Q, QF, F>
where
F::Fetch: FilterFetch,
QF: Fetch<'w, 's, State = Q::State>,
QF: Fetch<'w, State = Q::State>,
{
/// # Safety
/// This does not check for mutable query correctness. To be safe, make sure mutable queries
@ -48,7 +46,7 @@ where
last_change_tick,
change_tick,
);
let filter = <F::Fetch as Fetch>::init(
let filter = QueryFetch::<F>::init(
world,
&query_state.filter_state,
last_change_tick,
@ -72,8 +70,7 @@ where
impl<'w, 's, Q: WorldQuery, QF, F: WorldQuery> Iterator for QueryIter<'w, 's, Q, QF, F>
where
F::Fetch: FilterFetch,
QF: Fetch<'w, 's, State = Q::State>,
QF: Fetch<'w, State = Q::State>,
{
type Item = QF::Item;
@ -83,7 +80,7 @@ where
#[inline(always)]
fn next(&mut self) -> Option<Self::Item> {
unsafe {
if QF::IS_DENSE && F::Fetch::IS_DENSE {
if QF::IS_DENSE && <QueryFetch<'static, F>>::IS_DENSE {
loop {
if self.current_index == self.current_len {
let table_id = self.table_id_iter.next()?;
@ -152,24 +149,15 @@ where
}
}
pub struct QueryCombinationIter<'w, 's, Q: WorldQuery, QF, F: WorldQuery, const K: usize>
where
QF: Fetch<'w, 's, State = Q::State>,
F::Fetch: FilterFetch,
{
pub struct QueryCombinationIter<'w, 's, Q: WorldQuery, F: WorldQuery, const K: usize> {
tables: &'w Tables,
archetypes: &'w Archetypes,
query_state: &'s QueryState<Q, F>,
world: &'w World,
cursors: [QueryIterationCursor<'w, 's, Q, QF, F>; K],
cursors: [QueryIterationCursor<'w, 's, Q, QueryFetch<'w, Q>, F>; K],
}
impl<'w, 's, Q: WorldQuery, QF, F: WorldQuery, const K: usize>
QueryCombinationIter<'w, 's, Q, QF, F, K>
where
QF: Fetch<'w, 's, State = Q::State>,
F::Fetch: FilterFetch,
{
impl<'w, 's, Q: WorldQuery, F: WorldQuery, const K: usize> QueryCombinationIter<'w, 's, Q, F, K> {
/// # Safety
/// This does not check for mutable query correctness. To be safe, make sure mutable queries
/// have unique access to the components they query.
@ -184,36 +172,34 @@ where
// Initialize array with cursors.
// There is no FromIterator on arrays, so instead initialize it manually with MaybeUninit
// TODO: use MaybeUninit::uninit_array if it stabilizes
let mut cursors: [MaybeUninit<QueryIterationCursor<'w, 's, Q, QF, F>>; K] =
MaybeUninit::uninit().assume_init();
for (i, cursor) in cursors.iter_mut().enumerate() {
match i {
0 => cursor.as_mut_ptr().write(QueryIterationCursor::init(
world,
query_state,
last_change_tick,
change_tick,
)),
_ => cursor.as_mut_ptr().write(QueryIterationCursor::init_empty(
world,
query_state,
last_change_tick,
change_tick,
)),
}
let mut array: MaybeUninit<[QueryIterationCursor<'w, 's, Q, QueryFetch<'w, Q>, F>; K]> =
MaybeUninit::uninit();
let ptr = array
.as_mut_ptr()
.cast::<QueryIterationCursor<'w, 's, Q, QueryFetch<'w, Q>, F>>();
if K != 0 {
ptr.write(QueryIterationCursor::init(
world,
query_state,
last_change_tick,
change_tick,
));
}
for slot in (1..K).map(|offset| ptr.add(offset)) {
slot.write(QueryIterationCursor::init_empty(
world,
query_state,
last_change_tick,
change_tick,
));
}
// TODO: use MaybeUninit::array_assume_init if it stabilizes
let cursors: [QueryIterationCursor<'w, 's, Q, QF, F>; K] =
(&cursors as *const _ as *const [QueryIterationCursor<'w, 's, Q, QF, F>; K]).read();
QueryCombinationIter {
world,
query_state,
tables: &world.storages().tables,
archetypes: &world.archetypes,
cursors,
cursors: array.assume_init(),
}
}
@ -223,10 +209,10 @@ where
/// references to the same component, leading to unique reference aliasing.
///.
/// It is always safe for shared access.
unsafe fn fetch_next_aliased_unchecked(&mut self) -> Option<[QF::Item; K]>
unsafe fn fetch_next_aliased_unchecked(&mut self) -> Option<[QueryItem<'w, Q>; K]>
where
QF: Clone,
F::Fetch: Clone,
QueryFetch<'w, Q>: Clone,
QueryFetch<'w, F>: Clone,
{
if K == 0 {
return None;
@ -252,43 +238,43 @@ where
}
}
// TODO: use MaybeUninit::uninit_array if it stabilizes
let mut values: [MaybeUninit<QF::Item>; K] = MaybeUninit::uninit().assume_init();
let mut values = MaybeUninit::<[QueryItem<'w, Q>; K]>::uninit();
for (value, cursor) in values.iter_mut().zip(&mut self.cursors) {
value.as_mut_ptr().write(cursor.peek_last().unwrap());
let ptr = values.as_mut_ptr().cast::<QueryItem<'w, Q>>();
for (offset, cursor) in self.cursors.iter_mut().enumerate() {
ptr.add(offset).write(cursor.peek_last().unwrap())
}
// TODO: use MaybeUninit::array_assume_init if it stabilizes
let values: [QF::Item; K] = (&values as *const _ as *const [QF::Item; K]).read();
Some(values)
Some(values.assume_init())
}
/// Get next combination of queried components
#[inline]
pub fn fetch_next(&mut self) -> Option<[QF::Item; K]>
pub fn fetch_next(&mut self) -> Option<[QueryItem<'_, Q>; K]>
where
QF: Clone,
F::Fetch: Clone,
QueryFetch<'w, Q>: Clone,
QueryFetch<'w, F>: Clone,
{
// safety: we are limiting the returned reference to self,
// making sure this method cannot be called multiple times without getting rid
// of any previously returned unique references first, thus preventing aliasing.
unsafe { self.fetch_next_aliased_unchecked() }
unsafe {
self.fetch_next_aliased_unchecked()
.map(|array| array.map(Q::shrink))
}
}
}
// Iterator type is intentionally implemented only for read-only access.
// Doing so for mutable references would be unsound, because calling `next`
// multiple times would allow multiple owned references to the same data to exist.
impl<'w, 's, Q: WorldQuery, QF, F: WorldQuery, const K: usize> Iterator
for QueryCombinationIter<'w, 's, Q, QF, F, K>
impl<'w, 's, Q: WorldQuery, F: WorldQuery, const K: usize> Iterator
for QueryCombinationIter<'w, 's, Q, F, K>
where
QF: Fetch<'w, 's, State = Q::State> + Clone + ReadOnlyFetch,
F::Fetch: Clone + FilterFetch + ReadOnlyFetch,
QueryFetch<'w, Q>: Clone + ReadOnlyFetch,
QueryFetch<'w, F>: Clone + ReadOnlyFetch,
{
type Item = [QF::Item; K];
type Item = [QueryItem<'w, Q>; K];
#[inline]
fn next(&mut self) -> Option<Self::Item> {
@ -320,7 +306,7 @@ where
n / k_factorial
});
let archetype_query = F::Fetch::IS_ARCHETYPAL && QF::IS_ARCHETYPAL;
let archetype_query = F::Fetch::IS_ARCHETYPAL && Q::Fetch::IS_ARCHETYPAL;
let min_combinations = if archetype_query { max_size } else { 0 };
(min_combinations, max_combinations)
}
@ -334,7 +320,7 @@ where
// This would need to be added to all types that implement Filter with Filter::IS_ARCHETYPAL = true
impl<'w, 's, Q: WorldQuery, QF> ExactSizeIterator for QueryIter<'w, 's, Q, QF, ()>
where
QF: Fetch<'w, 's, State = Q::State>,
QF: Fetch<'w, State = Q::State>,
{
fn len(&self) -> usize {
self.query_state
@ -345,17 +331,11 @@ where
}
}
struct QueryIterationCursor<
'w,
's,
Q: WorldQuery,
QF: Fetch<'w, 's, State = Q::State>,
F: WorldQuery,
> {
struct QueryIterationCursor<'w, 's, Q: WorldQuery, QF: Fetch<'w, State = Q::State>, F: WorldQuery> {
table_id_iter: std::slice::Iter<'s, TableId>,
archetype_id_iter: std::slice::Iter<'s, ArchetypeId>,
fetch: QF,
filter: F::Fetch,
filter: QueryFetch<'w, F>,
current_len: usize,
current_index: usize,
phantom: PhantomData<&'w Q>,
@ -363,8 +343,8 @@ struct QueryIterationCursor<
impl<'w, 's, Q: WorldQuery, QF, F: WorldQuery> Clone for QueryIterationCursor<'w, 's, Q, QF, F>
where
QF: Fetch<'w, 's, State = Q::State> + Clone,
F::Fetch: Clone,
QF: Fetch<'w, State = Q::State> + Clone,
QueryFetch<'w, F>: Clone,
{
fn clone(&self) -> Self {
Self {
@ -381,11 +361,10 @@ where
impl<'w, 's, Q: WorldQuery, QF, F: WorldQuery> QueryIterationCursor<'w, 's, Q, QF, F>
where
QF: Fetch<'w, 's, State = Q::State>,
F::Fetch: FilterFetch,
QF: Fetch<'w, State = Q::State>,
{
unsafe fn init_empty(
world: &World,
world: &'w World,
query_state: &'s QueryState<Q, F>,
last_change_tick: u32,
change_tick: u32,
@ -398,7 +377,7 @@ where
}
unsafe fn init(
world: &World,
world: &'w World,
query_state: &'s QueryState<Q, F>,
last_change_tick: u32,
change_tick: u32,
@ -409,7 +388,7 @@ where
last_change_tick,
change_tick,
);
let filter = <F::Fetch as Fetch>::init(
let filter = QueryFetch::<F>::init(
world,
&query_state.filter_state,
last_change_tick,
@ -430,7 +409,7 @@ where
#[inline]
unsafe fn peek_last(&mut self) -> Option<QF::Item> {
if self.current_index > 0 {
if QF::IS_DENSE && F::Fetch::IS_DENSE {
if QF::IS_DENSE && <QueryFetch<'static, F>>::IS_DENSE {
Some(self.fetch.table_fetch(self.current_index - 1))
} else {
Some(self.fetch.archetype_fetch(self.current_index - 1))
@ -450,7 +429,7 @@ where
archetypes: &'w Archetypes,
query_state: &'s QueryState<Q, F>,
) -> Option<QF::Item> {
if QF::IS_DENSE && F::Fetch::IS_DENSE {
if QF::IS_DENSE && <QueryFetch<'static, F>>::IS_DENSE {
loop {
if self.current_index == self.current_len {
let table_id = self.table_id_iter.next()?;

21
crates/bevy_ecs/src/query/mod.rs
View file

@ -10,6 +10,13 @@ pub use filter::*;
pub use iter::*;
pub use state::*;
#[allow(unreachable_code)]
unsafe fn debug_checked_unreachable() -> ! {
#[cfg(debug_assertions)]
unreachable!();
std::hint::unreachable_unchecked();
}
#[cfg(test)]
mod tests {
use super::AnyOf;
@ -65,8 +72,8 @@ mod tests {
assert_eq!(a_query.iter_combinations::<4>(w).size_hint().1, Some(1));
assert_eq!(a_query.iter_combinations::<5>(w).count(), 0);
assert_eq!(a_query.iter_combinations::<5>(w).size_hint().1, Some(0));
assert_eq!(a_query.iter_combinations::<1024>(w).count(), 0);
assert_eq!(a_query.iter_combinations::<1024>(w).size_hint().1, Some(0));
assert_eq!(a_query.iter_combinations::<128>(w).count(), 0);
assert_eq!(a_query.iter_combinations::<128>(w).size_hint().1, Some(0));
let values: Vec<[&A; 2]> = world.query::<&A>().iter_combinations(&world).collect();
assert_eq!(
@ -146,8 +153,8 @@ mod tests {
assert_eq!(a_with_b.iter_combinations::<4>(w).size_hint().1, Some(0));
assert_eq!(a_with_b.iter_combinations::<5>(w).count(), 0);
assert_eq!(a_with_b.iter_combinations::<5>(w).size_hint().1, Some(0));
assert_eq!(a_with_b.iter_combinations::<1024>(w).count(), 0);
assert_eq!(a_with_b.iter_combinations::<1024>(w).size_hint().1, Some(0));
assert_eq!(a_with_b.iter_combinations::<128>(w).count(), 0);
assert_eq!(a_with_b.iter_combinations::<128>(w).size_hint().1, Some(0));
let mut a_wout_b = world.query_filtered::<&A, Without<B>>();
let w = &world;
@ -163,13 +170,13 @@ mod tests {
assert_eq!(a_wout_b.iter_combinations::<4>(w).size_hint().1, Some(0));
assert_eq!(a_wout_b.iter_combinations::<5>(w).count(), 0);
assert_eq!(a_wout_b.iter_combinations::<5>(w).size_hint().1, Some(0));
assert_eq!(a_wout_b.iter_combinations::<1024>(w).count(), 0);
assert_eq!(a_wout_b.iter_combinations::<1024>(w).size_hint().1, Some(0));
assert_eq!(a_wout_b.iter_combinations::<128>(w).count(), 0);
assert_eq!(a_wout_b.iter_combinations::<128>(w).size_hint().1, Some(0));
let values: HashSet<[&A; 2]> = a_wout_b.iter_combinations(&world).collect();
assert_eq!(
values,
[[&A(2), &A(3)], [&A(2), &A(4)], [&A(3), &A(4)],]
[[&A(2), &A(3)], [&A(2), &A(4)], [&A(3), &A(4)]]
.into_iter()
.collect::<HashSet<_>>()
);

200
crates/bevy_ecs/src/query/state.rs
View file

@ -4,8 +4,8 @@ use crate::{
entity::Entity,
prelude::FromWorld,
query::{
Access, Fetch, FetchState, FilterFetch, FilteredAccess, NopFetch, QueryCombinationIter,
QueryIter, WorldQuery,
Access, Fetch, FetchState, FilteredAccess, NopFetch, QueryCombinationIter, QueryIter,
WorldQuery,
},
storage::TableId,
world::{World, WorldId},
@ -14,11 +14,10 @@ use bevy_tasks::TaskPool;
use fixedbitset::FixedBitSet;
use std::fmt;
use super::{QueryFetch, QueryItem, ROQueryFetch, ROQueryItem};
/// Provides scoped access to a [`World`] state according to a given [`WorldQuery`] and query filter.
pub struct QueryState<Q: WorldQuery, F: WorldQuery = ()>
where
F::Fetch: FilterFetch,
{
pub struct QueryState<Q: WorldQuery, F: WorldQuery = ()> {
world_id: WorldId,
pub(crate) archetype_generation: ArchetypeGeneration,
pub(crate) matched_tables: FixedBitSet,
@ -33,19 +32,13 @@ where
pub(crate) filter_state: F::State,
}
impl<Q: WorldQuery, F: WorldQuery> FromWorld for QueryState<Q, F>
where
F::Fetch: FilterFetch,
{
impl<Q: WorldQuery, F: WorldQuery> FromWorld for QueryState<Q, F> {
fn from_world(world: &mut World) -> Self {
world.query_filtered()
}
}
impl<Q: WorldQuery, F: WorldQuery> QueryState<Q, F>
where
F::Fetch: FilterFetch,
{
impl<Q: WorldQuery, F: WorldQuery> QueryState<Q, F> {
/// Creates a new [`QueryState`] from a given [`World`] and inherits the result of `world.id()`.
pub fn new(world: &mut World) -> Self {
let fetch_state = <Q::State as FetchState>::init(world);
@ -146,15 +139,15 @@ where
///
/// This can only be called for read-only queries, see [`Self::get_mut`] for write-queries.
#[inline]
pub fn get<'w, 's>(
&'s mut self,
pub fn get<'w>(
&mut self,
world: &'w World,
entity: Entity,
) -> Result<<Q::ReadOnlyFetch as Fetch<'w, 's>>::Item, QueryEntityError> {
) -> Result<ROQueryItem<'w, Q>, QueryEntityError> {
self.update_archetypes(world);
// SAFETY: query is read only
unsafe {
self.get_unchecked_manual::<Q::ReadOnlyFetch>(
self.get_unchecked_manual::<ROQueryFetch<'w, Q>>(
world,
entity,
world.last_change_tick(),
@ -196,11 +189,11 @@ where
/// assert_eq!(query_state.get_many(&world, [wrong_entity]), Err(QueryEntityError::NoSuchEntity(wrong_entity)));
/// ```
#[inline]
pub fn get_many<'w, 's, const N: usize>(
&'s mut self,
pub fn get_many<'w, const N: usize>(
&mut self,
world: &'w World,
entities: [Entity; N],
) -> Result<[<Q::ReadOnlyFetch as Fetch<'w, 's>>::Item; N], QueryEntityError> {
) -> Result<[ROQueryItem<'w, Q>; N], QueryEntityError> {
self.update_archetypes(world);
// SAFE: update_archetypes validates the `World` matches
@ -216,15 +209,15 @@ where
/// Gets the query result for the given [`World`] and [`Entity`].
#[inline]
pub fn get_mut<'w, 's>(
&'s mut self,
pub fn get_mut<'w>(
&mut self,
world: &'w mut World,
entity: Entity,
) -> Result<<Q::Fetch as Fetch<'w, 's>>::Item, QueryEntityError> {
) -> Result<QueryItem<'w, Q>, QueryEntityError> {
self.update_archetypes(world);
// SAFETY: query has unique world access
unsafe {
self.get_unchecked_manual::<Q::Fetch>(
self.get_unchecked_manual::<QueryFetch<'w, Q>>(
world,
entity,
world.last_change_tick(),
@ -272,11 +265,11 @@ where
/// assert_eq!(query_state.get_many_mut(&mut world, [entities[0], entities[0]]).unwrap_err(), QueryEntityError::AliasedMutability(entities[0]));
/// ```
#[inline]
pub fn get_many_mut<'w, 's, const N: usize>(
&'s mut self,
pub fn get_many_mut<'w, const N: usize>(
&mut self,
world: &'w mut World,
entities: [Entity; N],
) -> Result<[<Q::Fetch as Fetch<'w, 's>>::Item; N], QueryEntityError> {
) -> Result<[QueryItem<'w, Q>; N], QueryEntityError> {
self.update_archetypes(world);
// SAFE: method requires exclusive world access
@ -292,15 +285,15 @@ where
}
#[inline]
pub fn get_manual<'w, 's>(
&'s self,
pub fn get_manual<'w>(
&self,
world: &'w World,
entity: Entity,
) -> Result<<Q::ReadOnlyFetch as Fetch<'w, 's>>::Item, QueryEntityError> {
) -> Result<ROQueryItem<'w, Q>, QueryEntityError> {
self.validate_world(world);
// SAFETY: query is read only and world is validated
unsafe {
self.get_unchecked_manual::<Q::ReadOnlyFetch>(
self.get_unchecked_manual::<ROQueryFetch<'w, Q>>(
world,
entity,
world.last_change_tick(),
@ -316,13 +309,13 @@ where
/// This does not check for mutable query correctness. To be safe, make sure mutable queries
/// have unique access to the components they query.
#[inline]
pub unsafe fn get_unchecked<'w, 's>(
&'s mut self,
pub unsafe fn get_unchecked<'w>(
&mut self,
world: &'w World,
entity: Entity,
) -> Result<<Q::Fetch as Fetch<'w, 's>>::Item, QueryEntityError> {
) -> Result<QueryItem<'w, Q>, QueryEntityError> {
self.update_archetypes(world);
self.get_unchecked_manual::<Q::Fetch>(
self.get_unchecked_manual::<QueryFetch<'w, Q>>(
world,
entity,
world.last_change_tick(),
@ -340,8 +333,8 @@ where
///
/// This must be called on the same `World` that the `Query` was generated from:
/// use `QueryState::validate_world` to verify this.
pub(crate) unsafe fn get_unchecked_manual<'w, 's, QF: Fetch<'w, 's, State = Q::State>>(
&'s self,
pub(crate) unsafe fn get_unchecked_manual<'w, QF: Fetch<'w, State = Q::State>>(
&self,
world: &'w World,
entity: Entity,
last_change_tick: u32,
@ -359,8 +352,12 @@ where
}
let archetype = &world.archetypes[location.archetype_id];
let mut fetch = QF::init(world, &self.fetch_state, last_change_tick, change_tick);
let mut filter =
<F::Fetch as Fetch>::init(world, &self.filter_state, last_change_tick, change_tick);
let mut filter = <QueryFetch<F> as Fetch>::init(
world,
&self.filter_state,
last_change_tick,
change_tick,
);
fetch.set_archetype(&self.fetch_state, archetype, &world.storages().tables);
filter.set_archetype(&self.filter_state, archetype, &world.storages().tables);
@ -378,17 +375,17 @@ where
///
/// This must be called on the same `World` that the `Query` was generated from:
/// use `QueryState::validate_world` to verify this.
pub(crate) unsafe fn get_many_read_only_manual<'s, 'w, const N: usize>(
&'s self,
pub(crate) unsafe fn get_many_read_only_manual<'w, const N: usize>(
&self,
world: &'w World,
entities: [Entity; N],
last_change_tick: u32,
change_tick: u32,
) -> Result<[<Q::ReadOnlyFetch as Fetch<'w, 's>>::Item; N], QueryEntityError> {
) -> Result<[ROQueryItem<'w, Q>; N], QueryEntityError> {
// SAFE: fetch is read-only
// and world must be validated
let array_of_results = entities.map(|entity| {
self.get_unchecked_manual::<Q::ReadOnlyFetch>(
self.get_unchecked_manual::<ROQueryFetch<'w, Q>>(
world,
entity,
last_change_tick,
@ -419,13 +416,13 @@ where
///
/// This must be called on the same `World` that the `Query` was generated from:
/// use `QueryState::validate_world` to verify this.
pub(crate) unsafe fn get_many_unchecked_manual<'s, 'w, const N: usize>(
&'s self,
pub(crate) unsafe fn get_many_unchecked_manual<'w, const N: usize>(
&self,
world: &'w World,
entities: [Entity; N],
last_change_tick: u32,
change_tick: u32,
) -> Result<[<Q::Fetch as Fetch<'w, 's>>::Item; N], QueryEntityError> {
) -> Result<[QueryItem<'w, Q>; N], QueryEntityError> {
// Verify that all entities are unique
for i in 0..N {
for j in 0..i {
@ -436,7 +433,12 @@ where
}
let array_of_results = entities.map(|entity| {
self.get_unchecked_manual::<Q::Fetch>(world, entity, last_change_tick, change_tick)
self.get_unchecked_manual::<QueryFetch<'w, Q>>(
world,
entity,
last_change_tick,
change_tick,
)
});
// If any of the get calls failed, bubble up the error
@ -458,7 +460,7 @@ where
pub fn iter<'w, 's>(
&'s mut self,
world: &'w World,
) -> QueryIter<'w, 's, Q, Q::ReadOnlyFetch, F> {
) -> QueryIter<'w, 's, Q, ROQueryFetch<'w, Q>, F> {
// SAFETY: query is read only
unsafe {
self.update_archetypes(world);
@ -471,7 +473,7 @@ where
pub fn iter_mut<'w, 's>(
&'s mut self,
world: &'w mut World,
) -> QueryIter<'w, 's, Q, Q::Fetch, F> {
) -> QueryIter<'w, 's, Q, QueryFetch<'w, Q>, F> {
// SAFETY: query has unique world access
unsafe {
self.update_archetypes(world);
@ -487,7 +489,7 @@ where
pub fn iter_manual<'w, 's>(
&'s self,
world: &'w World,
) -> QueryIter<'w, 's, Q, Q::ReadOnlyFetch, F> {
) -> QueryIter<'w, 's, Q, ROQueryFetch<'w, Q>, F> {
self.validate_world(world);
// SAFETY: query is read only and world is validated
unsafe {
@ -509,7 +511,7 @@ where
pub fn iter_combinations<'w, 's, const K: usize>(
&'s mut self,
world: &'w World,
) -> QueryCombinationIter<'w, 's, Q, Q::ReadOnlyFetch, F, K> {
) -> QueryCombinationIter<'w, 's, Q, F, K> {
// SAFE: query is read only
unsafe {
self.update_archetypes(world);
@ -532,7 +534,7 @@ where
pub fn iter_combinations_mut<'w, 's, const K: usize>(
&'s mut self,
world: &'w mut World,
) -> QueryCombinationIter<'w, 's, Q, Q::Fetch, F, K> {
) -> QueryCombinationIter<'w, 's, Q, F, K> {
// SAFE: query has unique world access
unsafe {
self.update_archetypes(world);
@ -554,7 +556,7 @@ where
pub unsafe fn iter_unchecked<'w, 's>(
&'s mut self,
world: &'w World,
) -> QueryIter<'w, 's, Q, Q::Fetch, F> {
) -> QueryIter<'w, 's, Q, QueryFetch<'w, Q>, F> {
self.update_archetypes(world);
self.iter_unchecked_manual(world, world.last_change_tick(), world.read_change_tick())
}
@ -571,7 +573,7 @@ where
pub unsafe fn iter_combinations_unchecked<'w, 's, const K: usize>(
&'s mut self,
world: &'w World,
) -> QueryCombinationIter<'w, 's, Q, Q::Fetch, F, K> {
) -> QueryCombinationIter<'w, 's, Q, F, K> {
self.update_archetypes(world);
self.iter_combinations_unchecked_manual(
world,
@ -590,7 +592,7 @@ where
/// This does not validate that `world.id()` matches `self.world_id`. Calling this on a `world`
/// with a mismatched [`WorldId`] is unsound.
#[inline]
pub(crate) unsafe fn iter_unchecked_manual<'w, 's, QF: Fetch<'w, 's, State = Q::State>>(
pub(crate) unsafe fn iter_unchecked_manual<'w, 's, QF: Fetch<'w, State = Q::State>>(
&'s self,
world: &'w World,
last_change_tick: u32,
@ -610,17 +612,12 @@ where
/// This does not validate that `world.id()` matches `self.world_id`. Calling this on a `world`
/// with a mismatched [`WorldId`] is unsound.
#[inline]
pub(crate) unsafe fn iter_combinations_unchecked_manual<
'w,
's,
QF: Fetch<'w, 's, State = Q::State>,
const K: usize,
>(
pub(crate) unsafe fn iter_combinations_unchecked_manual<'w, 's, const K: usize>(
&'s self,
world: &'w World,
last_change_tick: u32,
change_tick: u32,
) -> QueryCombinationIter<'w, 's, Q, QF, F, K> {
) -> QueryCombinationIter<'w, 's, Q, F, K> {
QueryCombinationIter::new(world, self, last_change_tick, change_tick)
}
@ -629,15 +626,11 @@ where
///
/// This can only be called for read-only queries, see [`Self::for_each_mut`] for write-queries.
#[inline]
pub fn for_each<'w, 's, FN: FnMut(<Q::ReadOnlyFetch as Fetch<'w, 's>>::Item)>(
&'s mut self,
world: &'w World,
func: FN,
) {
pub fn for_each<'w, FN: FnMut(ROQueryItem<'w, Q>)>(&mut self, world: &'w World, func: FN) {
// SAFETY: query is read only
unsafe {
self.update_archetypes(world);
self.for_each_unchecked_manual::<Q::ReadOnlyFetch, FN>(
self.for_each_unchecked_manual::<ROQueryFetch<Q>, FN>(
world,
func,
world.last_change_tick(),
@ -649,15 +642,15 @@ where
/// Runs `func` on each query result for the given [`World`]. This is faster than the equivalent
/// `iter_mut()` method, but cannot be chained like a normal [`Iterator`].
#[inline]
pub fn for_each_mut<'w, 's, FN: FnMut(<Q::Fetch as Fetch<'w, 's>>::Item)>(
&'s mut self,
pub fn for_each_mut<'w, FN: FnMut(QueryItem<'w, Q>)>(
&mut self,
world: &'w mut World,
func: FN,
) {
// SAFETY: query has unique world access
unsafe {
self.update_archetypes(world);
self.for_each_unchecked_manual::<Q::Fetch, FN>(
self.for_each_unchecked_manual::<QueryFetch<Q>, FN>(
world,
func,
world.last_change_tick(),
@ -676,13 +669,13 @@ where
/// This does not check for mutable query correctness. To be safe, make sure mutable queries
/// have unique access to the components they query.
#[inline]
pub unsafe fn for_each_unchecked<'w, 's, FN: FnMut(<Q::Fetch as Fetch<'w, 's>>::Item)>(
&'s mut self,
pub unsafe fn for_each_unchecked<'w, FN: FnMut(QueryItem<'w, Q>)>(
&mut self,
world: &'w World,
func: FN,
) {
self.update_archetypes(world);
self.for_each_unchecked_manual::<Q::Fetch, FN>(
self.for_each_unchecked_manual::<QueryFetch<Q>, FN>(
world,
func,
world.last_change_tick(),
@ -695,12 +688,8 @@ where
/// This can only be called for read-only queries, see [`Self::par_for_each_mut`] for
/// write-queries.
#[inline]
pub fn par_for_each<
'w,
's,
FN: Fn(<Q::ReadOnlyFetch as Fetch<'w, 's>>::Item) + Send + Sync + Clone,
>(
&'s mut self,
pub fn par_for_each<'w, FN: Fn(ROQueryItem<'w, Q>) + Send + Sync + Clone>(
&mut self,
world: &'w World,
task_pool: &TaskPool,
batch_size: usize,
@ -709,7 +698,7 @@ where
// SAFETY: query is read only
unsafe {
self.update_archetypes(world);
self.par_for_each_unchecked_manual::<Q::ReadOnlyFetch, FN>(
self.par_for_each_unchecked_manual::<ROQueryFetch<Q>, FN>(
world,
task_pool,
batch_size,
@ -722,12 +711,8 @@ where
/// Runs `func` on each query result in parallel using the given `task_pool`.
#[inline]
pub fn par_for_each_mut<
'w,
's,
FN: Fn(<Q::Fetch as Fetch<'w, 's>>::Item) + Send + Sync + Clone,
>(
&'s mut self,
pub fn par_for_each_mut<'w, FN: Fn(QueryItem<'w, Q>) + Send + Sync + Clone>(
&mut self,
world: &'w mut World,
task_pool: &TaskPool,
batch_size: usize,
@ -736,7 +721,7 @@ where
// SAFETY: query has unique world access
unsafe {
self.update_archetypes(world);
self.par_for_each_unchecked_manual::<Q::Fetch, FN>(
self.par_for_each_unchecked_manual::<QueryFetch<Q>, FN>(
world,
task_pool,
batch_size,
@ -756,19 +741,15 @@ where
/// This does not check for mutable query correctness. To be safe, make sure mutable queries
/// have unique access to the components they query.
#[inline]
pub unsafe fn par_for_each_unchecked<
'w,
's,
FN: Fn(<Q::Fetch as Fetch<'w, 's>>::Item) + Send + Sync + Clone,
>(
&'s mut self,
pub unsafe fn par_for_each_unchecked<'w, FN: Fn(QueryItem<'w, Q>) + Send + Sync + Clone>(
&mut self,
world: &'w World,
task_pool: &TaskPool,
batch_size: usize,
func: FN,
) {
self.update_archetypes(world);
self.par_for_each_unchecked_manual::<Q::Fetch, FN>(
self.par_for_each_unchecked_manual::<QueryFetch<Q>, FN>(
world,
task_pool,
batch_size,
@ -790,11 +771,10 @@ where
/// with a mismatched [`WorldId`] is unsound.
pub(crate) unsafe fn for_each_unchecked_manual<
'w,
's,
QF: Fetch<'w, 's, State = Q::State>,
QF: Fetch<'w, State = Q::State>,
FN: FnMut(QF::Item),
>(
&'s self,
&self,
world: &'w World,
mut func: FN,
last_change_tick: u32,
@ -803,9 +783,14 @@ where
// NOTE: If you are changing query iteration code, remember to update the following places, where relevant:
// QueryIter, QueryIterationCursor, QueryState::for_each_unchecked_manual, QueryState::par_for_each_unchecked_manual
let mut fetch = QF::init(world, &self.fetch_state, last_change_tick, change_tick);
let mut filter =
<F::Fetch as Fetch>::init(world, &self.filter_state, last_change_tick, change_tick);
if Q::Fetch::IS_DENSE && F::Fetch::IS_DENSE {
let mut filter = <QueryFetch<F> as Fetch>::init(
world,
&self.filter_state,
last_change_tick,
change_tick,
);
if <QueryFetch<'static, Q>>::IS_DENSE && <QueryFetch<'static, F>>::IS_DENSE {
let tables = &world.storages().tables;
for table_id in &self.matched_table_ids {
let table = &tables[*table_id];
@ -850,11 +835,10 @@ where
/// with a mismatched [`WorldId`] is unsound.
pub(crate) unsafe fn par_for_each_unchecked_manual<
'w,
's,
QF: Fetch<'w, 's, State = Q::State>,
QF: Fetch<'w, State = Q::State>,
FN: Fn(QF::Item) + Send + Sync + Clone,
>(
&'s self,
&self,
world: &'w World,
task_pool: &TaskPool,
batch_size: usize,
@ -865,7 +849,7 @@ where
// NOTE: If you are changing query iteration code, remember to update the following places, where relevant:
// QueryIter, QueryIterationCursor, QueryState::for_each_unchecked_manual, QueryState::par_for_each_unchecked_manual
task_pool.scope(|scope| {
if QF::IS_DENSE && F::Fetch::IS_DENSE {
if QF::IS_DENSE && <QueryFetch<'static, F>>::IS_DENSE {
let tables = &world.storages().tables;
for table_id in &self.matched_table_ids {
let table = &tables[*table_id];
@ -875,7 +859,7 @@ where
scope.spawn(async move {
let mut fetch =
QF::init(world, &self.fetch_state, last_change_tick, change_tick);
let mut filter = <F::Fetch as Fetch>::init(
let mut filter = <QueryFetch<F> as Fetch>::init(
world,
&self.filter_state,
last_change_tick,
@ -907,7 +891,7 @@ where
scope.spawn(async move {
let mut fetch =
QF::init(world, &self.fetch_state, last_change_tick, change_tick);
let mut filter = <F::Fetch as Fetch>::init(
let mut filter = <QueryFetch<F> as Fetch>::init(
world,
&self.filter_state,
last_change_tick,

187
crates/bevy_ecs/src/storage/blob_vec.rs
View file

@ -1,23 +1,46 @@
use std::{
alloc::{handle_alloc_error, Layout},
cell::UnsafeCell,
ptr::NonNull,
};
use crate::ptr::{OwningPtr, Ptr, PtrMut};
/// A flat, type-erased data storage type
///
/// Used to densely store homogeneous ECS data.
#[derive(Debug)]
pub struct BlobVec {
item_layout: Layout,
capacity: usize,
/// Number of elements, not bytes
len: usize,
data: NonNull<u8>,
swap_scratch: NonNull<u8>,
drop: unsafe fn(*mut u8),
drop: unsafe fn(OwningPtr<'_>),
}
// We want to ignore the `drop` field in our `Debug` impl
impl std::fmt::Debug for BlobVec {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("BlobVec")
.field("item_layout", &self.item_layout)
.field("capacity", &self.capacity)
.field("len", &self.len)
.field("data", &self.data)
.field("swap_scratch", &self.swap_scratch)
.finish()
}
}
impl BlobVec {
pub fn new(item_layout: Layout, drop: unsafe fn(*mut u8), capacity: usize) -> BlobVec {
/// # Safety
///
/// `drop` should be safe to call with an [`OwningPtr`] pointing to any item that's been pushed into this [`BlobVec`].
pub unsafe fn new(
item_layout: Layout,
drop: unsafe fn(OwningPtr<'_>),
capacity: usize,
) -> BlobVec {
if item_layout.size() == 0 {
BlobVec {
swap_scratch: NonNull::dangling(),
@ -28,7 +51,7 @@ impl BlobVec {
drop,
}
} else {
let swap_scratch = NonNull::new(unsafe { std::alloc::alloc(item_layout) })
let swap_scratch = NonNull::new(std::alloc::alloc(item_layout))
.unwrap_or_else(|| std::alloc::handle_alloc_error(item_layout));
let mut blob_vec = BlobVec {
swap_scratch,
@ -65,6 +88,8 @@ impl BlobVec {
}
}
// FIXME: this should probably be an unsafe fn as it shouldn't be called if the layout
// is for a ZST
fn grow_exact(&mut self, increment: usize) {
debug_assert!(self.item_layout.size() != 0);
@ -76,7 +101,7 @@ impl BlobVec {
std::alloc::alloc(new_layout)
} else {
std::alloc::realloc(
self.get_ptr().as_ptr(),
self.get_ptr_mut().inner().as_ptr(),
array_layout(&self.item_layout, self.capacity)
.expect("array layout should be valid"),
new_layout.size(),
@ -90,17 +115,17 @@ impl BlobVec {
/// # Safety
/// - index must be in bounds
/// - the memory in the `BlobVec` starting at index `index`, of a size matching this `BlobVec`'s
/// `item_layout`, must have been previously allocated, but not initialized yet
/// - the memory at `*value` must be previously initialized with an item matching this
/// `BlobVec`'s `item_layout`
/// - the item that was stored in `*value` is left logically uninitialised/moved out of after
/// calling this function, and as such should not be used or dropped by the caller.
/// - the memory in the [`BlobVec`] starting at index `index`, of a size matching this [`BlobVec`]'s
/// `item_layout`, must have been previously allocated.
#[inline]
pub unsafe fn initialize_unchecked(&mut self, index: usize, value: *mut u8) {
pub unsafe fn initialize_unchecked(&mut self, index: usize, value: OwningPtr<'_>) {
debug_assert!(index < self.len());
let ptr = self.get_unchecked(index);
std::ptr::copy_nonoverlapping(value, ptr, self.item_layout.size());
let ptr = self.get_unchecked_mut(index);
std::ptr::copy_nonoverlapping::<u8>(
value.inner().as_ptr(),
ptr.inner().as_ptr(),
self.item_layout.size(),
);
}
/// # Safety
@ -110,32 +135,35 @@ impl BlobVec {
/// this [`BlobVec`]'s `item_layout`
/// - the memory at `*value` must also be previously initialized with an item matching this
/// [`BlobVec`]'s `item_layout`
/// - the item that was stored in `*value` is left logically uninitialised/moved out of after
/// calling this function, and as such should not be used or dropped by the caller.
pub unsafe fn replace_unchecked(&mut self, index: usize, value: *mut u8) {
pub unsafe fn replace_unchecked(&mut self, index: usize, value: OwningPtr<'_>) {
debug_assert!(index < self.len());
let ptr = self.get_unchecked(index);
// If `drop` panics, then when the collection is dropped during stack unwinding, the
// collection's `Drop` impl will call `drop` again for the old value (which is still stored
// in the collection), so we get a double drop. To prevent that, we set len to 0 until we're
// done.
let old_len = std::mem::replace(&mut self.len, 0);
(self.drop)(ptr);
std::ptr::copy_nonoverlapping(value, ptr, self.item_layout.size());
let old_len = self.len;
let ptr = self.get_unchecked_mut(index).promote().inner();
self.len = 0;
// Drop the old value, then write back, justifying the promotion
(self.drop)(OwningPtr::new(ptr));
std::ptr::copy_nonoverlapping::<u8>(
value.inner().as_ptr(),
ptr.as_ptr(),
self.item_layout.size(),
);
self.len = old_len;
}
/// Increases the length by one (and grows the vec if needed) with uninitialized memory and
/// returns the index
/// Pushes a value to the [`BlobVec`].
///
/// # Safety
/// the newly allocated space must be immediately populated with a valid value
/// `value` must be valid to add to this [`BlobVec`]
#[inline]
pub unsafe fn push_uninit(&mut self) -> usize {
pub unsafe fn push(&mut self, value: OwningPtr<'_>) {
self.reserve_exact(1);
let index = self.len;
self.len += 1;
index
self.initialize_unchecked(index, value);
}
/// # Safety
@ -154,50 +182,77 @@ impl BlobVec {
///
/// # Safety
/// It is the caller's responsibility to ensure that `index` is < `self.len()`
/// Callers should _only_ access the returned pointer immediately after calling this function.
#[inline]
pub unsafe fn swap_remove_and_forget_unchecked(&mut self, index: usize) -> *mut u8 {
#[must_use = "The returned pointer should be used to dropped the removed element"]
pub unsafe fn swap_remove_and_forget_unchecked(&mut self, index: usize) -> OwningPtr<'_> {
// FIXME: This should probably just use `core::ptr::swap` and return an `OwningPtr`
// into the underlying `BlobVec` allocation, and remove swap_scratch
debug_assert!(index < self.len());
let last = self.len - 1;
let swap_scratch = self.swap_scratch.as_ptr();
std::ptr::copy_nonoverlapping(
self.get_unchecked(index),
std::ptr::copy_nonoverlapping::<u8>(
self.get_unchecked_mut(index).inner().as_ptr(),
swap_scratch,
self.item_layout.size(),
);
std::ptr::copy(
self.get_unchecked(last),
self.get_unchecked(index),
std::ptr::copy::<u8>(
self.get_unchecked_mut(last).inner().as_ptr(),
self.get_unchecked_mut(index).inner().as_ptr(),
self.item_layout.size(),
);
self.len -= 1;
swap_scratch
}
/// # Safety
/// index must be in-bounds
#[inline]
pub unsafe fn swap_remove_and_drop_unchecked(&mut self, index: usize) {
debug_assert!(index < self.len());
let value = self.swap_remove_and_forget_unchecked(index);
(self.drop)(value);
OwningPtr::new(self.swap_scratch)
}
/// # Safety
/// It is the caller's responsibility to ensure that `index` is < self.len()
#[inline]
pub unsafe fn get_unchecked(&self, index: usize) -> *mut u8 {
pub unsafe fn swap_remove_and_drop_unchecked(&mut self, index: usize) {
debug_assert!(index < self.len());
self.get_ptr().as_ptr().add(index * self.item_layout.size())
let drop = self.drop;
let value = self.swap_remove_and_forget_unchecked(index);
(drop)(value);
}
/// Gets a pointer to the start of the vec
/// # Safety
/// It is the caller's responsibility to ensure that `index` is < self.len()
#[inline]
pub unsafe fn get_unchecked(&self, index: usize) -> Ptr<'_> {
debug_assert!(index < self.len());
self.get_ptr().add(index * self.item_layout.size())
}
/// # Safety
/// It is the caller's responsibility to ensure that `index` is < self.len()
#[inline]
pub unsafe fn get_unchecked_mut(&mut self, index: usize) -> PtrMut<'_> {
debug_assert!(index < self.len());
let layout_size = self.item_layout.size();
self.get_ptr_mut().add(index * layout_size)
}
/// Gets a [`Ptr`] to the start of the vec
#[inline]
pub fn get_ptr(&self) -> Ptr<'_> {
// SAFE: the inner data will remain valid for as long as 'self.
unsafe { Ptr::new(self.data) }
}
/// Gets a [`PtrMut`] to the start of the vec
#[inline]
pub fn get_ptr_mut(&mut self) -> PtrMut<'_> {
// SAFE: the inner data will remain valid for as long as 'self.
unsafe { PtrMut::new(self.data) }
}
/// Get a reference to the entire [`BlobVec`] as if it were an array with elements of type `T`
///
/// # Safety
/// must ensure rust mutability rules are not violated
#[inline]
pub unsafe fn get_ptr(&self) -> NonNull<u8> {
self.data
/// The type `T` must be the type of the items in this [`BlobVec`].
pub unsafe fn get_slice<T>(&self) -> &[UnsafeCell<T>] {
// SAFE: the inner data will remain valid for as long as 'self.
std::slice::from_raw_parts(self.data.as_ptr() as *const UnsafeCell<T>, self.len)
}
pub fn clear(&mut self) {
@ -205,12 +260,14 @@ impl BlobVec {
// We set len to 0 _before_ dropping elements for unwind safety. This ensures we don't
// accidentally drop elements twice in the event of a drop impl panicking.
self.len = 0;
let drop = self.drop;
let layout_size = self.item_layout.size();
for i in 0..len {
unsafe {
// NOTE: this doesn't use self.get_unchecked(i) because the debug_assert on index
// will panic here due to self.len being set to 0
let ptr = self.get_ptr().as_ptr().add(i * self.item_layout.size());
(self.drop)(ptr);
let ptr = self.get_ptr_mut().add(i * layout_size).promote();
(drop)(ptr);
}
}
}
@ -223,7 +280,7 @@ impl Drop for BlobVec {
array_layout(&self.item_layout, self.capacity).expect("array layout should be valid");
if array_layout.size() > 0 {
unsafe {
std::alloc::dealloc(self.get_ptr().as_ptr(), array_layout);
std::alloc::dealloc(self.get_ptr_mut().inner().as_ptr(), array_layout);
std::alloc::dealloc(self.swap_scratch.as_ptr(), self.item_layout);
}
}
@ -286,21 +343,23 @@ const fn padding_needed_for(layout: &Layout, align: usize) -> usize {
#[cfg(test)]
mod tests {
use crate::ptr::OwningPtr;
use super::BlobVec;
use std::{alloc::Layout, cell::RefCell, rc::Rc};
// SAFETY: The pointer points to a valid value of type `T` and it is safe to drop this value.
unsafe fn drop_ptr<T>(x: *mut u8) {
x.cast::<T>().drop_in_place();
unsafe fn drop_ptr<T>(x: OwningPtr<'_>) {
x.inner().cast::<T>().as_ptr().drop_in_place()
}
/// # Safety
///
/// `blob_vec` must have a layout that matches `Layout::new::<T>()`
unsafe fn push<T>(blob_vec: &mut BlobVec, mut value: T) {
let index = blob_vec.push_uninit();
blob_vec.initialize_unchecked(index, (&mut value as *mut T).cast::<u8>());
std::mem::forget(value);
unsafe fn push<T>(blob_vec: &mut BlobVec, value: T) {
OwningPtr::make(value, |ptr| {
blob_vec.push(ptr);
});
}
/// # Safety
@ -309,7 +368,7 @@ mod tests {
unsafe fn swap_remove<T>(blob_vec: &mut BlobVec, index: usize) -> T {
assert!(index < blob_vec.len());
let value = blob_vec.swap_remove_and_forget_unchecked(index);
value.cast::<T>().read()
value.read::<T>()
}
/// # Safety
@ -318,14 +377,14 @@ mod tests {
/// value at the given `index`
unsafe fn get_mut<T>(blob_vec: &mut BlobVec, index: usize) -> &mut T {
assert!(index < blob_vec.len());
&mut *blob_vec.get_unchecked(index).cast::<T>()
blob_vec.get_unchecked_mut(index).deref_mut::<T>()
}
#[test]
fn resize_test() {
let item_layout = Layout::new::<usize>();
let drop = drop_ptr::<usize>;
let mut blob_vec = BlobVec::new(item_layout, drop, 64);
let mut blob_vec = unsafe { BlobVec::new(item_layout, drop, 64) };
unsafe {
for i in 0..1_000 {
push(&mut blob_vec, i as usize);
@ -355,7 +414,7 @@ mod tests {
{
let item_layout = Layout::new::<Foo>();
let drop = drop_ptr::<Foo>;
let mut blob_vec = BlobVec::new(item_layout, drop, 2);
let mut blob_vec = unsafe { BlobVec::new(item_layout, drop, 2) };
assert_eq!(blob_vec.capacity(), 2);
unsafe {
let foo1 = Foo {
@ -415,6 +474,6 @@ mod tests {
fn blob_vec_drop_empty_capacity() {
let item_layout = Layout::new::<Foo>();
let drop = drop_ptr::<Foo>;
let _ = BlobVec::new(item_layout, drop, 0);
let _ = unsafe { BlobVec::new(item_layout, drop, 0) };
}
}

52
crates/bevy_ecs/src/storage/sparse_set.rs
View file

@ -1,6 +1,7 @@
use crate::{
component::{ComponentId, ComponentInfo, ComponentTicks},
entity::Entity,
ptr::{OwningPtr, Ptr},
storage::BlobVec,
};
use std::{cell::UnsafeCell, marker::PhantomData};
@ -102,7 +103,10 @@ pub struct ComponentSparseSet {
impl ComponentSparseSet {
pub fn new(component_info: &ComponentInfo, capacity: usize) -> Self {
Self {
dense: BlobVec::new(component_info.layout(), component_info.drop(), capacity),
// SAFE: component_info.drop() is compatible with the items that will be inserted.
dense: unsafe {
BlobVec::new(component_info.layout(), component_info.drop(), capacity)
},
ticks: Vec::with_capacity(capacity),
entities: Vec::with_capacity(capacity),
sparse: Default::default(),
@ -127,25 +131,19 @@ impl ComponentSparseSet {
}
/// Inserts the `entity` key and component `value` pair into this sparse
/// set. This collection takes ownership of the contents of `value`, and
/// will drop the value when needed. Also, it may overwrite the contents of
/// the `value` pointer if convenient. The caller is responsible for
/// ensuring it does not drop `*value` after calling `insert`.
/// set.
///
/// # Safety
/// * The `value` pointer must point to a valid address that matches the
/// `Layout` inside the `ComponentInfo` given when constructing this
/// sparse set.
/// * The caller is responsible for ensuring it does not drop `*value` after
/// calling `insert`.
pub unsafe fn insert(&mut self, entity: Entity, value: *mut u8, change_tick: u32) {
/// The `value` pointer must point to a valid address that matches the [`Layout`](std::alloc::Layout)
/// inside the [`ComponentInfo`] given when constructing this sparse set.
pub unsafe fn insert(&mut self, entity: Entity, value: OwningPtr<'_>, change_tick: u32) {
if let Some(&dense_index) = self.sparse.get(entity) {
self.dense.replace_unchecked(dense_index, value);
*self.ticks.get_unchecked_mut(dense_index) =
UnsafeCell::new(ComponentTicks::new(change_tick));
} else {
let dense_index = self.dense.push_uninit();
self.dense.initialize_unchecked(dense_index, value);
let dense_index = self.dense.len();
self.dense.push(value);
self.sparse.insert(entity, dense_index);
debug_assert_eq!(self.ticks.len(), dense_index);
debug_assert_eq!(self.entities.len(), dense_index);
@ -160,39 +158,37 @@ impl ComponentSparseSet {
self.sparse.contains(entity)
}
/// # Safety
/// ensure the same entity is not accessed twice at the same time
#[inline]
pub fn get(&self, entity: Entity) -> Option<*mut u8> {
pub fn get(&self, entity: Entity) -> Option<Ptr<'_>> {
self.sparse.get(entity).map(|dense_index| {
// SAFE: if the sparse index points to something in the dense vec, it exists
unsafe { self.dense.get_unchecked(*dense_index) }
})
}
/// # Safety
/// ensure the same entity is not accessed twice at the same time
#[inline]
pub unsafe fn get_with_ticks(&self, entity: Entity) -> Option<(*mut u8, *mut ComponentTicks)> {
pub fn get_with_ticks(&self, entity: Entity) -> Option<(Ptr<'_>, &UnsafeCell<ComponentTicks>)> {
let dense_index = *self.sparse.get(entity)?;
// SAFE: if the sparse index points to something in the dense vec, it exists
Some((
self.dense.get_unchecked(dense_index),
self.ticks.get_unchecked(dense_index).get(),
))
unsafe {
Some((
self.dense.get_unchecked(dense_index),
self.ticks.get_unchecked(dense_index),
))
}
}
#[inline]
pub fn get_ticks(&self, entity: Entity) -> Option<&ComponentTicks> {
pub fn get_ticks(&self, entity: Entity) -> Option<&UnsafeCell<ComponentTicks>> {
let dense_index = *self.sparse.get(entity)?;
// SAFE: if the sparse index points to something in the dense vec, it exists
unsafe { Some(&*self.ticks.get_unchecked(dense_index).get()) }
unsafe { Some(self.ticks.get_unchecked(dense_index)) }
}
/// Removes the `entity` from this sparse set and returns a pointer to the associated value (if
/// it exists). It is the caller's responsibility to drop the returned ptr (if Some is
/// returned).
pub fn remove_and_forget(&mut self, entity: Entity) -> Option<*mut u8> {
/// it exists).
#[must_use = "The returned pointer must be used to drop the removed component."]
pub fn remove_and_forget(&mut self, entity: Entity) -> Option<OwningPtr<'_>> {
self.sparse.remove(entity).map(|dense_index| {
self.ticks.swap_remove(dense_index);
self.entities.swap_remove(dense_index);

72
crates/bevy_ecs/src/storage/table.rs
View file

@ -1,13 +1,13 @@
use crate::{
component::{ComponentId, ComponentInfo, ComponentTicks, Components},
entity::Entity,
ptr::{OwningPtr, Ptr, PtrMut},
storage::{BlobVec, SparseSet},
};
use bevy_utils::HashMap;
use std::{
cell::UnsafeCell,
ops::{Index, IndexMut},
ptr::NonNull,
};
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
@ -41,7 +41,8 @@ impl Column {
pub fn with_capacity(component_info: &ComponentInfo, capacity: usize) -> Self {
Column {
component_id: component_info.id(),
data: BlobVec::new(component_info.layout(), component_info.drop(), capacity),
// SAFE: component_info.drop() is valid for the types that will be inserted.
data: unsafe { BlobVec::new(component_info.layout(), component_info.drop(), capacity) },
ticks: Vec::with_capacity(capacity),
}
}
@ -53,7 +54,7 @@ impl Column {
/// # Safety
/// Assumes data has already been allocated for the given row.
#[inline]
pub unsafe fn initialize(&mut self, row: usize, data: *mut u8, ticks: ComponentTicks) {
pub unsafe fn initialize(&mut self, row: usize, data: OwningPtr<'_>, ticks: ComponentTicks) {
debug_assert!(row < self.len());
self.data.initialize_unchecked(row, data);
*self.ticks.get_unchecked_mut(row).get_mut() = ticks;
@ -65,7 +66,7 @@ impl Column {
/// # Safety
/// Assumes data has already been allocated for the given row.
#[inline]
pub unsafe fn replace(&mut self, row: usize, data: *mut u8, change_tick: u32) {
pub unsafe fn replace(&mut self, row: usize, data: OwningPtr<'_>, change_tick: u32) {
debug_assert!(row < self.len());
self.data.replace_unchecked(row, data);
self.ticks
@ -77,7 +78,7 @@ impl Column {
/// # Safety
/// Assumes data has already been allocated for the given row.
#[inline]
pub unsafe fn initialize_data(&mut self, row: usize, data: *mut u8) {
pub unsafe fn initialize_data(&mut self, row: usize, data: OwningPtr<'_>) {
debug_assert!(row < self.len());
self.data.initialize_unchecked(row, data);
}
@ -109,10 +110,11 @@ impl Column {
}
#[inline]
#[must_use = "The returned pointer should be used to dropped the removed component"]
pub(crate) unsafe fn swap_remove_and_forget_unchecked(
&mut self,
row: usize,
) -> (*mut u8, ComponentTicks) {
) -> (OwningPtr<'_>, ComponentTicks) {
let data = self.data.swap_remove_and_forget_unchecked(row);
let ticks = self.ticks.swap_remove(row).into_inner();
(data, ticks)
@ -120,9 +122,8 @@ impl Column {
// # Safety
// - ptr must point to valid data of this column's component type
pub(crate) unsafe fn push(&mut self, ptr: *mut u8, ticks: ComponentTicks) {
let row = self.data.push_uninit();
self.data.initialize_unchecked(row, ptr);
pub(crate) unsafe fn push(&mut self, ptr: OwningPtr<'_>, ticks: ComponentTicks) {
self.data.push(ptr);
self.ticks.push(UnsafeCell::new(ticks));
}
@ -132,50 +133,46 @@ impl Column {
self.ticks.reserve_exact(additional);
}
/// # Safety
/// must ensure rust mutability rules are not violated
#[inline]
pub unsafe fn get_data_ptr(&self) -> NonNull<u8> {
pub fn get_data_ptr(&self) -> Ptr<'_> {
self.data.get_ptr()
}
#[inline]
pub fn get_ticks_ptr(&self) -> *const UnsafeCell<ComponentTicks> {
self.ticks.as_ptr()
/// # Safety
/// The type `T` must be the type of the items in this column.
pub unsafe fn get_data_slice<T>(&self) -> &[UnsafeCell<T>] {
self.data.get_slice()
}
#[inline]
pub fn get_ticks_const_ptr(&self) -> *const ComponentTicks {
// cast is valid, because UnsafeCell is repr(transparent)
self.get_ticks_ptr() as *const ComponentTicks
pub fn get_ticks_slice(&self) -> &[UnsafeCell<ComponentTicks>] {
&self.ticks
}
/// # Safety
/// - index must be in-bounds
/// - no other reference to the data of the same row can exist at the same time
/// - pointer cannot be dereferenced after mutable reference to this `Column` was live
#[inline]
pub unsafe fn get_data_unchecked(&self, row: usize) -> *mut u8 {
pub unsafe fn get_data_unchecked(&self, row: usize) -> Ptr<'_> {
debug_assert!(row < self.data.len());
self.data.get_unchecked(row)
}
/// # Safety
/// index must be in-bounds
/// - index must be in-bounds
/// - no other reference to the data of the same row can exist at the same time
#[inline]
pub unsafe fn get_ticks_unchecked(&self, row: usize) -> &ComponentTicks {
debug_assert!(row < self.ticks.len());
&*self.ticks.get_unchecked(row).get()
pub unsafe fn get_data_unchecked_mut(&mut self, row: usize) -> PtrMut<'_> {
debug_assert!(row < self.data.len());
self.data.get_unchecked_mut(row)
}
/// # Safety
/// - index must be in-bounds
/// - no other reference to the ticks of the same row can exist at the same time
/// - pointer cannot be dereferenced after mutable reference to this column was live
/// index must be in-bounds
#[inline]
pub unsafe fn get_ticks_mut_ptr_unchecked(&self, row: usize) -> *mut ComponentTicks {
pub unsafe fn get_ticks_unchecked(&self, row: usize) -> &UnsafeCell<ComponentTicks> {
debug_assert!(row < self.ticks.len());
self.ticks.get_unchecked(row).get()
self.ticks.get_unchecked(row)
}
pub fn clear(&mut self) {
@ -257,8 +254,9 @@ impl Table {
let is_last = row == self.entities.len() - 1;
let new_row = new_table.allocate(self.entities.swap_remove(row));
for column in self.columns.values_mut() {
let component_id = column.component_id;
let (data, ticks) = column.swap_remove_and_forget_unchecked(row);
if let Some(new_column) = new_table.get_column_mut(column.component_id) {
if let Some(new_column) = new_table.get_column_mut(component_id) {
new_column.initialize(new_row, data, ticks);
}
}
@ -529,6 +527,7 @@ impl IndexMut<TableId> for Tables {
mod tests {
use crate as bevy_ecs;
use crate::component::Component;
use crate::ptr::OwningPtr;
use crate::storage::Storages;
use crate::{component::Components, entity::Entity, storage::Table};
#[derive(Component)]
@ -547,12 +546,13 @@ mod tests {
// SAFE: we allocate and immediately set data afterwards
unsafe {
let row = table.allocate(*entity);
let mut value = row;
let value_ptr = ((&mut value) as *mut usize).cast::<u8>();
table
.get_column_mut(component_id)
.unwrap()
.initialize_data(row, value_ptr);
let value: W<usize> = W(row);
OwningPtr::make(value, |value_ptr| {
table
.get_column_mut(component_id)
.unwrap()
.initialize_data(row, value_ptr);
});
};
}

118
crates/bevy_ecs/src/system/query.rs
View file

@ -2,8 +2,8 @@ use crate::{
component::Component,
entity::Entity,
query::{
Fetch, FilterFetch, NopFetch, QueryCombinationIter, QueryEntityError, QueryIter,
QueryState, ReadOnlyFetch, WorldQuery,
NopFetch, QueryCombinationIter, QueryEntityError, QueryFetch, QueryItem, QueryIter,
QueryState, ROQueryFetch, ROQueryItem, ReadOnlyFetch, WorldQuery,
},
world::{Mut, World},
};
@ -239,20 +239,14 @@ use std::{any::TypeId, fmt::Debug};
/// methods instead. Keep in mind though that they will return a [`QuerySingleError`] if the
/// number of query results differ from being exactly one. If that's the case, use `iter.next()`
/// (or `iter_mut.next()`) to only get the first query result.
pub struct Query<'world, 'state, Q: WorldQuery, F: WorldQuery = ()>
where
F::Fetch: FilterFetch,
{
pub struct Query<'world, 'state, Q: WorldQuery, F: WorldQuery = ()> {
pub(crate) world: &'world World,
pub(crate) state: &'state QueryState<Q, F>,
pub(crate) last_change_tick: u32,
pub(crate) change_tick: u32,
}
impl<'w, 's, Q: WorldQuery, F: WorldQuery> Query<'w, 's, Q, F>
where
F::Fetch: FilterFetch,
{
impl<'w, 's, Q: WorldQuery, F: WorldQuery> Query<'w, 's, Q, F> {
/// Creates a new query.
///
/// # Safety
@ -298,7 +292,7 @@ where
/// # bevy_ecs::system::assert_is_system(report_names_system);
/// ```
#[inline]
pub fn iter(&self) -> QueryIter<'_, 's, Q, Q::ReadOnlyFetch, F> {
pub fn iter(&self) -> QueryIter<'_, 's, Q, ROQueryFetch<'_, Q>, F> {
// SAFE: system runs without conflicts with other systems.
// same-system queries have runtime borrow checks when they conflict
unsafe {
@ -328,7 +322,7 @@ where
/// # bevy_ecs::system::assert_is_system(gravity_system);
/// ```
#[inline]
pub fn iter_mut(&mut self) -> QueryIter<'_, '_, Q, Q::Fetch, F> {
pub fn iter_mut(&mut self) -> QueryIter<'_, '_, Q, QueryFetch<'_, Q>, F> {
// SAFE: system runs without conflicts with other systems.
// same-system queries have runtime borrow checks when they conflict
unsafe {
@ -345,9 +339,7 @@ where
/// - if K < N: all possible K-sized combinations of query results, without repetition
/// - if K > N: empty set (no K-sized combinations exist)
#[inline]
pub fn iter_combinations<const K: usize>(
&self,
) -> QueryCombinationIter<'_, '_, Q, Q::ReadOnlyFetch, F, K> {
pub fn iter_combinations<const K: usize>(&self) -> QueryCombinationIter<'_, '_, Q, F, K> {
// SAFE: system runs without conflicts with other systems.
// same-system queries have runtime borrow checks when they conflict
unsafe {
@ -384,7 +376,7 @@ where
#[inline]
pub fn iter_combinations_mut<const K: usize>(
&mut self,
) -> QueryCombinationIter<'_, '_, Q, Q::Fetch, F, K> {
) -> QueryCombinationIter<'_, '_, Q, F, K> {
// SAFE: system runs without conflicts with other systems.
// same-system queries have runtime borrow checks when they conflict
unsafe {
@ -403,7 +395,7 @@ where
/// This function makes it possible to violate Rust's aliasing guarantees. You must make sure
/// this call does not result in multiple mutable references to the same component
#[inline]
pub unsafe fn iter_unsafe(&'s self) -> QueryIter<'w, 's, Q, Q::Fetch, F> {
pub unsafe fn iter_unsafe(&'s self) -> QueryIter<'w, 's, Q, QueryFetch<'w, Q>, F> {
// SEMI-SAFE: system runs without conflicts with other systems.
// same-system queries have runtime borrow checks when they conflict
self.state
@ -419,7 +411,7 @@ where
#[inline]
pub unsafe fn iter_combinations_unsafe<const K: usize>(
&self,
) -> QueryCombinationIter<'_, '_, Q, Q::Fetch, F, K> {
) -> QueryCombinationIter<'_, '_, Q, F, K> {
// SEMI-SAFE: system runs without conflicts with other systems.
// same-system queries have runtime borrow checks when they conflict
self.state.iter_combinations_unchecked_manual(
@ -453,14 +445,11 @@ where
/// # bevy_ecs::system::assert_is_system(report_names_system);
/// ```
#[inline]
pub fn for_each<'this>(
&'this self,
f: impl FnMut(<Q::ReadOnlyFetch as Fetch<'this, 's>>::Item),
) {
pub fn for_each<'this>(&'this self, f: impl FnMut(ROQueryItem<'this, Q>)) {
// SAFE: system runs without conflicts with other systems.
// same-system queries have runtime borrow checks when they conflict
unsafe {
self.state.for_each_unchecked_manual::<Q::ReadOnlyFetch, _>(
self.state.for_each_unchecked_manual::<ROQueryFetch<Q>, _>(
self.world,
f,
self.last_change_tick,
@ -491,11 +480,11 @@ where
/// # bevy_ecs::system::assert_is_system(gravity_system);
/// ```
#[inline]
pub fn for_each_mut<'a, FN: FnMut(<Q::Fetch as Fetch<'a, 'a>>::Item)>(&'a mut self, f: FN) {
pub fn for_each_mut<'a, FN: FnMut(QueryItem<'a, Q>)>(&'a mut self, f: FN) {
// SAFE: system runs without conflicts with other systems. same-system queries have runtime
// borrow checks when they conflict
unsafe {
self.state.for_each_unchecked_manual::<Q::Fetch, FN>(
self.state.for_each_unchecked_manual::<QueryFetch<Q>, FN>(
self.world,
f,
self.last_change_tick,
@ -529,13 +518,13 @@ where
&'this self,
task_pool: &TaskPool,
batch_size: usize,
f: impl Fn(<Q::ReadOnlyFetch as Fetch<'this, 's>>::Item) + Send + Sync + Clone,
f: impl Fn(ROQueryItem<'this, Q>) + Send + Sync + Clone,
) {
// SAFE: system runs without conflicts with other systems. same-system queries have runtime
// borrow checks when they conflict
unsafe {
self.state
.par_for_each_unchecked_manual::<Q::ReadOnlyFetch, _>(
.par_for_each_unchecked_manual::<ROQueryFetch<Q>, _>(
self.world,
task_pool,
batch_size,
@ -549,7 +538,7 @@ where
/// Runs `f` on each query result in parallel using the given [`TaskPool`].
/// See [`Self::par_for_each`] for more details.
#[inline]
pub fn par_for_each_mut<'a, FN: Fn(<Q::Fetch as Fetch<'a, 'a>>::Item) + Send + Sync + Clone>(
pub fn par_for_each_mut<'a, FN: Fn(QueryItem<'a, Q>) + Send + Sync + Clone>(
&'a mut self,
task_pool: &TaskPool,
batch_size: usize,
@ -558,14 +547,15 @@ where
// SAFE: system runs without conflicts with other systems. same-system queries have runtime
// borrow checks when they conflict
unsafe {
self.state.par_for_each_unchecked_manual::<Q::Fetch, FN>(
self.world,
task_pool,
batch_size,
f,
self.last_change_tick,
self.change_tick,
);
self.state
.par_for_each_unchecked_manual::<QueryFetch<Q>, FN>(
self.world,
task_pool,
batch_size,
f,
self.last_change_tick,
self.change_tick,
)
};
}
@ -601,14 +591,11 @@ where
/// # bevy_ecs::system::assert_is_system(print_selected_character_name_system);
/// ```
#[inline]
pub fn get(
&self,
entity: Entity,
) -> Result<<Q::ReadOnlyFetch as Fetch<'_, 's>>::Item, QueryEntityError> {
pub fn get(&self, entity: Entity) -> Result<ROQueryItem<'_, Q>, QueryEntityError> {
// SAFE: system runs without conflicts with other systems.
// same-system queries have runtime borrow checks when they conflict
unsafe {
self.state.get_unchecked_manual::<Q::ReadOnlyFetch>(
self.state.get_unchecked_manual::<ROQueryFetch<Q>>(
self.world,
entity,
self.last_change_tick,
@ -629,7 +616,7 @@ where
pub fn get_many<const N: usize>(
&self,
entities: [Entity; N],
) -> Result<[<Q::ReadOnlyFetch as Fetch<'_, 's>>::Item; N], QueryEntityError> {
) -> Result<[ROQueryItem<'_, Q>; N], QueryEntityError> {
// SAFE: it is the scheduler's responsibility to ensure that `Query` is never handed out on the wrong `World`.
unsafe {
self.state.get_many_read_only_manual(
@ -677,10 +664,7 @@ where
/// }
/// ```
#[inline]
pub fn many<const N: usize>(
&self,
entities: [Entity; N],
) -> [<Q::ReadOnlyFetch as Fetch<'_, 's>>::Item; N] {
pub fn many<const N: usize>(&self, entities: [Entity; N]) -> [ROQueryItem<'_, Q>; N] {
self.get_many(entities).unwrap()
}
@ -709,14 +693,11 @@ where
/// # bevy_ecs::system::assert_is_system(poison_system);
/// ```
#[inline]
pub fn get_mut(
&mut self,
entity: Entity,
) -> Result<<Q::Fetch as Fetch>::Item, QueryEntityError> {
pub fn get_mut(&mut self, entity: Entity) -> Result<QueryItem<'_, Q>, QueryEntityError> {
// SAFE: system runs without conflicts with other systems.
// same-system queries have runtime borrow checks when they conflict
unsafe {
self.state.get_unchecked_manual::<Q::Fetch>(
self.state.get_unchecked_manual::<QueryFetch<Q>>(
self.world,
entity,
self.last_change_tick,
@ -735,7 +716,7 @@ where
pub fn get_many_mut<const N: usize>(
&mut self,
entities: [Entity; N],
) -> Result<[<Q::Fetch as Fetch<'_, 's>>::Item; N], QueryEntityError> {
) -> Result<[QueryItem<'_, Q>; N], QueryEntityError> {
// SAFE: scheduler ensures safe Query world access
unsafe {
self.state.get_many_unchecked_manual(
@ -789,10 +770,7 @@ where
/// }
/// ```
#[inline]
pub fn many_mut<const N: usize>(
&mut self,
entities: [Entity; N],
) -> [<Q::Fetch as Fetch<'_, 's>>::Item; N] {
pub fn many_mut<const N: usize>(&mut self, entities: [Entity; N]) -> [QueryItem<'_, Q>; N] {
self.get_many_mut(entities).unwrap()
}
@ -809,10 +787,10 @@ where
pub unsafe fn get_unchecked(
&'s self,
entity: Entity,
) -> Result<<Q::Fetch as Fetch<'w, 's>>::Item, QueryEntityError> {
) -> Result<QueryItem<'w, Q>, QueryEntityError> {
// SEMI-SAFE: system runs without conflicts with other systems.
// same-system queries have runtime borrow checks when they conflict
self.state.get_unchecked_manual::<Q::Fetch>(
self.state.get_unchecked_manual::<QueryFetch<Q>>(
self.world,
entity,
self.last_change_tick,
@ -973,7 +951,7 @@ where
/// Panics if the number of query results is not exactly one. Use
/// [`get_single`](Self::get_single) to return a `Result` instead of panicking.
#[track_caller]
pub fn single(&self) -> <Q::ReadOnlyFetch as Fetch<'_, 's>>::Item {
pub fn single(&self) -> ROQueryItem<'_, Q> {
self.get_single().unwrap()
}
@ -1008,9 +986,7 @@ where
/// }
/// # bevy_ecs::system::assert_is_system(player_scoring_system);
/// ```
pub fn get_single(
&self,
) -> Result<<Q::ReadOnlyFetch as Fetch<'_, 's>>::Item, QuerySingleError> {
pub fn get_single(&self) -> Result<ROQueryItem<'_, Q>, QuerySingleError> {
let mut query = self.iter();
let first = query.next();
let extra = query.next().is_some();
@ -1049,7 +1025,7 @@ where
/// Panics if the number of query results is not exactly one. Use
/// [`get_single_mut`](Self::get_single_mut) to return a `Result` instead of panicking.
#[track_caller]
pub fn single_mut(&mut self) -> <Q::Fetch as Fetch<'_, '_>>::Item {
pub fn single_mut(&mut self) -> QueryItem<'_, Q> {
self.get_single_mut().unwrap()
}
@ -1075,9 +1051,7 @@ where
/// }
/// # bevy_ecs::system::assert_is_system(regenerate_player_health_system);
/// ```
pub fn get_single_mut(
&mut self,
) -> Result<<Q::Fetch as Fetch<'_, '_>>::Item, QuerySingleError> {
pub fn get_single_mut(&mut self) -> Result<QueryItem<'_, Q>, QuerySingleError> {
let mut query = self.iter_mut();
let first = query.next();
let extra = query.next().is_some();
@ -1213,8 +1187,7 @@ impl std::fmt::Display for QuerySingleError {
}
impl<'w, 's, Q: WorldQuery, F: WorldQuery> Query<'w, 's, Q, F>
where
F::Fetch: FilterFetch,
Q::Fetch: ReadOnlyFetch,
QueryFetch<'w, Q>: ReadOnlyFetch,
{
/// Returns the query result for the given [`Entity`], with the actual "inner" world lifetime.
///
@ -1248,14 +1221,11 @@ where
/// # bevy_ecs::system::assert_is_system(print_selected_character_name_system);
/// ```
#[inline]
pub fn get_inner(
&'s self,
entity: Entity,
) -> Result<<Q::ReadOnlyFetch as Fetch<'w, 's>>::Item, QueryEntityError> {
pub fn get_inner(&'s self, entity: Entity) -> Result<ROQueryItem<'w, Q>, QueryEntityError> {
// SAFE: system runs without conflicts with other systems.
// same-system queries have runtime borrow checks when they conflict
unsafe {
self.state.get_unchecked_manual::<Q::ReadOnlyFetch>(
self.state.get_unchecked_manual::<ROQueryFetch<'w, Q>>(
self.world,
entity,
self.last_change_tick,
@ -1288,7 +1258,7 @@ where
/// # bevy_ecs::system::assert_is_system(report_names_system);
/// ```
#[inline]
pub fn iter_inner(&'s self) -> QueryIter<'w, 's, Q, Q::ReadOnlyFetch, F> {
pub fn iter_inner(&'s self) -> QueryIter<'w, 's, Q, ROQueryFetch<'w, Q>, F> {
// SAFE: system runs without conflicts with other systems.
// same-system queries have runtime borrow checks when they conflict
unsafe {

52
crates/bevy_ecs/src/system/system_param.rs
View file

@ -5,8 +5,9 @@ use crate::{
change_detection::Ticks,
component::{Component, ComponentId, ComponentTicks, Components},
entity::{Entities, Entity},
ptr::UnsafeCellDeref,
query::{
Access, FilterFetch, FilteredAccess, FilteredAccessSet, QueryState, ReadOnlyFetch,
Access, FilteredAccess, FilteredAccessSet, QueryFetch, QueryState, ReadOnlyFetch,
WorldQuery,
},
system::{CommandQueue, Commands, Query, SystemMeta},
@ -87,26 +88,20 @@ pub trait SystemParamFetch<'world, 'state>: SystemParamState {
) -> Self::Item;
}
impl<'w, 's, Q: WorldQuery + 'static, F: WorldQuery + 'static> SystemParam for Query<'w, 's, Q, F>
where
F::Fetch: FilterFetch,
{
impl<'w, 's, Q: WorldQuery + 'static, F: WorldQuery + 'static> SystemParam for Query<'w, 's, Q, F> {
type Fetch = QueryState<Q, F>;
}
// SAFE: QueryState is constrained to read-only fetches, so it only reads World.
unsafe impl<Q: WorldQuery, F: WorldQuery> ReadOnlySystemParamFetch for QueryState<Q, F>
where
Q::Fetch: ReadOnlyFetch,
F::Fetch: FilterFetch,
unsafe impl<Q: WorldQuery, F: WorldQuery> ReadOnlySystemParamFetch for QueryState<Q, F> where
for<'x> QueryFetch<'x, Q>: ReadOnlyFetch
{
}
// SAFE: Relevant query ComponentId and ArchetypeComponentId access is applied to SystemMeta. If
// this QueryState conflicts with any prior access, a panic will occur.
unsafe impl<Q: WorldQuery + 'static, F: WorldQuery + 'static> SystemParamState for QueryState<Q, F>
where
F::Fetch: FilterFetch,
unsafe impl<Q: WorldQuery + 'static, F: WorldQuery + 'static> SystemParamState
for QueryState<Q, F>
{
fn init(world: &mut World, system_meta: &mut SystemMeta) -> Self {
let state = QueryState::new(world);
@ -137,8 +132,6 @@ where
impl<'w, 's, Q: WorldQuery + 'static, F: WorldQuery + 'static> SystemParamFetch<'w, 's>
for QueryState<Q, F>
where
F::Fetch: FilterFetch,
{
type Item = Query<'w, 's, Q, F>;
@ -323,8 +316,8 @@ impl<'w, 's, T: Resource> SystemParamFetch<'w, 's> for ResState<T> {
)
});
Res {
value: &*column.get_data_ptr().cast::<T>().as_ptr(),
ticks: column.get_ticks_unchecked(0),
value: column.get_data_ptr().deref::<T>(),
ticks: column.get_ticks_unchecked(0).deref(),
last_change_tick: system_meta.last_change_tick,
change_tick,
}
@ -362,8 +355,8 @@ impl<'w, 's, T: Resource> SystemParamFetch<'w, 's> for OptionResState<T> {
world
.get_populated_resource_column(state.0.component_id)
.map(|column| Res {
value: &*column.get_data_ptr().cast::<T>().as_ptr(),
ticks: column.get_ticks_unchecked(0),
value: column.get_data_ptr().deref::<T>(),
ticks: column.get_ticks_unchecked(0).deref(),
last_change_tick: system_meta.last_change_tick,
change_tick,
})
@ -881,8 +874,8 @@ impl<'w, 's, T: 'static> SystemParamFetch<'w, 's> for NonSendState<T> {
});
NonSend {
value: &*column.get_data_ptr().cast::<T>().as_ptr(),
ticks: column.get_ticks_unchecked(0).clone(),
value: column.get_data_ptr().deref::<T>(),
ticks: column.get_ticks_unchecked(0).read(),
last_change_tick: system_meta.last_change_tick,
change_tick,
}
@ -921,8 +914,8 @@ impl<'w, 's, T: 'static> SystemParamFetch<'w, 's> for OptionNonSendState<T> {
world
.get_populated_resource_column(state.0.component_id)
.map(|column| NonSend {
value: &*column.get_data_ptr().cast::<T>().as_ptr(),
ticks: column.get_ticks_unchecked(0).clone(),
value: column.get_data_ptr().deref::<T>(),
ticks: column.get_ticks_unchecked(0).read(),
last_change_tick: system_meta.last_change_tick,
change_tick,
})
@ -994,9 +987,9 @@ impl<'w, 's, T: 'static> SystemParamFetch<'w, 's> for NonSendMutState<T> {
)
});
NonSendMut {
value: &mut *column.get_data_ptr().cast::<T>().as_ptr(),
value: column.get_data_ptr().assert_unique().deref_mut::<T>(),
ticks: Ticks {
component_ticks: &mut *column.get_ticks_mut_ptr_unchecked(0),
component_ticks: column.get_ticks_unchecked(0).deref_mut(),
last_change_tick: system_meta.last_change_tick,
change_tick,
},
@ -1033,9 +1026,9 @@ impl<'w, 's, T: 'static> SystemParamFetch<'w, 's> for OptionNonSendMutState<T> {
world
.get_populated_resource_column(state.0.component_id)
.map(|column| NonSendMut {
value: &mut *column.get_data_ptr().cast::<T>().as_ptr(),
value: column.get_data_ptr().assert_unique().deref_mut::<T>(),
ticks: Ticks {
component_ticks: &mut *column.get_ticks_mut_ptr_unchecked(0),
component_ticks: column.get_ticks_unchecked(0).deref_mut(),
last_change_tick: system_meta.last_change_tick,
change_tick,
},
@ -1404,7 +1397,7 @@ mod tests {
use super::SystemParam;
use crate::{
self as bevy_ecs, // Necessary for the `SystemParam` Derive when used inside `bevy_ecs`.
query::{FilterFetch, WorldQuery},
query::WorldQuery,
system::Query,
};
@ -1415,10 +1408,7 @@ mod tests {
's,
Q: WorldQuery + Send + Sync + 'static,
F: WorldQuery + Send + Sync + 'static = (),
>
where
F::Fetch: FilterFetch,
{
> {
_query: Query<'w, 's, Q, F>,
}
}

64
crates/bevy_ecs/src/world/entity_ref.rs
View file

@ -4,10 +4,11 @@ use crate::{
change_detection::Ticks,
component::{Component, ComponentId, ComponentTicks, Components, StorageType},
entity::{Entities, Entity, EntityLocation},
ptr::{OwningPtr, Ptr},
storage::{SparseSet, Storages},
world::{Mut, World},
};
use std::any::TypeId;
use std::{any::TypeId, cell::UnsafeCell};
/// A read-only reference to a particular [`Entity`] and all of its components
pub struct EntityRef<'w> {
@ -67,7 +68,7 @@ impl<'w> EntityRef<'w> {
// SAFE: entity location is valid and returned component is of type T
unsafe {
get_component_with_type(self.world, TypeId::of::<T>(), self.entity, self.location)
.map(|value| &*value.cast::<T>())
.map(|value| value.deref::<T>())
}
}
@ -89,9 +90,9 @@ impl<'w> EntityRef<'w> {
) -> Option<Mut<'w, T>> {
get_component_and_ticks_with_type(self.world, TypeId::of::<T>(), self.entity, self.location)
.map(|(value, ticks)| Mut {
value: &mut *value.cast::<T>(),
value: value.assert_unique().deref_mut::<T>(),
ticks: Ticks {
component_ticks: &mut *ticks,
component_ticks: &mut *ticks.get(),
last_change_tick,
change_tick,
},
@ -158,7 +159,7 @@ impl<'w> EntityMut<'w> {
// SAFE: lifetimes enforce correct usage of returned borrow
unsafe {
get_component_with_type(self.world, TypeId::of::<T>(), self.entity, self.location)
.map(|value| &*value.cast::<T>())
.map(|value| value.deref::<T>())
}
}
@ -179,12 +180,12 @@ impl<'w> EntityMut<'w> {
/// may happen from **any** `insert_component`, `remove_component` or `despawn`
/// operation on this world (non-exhaustive list).
#[inline]
pub unsafe fn get_unchecked_mut<T: Component>(&self) -> Option<Mut<'w, T>> {
pub unsafe fn get_unchecked_mut<T: Component>(&self) -> Option<Mut<'_, T>> {
get_component_and_ticks_with_type(self.world, TypeId::of::<T>(), self.entity, self.location)
.map(|(value, ticks)| Mut {
value: &mut *value.cast::<T>(),
value: value.assert_unique().deref_mut::<T>(),
ticks: Ticks {
component_ticks: &mut *ticks,
component_ticks: &mut *ticks.get(),
last_change_tick: self.world.last_change_tick(),
change_tick: self.world.read_change_tick(),
},
@ -244,7 +245,7 @@ impl<'w> EntityMut<'w> {
// SAFE: bundle components are iterated in order, which guarantees that the component type
// matches
let result = unsafe {
T::from_components(|| {
T::from_components(storages, |storages| {
let component_id = bundle_components.next().unwrap();
// SAFE: entity location is valid and table row is removed below
take_component(
@ -477,7 +478,7 @@ unsafe fn get_component(
component_id: ComponentId,
entity: Entity,
location: EntityLocation,
) -> Option<*mut u8> {
) -> Option<Ptr<'_>> {
let archetype = &world.archetypes[location.archetype_id];
// SAFE: component_id exists and is therefore valid
let component_info = world.components.get_info_unchecked(component_id);
@ -508,7 +509,7 @@ unsafe fn get_component_and_ticks(
component_id: ComponentId,
entity: Entity,
location: EntityLocation,
) -> Option<(*mut u8, *mut ComponentTicks)> {
) -> Option<(Ptr<'_>, &UnsafeCell<ComponentTicks>)> {
let archetype = &world.archetypes[location.archetype_id];
let component_info = world.components.get_info_unchecked(component_id);
match component_info.storage_type() {
@ -519,7 +520,7 @@ unsafe fn get_component_and_ticks(
// SAFE: archetypes only store valid table_rows and the stored component type is T
Some((
components.get_data_unchecked(table_row),
components.get_ticks_mut_ptr_unchecked(table_row),
components.get_ticks_unchecked(table_row),
))
}
StorageType::SparseSet => world
@ -542,26 +543,26 @@ unsafe fn get_component_and_ticks(
/// - `component_id` must be valid
/// - The relevant table row **must be removed** by the caller once all components are taken
#[inline]
unsafe fn take_component(
unsafe fn take_component<'a>(
components: &Components,
storages: &mut Storages,
storages: &'a mut Storages,
archetype: &Archetype,
removed_components: &mut SparseSet<ComponentId, Vec<Entity>>,
component_id: ComponentId,
entity: Entity,
location: EntityLocation,
) -> *mut u8 {
) -> OwningPtr<'a> {
let component_info = components.get_info_unchecked(component_id);
let removed_components = removed_components.get_or_insert_with(component_id, Vec::new);
removed_components.push(entity);
match component_info.storage_type() {
StorageType::Table => {
let table = &storages.tables[archetype.table_id()];
let table = &mut storages.tables[archetype.table_id()];
// SAFE: archetypes will always point to valid columns
let components = table.get_column(component_id).unwrap();
let components = table.get_column_mut(component_id).unwrap();
let table_row = archetype.entity_table_row(location.index);
// SAFE: archetypes only store valid table_rows and the stored component type is T
components.get_data_unchecked(table_row)
components.get_data_unchecked_mut(table_row).promote()
}
StorageType::SparseSet => storages
.sparse_sets
@ -581,7 +582,7 @@ unsafe fn get_component_with_type(
type_id: TypeId,
entity: Entity,
location: EntityLocation,
) -> Option<*mut u8> {
) -> Option<Ptr<'_>> {
let component_id = world.components.get_id(type_id)?;
get_component(world, component_id, entity, location)
}
@ -595,7 +596,7 @@ pub(crate) unsafe fn get_component_and_ticks_with_type(
type_id: TypeId,
entity: Entity,
location: EntityLocation,
) -> Option<(*mut u8, *mut ComponentTicks)> {
) -> Option<(Ptr<'_>, &UnsafeCell<ComponentTicks>)> {
let component_id = world.components.get_id(type_id)?;
get_component_and_ticks(world, component_id, entity, location)
}
@ -733,6 +734,29 @@ fn sorted_remove<T: Eq + Ord + Copy>(source: &mut Vec<T>, remove: &[T]) {
});
}
// SAFETY: EntityLocation must be valid
#[inline]
pub(crate) unsafe fn get_mut<T: Component>(
world: &mut World,
entity: Entity,
location: EntityLocation,
) -> Option<Mut<'_, T>> {
// SAFE: world access is unique, entity location is valid, and returned component is of type
// T
let change_tick = world.change_tick();
let last_change_tick = world.last_change_tick();
get_component_and_ticks_with_type(world, TypeId::of::<T>(), entity, location).map(
|(value, ticks)| Mut {
value: value.assert_unique().deref_mut::<T>(),
ticks: Ticks {
component_ticks: &mut *ticks.get(),
last_change_tick,
change_tick,
},
},
)
}
#[cfg(test)]
mod tests {
#[test]

65
crates/bevy_ecs/src/world/mod.rs
View file

@ -13,7 +13,8 @@ use crate::{
change_detection::Ticks,
component::{Component, ComponentId, ComponentTicks, Components, StorageType},
entity::{AllocAtWithoutReplacement, Entities, Entity},
query::{FilterFetch, QueryState, WorldQuery},
ptr::{OwningPtr, UnsafeCellDeref},
query::{QueryState, WorldQuery},
storage::{Column, SparseSet, Storages},
system::Resource,
};
@ -21,7 +22,6 @@ use bevy_utils::tracing::debug;
use std::{
any::TypeId,
fmt,
mem::ManuallyDrop,
sync::atomic::{AtomicU32, Ordering},
};
mod identifier;
@ -438,7 +438,7 @@ impl World {
#[inline]
pub fn get_mut<T: Component>(&mut self, entity: Entity) -> Option<Mut<T>> {
// SAFE: lifetimes enforce correct usage of returned borrow
unsafe { self.get_entity_mut(entity)?.get_unchecked_mut::<T>() }
unsafe { get_mut(self, entity, self.get_entity(entity)?.location()) }
}
/// Despawns the given `entity`, if it exists. This will also remove all of the entity's
@ -545,7 +545,7 @@ impl World {
/// ```
#[inline]
pub fn query<Q: WorldQuery>(&mut self) -> QueryState<Q, ()> {
QueryState::new(self)
self.query_filtered::<Q, ()>()
}
/// Returns [`QueryState`] for the given filtered [`WorldQuery`], which is used to efficiently
@ -568,10 +568,7 @@ impl World {
/// assert_eq!(matching_entities, vec![e2]);
/// ```
#[inline]
pub fn query_filtered<Q: WorldQuery, F: WorldQuery>(&mut self) -> QueryState<Q, F>
where
F::Fetch: FilterFetch,
{
pub fn query_filtered<Q: WorldQuery, F: WorldQuery>(&mut self) -> QueryState<Q, F> {
QueryState::new(self)
}
@ -684,7 +681,7 @@ impl World {
// ptr value / drop is called when R is dropped
let (ptr, _) = unsafe { column.swap_remove_and_forget_unchecked(0) };
// SAFE: column is of type R
Some(unsafe { ptr.cast::<R>().read() })
Some(unsafe { ptr.read::<R>() })
}
/// Returns `true` if a resource of type `R` exists. Otherwise returns `false`.
@ -712,7 +709,7 @@ impl World {
return false;
};
// SAFE: resources table always have row 0
let ticks = unsafe { column.get_ticks_unchecked(0) };
let ticks = unsafe { column.get_ticks_unchecked(0).deref() };
ticks.is_added(self.last_change_tick(), self.read_change_tick())
}
@ -729,7 +726,7 @@ impl World {
return false;
};
// SAFE: resources table always have row 0
let ticks = unsafe { column.get_ticks_unchecked(0) };
let ticks = unsafe { column.get_ticks_unchecked(0).deref() };
ticks.is_changed(self.last_change_tick(), self.read_change_tick())
}
@ -1038,6 +1035,9 @@ impl World {
/// assert_eq!(world.get_resource::<A>().unwrap().0, 2);
/// ```
pub fn resource_scope<R: Resource, U>(&mut self, f: impl FnOnce(&mut World, Mut<R>) -> U) -> U {
let last_change_tick = self.last_change_tick();
let change_tick = self.change_tick();
let component_id = self
.components
.get_resource_id(TypeId::of::<R>())
@ -1057,31 +1057,32 @@ impl World {
// the ptr value / drop is called when R is dropped
unsafe { column.swap_remove_and_forget_unchecked(0) }
};
// SAFE: pointer is of type T and valid to move out of
// SAFE: pointer is of type R
// Read the value onto the stack to avoid potential mut aliasing.
let mut value = unsafe { std::ptr::read(ptr.cast::<R>()) };
let mut value = unsafe { ptr.read::<R>() };
let value_mut = Mut {
value: &mut value,
ticks: Ticks {
component_ticks: &mut ticks,
last_change_tick: self.last_change_tick(),
change_tick: self.change_tick(),
last_change_tick,
change_tick,
},
};
let result = f(self, value_mut);
assert!(!self.contains_resource::<R>());
let resource_archetype = self.archetypes.resource_mut();
let unique_components = resource_archetype.unique_components_mut();
let column = unique_components
.get_mut(component_id)
.unwrap_or_else(|| panic!("resource does not exist: {}", std::any::type_name::<R>()));
// Wrap the value in MaybeUninit to prepare for passing the value back into the ECS
let mut nodrop_wrapped_value = std::mem::MaybeUninit::new(value);
unsafe {
// SAFE: pointer is of type T, and valid to move out of
column.push(nodrop_wrapped_value.as_mut_ptr() as *mut _, ticks);
}
OwningPtr::make(value, |ptr| {
unsafe {
// SAFE: pointer is of type R
column.push(ptr, ticks);
}
});
result
}
@ -1093,7 +1094,7 @@ impl World {
component_id: ComponentId,
) -> Option<&R> {
let column = self.get_populated_resource_column(component_id)?;
Some(&*column.get_data_ptr().as_ptr().cast::<R>())
Some(column.get_data_ptr().deref::<R>())
}
/// # Safety
@ -1106,9 +1107,9 @@ impl World {
) -> Option<Mut<'_, R>> {
let column = self.get_populated_resource_column(component_id)?;
Some(Mut {
value: &mut *column.get_data_ptr().cast::<R>().as_ptr(),
value: column.get_data_ptr().assert_unique().deref_mut(),
ticks: Ticks {
component_ticks: &mut *column.get_ticks_mut_ptr_unchecked(0),
component_ticks: column.get_ticks_unchecked(0).deref_mut(),
last_change_tick: self.last_change_tick(),
change_tick: self.read_change_tick(),
},
@ -1145,13 +1146,13 @@ impl World {
let change_tick = self.change_tick();
let column = self.initialize_resource_internal(component_id);
if column.is_empty() {
let mut value = ManuallyDrop::new(value);
// SAFE: column is of type R and has been allocated above
let data = (&mut *value as *mut R).cast::<u8>();
column.push(data, ComponentTicks::new(change_tick));
OwningPtr::make(value, |ptr| {
column.push(ptr, ComponentTicks::new(change_tick));
});
} else {
// SAFE: column is of type R and has already been allocated
*column.get_data_unchecked(0).cast::<R>() = value;
*column.get_data_unchecked_mut(0).deref_mut::<R>() = value;
column.get_ticks_unchecked_mut(0).set_changed(change_tick);
}
}
@ -1422,10 +1423,7 @@ mod tests {
}
pub fn finish(self, panic_res: std::thread::Result<()>) -> Vec<DropLogItem> {
let drop_log = Arc::try_unwrap(self.drop_log)
.unwrap()
.into_inner()
.unwrap();
let drop_log = self.drop_log.lock().unwrap();
let expected_panic_flag = self.expected_panic_flag.load(Ordering::SeqCst);
if !expected_panic_flag {
@ -1435,7 +1433,7 @@ mod tests {
}
}
drop_log
drop_log.to_owned()
}
}
@ -1461,7 +1459,6 @@ mod tests {
DropLogItem::Create(0),
DropLogItem::Create(1),
DropLogItem::Drop(0),
DropLogItem::Drop(1)
]
);
}

11
crates/bevy_render/src/render_component.rs
View file

@ -8,7 +8,7 @@ use bevy_asset::{Asset, Handle};
use bevy_ecs::{
component::Component,
prelude::*,
query::{FilterFetch, QueryItem, WorldQuery},
query::{QueryItem, WorldQuery},
system::{lifetimeless::Read, StaticSystemParam},
};
use std::{marker::PhantomData, ops::Deref};
@ -139,10 +139,7 @@ impl<C, F> Default for ExtractComponentPlugin<C, F> {
}
}
impl<C: ExtractComponent> Plugin for ExtractComponentPlugin<C>
where
<C::Filter as WorldQuery>::Fetch: FilterFetch,
{
impl<C: ExtractComponent> Plugin for ExtractComponentPlugin<C> {
fn build(&self, app: &mut App) {
if let Ok(render_app) = app.get_sub_app_mut(RenderApp) {
render_app.add_system_to_stage(RenderStage::Extract, extract_components::<C>);
@ -165,9 +162,7 @@ fn extract_components<C: ExtractComponent>(
mut commands: Commands,
mut previous_len: Local<usize>,
mut query: StaticSystemParam<Query<(Entity, C::Query), C::Filter>>,
) where
<C::Filter as WorldQuery>::Fetch: FilterFetch,
{
) {
let mut values = Vec::with_capacity(*previous_len);
for (entity, query_item) in query.iter_mut() {
values.push((entity, (C::extract_component(query_item),)));

6
crates/bevy_ui/src/flex/mod.rs
View file

@ -4,7 +4,7 @@ use crate::{CalculatedSize, Node, Style};
use bevy_ecs::{
entity::Entity,
event::EventReader,
query::{Changed, FilterFetch, With, Without, WorldQuery},
query::{Changed, With, Without, WorldQuery},
system::{Query, Res, ResMut},
};
use bevy_hierarchy::{Children, Parent};
@ -235,9 +235,7 @@ pub fn flex_node_system(
flex_surface: &mut FlexSurface,
scaling_factor: f64,
query: Query<(Entity, &Style, Option<&CalculatedSize>), F>,
) where
F::Fetch: FilterFetch,
{
) {
// update changed nodes
for (entity, style, calculated_size) in query.iter() {
// TODO: remove node from old hierarchy if its root has changed

54
examples/ecs/custom_query_param.rs
View file

@ -2,7 +2,7 @@ use bevy::{
ecs::{component::Component, query::WorldQuery},
prelude::*,
};
use std::{fmt::Debug, marker::PhantomData};
use std::fmt::Debug;
/// This examples illustrates the usage of the `WorldQuery` derive macro, which allows
/// defining custom query and filter types.
@ -40,15 +40,15 @@ struct ComponentZ;
#[derive(WorldQuery)]
#[world_query(derive(Debug))]
struct ReadOnlyCustomQuery<'w, T: Component + Debug, P: Component + Debug> {
struct ReadOnlyCustomQuery<T: Component + Debug, P: Component + Debug> {
entity: Entity,
a: &'w ComponentA,
b: Option<&'w ComponentB>,
nested: NestedQuery<'w>,
optional_nested: Option<NestedQuery<'w>>,
optional_tuple: Option<(&'w ComponentB, &'w ComponentZ)>,
generic: GenericQuery<'w, T, P>,
empty: EmptyQuery<'w>,
a: &'static ComponentA,
b: Option<&'static ComponentB>,
nested: NestedQuery,
optional_nested: Option<NestedQuery>,
optional_tuple: Option<(&'static ComponentB, &'static ComponentZ)>,
generic: GenericQuery<T, P>,
empty: EmptyQuery,
}
fn print_components_read_only(
@ -74,49 +74,43 @@ fn print_components_read_only(
// using the `derive` attribute.
#[derive(WorldQuery)]
#[world_query(mutable, derive(Debug))]
struct CustomQuery<'w, T: Component + Debug, P: Component + Debug> {
struct CustomQuery<T: Component + Debug, P: Component + Debug> {
entity: Entity,
a: &'w mut ComponentA,
b: Option<&'w mut ComponentB>,
nested: NestedQuery<'w>,
optional_nested: Option<NestedQuery<'w>>,
optional_tuple: Option<(NestedQuery<'w>, &'w mut ComponentZ)>,
generic: GenericQuery<'w, T, P>,
empty: EmptyQuery<'w>,
a: &'static mut ComponentA,
b: Option<&'static mut ComponentB>,
nested: NestedQuery,
optional_nested: Option<NestedQuery>,
optional_tuple: Option<(NestedQuery, &'static mut ComponentZ)>,
generic: GenericQuery<T, P>,
empty: EmptyQuery,
}
// This is a valid query as well, which would iterate over every entity.
#[derive(WorldQuery)]
#[world_query(derive(Debug))]
struct EmptyQuery<'w> {
// The derive macro expect a lifetime. As Rust doesn't allow unused lifetimes, we need
// to use `PhantomData` as a work around.
#[world_query(ignore)]
_w: std::marker::PhantomData<&'w ()>,
struct EmptyQuery {
empty: (),
}
#[derive(WorldQuery)]
#[world_query(derive(Debug))]
struct NestedQuery<'w> {
c: &'w ComponentC,
d: Option<&'w ComponentD>,
struct NestedQuery {
c: &'static ComponentC,
d: Option<&'static ComponentD>,
}
#[derive(WorldQuery)]
#[world_query(derive(Debug))]
struct GenericQuery<'w, T: Component, P: Component> {
generic: (&'w T, &'w P),
struct GenericQuery<T: Component, P: Component> {
generic: (&'static T, &'static P),
}
#[derive(WorldQuery)]
#[world_query(filter)]
struct QueryFilter<T: Component, P: Component> {
_c: With<ComponentC>,
_d: With<ComponentD>,
_or: Or<(Added<ComponentC>, Changed<ComponentD>, Without<ComponentZ>)>,
_generic_tuple: (With<T>, With<P>),
#[world_query(ignore)]
_tp: PhantomData<(T, P)>,
}
fn spawn(mut commands: Commands) {