Proper prehashing (#3963)
For some keys, it is too expensive to hash them on every lookup. Historically in Bevy, we have regrettably done the "wrong" thing in these cases (pre-computing hashes, then re-hashing them) because Rust's built in hashed collections don't give us the tools we need to do otherwise. Doing this is "wrong" because two different values can result in the same hash. Hashed collections generally get around this by falling back to equality checks on hash collisions. You can't do that if the key _is_ the hash. Additionally, re-hashing a hash increase the odds of collision!
#3959 needs pre-hashing to be viable, so I decided to finally properly solve the problem. The solution involves two different changes:
1. A new generalized "pre-hashing" solution in bevy_utils: `Hashed<T>` types, which store a value alongside a pre-computed hash. And `PreHashMap<K, V>` (which uses `Hashed<T>` internally) . `PreHashMap` is just an alias for a normal HashMap that uses `Hashed<T>` as the key and a new `PassHash` implementation as the Hasher.
2. Replacing the `std::collections` re-exports in `bevy_utils` with equivalent `hashbrown` impls. Avoiding re-hashes requires the `raw_entry_mut` api, which isn't stabilized yet (and may never be ... `entry_ref` has favor now, but also isn't available yet). If std's HashMap ever provides the tools we need, we can move back to that. The latest version of `hashbrown` adds support for the `entity_ref` api, so we can move to that in preparation for an std migration, if thats the direction they seem to be going in. Note that adding hashbrown doesn't increase our dependency count because it was already in our tree.
In addition to providing these core tools, I also ported the "table identity hashing" in `bevy_ecs` to `raw_entry_mut`, which was a particularly egregious case.
The biggest outstanding case is `AssetPathId`, which stores a pre-hash. We need AssetPathId to be cheaply clone-able (and ideally Copy), but `Hashed<AssetPath>` requires ownership of the AssetPath, which makes cloning ids way more expensive. We could consider doing `Hashed<Arc<AssetPath>>`, but cloning an arc is still a non-trivial expensive that needs to be considered. I would like to handle this in a separate PR. And given that we will be re-evaluating the Bevy Assets implementation in the very near future, I'd prefer to hold off until after that conversation is concluded.
2022-02-18 03:26:01 +00:00
|
|
|
use std::any::Any;
|
2020-11-28 00:39:59 +00:00
|
|
|
|
Proper prehashing (#3963)
For some keys, it is too expensive to hash them on every lookup. Historically in Bevy, we have regrettably done the "wrong" thing in these cases (pre-computing hashes, then re-hashing them) because Rust's built in hashed collections don't give us the tools we need to do otherwise. Doing this is "wrong" because two different values can result in the same hash. Hashed collections generally get around this by falling back to equality checks on hash collisions. You can't do that if the key _is_ the hash. Additionally, re-hashing a hash increase the odds of collision!
#3959 needs pre-hashing to be viable, so I decided to finally properly solve the problem. The solution involves two different changes:
1. A new generalized "pre-hashing" solution in bevy_utils: `Hashed<T>` types, which store a value alongside a pre-computed hash. And `PreHashMap<K, V>` (which uses `Hashed<T>` internally) . `PreHashMap` is just an alias for a normal HashMap that uses `Hashed<T>` as the key and a new `PassHash` implementation as the Hasher.
2. Replacing the `std::collections` re-exports in `bevy_utils` with equivalent `hashbrown` impls. Avoiding re-hashes requires the `raw_entry_mut` api, which isn't stabilized yet (and may never be ... `entry_ref` has favor now, but also isn't available yet). If std's HashMap ever provides the tools we need, we can move back to that. The latest version of `hashbrown` adds support for the `entity_ref` api, so we can move to that in preparation for an std migration, if thats the direction they seem to be going in. Note that adding hashbrown doesn't increase our dependency count because it was already in our tree.
In addition to providing these core tools, I also ported the "table identity hashing" in `bevy_ecs` to `raw_entry_mut`, which was a particularly egregious case.
The biggest outstanding case is `AssetPathId`, which stores a pre-hash. We need AssetPathId to be cheaply clone-able (and ideally Copy), but `Hashed<AssetPath>` requires ownership of the AssetPath, which makes cloning ids way more expensive. We could consider doing `Hashed<Arc<AssetPath>>`, but cloning an arc is still a non-trivial expensive that needs to be considered. I would like to handle this in a separate PR. And given that we will be re-evaluating the Bevy Assets implementation in the very near future, I'd prefer to hold off until after that conversation is concluded.
2022-02-18 03:26:01 +00:00
|
|
|
use bevy_utils::{Entry, HashMap};
|
2020-11-28 00:39:59 +00:00
|
|
|
|
|
|
|
use crate::{serde::Serializable, Reflect, ReflectMut, ReflectRef};
|
|
|
|
|
2022-01-14 19:09:44 +00:00
|
|
|
/// An ordered mapping between [`Reflect`] values.
|
|
|
|
///
|
|
|
|
/// Because the values are reflected, the underlying types of keys and values
|
|
|
|
/// may differ between entries.
|
|
|
|
///
|
|
|
|
///`ReflectValue` `Keys` are assumed to return a non-`None` hash. The ordering
|
|
|
|
/// of `Map` entries is not guaranteed to be stable across runs or between
|
|
|
|
/// instances.
|
|
|
|
///
|
|
|
|
/// This trait corresponds to types like [`std::collections::HashMap`].
|
2020-11-28 00:39:59 +00:00
|
|
|
pub trait Map: Reflect {
|
2022-01-14 19:09:44 +00:00
|
|
|
/// Returns a reference to the value associated with the given key.
|
|
|
|
///
|
|
|
|
/// If no value is associated with `key`, returns `None`.
|
2020-11-28 00:39:59 +00:00
|
|
|
fn get(&self, key: &dyn Reflect) -> Option<&dyn Reflect>;
|
2022-01-14 19:09:44 +00:00
|
|
|
|
|
|
|
/// Returns a mutable reference to the value associated with the given key.
|
|
|
|
///
|
|
|
|
/// If no value is associated with `key`, returns `None`.
|
2020-11-28 00:39:59 +00:00
|
|
|
fn get_mut(&mut self, key: &dyn Reflect) -> Option<&mut dyn Reflect>;
|
2022-01-14 19:09:44 +00:00
|
|
|
|
|
|
|
/// Returns the key-value pair at `index` by reference, or `None` if out of bounds.
|
2020-11-28 00:39:59 +00:00
|
|
|
fn get_at(&self, index: usize) -> Option<(&dyn Reflect, &dyn Reflect)>;
|
2022-01-14 19:09:44 +00:00
|
|
|
|
|
|
|
/// Returns the number of elements in the map.
|
2020-11-28 00:39:59 +00:00
|
|
|
fn len(&self) -> usize;
|
2022-01-14 19:09:44 +00:00
|
|
|
|
|
|
|
/// Returns `true` if the list contains no elements.
|
2020-11-28 00:39:59 +00:00
|
|
|
fn is_empty(&self) -> bool {
|
|
|
|
self.len() == 0
|
|
|
|
}
|
2022-01-14 19:09:44 +00:00
|
|
|
|
|
|
|
/// Returns an iterator over the key-value pairs of the map.
|
2020-11-28 00:39:59 +00:00
|
|
|
fn iter(&self) -> MapIter;
|
2022-01-14 19:09:44 +00:00
|
|
|
|
|
|
|
/// Clones the map, producing a [`DynamicMap`].
|
2020-11-28 00:39:59 +00:00
|
|
|
fn clone_dynamic(&self) -> DynamicMap;
|
|
|
|
}
|
|
|
|
|
|
|
|
const HASH_ERROR: &str = "the given key does not support hashing";
|
|
|
|
|
2022-01-14 19:09:44 +00:00
|
|
|
/// An ordered mapping between reflected values.
|
2020-11-28 00:39:59 +00:00
|
|
|
#[derive(Default)]
|
|
|
|
pub struct DynamicMap {
|
2021-02-02 21:57:26 +00:00
|
|
|
name: String,
|
|
|
|
values: Vec<(Box<dyn Reflect>, Box<dyn Reflect>)>,
|
|
|
|
indices: HashMap<u64, usize>,
|
2020-11-28 00:39:59 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
impl DynamicMap {
|
2022-01-14 19:09:44 +00:00
|
|
|
/// Returns the type name of the map.
|
|
|
|
///
|
|
|
|
/// The value returned by this method is the same value returned by
|
|
|
|
/// [`Reflect::type_name`].
|
2021-02-02 21:57:26 +00:00
|
|
|
pub fn name(&self) -> &str {
|
|
|
|
&self.name
|
|
|
|
}
|
|
|
|
|
2022-01-14 19:09:44 +00:00
|
|
|
/// Sets the type name of the map.
|
|
|
|
///
|
|
|
|
/// The value set by this method is the same value returned by
|
|
|
|
/// [`Reflect::type_name`].
|
2021-02-02 21:57:26 +00:00
|
|
|
pub fn set_name(&mut self, name: String) {
|
|
|
|
self.name = name;
|
|
|
|
}
|
|
|
|
|
2022-01-14 19:09:44 +00:00
|
|
|
/// Inserts a typed key-value pair into the map.
|
2020-11-28 00:39:59 +00:00
|
|
|
pub fn insert<K: Reflect, V: Reflect>(&mut self, key: K, value: V) {
|
|
|
|
self.insert_boxed(Box::new(key), Box::new(value));
|
|
|
|
}
|
|
|
|
|
2022-01-14 19:09:44 +00:00
|
|
|
/// Inserts a key-value pair of [`Reflect`] values into the map.
|
2020-11-28 00:39:59 +00:00
|
|
|
pub fn insert_boxed(&mut self, key: Box<dyn Reflect>, value: Box<dyn Reflect>) {
|
2020-12-01 19:15:07 +00:00
|
|
|
match self.indices.entry(key.reflect_hash().expect(HASH_ERROR)) {
|
2020-11-28 00:39:59 +00:00
|
|
|
Entry::Occupied(entry) => {
|
|
|
|
self.values[*entry.get()] = (key, value);
|
|
|
|
}
|
|
|
|
Entry::Vacant(entry) => {
|
|
|
|
entry.insert(self.values.len());
|
|
|
|
self.values.push((key, value));
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
impl Map for DynamicMap {
|
|
|
|
fn get(&self, key: &dyn Reflect) -> Option<&dyn Reflect> {
|
|
|
|
self.indices
|
2020-12-01 19:15:07 +00:00
|
|
|
.get(&key.reflect_hash().expect(HASH_ERROR))
|
2020-11-28 00:39:59 +00:00
|
|
|
.map(|index| &*self.values.get(*index).unwrap().1)
|
|
|
|
}
|
|
|
|
|
|
|
|
fn get_mut(&mut self, key: &dyn Reflect) -> Option<&mut dyn Reflect> {
|
|
|
|
self.indices
|
2020-12-01 19:15:07 +00:00
|
|
|
.get(&key.reflect_hash().expect(HASH_ERROR))
|
2020-11-28 00:39:59 +00:00
|
|
|
.cloned()
|
|
|
|
.map(move |index| &mut *self.values.get_mut(index).unwrap().1)
|
|
|
|
}
|
|
|
|
|
|
|
|
fn len(&self) -> usize {
|
|
|
|
self.values.len()
|
|
|
|
}
|
|
|
|
|
|
|
|
fn clone_dynamic(&self) -> DynamicMap {
|
|
|
|
DynamicMap {
|
2021-02-02 21:57:26 +00:00
|
|
|
name: self.name.clone(),
|
2020-11-28 00:39:59 +00:00
|
|
|
values: self
|
|
|
|
.values
|
|
|
|
.iter()
|
|
|
|
.map(|(key, value)| (key.clone_value(), value.clone_value()))
|
|
|
|
.collect(),
|
|
|
|
indices: self.indices.clone(),
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
fn iter(&self) -> MapIter {
|
|
|
|
MapIter {
|
|
|
|
map: self,
|
|
|
|
index: 0,
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
fn get_at(&self, index: usize) -> Option<(&dyn Reflect, &dyn Reflect)> {
|
|
|
|
self.values
|
|
|
|
.get(index)
|
|
|
|
.map(|(key, value)| (&**key, &**value))
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2021-03-17 22:46:46 +00:00
|
|
|
// SAFE: any and any_mut both return self
|
|
|
|
unsafe impl Reflect for DynamicMap {
|
2020-11-28 00:39:59 +00:00
|
|
|
fn type_name(&self) -> &str {
|
2021-02-02 21:57:26 +00:00
|
|
|
&self.name
|
2020-11-28 00:39:59 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
fn any(&self) -> &dyn Any {
|
|
|
|
self
|
|
|
|
}
|
|
|
|
|
|
|
|
fn any_mut(&mut self) -> &mut dyn Any {
|
|
|
|
self
|
|
|
|
}
|
|
|
|
|
2022-04-25 13:54:48 +00:00
|
|
|
#[inline]
|
|
|
|
fn as_reflect(&self) -> &dyn Reflect {
|
|
|
|
self
|
|
|
|
}
|
|
|
|
|
|
|
|
#[inline]
|
|
|
|
fn as_reflect_mut(&mut self) -> &mut dyn Reflect {
|
|
|
|
self
|
|
|
|
}
|
|
|
|
|
2020-11-28 00:39:59 +00:00
|
|
|
fn apply(&mut self, value: &dyn Reflect) {
|
|
|
|
if let ReflectRef::Map(map_value) = value.reflect_ref() {
|
|
|
|
for (key, value) in map_value.iter() {
|
|
|
|
if let Some(v) = self.get_mut(key) {
|
2022-02-13 22:33:55 +00:00
|
|
|
v.apply(value);
|
2020-11-28 00:39:59 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
} else {
|
2020-12-02 19:31:16 +00:00
|
|
|
panic!("Attempted to apply a non-map type to a map type.");
|
2020-11-28 00:39:59 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
fn set(&mut self, value: Box<dyn Reflect>) -> Result<(), Box<dyn Reflect>> {
|
|
|
|
*self = value.take()?;
|
|
|
|
Ok(())
|
|
|
|
}
|
|
|
|
|
|
|
|
fn reflect_ref(&self) -> ReflectRef {
|
|
|
|
ReflectRef::Map(self)
|
|
|
|
}
|
|
|
|
|
|
|
|
fn reflect_mut(&mut self) -> ReflectMut {
|
|
|
|
ReflectMut::Map(self)
|
|
|
|
}
|
|
|
|
|
|
|
|
fn clone_value(&self) -> Box<dyn Reflect> {
|
|
|
|
Box::new(self.clone_dynamic())
|
|
|
|
}
|
|
|
|
|
2020-12-01 19:15:07 +00:00
|
|
|
fn reflect_hash(&self) -> Option<u64> {
|
2020-11-28 00:39:59 +00:00
|
|
|
None
|
|
|
|
}
|
|
|
|
|
2020-12-01 19:15:07 +00:00
|
|
|
fn reflect_partial_eq(&self, value: &dyn Reflect) -> Option<bool> {
|
2020-11-28 00:39:59 +00:00
|
|
|
map_partial_eq(self, value)
|
|
|
|
}
|
|
|
|
|
|
|
|
fn serializable(&self) -> Option<Serializable> {
|
|
|
|
None
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2022-01-14 19:09:44 +00:00
|
|
|
/// An iterator over the key-value pairs of a [`Map`].
|
2020-11-28 00:39:59 +00:00
|
|
|
pub struct MapIter<'a> {
|
|
|
|
pub(crate) map: &'a dyn Map,
|
|
|
|
pub(crate) index: usize,
|
|
|
|
}
|
|
|
|
|
|
|
|
impl<'a> Iterator for MapIter<'a> {
|
|
|
|
type Item = (&'a dyn Reflect, &'a dyn Reflect);
|
|
|
|
|
|
|
|
fn next(&mut self) -> Option<Self::Item> {
|
|
|
|
let value = self.map.get_at(self.index);
|
|
|
|
self.index += 1;
|
|
|
|
value
|
|
|
|
}
|
2021-04-13 01:28:14 +00:00
|
|
|
|
|
|
|
fn size_hint(&self) -> (usize, Option<usize>) {
|
|
|
|
let size = self.map.len();
|
|
|
|
(size, Some(size))
|
|
|
|
}
|
2020-11-28 00:39:59 +00:00
|
|
|
}
|
|
|
|
|
2022-04-26 00:17:38 +00:00
|
|
|
impl IntoIterator for DynamicMap {
|
|
|
|
type Item = (Box<dyn Reflect>, Box<dyn Reflect>);
|
|
|
|
type IntoIter = std::vec::IntoIter<Self::Item>;
|
|
|
|
|
|
|
|
fn into_iter(self) -> Self::IntoIter {
|
|
|
|
self.values.into_iter()
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2021-04-13 01:28:14 +00:00
|
|
|
impl<'a> ExactSizeIterator for MapIter<'a> {}
|
|
|
|
|
2022-01-14 19:09:44 +00:00
|
|
|
/// Compares a [`Map`] with a [`Reflect`] value.
|
|
|
|
///
|
|
|
|
/// Returns true if and only if all of the following are true:
|
|
|
|
/// - `b` is a map;
|
|
|
|
/// - `b` is the same length as `a`;
|
|
|
|
/// - For each key-value pair in `a`, `b` contains a value for the given key,
|
|
|
|
/// and [`Reflect::reflect_partial_eq`] returns `Some(true)` for the two values.
|
2020-11-28 00:39:59 +00:00
|
|
|
#[inline]
|
|
|
|
pub fn map_partial_eq<M: Map>(a: &M, b: &dyn Reflect) -> Option<bool> {
|
|
|
|
let map = if let ReflectRef::Map(map) = b.reflect_ref() {
|
|
|
|
map
|
|
|
|
} else {
|
|
|
|
return Some(false);
|
|
|
|
};
|
|
|
|
|
|
|
|
if a.len() != map.len() {
|
|
|
|
return Some(false);
|
|
|
|
}
|
|
|
|
|
|
|
|
for (key, value) in a.iter() {
|
|
|
|
if let Some(map_value) = map.get(key) {
|
2020-12-01 19:15:07 +00:00
|
|
|
if let Some(false) | None = value.reflect_partial_eq(map_value) {
|
2020-11-28 00:39:59 +00:00
|
|
|
return Some(false);
|
|
|
|
}
|
|
|
|
} else {
|
|
|
|
return Some(false);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
Some(true)
|
|
|
|
}
|
2022-04-26 00:17:38 +00:00
|
|
|
|
|
|
|
#[cfg(test)]
|
|
|
|
mod tests {
|
|
|
|
use super::DynamicMap;
|
|
|
|
|
|
|
|
#[test]
|
|
|
|
fn test_into_iter() {
|
|
|
|
let expected = vec!["foo", "bar", "baz"];
|
|
|
|
|
|
|
|
let mut map = DynamicMap::default();
|
|
|
|
map.insert(0usize, expected[0].to_string());
|
|
|
|
map.insert(1usize, expected[1].to_string());
|
|
|
|
map.insert(2usize, expected[2].to_string());
|
|
|
|
|
|
|
|
for (index, item) in map.into_iter().enumerate() {
|
|
|
|
let key = item.0.take::<usize>().expect("couldn't downcast to usize");
|
|
|
|
let value = item
|
|
|
|
.1
|
|
|
|
.take::<String>()
|
|
|
|
.expect("couldn't downcast to String");
|
|
|
|
assert_eq!(index, key);
|
|
|
|
assert_eq!(expected[index], value);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|