bevy/examples/asset/processing/asset_processing.rs
Joseph 5876352206
Optimize common usages of AssetReader (#14082)
# Objective

The `AssetReader` trait allows customizing the behavior of fetching
bytes for an `AssetPath`, and expects implementors to return `dyn
AsyncRead + AsyncSeek`. This gives implementors of `AssetLoader` great
flexibility to tightly integrate their asset loading behavior with the
asynchronous task system.

However, almost all implementors of `AssetLoader` don't use the async
functionality at all, and just call `AsyncReadExt::read_to_end(&mut
Vec<u8>)`. This is incredibly inefficient, as this method repeatedly
calls `poll_read` on the trait object, filling the vector 32 bytes at a
time. At my work we have assets that are hundreds of megabytes which
makes this a meaningful overhead.

## Solution

Turn the `Reader` type alias into an actual trait, with a provided
method `read_to_end`. This provided method should be more efficient than
the existing extension method, as the compiler will know the underlying
type of `Reader` when generating this function, which removes the
repeated dynamic dispatches and allows the compiler to make further
optimizations after inlining. Individual implementors are able to
override the provided implementation -- for simple asset readers that
just copy bytes from one buffer to another, this allows removing a large
amount of overhead from the provided implementation.

Now that `Reader` is an actual trait, I also improved the ergonomics for
implementing `AssetReader`. Currently, implementors are expected to box
their reader and return it as a trait object, which adds unnecessary
boilerplate to implementations. This PR changes that trait method to
return a pseudo trait alias, which allows implementors to return `impl
Reader` instead of `Box<dyn Reader>`. Now, the boilerplate for boxing
occurs in `ErasedAssetReader`.

## Testing

I made identical changes to my company's fork of bevy. Our app, which
makes heavy use of `read_to_end` for asset loading, still worked
properly after this. I am not aware if we have a more systematic way of
testing asset loading for correctness.

---

## Migration Guide

The trait method `bevy_asset::io::AssetReader::read` (and `read_meta`)
now return an opaque type instead of a boxed trait object. Implementors
of these methods should change the type signatures appropriately

```rust
impl AssetReader for MyReader {
    // Before
    async fn read<'a>(&'a self, path: &'a Path) -> Result<Box<Reader<'a>>, AssetReaderError> {
        let reader = // construct a reader
        Box::new(reader) as Box<Reader<'a>>
    }

    // After
    async fn read<'a>(&'a self, path: &'a Path) -> Result<impl Reader + 'a, AssetReaderError> {
        // create a reader
    }
}
```

`bevy::asset::io::Reader` is now a trait, rather than a type alias for a
trait object. Implementors of `AssetLoader::load` will need to adjust
the method signature accordingly

```rust
impl AssetLoader for MyLoader {
    async fn load<'a>(
        &'a self,
        // Before:
        reader: &'a mut bevy::asset::io::Reader,
        // After:
        reader: &'a mut dyn bevy::asset::io::Reader,
        _: &'a Self::Settings,
        load_context: &'a mut LoadContext<'_>,
    ) -> Result<Self::Asset, Self::Error> {
}
```

Additionally, implementors of `AssetReader` that return a type
implementing `futures_io::AsyncRead` and `AsyncSeek` might need to
explicitly implement `bevy::asset::io::Reader` for that type.

```rust
impl bevy::asset::io::Reader for MyAsyncReadAndSeek {}
```
2024-07-01 19:59:42 +00:00

264 lines
8.6 KiB
Rust

//! This example illustrates how to define custom `AssetLoader`s, `AssetTransformer`s, and `AssetSaver`s, how to configure them, and how to register asset processors.
use bevy::{
asset::{
embedded_asset,
io::{Reader, Writer},
processor::LoadTransformAndSave,
saver::{AssetSaver, SavedAsset},
transformer::{AssetTransformer, TransformedAsset},
AssetLoader, AsyncWriteExt, LoadContext,
},
prelude::*,
reflect::TypePath,
};
use serde::{Deserialize, Serialize};
use std::convert::Infallible;
use thiserror::Error;
fn main() {
App::new()
// Using the "processed" mode will configure the AssetPlugin to use asset processing.
// If you also enable the `asset_processor` cargo feature, this will run the AssetProcessor
// in the background, run them through configured asset processors, and write the results to
// the `imported_assets` folder. If you also enable the `file_watcher` cargo feature, changes to the
// source assets will be detected and they will be reprocessed.
//
// The AssetProcessor will create `.meta` files automatically for assets in the `assets` folder,
// which can then be used to configure how the asset will be processed.
.add_plugins((
DefaultPlugins.set(AssetPlugin {
mode: AssetMode::Processed,
// This is just overriding the default paths to scope this to the correct example folder
// You can generally skip this in your own projects
file_path: "examples/asset/processing/assets".to_string(),
processed_file_path: "examples/asset/processing/imported_assets/Default"
.to_string(),
..default()
}),
TextPlugin,
))
.add_systems(Startup, setup)
.add_systems(Update, print_text)
.run();
}
/// This [`TextPlugin`] defines two assets types:
/// * [`CoolText`]: a custom RON text format that supports dependencies and embedded dependencies
/// * [`Text`]: a "normal" plain text file
///
/// It also defines an asset processor that will load [`CoolText`], resolve embedded dependencies, and write the resulting
/// output to a "normal" plain text file. When the processed asset is loaded, it is loaded as a Text (plaintext) asset.
/// This illustrates that when you process an asset, you can change its type! However you don't _need_ to change the type.
struct TextPlugin;
impl Plugin for TextPlugin {
fn build(&self, app: &mut App) {
embedded_asset!(app, "examples/asset/processing/", "e.txt");
app.init_asset::<CoolText>()
.init_asset::<Text>()
.register_asset_loader(CoolTextLoader)
.register_asset_loader(TextLoader)
.register_asset_processor::<LoadTransformAndSave<CoolTextLoader, CoolTextTransformer, CoolTextSaver>>(
LoadTransformAndSave::new(CoolTextTransformer, CoolTextSaver),
)
.set_default_asset_processor::<LoadTransformAndSave<CoolTextLoader, CoolTextTransformer, CoolTextSaver>>("cool.ron");
}
}
#[derive(Asset, TypePath, Debug)]
struct Text(String);
#[derive(Default)]
struct TextLoader;
#[derive(Clone, Default, Serialize, Deserialize)]
struct TextSettings {
text_override: Option<String>,
}
impl AssetLoader for TextLoader {
type Asset = Text;
type Settings = TextSettings;
type Error = std::io::Error;
async fn load<'a>(
&'a self,
reader: &'a mut dyn Reader,
settings: &'a TextSettings,
_load_context: &'a mut LoadContext<'_>,
) -> Result<Text, Self::Error> {
let mut bytes = Vec::new();
reader.read_to_end(&mut bytes).await?;
let value = if let Some(ref text) = settings.text_override {
text.clone()
} else {
String::from_utf8(bytes).unwrap()
};
Ok(Text(value))
}
fn extensions(&self) -> &[&str] {
&["txt"]
}
}
#[derive(Serialize, Deserialize)]
struct CoolTextRon {
text: String,
dependencies: Vec<String>,
embedded_dependencies: Vec<String>,
dependencies_with_settings: Vec<(String, TextSettings)>,
}
#[derive(Asset, TypePath, Debug)]
struct CoolText {
text: String,
#[allow(unused)]
dependencies: Vec<Handle<Text>>,
}
#[derive(Default)]
struct CoolTextLoader;
#[derive(Debug, Error)]
enum CoolTextLoaderError {
#[error(transparent)]
Io(#[from] std::io::Error),
#[error(transparent)]
RonSpannedError(#[from] ron::error::SpannedError),
#[error(transparent)]
LoadDirectError(#[from] bevy::asset::LoadDirectError),
}
impl AssetLoader for CoolTextLoader {
type Asset = CoolText;
type Settings = ();
type Error = CoolTextLoaderError;
async fn load<'a>(
&'a self,
reader: &'a mut dyn Reader,
_settings: &'a Self::Settings,
load_context: &'a mut LoadContext<'_>,
) -> Result<CoolText, Self::Error> {
let mut bytes = Vec::new();
reader.read_to_end(&mut bytes).await?;
let ron: CoolTextRon = ron::de::from_bytes(&bytes)?;
let mut base_text = ron.text;
for embedded in ron.embedded_dependencies {
let loaded = load_context
.loader()
.direct()
.load::<Text>(&embedded)
.await?;
base_text.push_str(&loaded.get().0);
}
for (path, settings_override) in ron.dependencies_with_settings {
let loaded = load_context
.loader()
.with_settings(move |settings| {
*settings = settings_override.clone();
})
.direct()
.load::<Text>(&path)
.await?;
base_text.push_str(&loaded.get().0);
}
Ok(CoolText {
text: base_text,
dependencies: ron
.dependencies
.iter()
.map(|p| load_context.load(p))
.collect(),
})
}
fn extensions(&self) -> &[&str] {
&["cool.ron"]
}
}
#[derive(Default)]
struct CoolTextTransformer;
#[derive(Default, Serialize, Deserialize)]
struct CoolTextTransformerSettings {
appended: String,
}
impl AssetTransformer for CoolTextTransformer {
type AssetInput = CoolText;
type AssetOutput = CoolText;
type Settings = CoolTextTransformerSettings;
type Error = Infallible;
async fn transform<'a>(
&'a self,
mut asset: TransformedAsset<Self::AssetInput>,
settings: &'a Self::Settings,
) -> Result<TransformedAsset<Self::AssetOutput>, Self::Error> {
asset.text = format!("{}{}", asset.text, settings.appended);
Ok(asset)
}
}
struct CoolTextSaver;
impl AssetSaver for CoolTextSaver {
type Asset = CoolText;
type Settings = ();
type OutputLoader = TextLoader;
type Error = std::io::Error;
async fn save<'a>(
&'a self,
writer: &'a mut Writer,
asset: SavedAsset<'a, Self::Asset>,
_settings: &'a Self::Settings,
) -> Result<TextSettings, Self::Error> {
writer.write_all(asset.text.as_bytes()).await?;
Ok(TextSettings::default())
}
}
#[derive(Resource)]
struct TextAssets {
a: Handle<Text>,
b: Handle<Text>,
c: Handle<Text>,
d: Handle<Text>,
e: Handle<Text>,
}
fn setup(mut commands: Commands, assets: Res<AssetServer>) {
// This the final processed versions of `assets/a.cool.ron` and `assets/foo.c.cool.ron`
// Check out their counterparts in `imported_assets` to see what the outputs look like.
commands.insert_resource(TextAssets {
a: assets.load("a.cool.ron"),
b: assets.load("foo/b.cool.ron"),
c: assets.load("foo/c.cool.ron"),
d: assets.load("d.cool.ron"),
e: assets.load("embedded://asset_processing/e.txt"),
});
}
fn print_text(
handles: Res<TextAssets>,
texts: Res<Assets<Text>>,
mut asset_events: EventReader<AssetEvent<Text>>,
) {
if !asset_events.is_empty() {
// This prints the current values of the assets
// Hot-reloading is supported, so try modifying the source assets (and their meta files)!
println!("Current Values:");
println!(" a: {:?}", texts.get(&handles.a));
println!(" b: {:?}", texts.get(&handles.b));
println!(" c: {:?}", texts.get(&handles.c));
println!(" d: {:?}", texts.get(&handles.d));
println!(" e: {:?}", texts.get(&handles.e));
println!("(You can modify source assets and their .meta files to hot-reload changes!)");
println!();
asset_events.clear();
}
}