bevy/crates/bevy_render/src/texture/image.rs

794 lines
27 KiB
Rust
Raw Normal View History

#[cfg(feature = "basis-universal")]
use super::basis::*;
#[cfg(feature = "dds")]
use super::dds::*;
#[cfg(feature = "ktx2")]
use super::ktx2::*;
2021-06-21 23:28:52 +00:00
use super::image_texture_conversion::image_to_texture;
2021-06-25 03:04:28 +00:00
use crate::{
Modular Rendering (#2831) This changes how render logic is composed to make it much more modular. Previously, all extraction logic was centralized for a given "type" of rendered thing. For example, we extracted meshes into a vector of ExtractedMesh, which contained the mesh and material asset handles, the transform, etc. We looked up bindings for "drawn things" using their index in the `Vec<ExtractedMesh>`. This worked fine for built in rendering, but made it hard to reuse logic for "custom" rendering. It also prevented us from reusing things like "extracted transforms" across contexts. To make rendering more modular, I made a number of changes: * Entities now drive rendering: * We extract "render components" from "app components" and store them _on_ entities. No more centralized uber lists! We now have true "ECS-driven rendering" * To make this perform well, I implemented #2673 in upstream Bevy for fast batch insertions into specific entities. This was merged into the `pipelined-rendering` branch here: #2815 * Reworked the `Draw` abstraction: * Generic `PhaseItems`: each draw phase can define its own type of "rendered thing", which can define its own "sort key" * Ported the 2d, 3d, and shadow phases to the new PhaseItem impl (currently Transparent2d, Transparent3d, and Shadow PhaseItems) * `Draw` trait and and `DrawFunctions` are now generic on PhaseItem * Modular / Ergonomic `DrawFunctions` via `RenderCommands` * RenderCommand is a trait that runs an ECS query and produces one or more RenderPass calls. Types implementing this trait can be composed to create a final DrawFunction. For example the DrawPbr DrawFunction is created from the following DrawCommand tuple. Const generics are used to set specific bind group locations: ```rust pub type DrawPbr = ( SetPbrPipeline, SetMeshViewBindGroup<0>, SetStandardMaterialBindGroup<1>, SetTransformBindGroup<2>, DrawMesh, ); ``` * The new `custom_shader_pipelined` example illustrates how the commands above can be reused to create a custom draw function: ```rust type DrawCustom = ( SetCustomMaterialPipeline, SetMeshViewBindGroup<0>, SetTransformBindGroup<2>, DrawMesh, ); ``` * ExtractComponentPlugin and UniformComponentPlugin: * Simple, standardized ways to easily extract individual components and write them to GPU buffers * Ported PBR and Sprite rendering to the new primitives above. * Removed staging buffer from UniformVec in favor of direct Queue usage * Makes UniformVec much easier to use and more ergonomic. Completely removes the need for custom render graph nodes in these contexts (see the PbrNode and view Node removals and the much simpler call patterns in the relevant Prepare systems). * Added a many_cubes_pipelined example to benchmark baseline 3d rendering performance and ensure there were no major regressions during this port. Avoiding regressions was challenging given that the old approach of extracting into centralized vectors is basically the "optimal" approach. However thanks to a various ECS optimizations and render logic rephrasing, we pretty much break even on this benchmark! * Lifetimeless SystemParams: this will be a bit divisive, but as we continue to embrace "trait driven systems" (ex: ExtractComponentPlugin, UniformComponentPlugin, DrawCommand), the ergonomics of `(Query<'static, 'static, (&'static A, &'static B, &'static)>, Res<'static, C>)` were getting very hard to bear. As a compromise, I added "static type aliases" for the relevant SystemParams. The previous example can now be expressed like this: `(SQuery<(Read<A>, Read<B>)>, SRes<C>)`. If anyone has better ideas / conflicting opinions, please let me know! * RunSystem trait: a way to define Systems via a trait with a SystemParam associated type. This is used to implement the various plugins mentioned above. I also added SystemParamItem and QueryItem type aliases to make "trait stye" ecs interactions nicer on the eyes (and fingers). * RenderAsset retrying: ensures that render assets are only created when they are "ready" and allows us to create bind groups directly inside render assets (which significantly simplified the StandardMaterial code). I think ultimately we should swap this out on "asset dependency" events to wait for dependencies to load, but this will require significant asset system changes. * Updated some built in shaders to account for missing MeshUniform fields
2021-09-23 06:16:11 +00:00
render_asset::{PrepareAssetError, RenderAsset},
2021-06-25 03:04:28 +00:00
render_resource::{Sampler, Texture, TextureView},
renderer::{RenderDevice, RenderQueue},
texture::BevyDefault,
2021-06-25 03:04:28 +00:00
};
use bevy_asset::HandleUntyped;
use bevy_derive::{Deref, DerefMut};
Modular Rendering (#2831) This changes how render logic is composed to make it much more modular. Previously, all extraction logic was centralized for a given "type" of rendered thing. For example, we extracted meshes into a vector of ExtractedMesh, which contained the mesh and material asset handles, the transform, etc. We looked up bindings for "drawn things" using their index in the `Vec<ExtractedMesh>`. This worked fine for built in rendering, but made it hard to reuse logic for "custom" rendering. It also prevented us from reusing things like "extracted transforms" across contexts. To make rendering more modular, I made a number of changes: * Entities now drive rendering: * We extract "render components" from "app components" and store them _on_ entities. No more centralized uber lists! We now have true "ECS-driven rendering" * To make this perform well, I implemented #2673 in upstream Bevy for fast batch insertions into specific entities. This was merged into the `pipelined-rendering` branch here: #2815 * Reworked the `Draw` abstraction: * Generic `PhaseItems`: each draw phase can define its own type of "rendered thing", which can define its own "sort key" * Ported the 2d, 3d, and shadow phases to the new PhaseItem impl (currently Transparent2d, Transparent3d, and Shadow PhaseItems) * `Draw` trait and and `DrawFunctions` are now generic on PhaseItem * Modular / Ergonomic `DrawFunctions` via `RenderCommands` * RenderCommand is a trait that runs an ECS query and produces one or more RenderPass calls. Types implementing this trait can be composed to create a final DrawFunction. For example the DrawPbr DrawFunction is created from the following DrawCommand tuple. Const generics are used to set specific bind group locations: ```rust pub type DrawPbr = ( SetPbrPipeline, SetMeshViewBindGroup<0>, SetStandardMaterialBindGroup<1>, SetTransformBindGroup<2>, DrawMesh, ); ``` * The new `custom_shader_pipelined` example illustrates how the commands above can be reused to create a custom draw function: ```rust type DrawCustom = ( SetCustomMaterialPipeline, SetMeshViewBindGroup<0>, SetTransformBindGroup<2>, DrawMesh, ); ``` * ExtractComponentPlugin and UniformComponentPlugin: * Simple, standardized ways to easily extract individual components and write them to GPU buffers * Ported PBR and Sprite rendering to the new primitives above. * Removed staging buffer from UniformVec in favor of direct Queue usage * Makes UniformVec much easier to use and more ergonomic. Completely removes the need for custom render graph nodes in these contexts (see the PbrNode and view Node removals and the much simpler call patterns in the relevant Prepare systems). * Added a many_cubes_pipelined example to benchmark baseline 3d rendering performance and ensure there were no major regressions during this port. Avoiding regressions was challenging given that the old approach of extracting into centralized vectors is basically the "optimal" approach. However thanks to a various ECS optimizations and render logic rephrasing, we pretty much break even on this benchmark! * Lifetimeless SystemParams: this will be a bit divisive, but as we continue to embrace "trait driven systems" (ex: ExtractComponentPlugin, UniformComponentPlugin, DrawCommand), the ergonomics of `(Query<'static, 'static, (&'static A, &'static B, &'static)>, Res<'static, C>)` were getting very hard to bear. As a compromise, I added "static type aliases" for the relevant SystemParams. The previous example can now be expressed like this: `(SQuery<(Read<A>, Read<B>)>, SRes<C>)`. If anyone has better ideas / conflicting opinions, please let me know! * RunSystem trait: a way to define Systems via a trait with a SystemParam associated type. This is used to implement the various plugins mentioned above. I also added SystemParamItem and QueryItem type aliases to make "trait stye" ecs interactions nicer on the eyes (and fingers). * RenderAsset retrying: ensures that render assets are only created when they are "ready" and allows us to create bind groups directly inside render assets (which significantly simplified the StandardMaterial code). I think ultimately we should swap this out on "asset dependency" events to wait for dependencies to load, but this will require significant asset system changes. * Updated some built in shaders to account for missing MeshUniform fields
2021-09-23 06:16:11 +00:00
use bevy_ecs::system::{lifetimeless::SRes, SystemParamItem};
use bevy_math::Vec2;
2021-04-11 20:13:07 +00:00
use bevy_reflect::TypeUuid;
use std::hash::Hash;
2021-04-11 20:13:07 +00:00
use thiserror::Error;
2021-06-25 03:04:28 +00:00
use wgpu::{
Extent3d, ImageCopyTexture, ImageDataLayout, Origin3d, TextureDimension, TextureFormat,
TextureViewDescriptor,
};
2021-04-11 20:13:07 +00:00
pub const TEXTURE_ASSET_INDEX: u64 = 0;
pub const SAMPLER_ASSET_INDEX: u64 = 1;
pub const DEFAULT_IMAGE_HANDLE: HandleUntyped =
HandleUntyped::weak_from_u64(Image::TYPE_UUID, 13148262314052771789);
2021-04-11 20:13:07 +00:00
#[derive(Debug)]
pub enum ImageFormat {
Avif,
Basis,
Bmp,
Dds,
Farbfeld,
Gif,
Hdr,
Ico,
Jpeg,
Ktx2,
Png,
Pnm,
Tga,
Tiff,
WebP,
}
impl ImageFormat {
pub fn from_mime_type(mime_type: &str) -> Option<Self> {
Some(match mime_type.to_ascii_lowercase().as_str() {
"image/bmp" | "image/x-bmp" => ImageFormat::Bmp,
"image/vnd-ms.dds" => ImageFormat::Dds,
"image/jpeg" => ImageFormat::Jpeg,
"image/ktx2" => ImageFormat::Ktx2,
"image/png" => ImageFormat::Png,
"image/x-targa" | "image/x-tga" => ImageFormat::Tga,
_ => return None,
})
}
pub fn from_extension(extension: &str) -> Option<Self> {
Some(match extension.to_ascii_lowercase().as_str() {
"avif" => ImageFormat::Avif,
"basis" => ImageFormat::Basis,
"bmp" => ImageFormat::Bmp,
"dds" => ImageFormat::Dds,
"ff" | "farbfeld" => ImageFormat::Farbfeld,
"gif" => ImageFormat::Gif,
"hdr" => ImageFormat::Hdr,
"ico" => ImageFormat::Ico,
"jpg" | "jpeg" => ImageFormat::Jpeg,
"ktx2" => ImageFormat::Ktx2,
"pbm" | "pam" | "ppm" | "pgm" => ImageFormat::Pnm,
"png" => ImageFormat::Png,
"tga" => ImageFormat::Tga,
"tif" | "tiff" => ImageFormat::Tiff,
"webp" => ImageFormat::WebP,
_ => return None,
})
}
pub fn as_image_crate_format(&self) -> Option<image::ImageFormat> {
Some(match self {
ImageFormat::Avif => image::ImageFormat::Avif,
ImageFormat::Bmp => image::ImageFormat::Bmp,
ImageFormat::Dds => image::ImageFormat::Dds,
ImageFormat::Farbfeld => image::ImageFormat::Farbfeld,
ImageFormat::Gif => image::ImageFormat::Gif,
ImageFormat::Hdr => image::ImageFormat::Hdr,
ImageFormat::Ico => image::ImageFormat::Ico,
ImageFormat::Jpeg => image::ImageFormat::Jpeg,
ImageFormat::Png => image::ImageFormat::Png,
ImageFormat::Pnm => image::ImageFormat::Pnm,
ImageFormat::Tga => image::ImageFormat::Tga,
ImageFormat::Tiff => image::ImageFormat::Tiff,
ImageFormat::WebP => image::ImageFormat::WebP,
ImageFormat::Basis | ImageFormat::Ktx2 => return None,
})
}
}
2021-04-11 20:13:07 +00:00
#[derive(Debug, Clone, TypeUuid)]
#[uuid = "6ea26da6-6cf8-4ea2-9986-1d7bf6c17d6f"]
2021-06-21 23:28:52 +00:00
pub struct Image {
2021-04-11 20:13:07 +00:00
pub data: Vec<u8>,
2021-06-21 23:28:52 +00:00
// TODO: this nesting makes accessing Image metadata verbose. Either flatten out descriptor or add accessors
pub texture_descriptor: wgpu::TextureDescriptor<'static>,
/// The [`ImageSampler`] to use during rendering.
pub sampler_descriptor: ImageSampler,
2021-04-11 20:13:07 +00:00
}
/// Used in [`Image`], this determines what image sampler to use when rendering. The default setting,
/// [`ImageSampler::Default`], will read the sampler from the [`ImageSettings`] resource at runtime.
/// Setting this to [`ImageSampler::Descriptor`] will override the global default descriptor for this [`Image`].
#[derive(Debug, Default, Clone)]
pub enum ImageSampler {
/// Default image sampler, derived from the [`ImageSettings`] resource.
#[default]
Default,
/// Custom sampler for this image which will override global default.
Descriptor(wgpu::SamplerDescriptor<'static>),
}
impl ImageSampler {
/// Returns an image sampler with `Linear` min and mag filters
#[inline]
pub fn linear() -> ImageSampler {
ImageSampler::Descriptor(Self::linear_descriptor())
}
/// Returns an image sampler with `nearest` min and mag filters
#[inline]
pub fn nearest() -> ImageSampler {
ImageSampler::Descriptor(Self::nearest_descriptor())
}
/// Returns a sampler descriptor with `Linear` min and mag filters
#[inline]
pub fn linear_descriptor() -> wgpu::SamplerDescriptor<'static> {
wgpu::SamplerDescriptor {
mag_filter: wgpu::FilterMode::Linear,
min_filter: wgpu::FilterMode::Linear,
..Default::default()
}
}
/// Returns a sampler descriptor with `Nearest` min and mag filters
#[inline]
pub fn nearest_descriptor() -> wgpu::SamplerDescriptor<'static> {
wgpu::SamplerDescriptor {
mag_filter: wgpu::FilterMode::Nearest,
min_filter: wgpu::FilterMode::Nearest,
..Default::default()
}
}
}
/// Global resource for [`Image`] settings.
///
/// Can be set via `insert_resource` during app initialization to change the default settings.
pub struct ImageSettings {
/// The default image sampler to use when [`ImageSampler`] is set to `Default`.
pub default_sampler: wgpu::SamplerDescriptor<'static>,
}
impl Default for ImageSettings {
fn default() -> Self {
ImageSettings::default_linear()
}
}
impl ImageSettings {
/// Creates image settings with linear sampling by default.
pub fn default_linear() -> ImageSettings {
ImageSettings {
default_sampler: ImageSampler::linear_descriptor(),
}
}
/// Creates image settings with nearest sampling by default.
pub fn default_nearest() -> ImageSettings {
ImageSettings {
default_sampler: ImageSampler::nearest_descriptor(),
}
}
}
/// A rendering resource for the default image sampler which is set during renderer
/// initialization.
///
/// The [`ImageSettings`] resource can be set during app initialization to change the default
/// image sampler.
#[derive(Debug, Clone, Deref, DerefMut)]
pub struct DefaultImageSampler(pub(crate) Sampler);
2021-06-21 23:28:52 +00:00
impl Default for Image {
2021-04-11 20:13:07 +00:00
fn default() -> Self {
let format = wgpu::TextureFormat::bevy_default();
let data = vec![255; format.pixel_size() as usize];
2021-06-21 23:28:52 +00:00
Image {
data,
2021-06-21 23:28:52 +00:00
texture_descriptor: wgpu::TextureDescriptor {
size: wgpu::Extent3d {
width: 1,
height: 1,
depth_or_array_layers: 1,
},
format,
2021-06-21 23:28:52 +00:00
dimension: wgpu::TextureDimension::D2,
label: None,
mip_level_count: 1,
sample_count: 1,
usage: wgpu::TextureUsages::TEXTURE_BINDING | wgpu::TextureUsages::COPY_DST,
2021-04-11 20:13:07 +00:00
},
sampler_descriptor: ImageSampler::Default,
2021-04-11 20:13:07 +00:00
}
}
}
2021-06-21 23:28:52 +00:00
impl Image {
/// Creates a new image from raw binary data and the corresponding metadata.
///
/// # Panics
/// Panics if the length of the `data`, volume of the `size` and the size of the `format`
/// do not match.
2021-04-11 20:13:07 +00:00
pub fn new(
size: Extent3d,
dimension: TextureDimension,
data: Vec<u8>,
format: TextureFormat,
) -> Self {
debug_assert_eq!(
size.volume() * format.pixel_size(),
data.len(),
"Pixel data, size and format have to match",
);
2021-07-02 01:05:20 +00:00
let mut image = Self {
data,
..Default::default()
};
2021-06-21 23:28:52 +00:00
image.texture_descriptor.dimension = dimension;
image.texture_descriptor.size = size;
image.texture_descriptor.format = format;
image
2021-04-11 20:13:07 +00:00
}
/// Creates a new image from raw binary data and the corresponding metadata, by filling
/// the image data with the `pixel` data repeated multiple times.
///
/// # Panics
/// Panics if the size of the `format` is not a multiple of the length of the `pixel` data.
/// do not match.
2021-04-11 20:13:07 +00:00
pub fn new_fill(
size: Extent3d,
dimension: TextureDimension,
pixel: &[u8],
format: TextureFormat,
) -> Self {
2021-06-21 23:28:52 +00:00
let mut value = Image::default();
value.texture_descriptor.format = format;
value.texture_descriptor.dimension = dimension;
2021-04-11 20:13:07 +00:00
value.resize(size);
debug_assert_eq!(
pixel.len() % format.pixel_size(),
0,
"Must not have incomplete pixel data."
);
debug_assert!(
pixel.len() <= value.data.len(),
"Fill data must fit within pixel buffer."
);
for current_pixel in value.data.chunks_exact_mut(pixel.len()) {
current_pixel.copy_from_slice(pixel);
2021-04-11 20:13:07 +00:00
}
value
}
/// Returns the aspect ratio (height/width) of a 2D image.
2021-04-11 20:13:07 +00:00
pub fn aspect_2d(&self) -> f32 {
2021-06-21 23:28:52 +00:00
self.texture_descriptor.size.height as f32 / self.texture_descriptor.size.width as f32
2021-04-11 20:13:07 +00:00
}
/// Returns the size of a 2D image.
pub fn size(&self) -> Vec2 {
Vec2::new(
self.texture_descriptor.size.width as f32,
self.texture_descriptor.size.height as f32,
)
}
/// Resizes the image to the new size, by removing information or appending 0 to the `data`.
/// Does not properly resize the contents of the image, but only its internal `data` buffer.
2021-04-11 20:13:07 +00:00
pub fn resize(&mut self, size: Extent3d) {
2021-06-21 23:28:52 +00:00
self.texture_descriptor.size = size;
self.data.resize(
size.volume() * self.texture_descriptor.format.pixel_size(),
0,
);
2021-04-11 20:13:07 +00:00
}
/// Changes the `size`, asserting that the total number of data elements (pixels) remains the
/// same.
///
/// # Panics
/// Panics if the `new_size` does not have the same volume as to old one.
2021-04-11 20:13:07 +00:00
pub fn reinterpret_size(&mut self, new_size: Extent3d) {
assert!(
2021-06-21 23:28:52 +00:00
new_size.volume() == self.texture_descriptor.size.volume(),
2021-04-11 20:13:07 +00:00
"Incompatible sizes: old = {:?} new = {:?}",
2021-06-21 23:28:52 +00:00
self.texture_descriptor.size,
2021-04-11 20:13:07 +00:00
new_size
);
2021-06-21 23:28:52 +00:00
self.texture_descriptor.size = new_size;
2021-04-11 20:13:07 +00:00
}
/// Takes a 2D image containing vertically stacked images of the same size, and reinterprets
2021-04-11 20:13:07 +00:00
/// it as a 2D array texture, where each of the stacked images becomes one layer of the
/// array. This is primarily for use with the `texture2DArray` shader uniform type.
///
/// # Panics
/// Panics if the texture is not 2D, has more than one layers or is not evenly dividable into
/// the `layers`.
2021-04-11 20:13:07 +00:00
pub fn reinterpret_stacked_2d_as_array(&mut self, layers: u32) {
// Must be a stacked image, and the height must be divisible by layers.
2021-06-21 23:28:52 +00:00
assert!(self.texture_descriptor.dimension == TextureDimension::D2);
assert!(self.texture_descriptor.size.depth_or_array_layers == 1);
assert_eq!(self.texture_descriptor.size.height % layers, 0);
2021-04-11 20:13:07 +00:00
self.reinterpret_size(Extent3d {
2021-06-21 23:28:52 +00:00
width: self.texture_descriptor.size.width,
height: self.texture_descriptor.size.height / layers,
2021-04-11 20:13:07 +00:00
depth_or_array_layers: layers,
});
}
/// Convert a texture from a format to another
/// Only a few formats are supported as input and output:
/// - `TextureFormat::R8Unorm`
/// - `TextureFormat::Rg8Unorm`
/// - `TextureFormat::Rgba8UnormSrgb`
pub fn convert(&self, new_format: TextureFormat) -> Option<Self> {
super::image_texture_conversion::texture_to_image(self)
.and_then(|img| match new_format {
TextureFormat::R8Unorm => {
Some((image::DynamicImage::ImageLuma8(img.into_luma8()), false))
2021-04-11 20:13:07 +00:00
}
TextureFormat::Rg8Unorm => Some((
image::DynamicImage::ImageLumaA8(img.into_luma_alpha8()),
false,
)),
2021-04-11 20:13:07 +00:00
TextureFormat::Rgba8UnormSrgb => {
Some((image::DynamicImage::ImageRgba8(img.into_rgba8()), true))
2021-04-11 20:13:07 +00:00
}
_ => None,
})
.map(|(dyn_img, is_srgb)| {
super::image_texture_conversion::image_to_texture(dyn_img, is_srgb)
})
2021-04-11 20:13:07 +00:00
}
/// Load a bytes buffer in a [`Image`], according to type `image_type`, using the `image`
/// crate
pub fn from_buffer(
buffer: &[u8],
image_type: ImageType,
#[allow(unused_variables)] supported_compressed_formats: CompressedImageFormats,
is_srgb: bool,
) -> Result<Image, TextureError> {
let format = image_type.to_image_format()?;
2021-04-11 20:13:07 +00:00
// Load the image in the expected format.
// Some formats like PNG allow for R or RG textures too, so the texture
// format needs to be determined. For RGB textures an alpha channel
// needs to be added, so the image data needs to be converted in those
// cases.
match format {
#[cfg(feature = "basis-universal")]
ImageFormat::Basis => {
basis_buffer_to_image(buffer, supported_compressed_formats, is_srgb)
}
#[cfg(feature = "dds")]
ImageFormat::Dds => dds_buffer_to_image(buffer, supported_compressed_formats, is_srgb),
#[cfg(feature = "ktx2")]
ImageFormat::Ktx2 => {
ktx2_buffer_to_image(buffer, supported_compressed_formats, is_srgb)
}
_ => {
let image_crate_format = format.as_image_crate_format().ok_or_else(|| {
TextureError::UnsupportedTextureFormat(format!("{:?}", format))
})?;
let mut reader = image::io::Reader::new(std::io::Cursor::new(buffer));
reader.set_format(image_crate_format);
reader.no_limits();
let dyn_img = reader.decode()?;
Ok(image_to_texture(dyn_img, is_srgb))
}
}
}
/// Whether the texture format is compressed or uncompressed
pub fn is_compressed(&self) -> bool {
let format_description = self.texture_descriptor.format.describe();
format_description
.required_features
.contains(wgpu::Features::TEXTURE_COMPRESSION_ASTC_LDR)
|| format_description
.required_features
.contains(wgpu::Features::TEXTURE_COMPRESSION_BC)
|| format_description
.required_features
.contains(wgpu::Features::TEXTURE_COMPRESSION_ETC2)
2021-04-11 20:13:07 +00:00
}
}
#[derive(Clone, Copy, Debug)]
pub enum DataFormat {
Rgb,
Rgba,
Rrr,
Rrrg,
Rg,
}
#[derive(Clone, Copy, Debug)]
pub enum TranscodeFormat {
Etc1s,
// Has to be transcoded to Rgba8 for use with `wgpu`
Rgb8,
Uastc(DataFormat),
}
2021-04-11 20:13:07 +00:00
/// An error that occurs when loading a texture
#[derive(Error, Debug)]
pub enum TextureError {
#[error("invalid image mime type: {0}")]
2021-04-11 20:13:07 +00:00
InvalidImageMimeType(String),
#[error("invalid image extension: {0}")]
2021-04-11 20:13:07 +00:00
InvalidImageExtension(String),
#[error("failed to load an image: {0}")]
ImageError(#[from] image::ImageError),
#[error("unsupported texture format: {0}")]
UnsupportedTextureFormat(String),
#[error("supercompression not supported: {0}")]
SuperCompressionNotSupported(String),
#[error("failed to load an image: {0}")]
SuperDecompressionError(String),
#[error("invalid data: {0}")]
InvalidData(String),
#[error("transcode error: {0}")]
TranscodeError(String),
#[error("format requires transcoding: {0:?}")]
FormatRequiresTranscodingError(TranscodeFormat),
2021-04-11 20:13:07 +00:00
}
/// The type of a raw image buffer.
2021-04-11 20:13:07 +00:00
pub enum ImageType<'a> {
/// The mime type of an image, for example `"image/png"`.
2021-04-11 20:13:07 +00:00
MimeType(&'a str),
/// The extension of an image file, for example `"png"`.
2021-04-11 20:13:07 +00:00
Extension(&'a str),
}
2021-06-21 23:28:52 +00:00
impl<'a> ImageType<'a> {
pub fn to_image_format(&self) -> Result<ImageFormat, TextureError> {
match self {
ImageType::MimeType(mime_type) => ImageFormat::from_mime_type(mime_type)
.ok_or_else(|| TextureError::InvalidImageMimeType(mime_type.to_string())),
ImageType::Extension(extension) => ImageFormat::from_extension(extension)
.ok_or_else(|| TextureError::InvalidImageExtension(extension.to_string())),
}
}
}
/// Used to calculate the volume of an item.
2021-06-21 23:28:52 +00:00
pub trait Volume {
fn volume(&self) -> usize;
}
impl Volume for Extent3d {
/// Calculates the volume of the [`Extent3d`].
2021-06-21 23:28:52 +00:00
fn volume(&self) -> usize {
(self.width * self.height * self.depth_or_array_layers) as usize
}
}
/// Information about the pixel size in bytes and the number of different components.
2021-06-21 23:28:52 +00:00
pub struct PixelInfo {
/// The size of a component of a pixel in bytes.
2021-06-21 23:28:52 +00:00
pub type_size: usize,
/// The amount of different components (color channels).
2021-06-21 23:28:52 +00:00
pub num_components: usize,
}
/// Extends the wgpu [`TextureFormat`] with information about the pixel.
2021-06-21 23:28:52 +00:00
pub trait TextureFormatPixelInfo {
/// Returns the pixel information of the format.
2021-06-21 23:28:52 +00:00
fn pixel_info(&self) -> PixelInfo;
/// Returns the size of a pixel of the format.
2021-06-21 23:28:52 +00:00
fn pixel_size(&self) -> usize {
let info = self.pixel_info();
info.type_size * info.num_components
}
}
impl TextureFormatPixelInfo for TextureFormat {
#[allow(clippy::match_same_arms)]
2021-06-21 23:28:52 +00:00
fn pixel_info(&self) -> PixelInfo {
let type_size = match self {
// 8bit
TextureFormat::R8Unorm
| TextureFormat::R8Snorm
| TextureFormat::R8Uint
| TextureFormat::R8Sint
| TextureFormat::Rg8Unorm
| TextureFormat::Rg8Snorm
| TextureFormat::Rg8Uint
| TextureFormat::Rg8Sint
| TextureFormat::Rgba8Unorm
| TextureFormat::Rgba8UnormSrgb
| TextureFormat::Rgba8Snorm
| TextureFormat::Rgba8Uint
| TextureFormat::Rgba8Sint
| TextureFormat::Bgra8Unorm
| TextureFormat::Bgra8UnormSrgb => 1,
// 16bit
TextureFormat::R16Uint
| TextureFormat::R16Sint
| TextureFormat::R16Float
| TextureFormat::Rg16Uint
| TextureFormat::Rg16Sint
| TextureFormat::R16Unorm
2021-06-21 23:28:52 +00:00
| TextureFormat::Rg16Float
| TextureFormat::Rgba16Uint
| TextureFormat::Rgba16Sint
| TextureFormat::Rgba16Float => 2,
// 32bit
TextureFormat::R32Uint
| TextureFormat::R32Sint
| TextureFormat::R32Float
| TextureFormat::Rg32Uint
| TextureFormat::Rg32Sint
| TextureFormat::Rg32Float
| TextureFormat::Rgba32Uint
| TextureFormat::Rgba32Sint
| TextureFormat::Rgba32Float
| TextureFormat::Depth32Float => 4,
// special cases
TextureFormat::Rgb10a2Unorm => 4,
TextureFormat::Rg11b10Float => 4,
TextureFormat::Depth24Plus => 3, // FIXME is this correct?
TextureFormat::Depth24PlusStencil8 => 4,
// TODO: this is not good! this is a temporary step while porting bevy_render to direct wgpu usage
_ => panic!("cannot get pixel info for type"),
};
let components = match self {
TextureFormat::R8Unorm
| TextureFormat::R8Snorm
| TextureFormat::R8Uint
| TextureFormat::R8Sint
| TextureFormat::R16Uint
| TextureFormat::R16Sint
| TextureFormat::R16Unorm
2021-06-21 23:28:52 +00:00
| TextureFormat::R16Float
| TextureFormat::R32Uint
| TextureFormat::R32Sint
| TextureFormat::R32Float => 1,
TextureFormat::Rg8Unorm
| TextureFormat::Rg8Snorm
| TextureFormat::Rg8Uint
| TextureFormat::Rg8Sint
| TextureFormat::Rg16Uint
| TextureFormat::Rg16Sint
| TextureFormat::Rg16Float
| TextureFormat::Rg32Uint
| TextureFormat::Rg32Sint
| TextureFormat::Rg32Float => 2,
TextureFormat::Rgba8Unorm
| TextureFormat::Rgba8UnormSrgb
| TextureFormat::Rgba8Snorm
| TextureFormat::Rgba8Uint
| TextureFormat::Rgba8Sint
| TextureFormat::Bgra8Unorm
| TextureFormat::Bgra8UnormSrgb
| TextureFormat::Rgba16Uint
| TextureFormat::Rgba16Sint
| TextureFormat::Rgba16Float
| TextureFormat::Rgba32Uint
| TextureFormat::Rgba32Sint
| TextureFormat::Rgba32Float => 4,
// special cases
TextureFormat::Rgb10a2Unorm
| TextureFormat::Rg11b10Float
| TextureFormat::Depth32Float
| TextureFormat::Depth24Plus
| TextureFormat::Depth24PlusStencil8 => 1,
// TODO: this is not good! this is a temporary step while porting bevy_render to direct wgpu usage
_ => panic!("cannot get pixel info for type"),
};
PixelInfo {
type_size,
num_components: components,
}
}
}
2021-06-25 03:04:28 +00:00
/// The GPU-representation of an [`Image`].
/// Consists of the [`Texture`], its [`TextureView`] and the corresponding [`Sampler`], and the texture's size.
2021-06-25 03:04:28 +00:00
#[derive(Debug, Clone)]
pub struct GpuImage {
pub texture: Texture,
pub texture_view: TextureView,
pub texture_format: TextureFormat,
2021-06-25 03:04:28 +00:00
pub sampler: Sampler,
pub size: Vec2,
2021-06-25 03:04:28 +00:00
}
impl RenderAsset for Image {
type ExtractedAsset = Image;
type PreparedAsset = GpuImage;
type Param = (
SRes<RenderDevice>,
SRes<RenderQueue>,
SRes<DefaultImageSampler>,
);
2021-06-25 03:04:28 +00:00
/// Clones the Image.
2021-06-25 03:04:28 +00:00
fn extract_asset(&self) -> Self::ExtractedAsset {
self.clone()
}
/// Converts the extracted image into a [`GpuImage`].
2021-06-25 03:04:28 +00:00
fn prepare_asset(
image: Self::ExtractedAsset,
(render_device, render_queue, default_sampler): &mut SystemParamItem<Self::Param>,
Modular Rendering (#2831) This changes how render logic is composed to make it much more modular. Previously, all extraction logic was centralized for a given "type" of rendered thing. For example, we extracted meshes into a vector of ExtractedMesh, which contained the mesh and material asset handles, the transform, etc. We looked up bindings for "drawn things" using their index in the `Vec<ExtractedMesh>`. This worked fine for built in rendering, but made it hard to reuse logic for "custom" rendering. It also prevented us from reusing things like "extracted transforms" across contexts. To make rendering more modular, I made a number of changes: * Entities now drive rendering: * We extract "render components" from "app components" and store them _on_ entities. No more centralized uber lists! We now have true "ECS-driven rendering" * To make this perform well, I implemented #2673 in upstream Bevy for fast batch insertions into specific entities. This was merged into the `pipelined-rendering` branch here: #2815 * Reworked the `Draw` abstraction: * Generic `PhaseItems`: each draw phase can define its own type of "rendered thing", which can define its own "sort key" * Ported the 2d, 3d, and shadow phases to the new PhaseItem impl (currently Transparent2d, Transparent3d, and Shadow PhaseItems) * `Draw` trait and and `DrawFunctions` are now generic on PhaseItem * Modular / Ergonomic `DrawFunctions` via `RenderCommands` * RenderCommand is a trait that runs an ECS query and produces one or more RenderPass calls. Types implementing this trait can be composed to create a final DrawFunction. For example the DrawPbr DrawFunction is created from the following DrawCommand tuple. Const generics are used to set specific bind group locations: ```rust pub type DrawPbr = ( SetPbrPipeline, SetMeshViewBindGroup<0>, SetStandardMaterialBindGroup<1>, SetTransformBindGroup<2>, DrawMesh, ); ``` * The new `custom_shader_pipelined` example illustrates how the commands above can be reused to create a custom draw function: ```rust type DrawCustom = ( SetCustomMaterialPipeline, SetMeshViewBindGroup<0>, SetTransformBindGroup<2>, DrawMesh, ); ``` * ExtractComponentPlugin and UniformComponentPlugin: * Simple, standardized ways to easily extract individual components and write them to GPU buffers * Ported PBR and Sprite rendering to the new primitives above. * Removed staging buffer from UniformVec in favor of direct Queue usage * Makes UniformVec much easier to use and more ergonomic. Completely removes the need for custom render graph nodes in these contexts (see the PbrNode and view Node removals and the much simpler call patterns in the relevant Prepare systems). * Added a many_cubes_pipelined example to benchmark baseline 3d rendering performance and ensure there were no major regressions during this port. Avoiding regressions was challenging given that the old approach of extracting into centralized vectors is basically the "optimal" approach. However thanks to a various ECS optimizations and render logic rephrasing, we pretty much break even on this benchmark! * Lifetimeless SystemParams: this will be a bit divisive, but as we continue to embrace "trait driven systems" (ex: ExtractComponentPlugin, UniformComponentPlugin, DrawCommand), the ergonomics of `(Query<'static, 'static, (&'static A, &'static B, &'static)>, Res<'static, C>)` were getting very hard to bear. As a compromise, I added "static type aliases" for the relevant SystemParams. The previous example can now be expressed like this: `(SQuery<(Read<A>, Read<B>)>, SRes<C>)`. If anyone has better ideas / conflicting opinions, please let me know! * RunSystem trait: a way to define Systems via a trait with a SystemParam associated type. This is used to implement the various plugins mentioned above. I also added SystemParamItem and QueryItem type aliases to make "trait stye" ecs interactions nicer on the eyes (and fingers). * RenderAsset retrying: ensures that render assets are only created when they are "ready" and allows us to create bind groups directly inside render assets (which significantly simplified the StandardMaterial code). I think ultimately we should swap this out on "asset dependency" events to wait for dependencies to load, but this will require significant asset system changes. * Updated some built in shaders to account for missing MeshUniform fields
2021-09-23 06:16:11 +00:00
) -> Result<Self::PreparedAsset, PrepareAssetError<Self::ExtractedAsset>> {
let texture = if image.texture_descriptor.mip_level_count > 1 || image.is_compressed() {
render_device.create_texture_with_data(
render_queue,
&image.texture_descriptor,
&image.data,
)
} else {
let texture = render_device.create_texture(&image.texture_descriptor);
let format_size = image.texture_descriptor.format.pixel_size();
render_queue.write_texture(
ImageCopyTexture {
texture: &texture,
mip_level: 0,
origin: Origin3d::ZERO,
aspect: wgpu::TextureAspect::All,
},
&image.data,
ImageDataLayout {
offset: 0,
bytes_per_row: Some(
std::num::NonZeroU32::new(
image.texture_descriptor.size.width * format_size as u32,
)
.unwrap(),
),
rows_per_image: if image.texture_descriptor.size.depth_or_array_layers > 1 {
std::num::NonZeroU32::new(image.texture_descriptor.size.height)
} else {
None
},
},
image.texture_descriptor.size,
);
texture
};
2021-06-25 03:04:28 +00:00
let texture_view = texture.create_view(&TextureViewDescriptor::default());
let size = Vec2::new(
image.texture_descriptor.size.width as f32,
image.texture_descriptor.size.height as f32,
);
let sampler = match image.sampler_descriptor {
ImageSampler::Default => (***default_sampler).clone(),
ImageSampler::Descriptor(descriptor) => render_device.create_sampler(&descriptor),
};
Modular Rendering (#2831) This changes how render logic is composed to make it much more modular. Previously, all extraction logic was centralized for a given "type" of rendered thing. For example, we extracted meshes into a vector of ExtractedMesh, which contained the mesh and material asset handles, the transform, etc. We looked up bindings for "drawn things" using their index in the `Vec<ExtractedMesh>`. This worked fine for built in rendering, but made it hard to reuse logic for "custom" rendering. It also prevented us from reusing things like "extracted transforms" across contexts. To make rendering more modular, I made a number of changes: * Entities now drive rendering: * We extract "render components" from "app components" and store them _on_ entities. No more centralized uber lists! We now have true "ECS-driven rendering" * To make this perform well, I implemented #2673 in upstream Bevy for fast batch insertions into specific entities. This was merged into the `pipelined-rendering` branch here: #2815 * Reworked the `Draw` abstraction: * Generic `PhaseItems`: each draw phase can define its own type of "rendered thing", which can define its own "sort key" * Ported the 2d, 3d, and shadow phases to the new PhaseItem impl (currently Transparent2d, Transparent3d, and Shadow PhaseItems) * `Draw` trait and and `DrawFunctions` are now generic on PhaseItem * Modular / Ergonomic `DrawFunctions` via `RenderCommands` * RenderCommand is a trait that runs an ECS query and produces one or more RenderPass calls. Types implementing this trait can be composed to create a final DrawFunction. For example the DrawPbr DrawFunction is created from the following DrawCommand tuple. Const generics are used to set specific bind group locations: ```rust pub type DrawPbr = ( SetPbrPipeline, SetMeshViewBindGroup<0>, SetStandardMaterialBindGroup<1>, SetTransformBindGroup<2>, DrawMesh, ); ``` * The new `custom_shader_pipelined` example illustrates how the commands above can be reused to create a custom draw function: ```rust type DrawCustom = ( SetCustomMaterialPipeline, SetMeshViewBindGroup<0>, SetTransformBindGroup<2>, DrawMesh, ); ``` * ExtractComponentPlugin and UniformComponentPlugin: * Simple, standardized ways to easily extract individual components and write them to GPU buffers * Ported PBR and Sprite rendering to the new primitives above. * Removed staging buffer from UniformVec in favor of direct Queue usage * Makes UniformVec much easier to use and more ergonomic. Completely removes the need for custom render graph nodes in these contexts (see the PbrNode and view Node removals and the much simpler call patterns in the relevant Prepare systems). * Added a many_cubes_pipelined example to benchmark baseline 3d rendering performance and ensure there were no major regressions during this port. Avoiding regressions was challenging given that the old approach of extracting into centralized vectors is basically the "optimal" approach. However thanks to a various ECS optimizations and render logic rephrasing, we pretty much break even on this benchmark! * Lifetimeless SystemParams: this will be a bit divisive, but as we continue to embrace "trait driven systems" (ex: ExtractComponentPlugin, UniformComponentPlugin, DrawCommand), the ergonomics of `(Query<'static, 'static, (&'static A, &'static B, &'static)>, Res<'static, C>)` were getting very hard to bear. As a compromise, I added "static type aliases" for the relevant SystemParams. The previous example can now be expressed like this: `(SQuery<(Read<A>, Read<B>)>, SRes<C>)`. If anyone has better ideas / conflicting opinions, please let me know! * RunSystem trait: a way to define Systems via a trait with a SystemParam associated type. This is used to implement the various plugins mentioned above. I also added SystemParamItem and QueryItem type aliases to make "trait stye" ecs interactions nicer on the eyes (and fingers). * RenderAsset retrying: ensures that render assets are only created when they are "ready" and allows us to create bind groups directly inside render assets (which significantly simplified the StandardMaterial code). I think ultimately we should swap this out on "asset dependency" events to wait for dependencies to load, but this will require significant asset system changes. * Updated some built in shaders to account for missing MeshUniform fields
2021-09-23 06:16:11 +00:00
Ok(GpuImage {
2021-06-25 03:04:28 +00:00
texture,
texture_view,
texture_format: image.texture_descriptor.format,
2021-06-25 03:04:28 +00:00
sampler,
size,
Modular Rendering (#2831) This changes how render logic is composed to make it much more modular. Previously, all extraction logic was centralized for a given "type" of rendered thing. For example, we extracted meshes into a vector of ExtractedMesh, which contained the mesh and material asset handles, the transform, etc. We looked up bindings for "drawn things" using their index in the `Vec<ExtractedMesh>`. This worked fine for built in rendering, but made it hard to reuse logic for "custom" rendering. It also prevented us from reusing things like "extracted transforms" across contexts. To make rendering more modular, I made a number of changes: * Entities now drive rendering: * We extract "render components" from "app components" and store them _on_ entities. No more centralized uber lists! We now have true "ECS-driven rendering" * To make this perform well, I implemented #2673 in upstream Bevy for fast batch insertions into specific entities. This was merged into the `pipelined-rendering` branch here: #2815 * Reworked the `Draw` abstraction: * Generic `PhaseItems`: each draw phase can define its own type of "rendered thing", which can define its own "sort key" * Ported the 2d, 3d, and shadow phases to the new PhaseItem impl (currently Transparent2d, Transparent3d, and Shadow PhaseItems) * `Draw` trait and and `DrawFunctions` are now generic on PhaseItem * Modular / Ergonomic `DrawFunctions` via `RenderCommands` * RenderCommand is a trait that runs an ECS query and produces one or more RenderPass calls. Types implementing this trait can be composed to create a final DrawFunction. For example the DrawPbr DrawFunction is created from the following DrawCommand tuple. Const generics are used to set specific bind group locations: ```rust pub type DrawPbr = ( SetPbrPipeline, SetMeshViewBindGroup<0>, SetStandardMaterialBindGroup<1>, SetTransformBindGroup<2>, DrawMesh, ); ``` * The new `custom_shader_pipelined` example illustrates how the commands above can be reused to create a custom draw function: ```rust type DrawCustom = ( SetCustomMaterialPipeline, SetMeshViewBindGroup<0>, SetTransformBindGroup<2>, DrawMesh, ); ``` * ExtractComponentPlugin and UniformComponentPlugin: * Simple, standardized ways to easily extract individual components and write them to GPU buffers * Ported PBR and Sprite rendering to the new primitives above. * Removed staging buffer from UniformVec in favor of direct Queue usage * Makes UniformVec much easier to use and more ergonomic. Completely removes the need for custom render graph nodes in these contexts (see the PbrNode and view Node removals and the much simpler call patterns in the relevant Prepare systems). * Added a many_cubes_pipelined example to benchmark baseline 3d rendering performance and ensure there were no major regressions during this port. Avoiding regressions was challenging given that the old approach of extracting into centralized vectors is basically the "optimal" approach. However thanks to a various ECS optimizations and render logic rephrasing, we pretty much break even on this benchmark! * Lifetimeless SystemParams: this will be a bit divisive, but as we continue to embrace "trait driven systems" (ex: ExtractComponentPlugin, UniformComponentPlugin, DrawCommand), the ergonomics of `(Query<'static, 'static, (&'static A, &'static B, &'static)>, Res<'static, C>)` were getting very hard to bear. As a compromise, I added "static type aliases" for the relevant SystemParams. The previous example can now be expressed like this: `(SQuery<(Read<A>, Read<B>)>, SRes<C>)`. If anyone has better ideas / conflicting opinions, please let me know! * RunSystem trait: a way to define Systems via a trait with a SystemParam associated type. This is used to implement the various plugins mentioned above. I also added SystemParamItem and QueryItem type aliases to make "trait stye" ecs interactions nicer on the eyes (and fingers). * RenderAsset retrying: ensures that render assets are only created when they are "ready" and allows us to create bind groups directly inside render assets (which significantly simplified the StandardMaterial code). I think ultimately we should swap this out on "asset dependency" events to wait for dependencies to load, but this will require significant asset system changes. * Updated some built in shaders to account for missing MeshUniform fields
2021-09-23 06:16:11 +00:00
})
2021-06-25 03:04:28 +00:00
}
}
bitflags::bitflags! {
#[derive(Default)]
#[repr(transparent)]
pub struct CompressedImageFormats: u32 {
const NONE = 0;
const ASTC_LDR = (1 << 0);
const BC = (1 << 1);
const ETC2 = (1 << 2);
}
}
impl CompressedImageFormats {
pub fn from_features(features: wgpu::Features) -> Self {
let mut supported_compressed_formats = Self::default();
if features.contains(wgpu::Features::TEXTURE_COMPRESSION_ASTC_LDR) {
supported_compressed_formats |= Self::ASTC_LDR;
}
if features.contains(wgpu::Features::TEXTURE_COMPRESSION_BC) {
supported_compressed_formats |= Self::BC;
}
if features.contains(wgpu::Features::TEXTURE_COMPRESSION_ETC2) {
supported_compressed_formats |= Self::ETC2;
}
supported_compressed_formats
}
pub fn supports(&self, format: TextureFormat) -> bool {
match format {
TextureFormat::Bc1RgbaUnorm
| TextureFormat::Bc1RgbaUnormSrgb
| TextureFormat::Bc2RgbaUnorm
| TextureFormat::Bc2RgbaUnormSrgb
| TextureFormat::Bc3RgbaUnorm
| TextureFormat::Bc3RgbaUnormSrgb
| TextureFormat::Bc4RUnorm
| TextureFormat::Bc4RSnorm
| TextureFormat::Bc5RgUnorm
| TextureFormat::Bc5RgSnorm
| TextureFormat::Bc6hRgbUfloat
| TextureFormat::Bc6hRgbSfloat
| TextureFormat::Bc7RgbaUnorm
| TextureFormat::Bc7RgbaUnormSrgb => self.contains(CompressedImageFormats::BC),
TextureFormat::Etc2Rgb8Unorm
| TextureFormat::Etc2Rgb8UnormSrgb
| TextureFormat::Etc2Rgb8A1Unorm
| TextureFormat::Etc2Rgb8A1UnormSrgb
| TextureFormat::Etc2Rgba8Unorm
| TextureFormat::Etc2Rgba8UnormSrgb
| TextureFormat::EacR11Unorm
| TextureFormat::EacR11Snorm
| TextureFormat::EacRg11Unorm
| TextureFormat::EacRg11Snorm => self.contains(CompressedImageFormats::ETC2),
TextureFormat::Astc { .. } => self.contains(CompressedImageFormats::ASTC_LDR),
_ => true,
}
}
}
#[cfg(test)]
mod test {
use super::*;
#[test]
fn image_size() {
let size = Extent3d {
width: 200,
height: 100,
depth_or_array_layers: 1,
};
let image = Image::new_fill(
size,
TextureDimension::D2,
&[0, 0, 0, 255],
TextureFormat::Rgba8Unorm,
);
assert_eq!(
Vec2::new(size.width as f32, size.height as f32),
image.size()
);
}
#[test]
fn image_default_size() {
let image = Image::default();
assert_eq!(Vec2::ONE, image.size());
}
}