bevy/crates/bevy_audio/src/lib.rs

101 lines
2.9 KiB
Rust
Raw Normal View History

//! Audio support for the game engine Bevy
//!
//! ```no_run
bevy_audio: ECS-based API redesign (#8424) # Objective Improve the `bevy_audio` API to make it more user-friendly and ECS-idiomatic. This PR is a first-pass at addressing some of the most obvious (to me) problems. In the interest of keeping the scope small, further improvements can be done in future PRs. The current `bevy_audio` API is very clunky to work with, due to how it (ab)uses bevy assets to represent audio sinks. The user needs to write a lot of boilerplate (accessing `Res<Assets<AudioSink>>`) and deal with a lot of cognitive overhead (worry about strong vs. weak handles, etc.) in order to control audio playback. Audio playback is initiated via a centralized `Audio` resource, which makes it difficult to keep track of many different sounds playing in a typical game. Further, everything carries a generic type parameter for the sound source type, making it difficult to mix custom sound sources (such as procedurally generated audio or unofficial formats) with regular audio assets. Let's fix these issues. ## Solution Refactor `bevy_audio` to a more idiomatic ECS API. Remove the `Audio` resource. Do everything via entities and components instead. Audio playback data is now stored in components: - `PlaybackSettings`, `SpatialSettings`, `Handle<AudioSource>` are now components. The user inserts them to tell Bevy to play a sound and configure the initial playback parameters. - `AudioSink`, `SpatialAudioSink` are now components instead of special magical "asset" types. They are inserted by Bevy when it actually begins playing the sound, and can be queried for by the user in order to control the sound during playback. Bundles: `AudioBundle` and `SpatialAudioBundle` are available to make it easy for users to play sounds. Spawn an entity with one of these bundles (or insert them to a complex entity alongside other stuff) to play a sound. Each entity represents a sound to be played. There is also a new "auto-despawn" feature (activated using `PlaybackSettings`), which, if enabled, tells Bevy to despawn entities when the sink playback finishes. This allows for "fire-and-forget" sound playback. Users can simply spawn entities whenever they want to play sounds and not have to worry about leaking memory. ## Unsolved Questions I think the current design is *fine*. I'd be happy for it to be merged. It has some possibly-surprising usability pitfalls, but I think it is still much better than the old `bevy_audio`. Here are some discussion questions for things that we could further improve. I'm undecided on these questions, which is why I didn't implement them. We should decide which of these should be addressed in this PR, and what should be left for future PRs. Or if they should be addressed at all. ### What happens when sounds start playing? Currently, the audio sink components are inserted and the bundle components are kept. Should Bevy remove the bundle components? Something else? The current design allows an entity to be reused for playing the same sound with the same parameters repeatedly. This is a niche use case I'd like to be supported, but if we have to give it up for a simpler design, I'd be fine with that. ### What happens if users remove any of the components themselves? As described above, currently, entities can be reused. Removing the audio sink causes it to be "detached" (I kept the old `Drop` impl), so the sound keeps playing. However, if the audio bundle components are not removed, Bevy will detect this entity as a "queued" sound entity again (has the bundle compoenents, without a sink component), just like before playing the sound the first time, and start playing the sound again. This behavior might be surprising? Should we do something different? ### Should mutations to `PlaybackSettings` be applied to the audio sink? We currently do not do that. `PlaybackSettings` is just for the initial settings when the sound starts playing. This is clearly documented. Do we want to keep this behavior, or do we want to allow users to use `PlaybackSettings` instead of `AudioSink`/`SpatialAudioSink` to control sounds during playback too? I think I prefer for them to be kept separate. It is not a bad mental model once you understand it, and it is documented. ### Should `AudioSink` and `SpatialAudioSink` be unified into a single component type? They provide a similar API (via the `AudioSinkPlayback` trait) and it might be annoying for users to have to deal with both of them. The unification could be done using an enum that is matched on internally by the methods. Spatial audio has extra features, so this might make it harder to access. I think we shouldn't. ### Automatic synchronization of spatial sound properties from Transforms? Should Bevy automatically apply changes to Transforms to spatial audio entities? How do we distinguish between listener and emitter? Which one does the transform represent? Where should the other one come from? Alternatively, leave this problem for now, and address it in a future PR. Or do nothing, and let users deal with it, as shown in the `spatial_audio_2d` and `spatial_audio_3d` examples. --- ## Changelog Added: - `AudioBundle`/`SpatialAudioBundle`, add them to entities to play sounds. Removed: - The `Audio` resource. - `AudioOutput` is no longer `pub`. Changed: - `AudioSink`, `SpatialAudioSink` are now components instead of assets. ## Migration Guide // TODO: write a more detailed migration guide, after the "unsolved questions" are answered and this PR is finalized. Before: ```rust /// Need to store handles somewhere #[derive(Resource)] struct MyMusic { sink: Handle<AudioSink>, } fn play_music( asset_server: Res<AssetServer>, audio: Res<Audio>, audio_sinks: Res<Assets<AudioSink>>, mut commands: Commands, ) { let weak_handle = audio.play_with_settings( asset_server.load("music.ogg"), PlaybackSettings::LOOP.with_volume(0.5), ); // upgrade to strong handle and store it commands.insert_resource(MyMusic { sink: audio_sinks.get_handle(weak_handle), }); } fn toggle_pause_music( audio_sinks: Res<Assets<AudioSink>>, mymusic: Option<Res<MyMusic>>, ) { if let Some(mymusic) = &mymusic { if let Some(sink) = audio_sinks.get(&mymusic.sink) { sink.toggle(); } } } ``` Now: ```rust /// Marker component for our music entity #[derive(Component)] struct MyMusic; fn play_music( mut commands: Commands, asset_server: Res<AssetServer>, ) { commands.spawn(( AudioBundle::from_audio_source(asset_server.load("music.ogg")) .with_settings(PlaybackSettings::LOOP.with_volume(0.5)), MyMusic, )); } fn toggle_pause_music( // `AudioSink` will be inserted by Bevy when the audio starts playing query_music: Query<&AudioSink, With<MyMusic>>, ) { if let Ok(sink) = query.get_single() { sink.toggle(); } } ```
2023-07-07 23:01:17 +00:00
//! # use bevy_ecs::prelude::*;
//! # use bevy_audio::{AudioBundle, AudioPlugin, PlaybackSettings};
//! # use bevy_asset::{AssetPlugin, AssetServer};
//! # use bevy_app::{App, AppExit, NoopPluginGroup as MinimalPlugins, Startup};
//! fn main() {
//! App::new()
//! .add_plugins((MinimalPlugins, AssetPlugin::default(), AudioPlugin::default()))
//! .add_systems(Startup, play_background_audio)
//! .run();
//! }
//!
bevy_audio: ECS-based API redesign (#8424) # Objective Improve the `bevy_audio` API to make it more user-friendly and ECS-idiomatic. This PR is a first-pass at addressing some of the most obvious (to me) problems. In the interest of keeping the scope small, further improvements can be done in future PRs. The current `bevy_audio` API is very clunky to work with, due to how it (ab)uses bevy assets to represent audio sinks. The user needs to write a lot of boilerplate (accessing `Res<Assets<AudioSink>>`) and deal with a lot of cognitive overhead (worry about strong vs. weak handles, etc.) in order to control audio playback. Audio playback is initiated via a centralized `Audio` resource, which makes it difficult to keep track of many different sounds playing in a typical game. Further, everything carries a generic type parameter for the sound source type, making it difficult to mix custom sound sources (such as procedurally generated audio or unofficial formats) with regular audio assets. Let's fix these issues. ## Solution Refactor `bevy_audio` to a more idiomatic ECS API. Remove the `Audio` resource. Do everything via entities and components instead. Audio playback data is now stored in components: - `PlaybackSettings`, `SpatialSettings`, `Handle<AudioSource>` are now components. The user inserts them to tell Bevy to play a sound and configure the initial playback parameters. - `AudioSink`, `SpatialAudioSink` are now components instead of special magical "asset" types. They are inserted by Bevy when it actually begins playing the sound, and can be queried for by the user in order to control the sound during playback. Bundles: `AudioBundle` and `SpatialAudioBundle` are available to make it easy for users to play sounds. Spawn an entity with one of these bundles (or insert them to a complex entity alongside other stuff) to play a sound. Each entity represents a sound to be played. There is also a new "auto-despawn" feature (activated using `PlaybackSettings`), which, if enabled, tells Bevy to despawn entities when the sink playback finishes. This allows for "fire-and-forget" sound playback. Users can simply spawn entities whenever they want to play sounds and not have to worry about leaking memory. ## Unsolved Questions I think the current design is *fine*. I'd be happy for it to be merged. It has some possibly-surprising usability pitfalls, but I think it is still much better than the old `bevy_audio`. Here are some discussion questions for things that we could further improve. I'm undecided on these questions, which is why I didn't implement them. We should decide which of these should be addressed in this PR, and what should be left for future PRs. Or if they should be addressed at all. ### What happens when sounds start playing? Currently, the audio sink components are inserted and the bundle components are kept. Should Bevy remove the bundle components? Something else? The current design allows an entity to be reused for playing the same sound with the same parameters repeatedly. This is a niche use case I'd like to be supported, but if we have to give it up for a simpler design, I'd be fine with that. ### What happens if users remove any of the components themselves? As described above, currently, entities can be reused. Removing the audio sink causes it to be "detached" (I kept the old `Drop` impl), so the sound keeps playing. However, if the audio bundle components are not removed, Bevy will detect this entity as a "queued" sound entity again (has the bundle compoenents, without a sink component), just like before playing the sound the first time, and start playing the sound again. This behavior might be surprising? Should we do something different? ### Should mutations to `PlaybackSettings` be applied to the audio sink? We currently do not do that. `PlaybackSettings` is just for the initial settings when the sound starts playing. This is clearly documented. Do we want to keep this behavior, or do we want to allow users to use `PlaybackSettings` instead of `AudioSink`/`SpatialAudioSink` to control sounds during playback too? I think I prefer for them to be kept separate. It is not a bad mental model once you understand it, and it is documented. ### Should `AudioSink` and `SpatialAudioSink` be unified into a single component type? They provide a similar API (via the `AudioSinkPlayback` trait) and it might be annoying for users to have to deal with both of them. The unification could be done using an enum that is matched on internally by the methods. Spatial audio has extra features, so this might make it harder to access. I think we shouldn't. ### Automatic synchronization of spatial sound properties from Transforms? Should Bevy automatically apply changes to Transforms to spatial audio entities? How do we distinguish between listener and emitter? Which one does the transform represent? Where should the other one come from? Alternatively, leave this problem for now, and address it in a future PR. Or do nothing, and let users deal with it, as shown in the `spatial_audio_2d` and `spatial_audio_3d` examples. --- ## Changelog Added: - `AudioBundle`/`SpatialAudioBundle`, add them to entities to play sounds. Removed: - The `Audio` resource. - `AudioOutput` is no longer `pub`. Changed: - `AudioSink`, `SpatialAudioSink` are now components instead of assets. ## Migration Guide // TODO: write a more detailed migration guide, after the "unsolved questions" are answered and this PR is finalized. Before: ```rust /// Need to store handles somewhere #[derive(Resource)] struct MyMusic { sink: Handle<AudioSink>, } fn play_music( asset_server: Res<AssetServer>, audio: Res<Audio>, audio_sinks: Res<Assets<AudioSink>>, mut commands: Commands, ) { let weak_handle = audio.play_with_settings( asset_server.load("music.ogg"), PlaybackSettings::LOOP.with_volume(0.5), ); // upgrade to strong handle and store it commands.insert_resource(MyMusic { sink: audio_sinks.get_handle(weak_handle), }); } fn toggle_pause_music( audio_sinks: Res<Assets<AudioSink>>, mymusic: Option<Res<MyMusic>>, ) { if let Some(mymusic) = &mymusic { if let Some(sink) = audio_sinks.get(&mymusic.sink) { sink.toggle(); } } } ``` Now: ```rust /// Marker component for our music entity #[derive(Component)] struct MyMusic; fn play_music( mut commands: Commands, asset_server: Res<AssetServer>, ) { commands.spawn(( AudioBundle::from_audio_source(asset_server.load("music.ogg")) .with_settings(PlaybackSettings::LOOP.with_volume(0.5)), MyMusic, )); } fn toggle_pause_music( // `AudioSink` will be inserted by Bevy when the audio starts playing query_music: Query<&AudioSink, With<MyMusic>>, ) { if let Ok(sink) = query.get_single() { sink.toggle(); } } ```
2023-07-07 23:01:17 +00:00
//! fn play_background_audio(asset_server: Res<AssetServer>, mut commands: Commands) {
//! commands.spawn(AudioBundle {
//! source: asset_server.load("background_audio.ogg"),
//! settings: PlaybackSettings::LOOP,
//! });
//! }
//! ```
#![forbid(unsafe_code)]
#![allow(clippy::type_complexity)]
#![warn(missing_docs)]
mod audio;
mod audio_output;
mod audio_source;
mod pitch;
mod sinks;
#[allow(missing_docs)]
2020-07-17 02:20:51 +00:00
pub mod prelude {
#[doc(hidden)]
pub use crate::{
bevy_audio: ECS-based API redesign (#8424) # Objective Improve the `bevy_audio` API to make it more user-friendly and ECS-idiomatic. This PR is a first-pass at addressing some of the most obvious (to me) problems. In the interest of keeping the scope small, further improvements can be done in future PRs. The current `bevy_audio` API is very clunky to work with, due to how it (ab)uses bevy assets to represent audio sinks. The user needs to write a lot of boilerplate (accessing `Res<Assets<AudioSink>>`) and deal with a lot of cognitive overhead (worry about strong vs. weak handles, etc.) in order to control audio playback. Audio playback is initiated via a centralized `Audio` resource, which makes it difficult to keep track of many different sounds playing in a typical game. Further, everything carries a generic type parameter for the sound source type, making it difficult to mix custom sound sources (such as procedurally generated audio or unofficial formats) with regular audio assets. Let's fix these issues. ## Solution Refactor `bevy_audio` to a more idiomatic ECS API. Remove the `Audio` resource. Do everything via entities and components instead. Audio playback data is now stored in components: - `PlaybackSettings`, `SpatialSettings`, `Handle<AudioSource>` are now components. The user inserts them to tell Bevy to play a sound and configure the initial playback parameters. - `AudioSink`, `SpatialAudioSink` are now components instead of special magical "asset" types. They are inserted by Bevy when it actually begins playing the sound, and can be queried for by the user in order to control the sound during playback. Bundles: `AudioBundle` and `SpatialAudioBundle` are available to make it easy for users to play sounds. Spawn an entity with one of these bundles (or insert them to a complex entity alongside other stuff) to play a sound. Each entity represents a sound to be played. There is also a new "auto-despawn" feature (activated using `PlaybackSettings`), which, if enabled, tells Bevy to despawn entities when the sink playback finishes. This allows for "fire-and-forget" sound playback. Users can simply spawn entities whenever they want to play sounds and not have to worry about leaking memory. ## Unsolved Questions I think the current design is *fine*. I'd be happy for it to be merged. It has some possibly-surprising usability pitfalls, but I think it is still much better than the old `bevy_audio`. Here are some discussion questions for things that we could further improve. I'm undecided on these questions, which is why I didn't implement them. We should decide which of these should be addressed in this PR, and what should be left for future PRs. Or if they should be addressed at all. ### What happens when sounds start playing? Currently, the audio sink components are inserted and the bundle components are kept. Should Bevy remove the bundle components? Something else? The current design allows an entity to be reused for playing the same sound with the same parameters repeatedly. This is a niche use case I'd like to be supported, but if we have to give it up for a simpler design, I'd be fine with that. ### What happens if users remove any of the components themselves? As described above, currently, entities can be reused. Removing the audio sink causes it to be "detached" (I kept the old `Drop` impl), so the sound keeps playing. However, if the audio bundle components are not removed, Bevy will detect this entity as a "queued" sound entity again (has the bundle compoenents, without a sink component), just like before playing the sound the first time, and start playing the sound again. This behavior might be surprising? Should we do something different? ### Should mutations to `PlaybackSettings` be applied to the audio sink? We currently do not do that. `PlaybackSettings` is just for the initial settings when the sound starts playing. This is clearly documented. Do we want to keep this behavior, or do we want to allow users to use `PlaybackSettings` instead of `AudioSink`/`SpatialAudioSink` to control sounds during playback too? I think I prefer for them to be kept separate. It is not a bad mental model once you understand it, and it is documented. ### Should `AudioSink` and `SpatialAudioSink` be unified into a single component type? They provide a similar API (via the `AudioSinkPlayback` trait) and it might be annoying for users to have to deal with both of them. The unification could be done using an enum that is matched on internally by the methods. Spatial audio has extra features, so this might make it harder to access. I think we shouldn't. ### Automatic synchronization of spatial sound properties from Transforms? Should Bevy automatically apply changes to Transforms to spatial audio entities? How do we distinguish between listener and emitter? Which one does the transform represent? Where should the other one come from? Alternatively, leave this problem for now, and address it in a future PR. Or do nothing, and let users deal with it, as shown in the `spatial_audio_2d` and `spatial_audio_3d` examples. --- ## Changelog Added: - `AudioBundle`/`SpatialAudioBundle`, add them to entities to play sounds. Removed: - The `Audio` resource. - `AudioOutput` is no longer `pub`. Changed: - `AudioSink`, `SpatialAudioSink` are now components instead of assets. ## Migration Guide // TODO: write a more detailed migration guide, after the "unsolved questions" are answered and this PR is finalized. Before: ```rust /// Need to store handles somewhere #[derive(Resource)] struct MyMusic { sink: Handle<AudioSink>, } fn play_music( asset_server: Res<AssetServer>, audio: Res<Audio>, audio_sinks: Res<Assets<AudioSink>>, mut commands: Commands, ) { let weak_handle = audio.play_with_settings( asset_server.load("music.ogg"), PlaybackSettings::LOOP.with_volume(0.5), ); // upgrade to strong handle and store it commands.insert_resource(MyMusic { sink: audio_sinks.get_handle(weak_handle), }); } fn toggle_pause_music( audio_sinks: Res<Assets<AudioSink>>, mymusic: Option<Res<MyMusic>>, ) { if let Some(mymusic) = &mymusic { if let Some(sink) = audio_sinks.get(&mymusic.sink) { sink.toggle(); } } } ``` Now: ```rust /// Marker component for our music entity #[derive(Component)] struct MyMusic; fn play_music( mut commands: Commands, asset_server: Res<AssetServer>, ) { commands.spawn(( AudioBundle::from_audio_source(asset_server.load("music.ogg")) .with_settings(PlaybackSettings::LOOP.with_volume(0.5)), MyMusic, )); } fn toggle_pause_music( // `AudioSink` will be inserted by Bevy when the audio starts playing query_music: Query<&AudioSink, With<MyMusic>>, ) { if let Ok(sink) = query.get_single() { sink.toggle(); } } ```
2023-07-07 23:01:17 +00:00
AudioBundle, AudioSink, AudioSinkPlayback, AudioSource, AudioSourceBundle, Decodable,
GlobalVolume, Pitch, PitchBundle, PlaybackSettings, SpatialAudioBundle, SpatialAudioSink,
SpatialAudioSourceBundle, SpatialPitchBundle, SpatialSettings,
};
2020-07-17 02:20:51 +00:00
}
pub use audio::*;
pub use audio_source::*;
pub use pitch::*;
pub use rodio::cpal::Sample as CpalSample;
pub use rodio::source::Source;
pub use rodio::Sample;
pub use sinks::*;
2020-07-17 01:47:51 +00:00
use bevy_app::prelude::*;
use bevy_asset::{AddAsset, Asset};
bevy_audio: ECS-based API redesign (#8424) # Objective Improve the `bevy_audio` API to make it more user-friendly and ECS-idiomatic. This PR is a first-pass at addressing some of the most obvious (to me) problems. In the interest of keeping the scope small, further improvements can be done in future PRs. The current `bevy_audio` API is very clunky to work with, due to how it (ab)uses bevy assets to represent audio sinks. The user needs to write a lot of boilerplate (accessing `Res<Assets<AudioSink>>`) and deal with a lot of cognitive overhead (worry about strong vs. weak handles, etc.) in order to control audio playback. Audio playback is initiated via a centralized `Audio` resource, which makes it difficult to keep track of many different sounds playing in a typical game. Further, everything carries a generic type parameter for the sound source type, making it difficult to mix custom sound sources (such as procedurally generated audio or unofficial formats) with regular audio assets. Let's fix these issues. ## Solution Refactor `bevy_audio` to a more idiomatic ECS API. Remove the `Audio` resource. Do everything via entities and components instead. Audio playback data is now stored in components: - `PlaybackSettings`, `SpatialSettings`, `Handle<AudioSource>` are now components. The user inserts them to tell Bevy to play a sound and configure the initial playback parameters. - `AudioSink`, `SpatialAudioSink` are now components instead of special magical "asset" types. They are inserted by Bevy when it actually begins playing the sound, and can be queried for by the user in order to control the sound during playback. Bundles: `AudioBundle` and `SpatialAudioBundle` are available to make it easy for users to play sounds. Spawn an entity with one of these bundles (or insert them to a complex entity alongside other stuff) to play a sound. Each entity represents a sound to be played. There is also a new "auto-despawn" feature (activated using `PlaybackSettings`), which, if enabled, tells Bevy to despawn entities when the sink playback finishes. This allows for "fire-and-forget" sound playback. Users can simply spawn entities whenever they want to play sounds and not have to worry about leaking memory. ## Unsolved Questions I think the current design is *fine*. I'd be happy for it to be merged. It has some possibly-surprising usability pitfalls, but I think it is still much better than the old `bevy_audio`. Here are some discussion questions for things that we could further improve. I'm undecided on these questions, which is why I didn't implement them. We should decide which of these should be addressed in this PR, and what should be left for future PRs. Or if they should be addressed at all. ### What happens when sounds start playing? Currently, the audio sink components are inserted and the bundle components are kept. Should Bevy remove the bundle components? Something else? The current design allows an entity to be reused for playing the same sound with the same parameters repeatedly. This is a niche use case I'd like to be supported, but if we have to give it up for a simpler design, I'd be fine with that. ### What happens if users remove any of the components themselves? As described above, currently, entities can be reused. Removing the audio sink causes it to be "detached" (I kept the old `Drop` impl), so the sound keeps playing. However, if the audio bundle components are not removed, Bevy will detect this entity as a "queued" sound entity again (has the bundle compoenents, without a sink component), just like before playing the sound the first time, and start playing the sound again. This behavior might be surprising? Should we do something different? ### Should mutations to `PlaybackSettings` be applied to the audio sink? We currently do not do that. `PlaybackSettings` is just for the initial settings when the sound starts playing. This is clearly documented. Do we want to keep this behavior, or do we want to allow users to use `PlaybackSettings` instead of `AudioSink`/`SpatialAudioSink` to control sounds during playback too? I think I prefer for them to be kept separate. It is not a bad mental model once you understand it, and it is documented. ### Should `AudioSink` and `SpatialAudioSink` be unified into a single component type? They provide a similar API (via the `AudioSinkPlayback` trait) and it might be annoying for users to have to deal with both of them. The unification could be done using an enum that is matched on internally by the methods. Spatial audio has extra features, so this might make it harder to access. I think we shouldn't. ### Automatic synchronization of spatial sound properties from Transforms? Should Bevy automatically apply changes to Transforms to spatial audio entities? How do we distinguish between listener and emitter? Which one does the transform represent? Where should the other one come from? Alternatively, leave this problem for now, and address it in a future PR. Or do nothing, and let users deal with it, as shown in the `spatial_audio_2d` and `spatial_audio_3d` examples. --- ## Changelog Added: - `AudioBundle`/`SpatialAudioBundle`, add them to entities to play sounds. Removed: - The `Audio` resource. - `AudioOutput` is no longer `pub`. Changed: - `AudioSink`, `SpatialAudioSink` are now components instead of assets. ## Migration Guide // TODO: write a more detailed migration guide, after the "unsolved questions" are answered and this PR is finalized. Before: ```rust /// Need to store handles somewhere #[derive(Resource)] struct MyMusic { sink: Handle<AudioSink>, } fn play_music( asset_server: Res<AssetServer>, audio: Res<Audio>, audio_sinks: Res<Assets<AudioSink>>, mut commands: Commands, ) { let weak_handle = audio.play_with_settings( asset_server.load("music.ogg"), PlaybackSettings::LOOP.with_volume(0.5), ); // upgrade to strong handle and store it commands.insert_resource(MyMusic { sink: audio_sinks.get_handle(weak_handle), }); } fn toggle_pause_music( audio_sinks: Res<Assets<AudioSink>>, mymusic: Option<Res<MyMusic>>, ) { if let Some(mymusic) = &mymusic { if let Some(sink) = audio_sinks.get(&mymusic.sink) { sink.toggle(); } } } ``` Now: ```rust /// Marker component for our music entity #[derive(Component)] struct MyMusic; fn play_music( mut commands: Commands, asset_server: Res<AssetServer>, ) { commands.spawn(( AudioBundle::from_audio_source(asset_server.load("music.ogg")) .with_settings(PlaybackSettings::LOOP.with_volume(0.5)), MyMusic, )); } fn toggle_pause_music( // `AudioSink` will be inserted by Bevy when the audio starts playing query_music: Query<&AudioSink, With<MyMusic>>, ) { if let Ok(sink) = query.get_single() { sink.toggle(); } } ```
2023-07-07 23:01:17 +00:00
use bevy_ecs::prelude::*;
use audio_output::*;
/// Set for the audio playback systems, so they can share a run condition
#[derive(SystemSet, Debug, Default, Clone, Copy, PartialEq, Eq, Hash)]
struct AudioPlaySet;
/// Adds support for audio playback to a Bevy Application
///
bevy_audio: ECS-based API redesign (#8424) # Objective Improve the `bevy_audio` API to make it more user-friendly and ECS-idiomatic. This PR is a first-pass at addressing some of the most obvious (to me) problems. In the interest of keeping the scope small, further improvements can be done in future PRs. The current `bevy_audio` API is very clunky to work with, due to how it (ab)uses bevy assets to represent audio sinks. The user needs to write a lot of boilerplate (accessing `Res<Assets<AudioSink>>`) and deal with a lot of cognitive overhead (worry about strong vs. weak handles, etc.) in order to control audio playback. Audio playback is initiated via a centralized `Audio` resource, which makes it difficult to keep track of many different sounds playing in a typical game. Further, everything carries a generic type parameter for the sound source type, making it difficult to mix custom sound sources (such as procedurally generated audio or unofficial formats) with regular audio assets. Let's fix these issues. ## Solution Refactor `bevy_audio` to a more idiomatic ECS API. Remove the `Audio` resource. Do everything via entities and components instead. Audio playback data is now stored in components: - `PlaybackSettings`, `SpatialSettings`, `Handle<AudioSource>` are now components. The user inserts them to tell Bevy to play a sound and configure the initial playback parameters. - `AudioSink`, `SpatialAudioSink` are now components instead of special magical "asset" types. They are inserted by Bevy when it actually begins playing the sound, and can be queried for by the user in order to control the sound during playback. Bundles: `AudioBundle` and `SpatialAudioBundle` are available to make it easy for users to play sounds. Spawn an entity with one of these bundles (or insert them to a complex entity alongside other stuff) to play a sound. Each entity represents a sound to be played. There is also a new "auto-despawn" feature (activated using `PlaybackSettings`), which, if enabled, tells Bevy to despawn entities when the sink playback finishes. This allows for "fire-and-forget" sound playback. Users can simply spawn entities whenever they want to play sounds and not have to worry about leaking memory. ## Unsolved Questions I think the current design is *fine*. I'd be happy for it to be merged. It has some possibly-surprising usability pitfalls, but I think it is still much better than the old `bevy_audio`. Here are some discussion questions for things that we could further improve. I'm undecided on these questions, which is why I didn't implement them. We should decide which of these should be addressed in this PR, and what should be left for future PRs. Or if they should be addressed at all. ### What happens when sounds start playing? Currently, the audio sink components are inserted and the bundle components are kept. Should Bevy remove the bundle components? Something else? The current design allows an entity to be reused for playing the same sound with the same parameters repeatedly. This is a niche use case I'd like to be supported, but if we have to give it up for a simpler design, I'd be fine with that. ### What happens if users remove any of the components themselves? As described above, currently, entities can be reused. Removing the audio sink causes it to be "detached" (I kept the old `Drop` impl), so the sound keeps playing. However, if the audio bundle components are not removed, Bevy will detect this entity as a "queued" sound entity again (has the bundle compoenents, without a sink component), just like before playing the sound the first time, and start playing the sound again. This behavior might be surprising? Should we do something different? ### Should mutations to `PlaybackSettings` be applied to the audio sink? We currently do not do that. `PlaybackSettings` is just for the initial settings when the sound starts playing. This is clearly documented. Do we want to keep this behavior, or do we want to allow users to use `PlaybackSettings` instead of `AudioSink`/`SpatialAudioSink` to control sounds during playback too? I think I prefer for them to be kept separate. It is not a bad mental model once you understand it, and it is documented. ### Should `AudioSink` and `SpatialAudioSink` be unified into a single component type? They provide a similar API (via the `AudioSinkPlayback` trait) and it might be annoying for users to have to deal with both of them. The unification could be done using an enum that is matched on internally by the methods. Spatial audio has extra features, so this might make it harder to access. I think we shouldn't. ### Automatic synchronization of spatial sound properties from Transforms? Should Bevy automatically apply changes to Transforms to spatial audio entities? How do we distinguish between listener and emitter? Which one does the transform represent? Where should the other one come from? Alternatively, leave this problem for now, and address it in a future PR. Or do nothing, and let users deal with it, as shown in the `spatial_audio_2d` and `spatial_audio_3d` examples. --- ## Changelog Added: - `AudioBundle`/`SpatialAudioBundle`, add them to entities to play sounds. Removed: - The `Audio` resource. - `AudioOutput` is no longer `pub`. Changed: - `AudioSink`, `SpatialAudioSink` are now components instead of assets. ## Migration Guide // TODO: write a more detailed migration guide, after the "unsolved questions" are answered and this PR is finalized. Before: ```rust /// Need to store handles somewhere #[derive(Resource)] struct MyMusic { sink: Handle<AudioSink>, } fn play_music( asset_server: Res<AssetServer>, audio: Res<Audio>, audio_sinks: Res<Assets<AudioSink>>, mut commands: Commands, ) { let weak_handle = audio.play_with_settings( asset_server.load("music.ogg"), PlaybackSettings::LOOP.with_volume(0.5), ); // upgrade to strong handle and store it commands.insert_resource(MyMusic { sink: audio_sinks.get_handle(weak_handle), }); } fn toggle_pause_music( audio_sinks: Res<Assets<AudioSink>>, mymusic: Option<Res<MyMusic>>, ) { if let Some(mymusic) = &mymusic { if let Some(sink) = audio_sinks.get(&mymusic.sink) { sink.toggle(); } } } ``` Now: ```rust /// Marker component for our music entity #[derive(Component)] struct MyMusic; fn play_music( mut commands: Commands, asset_server: Res<AssetServer>, ) { commands.spawn(( AudioBundle::from_audio_source(asset_server.load("music.ogg")) .with_settings(PlaybackSettings::LOOP.with_volume(0.5)), MyMusic, )); } fn toggle_pause_music( // `AudioSink` will be inserted by Bevy when the audio starts playing query_music: Query<&AudioSink, With<MyMusic>>, ) { if let Ok(sink) = query.get_single() { sink.toggle(); } } ```
2023-07-07 23:01:17 +00:00
/// Insert an [`AudioBundle`] or [`SpatialAudioBundle`] onto your entities to play audio.
#[derive(Default)]
pub struct AudioPlugin {
bevy_audio: ECS-based API redesign (#8424) # Objective Improve the `bevy_audio` API to make it more user-friendly and ECS-idiomatic. This PR is a first-pass at addressing some of the most obvious (to me) problems. In the interest of keeping the scope small, further improvements can be done in future PRs. The current `bevy_audio` API is very clunky to work with, due to how it (ab)uses bevy assets to represent audio sinks. The user needs to write a lot of boilerplate (accessing `Res<Assets<AudioSink>>`) and deal with a lot of cognitive overhead (worry about strong vs. weak handles, etc.) in order to control audio playback. Audio playback is initiated via a centralized `Audio` resource, which makes it difficult to keep track of many different sounds playing in a typical game. Further, everything carries a generic type parameter for the sound source type, making it difficult to mix custom sound sources (such as procedurally generated audio or unofficial formats) with regular audio assets. Let's fix these issues. ## Solution Refactor `bevy_audio` to a more idiomatic ECS API. Remove the `Audio` resource. Do everything via entities and components instead. Audio playback data is now stored in components: - `PlaybackSettings`, `SpatialSettings`, `Handle<AudioSource>` are now components. The user inserts them to tell Bevy to play a sound and configure the initial playback parameters. - `AudioSink`, `SpatialAudioSink` are now components instead of special magical "asset" types. They are inserted by Bevy when it actually begins playing the sound, and can be queried for by the user in order to control the sound during playback. Bundles: `AudioBundle` and `SpatialAudioBundle` are available to make it easy for users to play sounds. Spawn an entity with one of these bundles (or insert them to a complex entity alongside other stuff) to play a sound. Each entity represents a sound to be played. There is also a new "auto-despawn" feature (activated using `PlaybackSettings`), which, if enabled, tells Bevy to despawn entities when the sink playback finishes. This allows for "fire-and-forget" sound playback. Users can simply spawn entities whenever they want to play sounds and not have to worry about leaking memory. ## Unsolved Questions I think the current design is *fine*. I'd be happy for it to be merged. It has some possibly-surprising usability pitfalls, but I think it is still much better than the old `bevy_audio`. Here are some discussion questions for things that we could further improve. I'm undecided on these questions, which is why I didn't implement them. We should decide which of these should be addressed in this PR, and what should be left for future PRs. Or if they should be addressed at all. ### What happens when sounds start playing? Currently, the audio sink components are inserted and the bundle components are kept. Should Bevy remove the bundle components? Something else? The current design allows an entity to be reused for playing the same sound with the same parameters repeatedly. This is a niche use case I'd like to be supported, but if we have to give it up for a simpler design, I'd be fine with that. ### What happens if users remove any of the components themselves? As described above, currently, entities can be reused. Removing the audio sink causes it to be "detached" (I kept the old `Drop` impl), so the sound keeps playing. However, if the audio bundle components are not removed, Bevy will detect this entity as a "queued" sound entity again (has the bundle compoenents, without a sink component), just like before playing the sound the first time, and start playing the sound again. This behavior might be surprising? Should we do something different? ### Should mutations to `PlaybackSettings` be applied to the audio sink? We currently do not do that. `PlaybackSettings` is just for the initial settings when the sound starts playing. This is clearly documented. Do we want to keep this behavior, or do we want to allow users to use `PlaybackSettings` instead of `AudioSink`/`SpatialAudioSink` to control sounds during playback too? I think I prefer for them to be kept separate. It is not a bad mental model once you understand it, and it is documented. ### Should `AudioSink` and `SpatialAudioSink` be unified into a single component type? They provide a similar API (via the `AudioSinkPlayback` trait) and it might be annoying for users to have to deal with both of them. The unification could be done using an enum that is matched on internally by the methods. Spatial audio has extra features, so this might make it harder to access. I think we shouldn't. ### Automatic synchronization of spatial sound properties from Transforms? Should Bevy automatically apply changes to Transforms to spatial audio entities? How do we distinguish between listener and emitter? Which one does the transform represent? Where should the other one come from? Alternatively, leave this problem for now, and address it in a future PR. Or do nothing, and let users deal with it, as shown in the `spatial_audio_2d` and `spatial_audio_3d` examples. --- ## Changelog Added: - `AudioBundle`/`SpatialAudioBundle`, add them to entities to play sounds. Removed: - The `Audio` resource. - `AudioOutput` is no longer `pub`. Changed: - `AudioSink`, `SpatialAudioSink` are now components instead of assets. ## Migration Guide // TODO: write a more detailed migration guide, after the "unsolved questions" are answered and this PR is finalized. Before: ```rust /// Need to store handles somewhere #[derive(Resource)] struct MyMusic { sink: Handle<AudioSink>, } fn play_music( asset_server: Res<AssetServer>, audio: Res<Audio>, audio_sinks: Res<Assets<AudioSink>>, mut commands: Commands, ) { let weak_handle = audio.play_with_settings( asset_server.load("music.ogg"), PlaybackSettings::LOOP.with_volume(0.5), ); // upgrade to strong handle and store it commands.insert_resource(MyMusic { sink: audio_sinks.get_handle(weak_handle), }); } fn toggle_pause_music( audio_sinks: Res<Assets<AudioSink>>, mymusic: Option<Res<MyMusic>>, ) { if let Some(mymusic) = &mymusic { if let Some(sink) = audio_sinks.get(&mymusic.sink) { sink.toggle(); } } } ``` Now: ```rust /// Marker component for our music entity #[derive(Component)] struct MyMusic; fn play_music( mut commands: Commands, asset_server: Res<AssetServer>, ) { commands.spawn(( AudioBundle::from_audio_source(asset_server.load("music.ogg")) .with_settings(PlaybackSettings::LOOP.with_volume(0.5)), MyMusic, )); } fn toggle_pause_music( // `AudioSink` will be inserted by Bevy when the audio starts playing query_music: Query<&AudioSink, With<MyMusic>>, ) { if let Ok(sink) = query.get_single() { sink.toggle(); } } ```
2023-07-07 23:01:17 +00:00
/// The global volume for all audio entities with a [`Volume::Relative`] volume.
pub global_volume: GlobalVolume,
}
2020-08-08 03:22:17 +00:00
impl Plugin for AudioPlugin {
fn build(&self, app: &mut App) {
bevy_audio: ECS-based API redesign (#8424) # Objective Improve the `bevy_audio` API to make it more user-friendly and ECS-idiomatic. This PR is a first-pass at addressing some of the most obvious (to me) problems. In the interest of keeping the scope small, further improvements can be done in future PRs. The current `bevy_audio` API is very clunky to work with, due to how it (ab)uses bevy assets to represent audio sinks. The user needs to write a lot of boilerplate (accessing `Res<Assets<AudioSink>>`) and deal with a lot of cognitive overhead (worry about strong vs. weak handles, etc.) in order to control audio playback. Audio playback is initiated via a centralized `Audio` resource, which makes it difficult to keep track of many different sounds playing in a typical game. Further, everything carries a generic type parameter for the sound source type, making it difficult to mix custom sound sources (such as procedurally generated audio or unofficial formats) with regular audio assets. Let's fix these issues. ## Solution Refactor `bevy_audio` to a more idiomatic ECS API. Remove the `Audio` resource. Do everything via entities and components instead. Audio playback data is now stored in components: - `PlaybackSettings`, `SpatialSettings`, `Handle<AudioSource>` are now components. The user inserts them to tell Bevy to play a sound and configure the initial playback parameters. - `AudioSink`, `SpatialAudioSink` are now components instead of special magical "asset" types. They are inserted by Bevy when it actually begins playing the sound, and can be queried for by the user in order to control the sound during playback. Bundles: `AudioBundle` and `SpatialAudioBundle` are available to make it easy for users to play sounds. Spawn an entity with one of these bundles (or insert them to a complex entity alongside other stuff) to play a sound. Each entity represents a sound to be played. There is also a new "auto-despawn" feature (activated using `PlaybackSettings`), which, if enabled, tells Bevy to despawn entities when the sink playback finishes. This allows for "fire-and-forget" sound playback. Users can simply spawn entities whenever they want to play sounds and not have to worry about leaking memory. ## Unsolved Questions I think the current design is *fine*. I'd be happy for it to be merged. It has some possibly-surprising usability pitfalls, but I think it is still much better than the old `bevy_audio`. Here are some discussion questions for things that we could further improve. I'm undecided on these questions, which is why I didn't implement them. We should decide which of these should be addressed in this PR, and what should be left for future PRs. Or if they should be addressed at all. ### What happens when sounds start playing? Currently, the audio sink components are inserted and the bundle components are kept. Should Bevy remove the bundle components? Something else? The current design allows an entity to be reused for playing the same sound with the same parameters repeatedly. This is a niche use case I'd like to be supported, but if we have to give it up for a simpler design, I'd be fine with that. ### What happens if users remove any of the components themselves? As described above, currently, entities can be reused. Removing the audio sink causes it to be "detached" (I kept the old `Drop` impl), so the sound keeps playing. However, if the audio bundle components are not removed, Bevy will detect this entity as a "queued" sound entity again (has the bundle compoenents, without a sink component), just like before playing the sound the first time, and start playing the sound again. This behavior might be surprising? Should we do something different? ### Should mutations to `PlaybackSettings` be applied to the audio sink? We currently do not do that. `PlaybackSettings` is just for the initial settings when the sound starts playing. This is clearly documented. Do we want to keep this behavior, or do we want to allow users to use `PlaybackSettings` instead of `AudioSink`/`SpatialAudioSink` to control sounds during playback too? I think I prefer for them to be kept separate. It is not a bad mental model once you understand it, and it is documented. ### Should `AudioSink` and `SpatialAudioSink` be unified into a single component type? They provide a similar API (via the `AudioSinkPlayback` trait) and it might be annoying for users to have to deal with both of them. The unification could be done using an enum that is matched on internally by the methods. Spatial audio has extra features, so this might make it harder to access. I think we shouldn't. ### Automatic synchronization of spatial sound properties from Transforms? Should Bevy automatically apply changes to Transforms to spatial audio entities? How do we distinguish between listener and emitter? Which one does the transform represent? Where should the other one come from? Alternatively, leave this problem for now, and address it in a future PR. Or do nothing, and let users deal with it, as shown in the `spatial_audio_2d` and `spatial_audio_3d` examples. --- ## Changelog Added: - `AudioBundle`/`SpatialAudioBundle`, add them to entities to play sounds. Removed: - The `Audio` resource. - `AudioOutput` is no longer `pub`. Changed: - `AudioSink`, `SpatialAudioSink` are now components instead of assets. ## Migration Guide // TODO: write a more detailed migration guide, after the "unsolved questions" are answered and this PR is finalized. Before: ```rust /// Need to store handles somewhere #[derive(Resource)] struct MyMusic { sink: Handle<AudioSink>, } fn play_music( asset_server: Res<AssetServer>, audio: Res<Audio>, audio_sinks: Res<Assets<AudioSink>>, mut commands: Commands, ) { let weak_handle = audio.play_with_settings( asset_server.load("music.ogg"), PlaybackSettings::LOOP.with_volume(0.5), ); // upgrade to strong handle and store it commands.insert_resource(MyMusic { sink: audio_sinks.get_handle(weak_handle), }); } fn toggle_pause_music( audio_sinks: Res<Assets<AudioSink>>, mymusic: Option<Res<MyMusic>>, ) { if let Some(mymusic) = &mymusic { if let Some(sink) = audio_sinks.get(&mymusic.sink) { sink.toggle(); } } } ``` Now: ```rust /// Marker component for our music entity #[derive(Component)] struct MyMusic; fn play_music( mut commands: Commands, asset_server: Res<AssetServer>, ) { commands.spawn(( AudioBundle::from_audio_source(asset_server.load("music.ogg")) .with_settings(PlaybackSettings::LOOP.with_volume(0.5)), MyMusic, )); } fn toggle_pause_music( // `AudioSink` will be inserted by Bevy when the audio starts playing query_music: Query<&AudioSink, With<MyMusic>>, ) { if let Ok(sink) = query.get_single() { sink.toggle(); } } ```
2023-07-07 23:01:17 +00:00
app.insert_resource(self.global_volume)
.configure_set(PostUpdate, AudioPlaySet.run_if(audio_output_available))
.init_resource::<AudioOutput>();
#[cfg(any(feature = "mp3", feature = "flac", feature = "wav", feature = "vorbis"))]
bevy_audio: ECS-based API redesign (#8424) # Objective Improve the `bevy_audio` API to make it more user-friendly and ECS-idiomatic. This PR is a first-pass at addressing some of the most obvious (to me) problems. In the interest of keeping the scope small, further improvements can be done in future PRs. The current `bevy_audio` API is very clunky to work with, due to how it (ab)uses bevy assets to represent audio sinks. The user needs to write a lot of boilerplate (accessing `Res<Assets<AudioSink>>`) and deal with a lot of cognitive overhead (worry about strong vs. weak handles, etc.) in order to control audio playback. Audio playback is initiated via a centralized `Audio` resource, which makes it difficult to keep track of many different sounds playing in a typical game. Further, everything carries a generic type parameter for the sound source type, making it difficult to mix custom sound sources (such as procedurally generated audio or unofficial formats) with regular audio assets. Let's fix these issues. ## Solution Refactor `bevy_audio` to a more idiomatic ECS API. Remove the `Audio` resource. Do everything via entities and components instead. Audio playback data is now stored in components: - `PlaybackSettings`, `SpatialSettings`, `Handle<AudioSource>` are now components. The user inserts them to tell Bevy to play a sound and configure the initial playback parameters. - `AudioSink`, `SpatialAudioSink` are now components instead of special magical "asset" types. They are inserted by Bevy when it actually begins playing the sound, and can be queried for by the user in order to control the sound during playback. Bundles: `AudioBundle` and `SpatialAudioBundle` are available to make it easy for users to play sounds. Spawn an entity with one of these bundles (or insert them to a complex entity alongside other stuff) to play a sound. Each entity represents a sound to be played. There is also a new "auto-despawn" feature (activated using `PlaybackSettings`), which, if enabled, tells Bevy to despawn entities when the sink playback finishes. This allows for "fire-and-forget" sound playback. Users can simply spawn entities whenever they want to play sounds and not have to worry about leaking memory. ## Unsolved Questions I think the current design is *fine*. I'd be happy for it to be merged. It has some possibly-surprising usability pitfalls, but I think it is still much better than the old `bevy_audio`. Here are some discussion questions for things that we could further improve. I'm undecided on these questions, which is why I didn't implement them. We should decide which of these should be addressed in this PR, and what should be left for future PRs. Or if they should be addressed at all. ### What happens when sounds start playing? Currently, the audio sink components are inserted and the bundle components are kept. Should Bevy remove the bundle components? Something else? The current design allows an entity to be reused for playing the same sound with the same parameters repeatedly. This is a niche use case I'd like to be supported, but if we have to give it up for a simpler design, I'd be fine with that. ### What happens if users remove any of the components themselves? As described above, currently, entities can be reused. Removing the audio sink causes it to be "detached" (I kept the old `Drop` impl), so the sound keeps playing. However, if the audio bundle components are not removed, Bevy will detect this entity as a "queued" sound entity again (has the bundle compoenents, without a sink component), just like before playing the sound the first time, and start playing the sound again. This behavior might be surprising? Should we do something different? ### Should mutations to `PlaybackSettings` be applied to the audio sink? We currently do not do that. `PlaybackSettings` is just for the initial settings when the sound starts playing. This is clearly documented. Do we want to keep this behavior, or do we want to allow users to use `PlaybackSettings` instead of `AudioSink`/`SpatialAudioSink` to control sounds during playback too? I think I prefer for them to be kept separate. It is not a bad mental model once you understand it, and it is documented. ### Should `AudioSink` and `SpatialAudioSink` be unified into a single component type? They provide a similar API (via the `AudioSinkPlayback` trait) and it might be annoying for users to have to deal with both of them. The unification could be done using an enum that is matched on internally by the methods. Spatial audio has extra features, so this might make it harder to access. I think we shouldn't. ### Automatic synchronization of spatial sound properties from Transforms? Should Bevy automatically apply changes to Transforms to spatial audio entities? How do we distinguish between listener and emitter? Which one does the transform represent? Where should the other one come from? Alternatively, leave this problem for now, and address it in a future PR. Or do nothing, and let users deal with it, as shown in the `spatial_audio_2d` and `spatial_audio_3d` examples. --- ## Changelog Added: - `AudioBundle`/`SpatialAudioBundle`, add them to entities to play sounds. Removed: - The `Audio` resource. - `AudioOutput` is no longer `pub`. Changed: - `AudioSink`, `SpatialAudioSink` are now components instead of assets. ## Migration Guide // TODO: write a more detailed migration guide, after the "unsolved questions" are answered and this PR is finalized. Before: ```rust /// Need to store handles somewhere #[derive(Resource)] struct MyMusic { sink: Handle<AudioSink>, } fn play_music( asset_server: Res<AssetServer>, audio: Res<Audio>, audio_sinks: Res<Assets<AudioSink>>, mut commands: Commands, ) { let weak_handle = audio.play_with_settings( asset_server.load("music.ogg"), PlaybackSettings::LOOP.with_volume(0.5), ); // upgrade to strong handle and store it commands.insert_resource(MyMusic { sink: audio_sinks.get_handle(weak_handle), }); } fn toggle_pause_music( audio_sinks: Res<Assets<AudioSink>>, mymusic: Option<Res<MyMusic>>, ) { if let Some(mymusic) = &mymusic { if let Some(sink) = audio_sinks.get(&mymusic.sink) { sink.toggle(); } } } ``` Now: ```rust /// Marker component for our music entity #[derive(Component)] struct MyMusic; fn play_music( mut commands: Commands, asset_server: Res<AssetServer>, ) { commands.spawn(( AudioBundle::from_audio_source(asset_server.load("music.ogg")) .with_settings(PlaybackSettings::LOOP.with_volume(0.5)), MyMusic, )); } fn toggle_pause_music( // `AudioSink` will be inserted by Bevy when the audio starts playing query_music: Query<&AudioSink, With<MyMusic>>, ) { if let Ok(sink) = query.get_single() { sink.toggle(); } } ```
2023-07-07 23:01:17 +00:00
{
app.add_audio_source::<AudioSource>();
app.init_asset_loader::<AudioLoader>();
}
app.add_audio_source::<Pitch>();
}
}
impl AddAudioSource for App {
fn add_audio_source<T>(&mut self) -> &mut Self
where
T: Decodable + Asset,
f32: rodio::cpal::FromSample<T::DecoderItem>,
{
bevy_audio: ECS-based API redesign (#8424) # Objective Improve the `bevy_audio` API to make it more user-friendly and ECS-idiomatic. This PR is a first-pass at addressing some of the most obvious (to me) problems. In the interest of keeping the scope small, further improvements can be done in future PRs. The current `bevy_audio` API is very clunky to work with, due to how it (ab)uses bevy assets to represent audio sinks. The user needs to write a lot of boilerplate (accessing `Res<Assets<AudioSink>>`) and deal with a lot of cognitive overhead (worry about strong vs. weak handles, etc.) in order to control audio playback. Audio playback is initiated via a centralized `Audio` resource, which makes it difficult to keep track of many different sounds playing in a typical game. Further, everything carries a generic type parameter for the sound source type, making it difficult to mix custom sound sources (such as procedurally generated audio or unofficial formats) with regular audio assets. Let's fix these issues. ## Solution Refactor `bevy_audio` to a more idiomatic ECS API. Remove the `Audio` resource. Do everything via entities and components instead. Audio playback data is now stored in components: - `PlaybackSettings`, `SpatialSettings`, `Handle<AudioSource>` are now components. The user inserts them to tell Bevy to play a sound and configure the initial playback parameters. - `AudioSink`, `SpatialAudioSink` are now components instead of special magical "asset" types. They are inserted by Bevy when it actually begins playing the sound, and can be queried for by the user in order to control the sound during playback. Bundles: `AudioBundle` and `SpatialAudioBundle` are available to make it easy for users to play sounds. Spawn an entity with one of these bundles (or insert them to a complex entity alongside other stuff) to play a sound. Each entity represents a sound to be played. There is also a new "auto-despawn" feature (activated using `PlaybackSettings`), which, if enabled, tells Bevy to despawn entities when the sink playback finishes. This allows for "fire-and-forget" sound playback. Users can simply spawn entities whenever they want to play sounds and not have to worry about leaking memory. ## Unsolved Questions I think the current design is *fine*. I'd be happy for it to be merged. It has some possibly-surprising usability pitfalls, but I think it is still much better than the old `bevy_audio`. Here are some discussion questions for things that we could further improve. I'm undecided on these questions, which is why I didn't implement them. We should decide which of these should be addressed in this PR, and what should be left for future PRs. Or if they should be addressed at all. ### What happens when sounds start playing? Currently, the audio sink components are inserted and the bundle components are kept. Should Bevy remove the bundle components? Something else? The current design allows an entity to be reused for playing the same sound with the same parameters repeatedly. This is a niche use case I'd like to be supported, but if we have to give it up for a simpler design, I'd be fine with that. ### What happens if users remove any of the components themselves? As described above, currently, entities can be reused. Removing the audio sink causes it to be "detached" (I kept the old `Drop` impl), so the sound keeps playing. However, if the audio bundle components are not removed, Bevy will detect this entity as a "queued" sound entity again (has the bundle compoenents, without a sink component), just like before playing the sound the first time, and start playing the sound again. This behavior might be surprising? Should we do something different? ### Should mutations to `PlaybackSettings` be applied to the audio sink? We currently do not do that. `PlaybackSettings` is just for the initial settings when the sound starts playing. This is clearly documented. Do we want to keep this behavior, or do we want to allow users to use `PlaybackSettings` instead of `AudioSink`/`SpatialAudioSink` to control sounds during playback too? I think I prefer for them to be kept separate. It is not a bad mental model once you understand it, and it is documented. ### Should `AudioSink` and `SpatialAudioSink` be unified into a single component type? They provide a similar API (via the `AudioSinkPlayback` trait) and it might be annoying for users to have to deal with both of them. The unification could be done using an enum that is matched on internally by the methods. Spatial audio has extra features, so this might make it harder to access. I think we shouldn't. ### Automatic synchronization of spatial sound properties from Transforms? Should Bevy automatically apply changes to Transforms to spatial audio entities? How do we distinguish between listener and emitter? Which one does the transform represent? Where should the other one come from? Alternatively, leave this problem for now, and address it in a future PR. Or do nothing, and let users deal with it, as shown in the `spatial_audio_2d` and `spatial_audio_3d` examples. --- ## Changelog Added: - `AudioBundle`/`SpatialAudioBundle`, add them to entities to play sounds. Removed: - The `Audio` resource. - `AudioOutput` is no longer `pub`. Changed: - `AudioSink`, `SpatialAudioSink` are now components instead of assets. ## Migration Guide // TODO: write a more detailed migration guide, after the "unsolved questions" are answered and this PR is finalized. Before: ```rust /// Need to store handles somewhere #[derive(Resource)] struct MyMusic { sink: Handle<AudioSink>, } fn play_music( asset_server: Res<AssetServer>, audio: Res<Audio>, audio_sinks: Res<Assets<AudioSink>>, mut commands: Commands, ) { let weak_handle = audio.play_with_settings( asset_server.load("music.ogg"), PlaybackSettings::LOOP.with_volume(0.5), ); // upgrade to strong handle and store it commands.insert_resource(MyMusic { sink: audio_sinks.get_handle(weak_handle), }); } fn toggle_pause_music( audio_sinks: Res<Assets<AudioSink>>, mymusic: Option<Res<MyMusic>>, ) { if let Some(mymusic) = &mymusic { if let Some(sink) = audio_sinks.get(&mymusic.sink) { sink.toggle(); } } } ``` Now: ```rust /// Marker component for our music entity #[derive(Component)] struct MyMusic; fn play_music( mut commands: Commands, asset_server: Res<AssetServer>, ) { commands.spawn(( AudioBundle::from_audio_source(asset_server.load("music.ogg")) .with_settings(PlaybackSettings::LOOP.with_volume(0.5)), MyMusic, )); } fn toggle_pause_music( // `AudioSink` will be inserted by Bevy when the audio starts playing query_music: Query<&AudioSink, With<MyMusic>>, ) { if let Ok(sink) = query.get_single() { sink.toggle(); } } ```
2023-07-07 23:01:17 +00:00
self.add_asset::<T>().add_systems(
PostUpdate,
play_queued_audio_system::<T>.in_set(AudioPlaySet),
);
self.add_systems(PostUpdate, cleanup_finished_audio::<T>.in_set(AudioPlaySet));
self
}
}