S_Startsound sound environment argument

[kodi]

Heya.
I would like to be able to supply a sound environment ID as an argument for A_StartSound or S_StartSound calls.
Use cases would be having interface audio play undistorted while in a reverberating sector, dynamically applying "radio" distortion to voice clips and allowing simple simulations of audio passing from one kind of environment to another.

I understand there may be big technical limitations in place here - perhaps it's impossible to apply sound environments before final mixing? Whatever the case may be, thanks in advance!

[Chris]

It is possible to add in theory, though I'm not really sure the ideal way to do it. Currently only one environment reverb can be active at a time, and sounds can be excluded from the environment reverb by using the CHANF_UI flag, but can't be given a unique environment. As far as OpenAL Soft (what GZDoom uses for mixing audio and effects) is concerned, it can have up to 64 unique effect processors (which include reverb, chorus, distortion, ring modulation, and some others). One effect can be used by multiple (or all) sound sources, and one sound source can use up to 6 effects in parallel at a given time (+1 dry/no effect output mix).

That said, there has also been a desire for making the audio effects stuff more directly controllable from scripts, which leads into the biggest hurdle for me doing anything. What should the interface be like that will be useful, work with the existing environment effect framework, and be extendable in the future while remaining compatible.

[kodi]

Thanks for the info, it's actually better news than I really expected!
I think the best solution for the modder-facing interface would be a sound environment argument in the sound playing functions, along with a flag or combination of flags that determine whether to add this environment to the chain, to use only this environment and ignore any other, or to discard it if any other environment would affect the sound. For the 64-at-a-time and 6-in-a-row limitations, I would make those global values that can be read (for future-proofing) and leave it to the modder to keep track of things. Any attempt to go past the current limitations would just play the sound without any processing.

[Chris]

The thing is, the sound environment IDs just specify effect parameter presets, not effect processors. Simply specifying an environment ID to a sound to play introduces ambiguity and can create unexpected costs depending on what environments are used (e.g. if you play 10 sounds with 10 unique environments, that'll be 10x the cost compared to playing those same 10 sounds with the same 1 environment). Dynamically managing which environment presets are active will also create additional costs, since an environment shouldn't stop immediately when sounds stop using it (there can be an audible decaying reverb tail that lasts for several seconds after the sound itself is done). And there's the issue that mods should be able to dynamically alter the effect parameters; for example, something like LiveReverb that continuously probes the room around the player to set the active environment, which currently can only select from the set of presets depending on the average distance to the walls, floor, and ceiling (with some checks for the sky), can be better realized by more precisely controlling the early and late reflection delays and gains, and even their general direction (e.g. panning the early reflections toward the closest surfaces) along with other reverb properties. Scripts probably shouldn't be able to modify presets, but they should be able to modify the parameters used.

A more useful interface may instead give scripts the ability to allocate effect processors and set the effect properties for them, while also specifying which sounds apply to which effects. This would allow scripts to control multiple simultaneous environments, for instance if the player is in some large cave, and there's a nearby opening to a sewer pipe, there can be one primary effect for the cave environment the player is in, and a secondary effect for the sewer pipe panned toward where the opening is, while each sound controls how much it contributes to each effect (a sound in the cave with the player will sound like it's in the cave with very little echoing into and back out of the pipe, while a sound in the pipe will sound like it's in the pipe with only some of the cave reverberation). This would also allow scripts to create/manage special individualized effects, such as a distortion effect that's only applied to a specific radio sound or something.

But figuring out how exactly to design an interface for this is what I don't know. I don't know if it's a good idea for scripts to directly allocate, and subsequently be responsible for deallocating, effect processors, considering each one can have a notable cost (honestly, 64 is quite generous, I wouldn't be surprised if even modern systems could struggle well before that limit). Sounds should also be able to adjust how much they apply to each effect as they move (a sound that moves out of or into the aforementioned pipe after it starts, for example), and currently I don't think there's a way for scripts to control individual sounds after they start. I also don't know how this should interact with the sound environment things that control the existing implicit effect processor, along with the underwater effect that's automatically applied when underwater. It might be that some of this is too low-level for scripts to be responsible for, and dynamic multi-effect stuff should be controlled at a higher level that lets the audio engine automatically work out the details. Or maybe not, I really don't know what would be best for GZDoom.

[kodi]

Thank you for the detailed explanation. I'm fairly ignorant about this stuff, so I'm now slowly grasping the complexities involved. The issue with cutting a sound straight off when it's supposed to still be reverberating due to the environment(s) is a very good point. I hope you can find someone knowledgeable enough about interfaces like this this to help you, since the there's so much potential already there in the sound engine - even playing some simple sine tones with the right environment active can make the game sound like a high-grade synthesizer