ZDoom with OpenAL

[randi]

ROLLOFF_Log implements I3DL2-compatible attenuation, but doesn't need to be modified before sending to FMOD. If FMOD is going to modify the volume to be on a perceptive volume scale, shouldn't it need to be modified just like ROLLOFF_Custom?

Maybe. I never compared it against FMOD_3D_LOGROLLOFF, but you could be right.

[Chris]

Here's an update against r1244. This adds FFmpeg decoding support, but there's a problem. For some reason, it refuses to open the flacs ZDoom has (as well as those in KDiZD). Even other programs that use FFmpeg (eg. mplayer) won't open them. I can play other flacs fine, and the command-line flac program decodes them fine (but FFmpeg still won't open when recompressed). It does, though, decode other compressed sounds (like the Oggs in KDiZD). So exactly what the problem is, I don't know.

Under Windows, you'll need to get the sources, probably including it with ZDoom's source and building a static lib out of it. I know that may be somewhat of a hefty requirement, but I really don't know of another good cross-platform audio decoding library that can handle all these formats. Under Linux, it should work fine to just use the system-installed version.

All the FFmpeg related stuff is contained in ffmpeg.cpp, which "exposes" a simple API, so switching it out and/or changing it won't need that much work.

The patch also contains changes needed to get it building in Linux/GCC again:

• Makes viewwidth/viewheight declared extern "C" in r_main.h, as that's how they're defined in r_draw.cpp
• Includes i_system.h in thingdef_parse.cpp for strlwr()
• Fixes the macros in thingdef.h (they contain ####, which I assume MSVC treats as ##<empty>##, but GCC tries to append ## and complains)
• Adds p_state.cpp and thingdef/thingdef_parse.cpp to the list of sources in CMakeLists.txt

[Graf Zahl]

Here's an update against r1244. This adds FFmpeg decoding support, but there's a problem. For some reason, it refuses to open the flacs ZDoom has (as well as those in KDiZD). Even other programs that use FFmpeg (eg. mplayer) won't open them. I can play other flacs fine, and the command-line flac program decodes them fine (but FFmpeg still won't open when recompressed). It does, though, decode other compressed sounds (like the Oggs in KDiZD). So exactly what the problem is, I don't know.[/list]

Well, congratulations for the only sound decoding library which I already ruled out completely for use in an OpenAL sound implementation. So I'd say don't bother to investigate. We need something that is easier to set up and compile. In my opinion Its distribution is a mess. Apparently its developers don't give a shit about non-Unix like systems and everything without a proper Windows setup is out by default. Don't forget who the main users of ZDoom are!

Regarding FLAC I suspect it's too restrictive with its format checks. ZDoom's FLACs are mostly 11025 8 bit mono sounds which is a non-standard format and even the FLAC converter only accepts to encode them with '--lax'.

Anyway, I consider it highly stupid to develop a separate decoder for a format that already has a freely usable reference implementation. Same for Ogg Vorbis.

[Chris]

We need something that is easier to set up and compile. In my opinion Its distribution is a mess. Apparently its developers don't give a shit about non-Unix like systems and everything without a proper Windows setup is out by default. Don't forget who the main users of ZDoom are!

Yeah, I know FFmpeg isn't the easiest thing on the planet, especially for Windows. And I'd gladly pick something else if there was something else that could replace it. But short of having ZDoom use the libs directly, there's no cross-platform libs that can decode a good selection of formats. Plus having ZDoom handle MP3 directly would bring in more license issues that you'd want to avoid if you're trying to become, or at least be compatible with, the GPL.

Anyway, I consider it highly stupid to develop a separate decoder for a format that already has a freely usable reference implementation. Same for Ogg Vorbis.

Agreed. But what else can we do, beyond using the various decoder libs directly? Honestly.. I'm open to ideas if there's anything better than FFmpeg..

[Graf Zahl]

I haven't found anything useful so far. Not one single library does all ZDoom needs - many are a nightmare to use, others are so inflexible that it's a waste of time thinking about it and the ones that look semi-promising show decoding errors which render them unusable.

That said, FFMPeg doesn't resolve any MP3 issues because on Windows it has to be distributed in binary form. No Windows user will be able to compile this mess of a source distribution.

[Chris]

Well, I've started moving things towards using the codecs directly, then. So far, I've got FLAC++ and libvorbisfile being used in ffmpeg.c (which should be renamed) to open and decode the files.

But, there's yet another issue. Now that flacs are working with FLAC++, a whole lot of continuous looping sounds play on some maps. The second level of KDiZD, for instance, uses up at least 32 channels simultaneously upon level start, for environment/ambient sounds. The third level uses up at least 64. Even with distance and priority culling (as well as distance-based priority culling), I only get the player sounds back.

If I increase snd_channels/the max number of sources to 256, then it works fine.. but that seems like brute-forcing the issue instead of a proper fix. Plus some hardware may be limitted to 32 or 64 sources, so it wouldn't work in all cases anyway.

[Graf Zahl]

HM. Looking at s_sound.cpp it looks like the sound code completely ignores this scenario and happily keeps an active channel for the looping sounds no matter what - even if they are not playing.

If that is the case I'd rather call it an implementation flaw that has to be resolved in the high level sound code. The engine shouldn't map low level channels to high level channels 1:1 if not all are actually used. It wouldn't surprise me the least if the same issues showed up when using these levels with the current FMOD sound implementation as well.

And yes, snd_channels 32 is definitely no longer sufficient with this setup. I think the number of high level channels has to be unlimited and the decisions which sounds to throw out be made by the low level code if it runs out of system channels.

I smell a large bunch of work coming...

[Chris]

HM. Looking at s_sound.cpp it looks like the sound code completely ignores this scenario and happily keeps an active channel for the looping sounds no matter what - even if they are not playing.

If a sound is physically stopped (eg. it reached the end and wasn't looping, or was explicitly stopped), it decouples the source from the high-level channel.. it clears the SysChannel field, and removes the buffer from the source. The next sound that's started can then freely use the source with its high-level channel.

The problem is, the sounds are playing. They're looping ambient sounds. From what I can tell, if a sound type has SOURCE_None, or if the actor is the camera, it's gets a priority of 40. All other sounds get a priority of 0. Because of this, if all sources are taken and a normal non-player sound tries to play, it'll never find a sound that it can kill because the other priorites will always be >= to it. Even if a sound is a quiet ambient wind at the edge of audibility, it'll prevent a barrel exploding in your face from making a sound. And even if you allowed a sound to overtake another sound with equal priority, it would only help for a split second, before the the other sounds are triggered again and re-take it.

EDIT:
What I think needs to happen is ZDoom needs to realize a sound is out of hearing range (ie. is beyond MaxDistance with a non-log rolloff type) and stop the channel itself, and not try to start it again until it comes back in range (for looping sounds). This doesn't seem to be something the individual audio systems need to do because they'll all behave the same.. the sound will never be heard because the attenuated volume must be 0. It could also help to be a bit more.. explicit with priority values, instead of just 40 or 0. Maybe it could have 40 for "camera" sounds, 20 for in-range sounds, and 0 for out-of-range sounds (coupled with not trying to restart ouf-of-range looping sounds).

EDIT2:
This seems to work. If the sound is stopped when it's beyond max distance for non-log rolloff types (calculated attenuation is 0), and I don't even attempt to start a sound that's beyond max distance for non-log rolloff, then even Z1M3 with 32 channels plays fine. But this seems to be something ZDoom should do itself, instead of relying on the sound backend to do it.

[randi]

But this seems to be something ZDoom should do itself, instead of relying on the sound backend to do it.

I disagree. Suppose the sound is just outside your hearing range and then you run toward it. For a lengthy sound, it does make a difference. And in the case of a looping sound, you need a start time to keep it in sync as it moves on and off an active channel.

It also sounds like OpenAL has a primitive and essentially broken priority system if you are having such problems with it. In FMOD, equal priority channels can steal from each other by using the sound volume of each channel as a sub-priority, and the sound code counts on this happening. If OpenAL doesn't do that, then I think you'll have to write your own priority system.

As far as s_sound.cpp is supposed to know, there are an infinite number of channels available. The low level code is responsible for channel management.

[Kate]

Hmm. This would help me get ZDoom working on FreeBSD, which FMOD/Ex is not supported on. I've gotten ZDoom to compile on it (though I had to change every single reference to malloc.h to use stdlib.h instead (malloc.h is deprecated on this platform and including it is an error), and had to fix one stale ifdef and disable a few gcc checks that caused problems) but it cannot link without FMOD, and the linux binaries rely on libraries that FreeBSD simply does not have nor see any use for.

[Chris]

It also sounds like OpenAL has a primitive and essentially broken priority system if you are having such problems with it.

That's the thing. It doesn't have *any* priority system. It just plays what you tell it to. If it doesn't have enough voices/resources, it fails. Otherwise, it succeeds. It relies on the program's higher level functions knowing what should happen when you run out of resources (eg. should another sound be killed, and if so which one, or should it forgo mixing and have its play position tracked, or should it just forget it, etc..). While I can manage a priority system on top of OpenAL here, the higher-level stuff would still have a better idea of what can/is allowed to happen than OpenAL/FMOD or my code could.

From ZDoom's point of view, it can just stop sound A if it's farther away than sound B (3d sounds are not given an explicit volume; they just fade according to distance), and can restart a sound from an easilly calculatable position

Code: Select all

(lastposition + (currentupdate-lastplayedupdate)*timeperupdate*pitch) % soundlength

whereas if OpenAL had to handle it, it would have to do some non-trivial set up (calculate 3d position, distance, attenuation, cones, filters, doppler, etc) which doesn't help if the goal is to not waste processing on too many sounds at once. OpenAL couldn't make assumptions of what ZDoom will do (eg. not use cones, or no explicit volume for 3d sounds), whereas ZDoom has a rather good idea of what OpenAL will do (will play the sound at the calculated rolloff volume, in the direction of source-listener).

As far as s_sound.cpp is supposed to know, there are an infinite number of channels available. The low level code is responsible for channel management.

Then is it safe to assume snd_channels is merely a hint and if more can be used, they should be? Or should it be treated as a ceiling to limit the number of simultaniously playing sounds? It's a bit useless if the audio system is free to take more channels if it can..

[randi]

That's the thing. It doesn't have *any* priority system.

In that case, you'll have to write one.

the higher-level stuff would still have a better idea of what can/is allowed to happen than OpenAL/FMOD or my code could.

FMOD's priority system is fine, AFAICS. I see no need to reinvent the wheel for it. Even if you need to do so for OpenAL, I don't think it should be blindly applied it to FMOD as well.

From ZDoom's point of view, it can just stop sound A if it's farther away than sound B (3d sounds are not given an explicit volume; they just fade according to distance)

FMOD already does this. From the documentation for Channel::setPriority:

By default the importance of the sound (whether it is audible or not when more channels are playing than exist) is based on the volume, or audiblity of the sound. This is determined by distance from the listener in 3d, the volume set with Channel::setVolume, channel group volume, and geometry occlusion factors.

What you are describing is basically all stuff FMOD already does.

Then is it safe to assume snd_channels is merely a hint and if more can be used, they should be? Or should it be treated as a ceiling to limit the number of simultaniously playing sounds?

Snd_channels specifies the number of software voices to mix. How you want to use (or ignore) it is up to you.

[Chris]

The main issue is there's no sense of timing in the soundrenderer.. so if something's killed because of StartSound, later when StopChannel is called or UpdateSounds detects a stopped source, I can try to restart it (though it seems ZDoom already does this, at least for ambient looping sounds).. but I have no idea how much time would have passed. Is there a global "CurrentTick" value I could read?

[Graf Zahl]

What kind of counter do you need?

One that only counts time in the game or one that counts globally?

[Chris]

The game I suppose, since pausing (eg. opening the menu) could effect it. I'd need to store the tick a sound was killed on with the channel, and the play position it was at, then when I get a free source later on, I can increment its play position according to the difference between the current tick and the tick it was killed on, and start it at that offset (wrapping the value if it's looping).