Cutscenes don't seem to look for audio as they should.

[sergeirocks100]

Hello.

According to the GZDoom wiki, a cutscene definition is supposed to automatically pair a video with an audio file, if it has the same name and directory as the video file.

I dropped a .ivf file into a PK3 folder named /movies, and i dropped a .ogg file with the same name into the folder as well.

However, when the cutscene started, the audio didn't seem to play.

Even when i created a duplicate .ogg file in /sounds, defined it in SNDINFO, and set the sound on the cutscene to it, it still wouldn't play any audio until i deleted the .ogg file from /movies.

Here's a .pk3 which demonstrates the problem (hit the exit level switch to play the movie):

https://amel.pw/trevie/bugreport1.pk3

As i said before, the ZDoom wiki claims that a cutscene will automatically play an audio file in the video directory with the same name as the video, but it doesn't.

In fact, doing it appears to stop the cutscene audio from playing entirely, even if you define the audio separately.

I'm using GZDoom 4.8.2.

[Enjay]

With both 4.8.2 and the latest Git build (21 September) , I can hear the sound. I think that's what the bug report was about?

I'm not sure that the sound is synchronised in the way that you want. Is the tick sound meant to coincide with the white ball hitting the scrolling white rectangle? If so, that doesn't happen, but I definitely hear it.

[sergeirocks100]

For me, i can't hear any audio from the cutscene unless i set it manually, and there's no audio file with the same name in the video folder.

I have no clue what could be causing it to work for you, and not for me, unless it's something on the hardware or operating system side.

I also had syncing issues with the video and audio as you mentioned, but i wanted to make a separate bug report for that.

I'll try testing it on the latest development build, and see if it still occurs.

EDIT: It still happens on the latest development build.

[Kappes Buur]

As played with GZDoom g4.9pre-206



I assume that the short delay is due to trying to sync

Also works with GZDoom 4.8.2
Does not work with GZDoom 4.7.1 where it reports an error

[Graf Zahl]

The delay with the audio comes from OpenAL's rather high latency. This may need input from Chris to solve - I have no idea how to get around it.

That demo file works fine for me as well.

[sergeirocks100]

I figured out why the audio wasn't playing for me.

I had music turned all the way down in the settings, and the audio on the cutscene, for some reason, uses the music volume slider if it's picked automatically. If you set it manually, however, it seems to use the regular volume slider.

Why one would put it to the music track, and the other to the regular audio track, is beyond me.

[Graf Zahl]

That is simply because the sound gets streamed through the music system to avoid loading them into memory. If you play frame specific sounds they are just done as regular sounds.

[Chris]

The delay with the audio comes from OpenAL's rather high latency. This may need input from Chris to solve - I have no idea how to get around it.

From a quick look in src/common/cutscenes/movieplayer.cpp, it looks like it may need an overhaul. The audio seems to be handled as fire-and-forget, and expects the sound to automatically synchronize with the video even though there's no mechanism in place to make it do so. Even if there was absolutely 0 latency, there's nothing stopping the audio from desyncing due to timer drift (the audio device's clock ticking at a subtly different rate than the main system's clock, which builds up over time). Different systems experience different amounts of drift between different devices, so what seems to work fine for some people may noticeably desync for others.

The movie's audio stream needs to synchronize with the same timing source as the video, and it works best when audio samples are streamed with a queue since it allows more precise timing control. The trick is to be able to measure and correct for the difference in playback position of the audio and video as they play (using the audio sample currently at the DAC, or the video frame being shown, not the decoder position) relative to some "master clock" (a timer used to indicate what timepoint the audio and video should be at for the movie). OpenAL Soft allows getting the sample position at (or relatively close to) the DAC for a given sound source, so it's capable of synchronizing an audio stream with video regardless of what latency there may be.

Audio plays asynchronously, so if you just play it while showing some video frames, there's no guarantee it'll play correctly. Instead, the movie player needs an audio handler to keep tabs on the audio stream playback position along with the movie's master clock. If the audio is detected to be running behind, you skip some samples the next time the audio stream wants some so it "fast forwards" and catches up, and if the audio is ahead, you push some extra samples into the stream to hold it back.

I'll try to play around with it when I get some time, but this is an area I'm unfamiliar with so it may take time.

[Graf Zahl]

I'm aware of this. The player as written was for Raze after all where none of the videos requires strict audio sync. Furthermore, we still need some proper modern video format where audio and video are in the same file. But I haven't made any progress on that front, mainly due to how these libraries are set up and lack of good example code.

[Chris]

Well, I'm making progress. At least, I have a basic understanding of MoviePlayer I think, and I have VpxPlayer using a custom stream for the "soundtrack" instead of the music API. There's no synchronization being done yet, so there's no real change in anything, but I should be able to start working on that tomorrow. I'll also get a draft PR up so others can take a look and make sure I'm not doing anything wrong, along with explaining how it seems it can all work.

[Graf Zahl]

This should word properly now with the recent update of the movie code.