by Chris » Wed Jun 21, 2017 12:29 am
Facínora wrote:this kind of info is kinda fascinating, even though it sounds like greek to me. is it related to the sound waves we see when editing audio?
Basically, yeah. Audio samples use a normalized range which have minimum and maximum possible amplitude values. For 16-bit samples, this is -32768 (the lowest point of a sound wave) to +32767 (the highest point of a sound wave) which is the limits of a signed 16-bit value. For floating point, the normalized range is -1 to +1 (this covers the same "loudness", it's just a different value scale; simply multiply by 32767 or 32768 to get 16-bit sample values). A feature of floating point samples is that they can go outside its normalized range temporarily, so when doing various audio processing tasks you don't need to worry about clipping or wrapping before the mix is done. Internally, most audio mixers use floating point for that reason.
The issue arises if you have floating point samples being converted to 16-bit, and you don't properly account for them being outside the normalized -1 to +1 range. For example, if you have a floating point sample value of +1.01, multiplying by 32767 would give you an integer value of +33094. That's larger than the maximum 16-bit value (32767), so it wraps around and becomes -32442. That's very different from the intended amplitude value, resulting in very audible noise. A more correct approach would be to clamp it to 32767 -- it would still be different than intended, but the noise would be far more subtle (other options include applying gain compression, dynamically reducing the volume around loud noises to prevent large sample values like that). So what could happen is, the game gives floating point samples to the system that are outside the normalized range, FRAPS opens a "read-back" capture device on the system to get the floating point samples being played on the device, and improperly converts them to 16-bit when compressing the audio stream.
Anyway, yes. I think it's a FRAPS issue, but it could capture sounds properly before GZDoom 3. Maybe there is something to do with OpenAL, I don't know.
Probably. By default, OpenAL Soft will output floating point samples if the system will accept them, with the expectation that the system's audio layer may do further processing (mixing sounds from another app, system-level volume control, filtering or some kind of compression, etc) in which case it's best to avoid clamping before the system is done with it (and even if it doesn't do any more processing, there's little harm in letting the system do the float->integer conversion since it's being done regardless). FMOD probably defaults to 16-bit output, implicitly clamping its output to the normalized range before the system (and thus FRAPS) gets it, which is why it didn't happen before.
[quote="Facínora"]this kind of info is kinda fascinating, even though it sounds like greek to me. is it related to the sound waves we see when editing audio?[/quote]
Basically, yeah. Audio samples use a normalized range which have minimum and maximum possible amplitude values. For 16-bit samples, this is -32768 (the lowest point of a sound wave) to +32767 (the highest point of a sound wave) which is the limits of a signed 16-bit value. For floating point, the normalized range is -1 to +1 (this covers the same "loudness", it's just a different value scale; simply multiply by 32767 or 32768 to get 16-bit sample values). A feature of floating point samples is that they can go outside its normalized range temporarily, so when doing various audio processing tasks you don't need to worry about clipping or wrapping before the mix is done. Internally, most audio mixers use floating point for that reason.
The issue arises if you have floating point samples being converted to 16-bit, and you don't properly account for them being outside the normalized -1 to +1 range. For example, if you have a floating point sample value of +1.01, multiplying by 32767 would give you an integer value of +33094. That's larger than the maximum 16-bit value (32767), so it wraps around and becomes -32442. That's very different from the intended amplitude value, resulting in very audible noise. A more correct approach would be to clamp it to 32767 -- it would still be different than intended, but the noise would be far more subtle (other options include applying gain compression, dynamically reducing the volume around loud noises to prevent large sample values like that). So what could happen is, the game gives floating point samples to the system that are outside the normalized range, FRAPS opens a "read-back" capture device on the system to get the floating point samples being played on the device, and improperly converts them to 16-bit when compressing the audio stream.
[quote]Anyway, yes. I think it's a FRAPS issue, but it could capture sounds properly before GZDoom 3. Maybe there is something to do with OpenAL, I don't know.[/quote]
Probably. By default, OpenAL Soft will output floating point samples if the system will accept them, with the expectation that the system's audio layer may do further processing (mixing sounds from another app, system-level volume control, filtering or some kind of compression, etc) in which case it's best to avoid clamping before the system is done with it (and even if it doesn't do any more processing, there's little harm in letting the system do the float->integer conversion since it's being done regardless). FMOD probably defaults to 16-bit output, implicitly clamping its output to the normalized range before the system (and thus FRAPS) gets it, which is why it didn't happen before.