Fog after after palette emulation

[markanini]

Currently Raze appears to apply fog before palette emulation. Applying it after instead would in theory improve banding, while at the same time keep the original color appearance that's lost in true color mode.

[Graf Zahl]

Fog is part of the shade table, so there is no separate step that could be done. The entire process is identical to what Hexen did with its swamp level.

I cross-checked with VoidSW - what it does is apply two layers of fog - the one baked into the shade table and regular hardware fog on top of it. But that's no longer "palette emulation".

[markanini]

That raises the question, how is fog handled in truecolor mode?
Maybe my idea could be better described as: applying palette emulation to sprites and tiles only.

[Graf Zahl]

In true color mode the shade tables are not used. Instead the defined fog color for the table gets used to create real fog.
Thinking about it, if you use a foggy shade table and want hardware fog it's just like taking shade 0 for everything and applying real fog to it, which is essentially the same as true color mode, just without texture filtering. That should be doable with reasonable effort.

[markanini]

That sounds good.

[Graf Zahl]

Palette emulated fog has been removed some time ago because there's no way to make it work right, even palette emulated mode uses true color fog now.

[markanini]

Are you referring to the Palette postprocessing or the Polymost Palette emulation?

[Graf Zahl]

Emulation.

[markanini]

I'm seeing some stepped fog. Does this mean the palette is applied on top of the parametric tables?

[Graf Zahl]

No, currently foggy sectors do not use any palette emulation. The reason is that if we want real fog we cannot use the light fading that's baked into the palette.
Keep in mind that palette emulation has absolutely no effecti if a sector is being rendered fullbright - but fog needs to be applied on fullbright colors.

[markanini]

Is it not trivial to separate the palettes effect on the light fading, perhaps by making it part of the fog instead?

[Graf Zahl]

They are totally inseparable. The fog is baked into the lookup tables as light diminishing. Foggy sectors therefore have no fade to black effect at all, they only have fade to fog color effect instead.

[markanini]

Is sounded messy to me in my head, but now I know for sure. I'll still provide some alternate postprocessing LUTs when you are ready, similar to Pixel Eaters Troo Cullers mod.