GZDoom 3.6.0 official and a git build from the 17th.
Release version of the mod and the development version.
My drivers are current. GeForce 1080 with driver version 416.34 released 11 October.
I've found a GZDoom setting that changes this. If Light Shadowmaps is set to "yes" I get the problem. If it's set to "no" the problem goes away.
Running around in MAP09 of Doom2 after picking up an Unreal Damage powerup (which creates a pink glow around the player), I noticed that things sometimes could go a bit screwy if I looked at the floor.
Should look like:
But sometimes looked like:
It's almost as if the light is z-fighting with the floor.
It's even easier to show on Doom E2M2. Just grab an unreal damage powerup (there should be one where the berserk pack usually is) then climb up on to some crates.
I have also noticed that the light flickers more than it should even if the major glitchiness shown above has not become apparent (a video shows this).
Update. The glitch does not seem to happen with a git build from 24th September. I'll see if I can track down the last git build that works.
[edit]
gzdoom-x64-g3.6pre-103-gd65d46226 from Oct 04 does not glitch (though it did crash the first time I tried to run it - but after that it was fine). All available git builds after this one glitch.
Okay I know partly what is going on here. It only happens when you pick up the amp. I suspect the amp is perfectly aligned with the floor. When you jump the effect goes away until you land again.
The glitch most likely happens when a dynamic light is placed in the same plane as the surface. In the second example you found the dynlight happens to be placed in exactly same plane as the ceiling.
Yup, just been looking at the map setup and dynamic light myself. So, sort-of the same setup as with the Doom Tournament light being exactly on the floor. Perhaps my initial "it looks a bit like zfighting" might not be far off the mark? Could the shadowmap and dynamic light be doing something like zfighting with GZDoom trying to draw them on the same plane?
Anyway, in case it helps, I've made a similar setup and light definition to make it easy to find an area that displays the problem. With any luck, it should be visible to anyone trying to test it. Just load with Doom2. Nothing else needed.
[edit] Improved version showing the effect on the floor and the ceiling [/edit]
I think the answer is "kind of". It looks like any situation that puts a dynamic light on exactly the same plane as a flat with a shadow map on it will cause this to happen. So your mod does it and maybe the light should be raised up a little but it can just as easily happen with regular mapping and dynamic lights if the various positions are right for it (like the test map above).