Page 1 of 2

[GZDoom 4.0.0] Glitchy rendering with FXAA on

PostPosted: Tue Apr 09, 2019 11:58 pm
by Cacodemon345
Turning on FXAA will cause the rendering to be glitched like this:
Image
Tested with a AMD Radeon RX 460.

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

PostPosted: Wed Apr 10, 2019 8:11 am
by dpJudas
Are you seeing this only with FXAA or does other postprocessing effects cause similar effects?

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

PostPosted: Wed Apr 10, 2019 8:11 am
by Cacodemon345
dpJudas wrote:Are you seeing this only with FXAA or does other postprocessing effects cause similar effects?
Only FXAA, so far.

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

PostPosted: Wed Apr 17, 2019 2:58 pm
by dpJudas
Finally got around to test this. It doesn't happen on my computer.

Which operating system are you using?

Also, I could use some help from someone with an AMD card to try FXAA and report back. Need to narrow down what the conditions are for this to happen.

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

PostPosted: Thu Apr 18, 2019 4:23 am
by Cacodemon345
dpJudas wrote:Which operating system are you using?
It happens with both Windows 10 and Windows 7.
Edit: Turning off Bloom postprocessing fixes the glitched rendering a bit, only the weapon sprites are affected.

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

PostPosted: Sat May 18, 2019 2:00 am
by MasterBeavis
Postprocessing is causing allot of issues for me

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

PostPosted: Mon Jun 03, 2019 2:01 pm
by Batandy
It's happening to me as well, FXAA set to low:
Image

GPU: RX 580, CPU: AMD FX 8320

If you want me to test anything i'm here.

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

PostPosted: Mon Jun 03, 2019 2:26 pm
by dpJudas
@Batandy: if you turn on dynamic lights and play E1M1 like Cacodemon did, are you also getting halos around the light sources? And is it also isolated to only being FXAA or does other postprocess effects create similar effects?

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

PostPosted: Mon Jun 03, 2019 7:01 pm
by MasterBeavis
Im getting exactly what Batandy is getting and if you turn on anything else in PP it gets worse. no halos though

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

PostPosted: Tue Jun 04, 2019 3:36 am
by _mental_
Fixed in addcad8.

Just out of curiosity, why do we have "Vulkan" in DFrameBuffer::gl_vendorstring instead of actual vendor?
Disabled discard seems to have zero influence on performance, so it's not a big deal anyway.

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

PostPosted: Tue Jun 04, 2019 3:40 am
by Graf Zahl
_mental_ wrote:Disabled discard seems to have zero influence on performance, so it's not a big deal anyway.


That entirely depends on the hardware. When I tested this on an Intel HD 4000 the difference between a shader with and without 'discard' was roughly 50%, this was why it was made optional, i.e. on OpenGL it should only be active when really needed.

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

PostPosted: Tue Jun 04, 2019 8:13 am
by dpJudas
_mental_ wrote:Just out of curiosity, why do we have "Vulkan" in DFrameBuffer::gl_vendorstring instead of actual vendor?

Mostly because I didn't quickly find the right field in the vulkan device properties at the time. You're welcome to map it up to the correct field if you know which matches.

I wonder why this fix works. Did the GLSL compiler miscompile? Does the AMD driver have a bug? Did the removal of the discard simply mean all the pixels now finish simultaneously and by doing so hides a bug in an image pipeline barrier? I have a feeling this bug may return - we'll see in time I guess. :)

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

PostPosted: Tue Jun 04, 2019 8:29 am
by Graf Zahl
My guess would be that NVidia has a lot mess involving image layouts and transitioning. I wouldn't be surprised if AMD needed some specific layouts for this to work.

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

PostPosted: Tue Jun 04, 2019 9:03 am
by _mental_
dpJudas wrote:Mostly because I didn't quickly find the right field in the vulkan device properties at the time. You're welcome to map it up to the correct field if you know which matches.

I mapped three well-known vendor IDs to their corresponding names.

dpJudas wrote:I wonder why this fix works. Did the GLSL compiler miscompile? Does the AMD driver have a bug? Did the removal of the discard simply mean all the pixels now finish simultaneously and by doing so hides a bug in an image pipeline barrier?

I have no idea what happens in their drivers. All I know is the glitch was caused by these lines. Complete removal of this block fixes the problem as well.

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

PostPosted: Tue Jun 04, 2019 10:14 am
by dpJudas
I found the reason for why removing the discard keyword made a difference.

According to PPFXAA::Render the output is a new frame buffer - not the original. The discard therefore leaves a gap in the new one.

As an optimization, the vulkan backend tells the driver it can safely discard the old contents when doing the image transition as we will be overwriting it anyway. I strongly suspect most image transitions do nothing on NVidia, which means the original contents were probably just the contents from last render (like with the OpenGL backend). AMD, on the other hand, took advantage of the hint and decided not to transition the image, leaving more chaotic pixels behind.

In other words: the rendering glitch is technically there also on NVidia and OpenGL - we just never noticed it as the old pixels were correct enough.