[GZDoom 4.0.0] Glitchy rendering with FXAA on

Forum rules
Please don't bump threads here if you have a problem - it will often be forgotten about if you do. Instead, make a new thread here.

Post a reply

Smilies
:D :) :( :o :shock: :? 8-) :lol: :x :P :oops: :cry: :evil: :twisted: :roll: :wink: :geek: :ugeek: :!: :?: :idea: :arrow: :| :mrgreen: :3: :wub: >:( :blergh:
View more smilies

BBCode is OFF
Smilies are ON

Topic review
   

Expand view Topic review: [GZDoom 4.0.0] Glitchy rendering with FXAA on

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

by Graf Zahl » Tue Jun 04, 2019 9:50 am

dpJudas wrote:I strongly suspect most image transitions do nothing on NVidia, which means the original contents were probably just the contents from last render (like with the OpenGL backend).
I once read in a technical paper from NVidia that they do not have different image layouts and therefore transitioning is a no-op on their current hardware.

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

by _mental_ » Tue Jun 04, 2019 9:45 am

Here is the description for this #define in the original source code.
Probably, I got it wrong during addition of FXAA. Or maybe it worked by occasion, and I left it defined.

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

by dpJudas » Tue Jun 04, 2019 9:37 am

Your fix is correct - removing the discard keyword solves the problem. I do wonder in which situations the discard would work, though. Maybe the FXAA shader was originally made with the assumption the last shader would be rendering into the original frame buffer image. Probably under the assumption the game engine used a more classical setup where all postprocess ends up in the swap chain backbuffer.

Maybe also why they made it optional with that define?

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

by _mental_ » Tue Jun 04, 2019 9:29 am

I see. What should we do with my "workaround" then?

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

by dpJudas » Tue Jun 04, 2019 9:14 am

I found the reason for why removing the discard keyword made a difference.

According to PPFXAA::Render the output is a new frame buffer - not the original. The discard therefore leaves a gap in the new one.

As an optimization, the vulkan backend tells the driver it can safely discard the old contents when doing the image transition as we will be overwriting it anyway. I strongly suspect most image transitions do nothing on NVidia, which means the original contents were probably just the contents from last render (like with the OpenGL backend). AMD, on the other hand, took advantage of the hint and decided not to transition the image, leaving more chaotic pixels behind.

In other words: the rendering glitch is technically there also on NVidia and OpenGL - we just never noticed it as the old pixels were correct enough.

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

by _mental_ » Tue Jun 04, 2019 8:03 am

dpJudas wrote:Mostly because I didn't quickly find the right field in the vulkan device properties at the time. You're welcome to map it up to the correct field if you know which matches.
I mapped three well-known vendor IDs to their corresponding names.
dpJudas wrote:I wonder why this fix works. Did the GLSL compiler miscompile? Does the AMD driver have a bug? Did the removal of the discard simply mean all the pixels now finish simultaneously and by doing so hides a bug in an image pipeline barrier?
I have no idea what happens in their drivers. All I know is the glitch was caused by these lines. Complete removal of this block fixes the problem as well.

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

by Graf Zahl » Tue Jun 04, 2019 7:29 am

My guess would be that NVidia has a lot mess involving image layouts and transitioning. I wouldn't be surprised if AMD needed some specific layouts for this to work.

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

by dpJudas » Tue Jun 04, 2019 7:13 am

_mental_ wrote:Just out of curiosity, why do we have "Vulkan" in DFrameBuffer::gl_vendorstring instead of actual vendor?
Mostly because I didn't quickly find the right field in the vulkan device properties at the time. You're welcome to map it up to the correct field if you know which matches.

I wonder why this fix works. Did the GLSL compiler miscompile? Does the AMD driver have a bug? Did the removal of the discard simply mean all the pixels now finish simultaneously and by doing so hides a bug in an image pipeline barrier? I have a feeling this bug may return - we'll see in time I guess. :)

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

by Graf Zahl » Tue Jun 04, 2019 2:40 am

_mental_ wrote: Disabled discard seems to have zero influence on performance, so it's not a big deal anyway.
That entirely depends on the hardware. When I tested this on an Intel HD 4000 the difference between a shader with and without 'discard' was roughly 50%, this was why it was made optional, i.e. on OpenGL it should only be active when really needed.

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

by _mental_ » Tue Jun 04, 2019 2:36 am

Fixed in addcad8.

Just out of curiosity, why do we have "Vulkan" in DFrameBuffer::gl_vendorstring instead of actual vendor?
Disabled discard seems to have zero influence on performance, so it's not a big deal anyway.

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

by MasterBeavis » Mon Jun 03, 2019 6:01 pm

Im getting exactly what Batandy is getting and if you turn on anything else in PP it gets worse. no halos though

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

by dpJudas » Mon Jun 03, 2019 1:26 pm

@Batandy: if you turn on dynamic lights and play E1M1 like Cacodemon did, are you also getting halos around the light sources? And is it also isolated to only being FXAA or does other postprocess effects create similar effects?

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

by Batandy » Mon Jun 03, 2019 1:01 pm

It's happening to me as well, FXAA set to low:
Image

GPU: RX 580, CPU: AMD FX 8320

If you want me to test anything i'm here.

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

by MasterBeavis » Sat May 18, 2019 1:00 am

Postprocessing is causing allot of issues for me

Re: [GZDoom 4.0.0] Glitchy rendering with FXAA on

by Cacodemon345 » Thu Apr 18, 2019 3:23 am

dpJudas wrote: Which operating system are you using?
It happens with both Windows 10 and Windows 7.
Edit: Turning off Bloom postprocessing fixes the glitched rendering a bit, only the weapon sprites are affected.

Top