Unclearable and constant SSAO lag

[Accensus]

Disclaimer: posting this on behalf of a buddy, so I don't know too much about the mod's internals, only what happens and when.

Tested on g4.5pre-61-g4de8851e0 x64. Can only reproduce with Vulkan. Tested with OpenGL but couldn't get it to lag. It might not be impossible, just really, really difficult to do so.
Mod author reports the same issue and according to him it has been present for quite a while. If I understood correctly, even back in 4.3.3.

There are no exact steps because it always seems to be random, but the closest I can come to those is:
1. Start a map, any map. I've had it happen on E1M1 and MAP01. Tested the most on E1M1.
2. IDFA/give all to get all weapons
3. Empty the BFG into a wall. Try shooting the listed weapons in various directions for a better chance of lag.
4. Empty a mag from the rocket launcher (slot 5).
5. Empty the entire flamethrower (slot 5). Use alt-fire as well as that seems to be a good trigger.

NOTE: If it doesn't lag, go to step 2. It's bound to lag. Haven't had to IDFA more than twice before it does.

By the time you're done, "stat gpu" should show about 30 ms for SSAO. It doesn't go up gradually. One moment it's 1 ms and the next it's 30-50 ms. This is NOT cleared when switching maps, which is the main reason I'm reporting this and not writing it down as a user error. The only way to get rid of it is to either restart the game or disable SSAO. Re-enabling it again starts lagging everything down to 4 FPS. It gets worse if you look at outdoor areas like E1M1's blue armor secret.

The mod is private, so link to DL the example will be sent to the developers via PM shortly.

NOTE: The mod is going to take a while to load. It ramps up my RAM usage to about 2 gigs before it goes down to ~800 MB. Starting a level is also going to take about 4-5 seconds.

[Accensus]

Update: Okay so the lag seems to be because I ran out of VRAM. I was using Trilinear with Normal6x and the mod ate 8 gigs of VRAM in no time.

[phantombeta]

I'd like to mention that running out of VRAM with Vulkan (at least on Win 10 with a 10-series Nvidia GPU) seems to be a complete crapshoot of undefined behaviour. In some rare times it'll crash with an "out of device memory" error, but most often it'll use system RAM as swap. (Somehow. Even though supposedly the library GZDoom is using doesn't handle swap)
In some rare cases it'll corrupt the shit out of everything. Which can be pretty terrifying:




Edit: I should probably also mention: That corruption is... Pretty hard to reproduce. I can't ever seem to trigger it on purpose. I've also had the screen (across both monitors) "flash" when it triggers. Maybe it has something to do with Windows? Maybe it only happens once every time you boot Windows...

[Rachael]

Considering that drivers are ring-0, the corruption sounds like memory corruption caused by the drivers, themselves. Your system is likely in an unstable state when this happens - if you see this corruption at any time, then when you finish your playing you should restart Windows just to be on the safe side.

[Graf Zahl]

This is a known issue with Vulkan on NVidia. I have no idea what kind of weed they were smoking when adding out-of-VRAM workarounds to their internal allocator but all this produces is instability in client code. If the hardware runs out of VRAM it needs to report an error, not do it the OpenGL way any keep things running at all costs. This has been one of the biggest issues with OpenGL because it won't allow a client to recognize and deal with memory pressure issues, leading to bad performance.

And TBH, if they do not fix this soon it may cause them some very serious trouble in the future when the word gets around that AMD is simply better for modern games.

[dpJudas]

It isn't that simple. On modern OSes the GPU memory is a shared resource between multiple applications and as a result the memory can always risk being swapped to system memory. All we see here is a poor implementation on Nvidia's behalf.

Fat chance on AMD being considered the better option for modern games. They haven't been able to produce a competitive GPU for years now.

[phantombeta]

Considering that drivers are ring-0, the corruption sounds like memory corruption caused by the drivers, themselves. Your system is likely in an unstable state when this happens - if you see this corruption at any time, then when you finish your playing you should restart Windows just to be on the safe side.

Unlikely. This has never caused any system instability on my end, everything works just fine after this happens. No other programs break like this after this happens to GZDoom, not even ones that use hardware acceleration. (e.g., Firefox, Discord, Steam, GOG Galaxy...)
(It's not the first time I've had it happen, and I've never restarted because of it. I don't restart often either, so there's no way this wouldn't have caused problems if it really were actual system RAM corruption.)

Based on how the corruption seems pretty... Consistent, I guess*, I'd guess Nvidia's drivers are activating some internal feature suddenly that always causes the same issue in GZDoom and/or the Vulkan memory management library GZDoom uses. (And it probably only gets disabled again on restart)

* (e.g., those black spots in the top right corner are always there and always in that spot, textures other than the sky don't really seem to get corrupted, etc...)

[Nash]

Is this "out of memory with Vulkan on nvidia" problem only unique to GZDoom, or can the same thing happen to other games that use Vulkan, like say, Doom 2016/Eternal?

[dpJudas]

You don't generally see this problem because newer games keep track on their total texture allocation budget and unloads textures if a new texture exceeds the budget. I.e. gzdoom could reload a 6x scaled texture at 3x or 1x if there isnt room for all of them. However, this requires improvements in how textures are handled.

In most games today the textures are always loaded in a low resolution (1x scale in gzd) and then the higher res are only streamed in for those close to the viewer. If you pay attention you can sometimes see this in the games where the texture is low res and then transforms into higher res as the texture loading thread catches up

[Graf Zahl]

Modern games do not offer such a wasteful feature like a 6x upscaler. This was utter nonsense to add and certainly not my idea. The problem is that this is an option those clueless users will actually use, even on a 2GB graphics card - which gets essentially reduced to a 50 MB card with such high scaling factors - and we all know that some modern maps can use quite a high number of textures.

[dpJudas]

128x128 * 6 = 768x768. That's far less than the 4096x4096 textures used by modern games (which are fully PBR, so there's 4 or more of them per material). It is a large part of the reason games nowadays can be over 80 gigabytes in size.

Note: I'm not commenting on whether GZDoom should support something like this or not. I was replying to Nash asking why modern games don't run into the memory limit. And the answer to that is because of the way they stream textures.