Out of VRAM freeze
Moderator: GZDoom Developers
-
- Posts: 8197
- Joined: Sun Jan 28, 2007 3:55 pm
- Preferred Pronouns: He/Him
- Location: QZDoom Maintenance Team
Re: Lockups occasionally occurring when rendering new things
VK_ERROR_DEVICE_LOST again. And that's all I seem to be getting. Though it does crash faster certainly. Less than a minute this time.
-
-
- Posts: 3145
- Joined: Sat May 28, 2016 1:01 pm
Re: Lockups occasionally occurring when rendering new things
It sounds like you might be running out of video memory (maybe somehow due to a memory leak?), although it is strange that you don't get an explicit out of memory error. The device lost error is unfortunately the most useless thing vulkan can report back.
-
- Posts: 8197
- Joined: Sun Jan 28, 2007 3:55 pm
- Preferred Pronouns: He/Him
- Location: QZDoom Maintenance Team
Re: Lockups occasionally occurring when rendering new things
Certainly seems to be. The higher the multiplier is, the quicker it'll crash.
I'd hesitate to call it an immediate memory leak. More along the lines of so many different sprites caching, because it's AEons of Death. And there are many... many sprites.
Furthermore I have a WIP addon included which includes some larger sprites (all of which I plan on compressing the sprite file sizes as soon as I'm happy with all of them and they don't need further changes done to them). The monsters in question have 16 sprite angles too. The strange thing is, OpenGL never has this issue. Ever. This makes the vulkan situation all the more bizarre...
I think at this stage, unless you have any other ideas, it may be time for a last resort. If you and/or _mental_ are ready and willing to try and trace it yourselves, I'll send you all the files in a bundle. I'll see about including a simple "LoadMe.pk3" file with GameInfo set up to load all the files for you to make this as smooth as possible for you.
I'll ensure everything's set up so you don't have to change any of the mod's options around to get the effect to replicate. All you'll have to do is start the game, attach the debugger and wait patiently for it to crash - which shouldn't take long at all.
I'd hesitate to call it an immediate memory leak. More along the lines of so many different sprites caching, because it's AEons of Death. And there are many... many sprites.
Furthermore I have a WIP addon included which includes some larger sprites (all of which I plan on compressing the sprite file sizes as soon as I'm happy with all of them and they don't need further changes done to them). The monsters in question have 16 sprite angles too. The strange thing is, OpenGL never has this issue. Ever. This makes the vulkan situation all the more bizarre...
I think at this stage, unless you have any other ideas, it may be time for a last resort. If you and/or _mental_ are ready and willing to try and trace it yourselves, I'll send you all the files in a bundle. I'll see about including a simple "LoadMe.pk3" file with GameInfo set up to load all the files for you to make this as smooth as possible for you.
I'll ensure everything's set up so you don't have to change any of the mod's options around to get the effect to replicate. All you'll have to do is start the game, attach the debugger and wait patiently for it to crash - which shouldn't take long at all.
-
-
- Posts: 3819
- Joined: Sun Aug 07, 2011 4:32 am
Re: Lockups occasionally occurring when rendering new things
OpenGL driver will silently use main RAM if video memory is exhausted. Vulkan renderer requires explicit memory management in this case.
Unfortunately, I cannot help to debug this issue because I don’t have NVIDIA card in Windows PC.
Unfortunately, I cannot help to debug this issue because I don’t have NVIDIA card in Windows PC.
-
- Posts: 3886
- Joined: Fri Feb 08, 2008 9:15 am
- Preferred Pronouns: She/Her
- Operating System Version (Optional): (btw I use) Arch
- Graphics Processor: nVidia with Vulkan support
- Location: Vigo, Galicia
Re: Lockups occasionally occurring when rendering new things
After checking that vulkaninfo output it really bothers me that on one of the memory heaps there's a larger budget than its size, which makes no sense and goes against the spec.
Code: Select all
memoryHeaps[1] :
size = 4228358144 (0x1fc07a000) (3.94 GiB)
budget = 7670994739
-
-
- Posts: 3145
- Joined: Sat May 28, 2016 1:01 pm
Re: Lockups occasionally occurring when rendering new things
The strange thing is that, if video memory is exhausted, I would expect our memory allocation library VMA to return an error.
-
-
- Posts: 3819
- Joined: Sun Aug 07, 2011 4:32 am
Re: Lockups occasionally occurring when rendering new things
I did some testing on Kubuntu 18.04 notebook with Intel UHD 620 and NVIDIA MX150 (proprietary drivers).
Played MAP30 of Doom II with AEoD 6.66 Alpha 8, texture upscale was 6x.
Intel driver handles out of RAM correctly, and GZDoom exits with the corresponding message.
NVIDIA driver locks up entire system though. It didn’t hang completely, but response to a single key press was like in ten seconds.
On Windows it seems to be more graceful, but still works like crap. Apparently, Vulkan isn’t the primary target for NVIDIA.
Played MAP30 of Doom II with AEoD 6.66 Alpha 8, texture upscale was 6x.
Intel driver handles out of RAM correctly, and GZDoom exits with the corresponding message.
NVIDIA driver locks up entire system though. It didn’t hang completely, but response to a single key press was like in ten seconds.
On Windows it seems to be more graceful, but still works like crap. Apparently, Vulkan isn’t the primary target for NVIDIA.
-
- Posts: 8197
- Joined: Sun Jan 28, 2007 3:55 pm
- Preferred Pronouns: He/Him
- Location: QZDoom Maintenance Team
Re: Lockups occasionally occurring when rendering new things
I'm glad it was at least reproducible... Yeah, on Windows, GZDoom itself locks up but nothing else does, thankfully.
-
- Posts: 8197
- Joined: Sun Jan 28, 2007 3:55 pm
- Preferred Pronouns: He/Him
- Location: QZDoom Maintenance Team
Re: Lockups occasionally occurring when rendering new things
Only curious but was this commit something I should test to see if the game stops freezing up whenever it runs into an error? Or was that to address something else?
-
-
- Posts: 3145
- Joined: Sat May 28, 2016 1:01 pm
Re: Lockups occasionally occurring when rendering new things
It has a chance of maybe being a fix for it. But only you can tell if it actually did.
-
- Posts: 8197
- Joined: Sun Jan 28, 2007 3:55 pm
- Preferred Pronouns: He/Him
- Location: QZDoom Maintenance Team
Re: Lockups occasionally occurring when rendering new things
It didn't help, it seems. But at the same time, I think I found a clue.
If you attempt to do error_fatal in console, instead of showing the box like it's supposed to, it keeps the window from the game and simply shrinks it down. You get nothing out of it otherwise. No dialogue box with the logs or anything.
If you attempt to do error_fatal in console, instead of showing the box like it's supposed to, it keeps the window from the game and simply shrinks it down. You get nothing out of it otherwise. No dialogue box with the logs or anything.
-
-
- Posts: 3819
- Joined: Sun Aug 07, 2011 4:32 am
Re: Lockups occasionally occurring when rendering new things
Is this with default settings? I cannot test on NVIDIA graphics hardware, but on AMD fatal errors work fine for me.
-
- Posts: 8197
- Joined: Sun Jan 28, 2007 3:55 pm
- Preferred Pronouns: He/Him
- Location: QZDoom Maintenance Team
Re: Lockups occasionally occurring when rendering new things
Tested with my regular config file and default. It always happens on my NVidia card with vulkan enabled.
But just so you know, If you test with default settings, vulkan is disabled by default. On top of that, laptops use the default choice between graphics card providers - my intel in this case.
So after generating a brand new config, setting Vulkan back to On and setting the graphics card to Nvidia, this is what it looks like -a shrunken screen with the 'ghost' image frozen in on it. But you can see at the top it says "Fatal Error", caused by me entering the error_fatal command.
But just so you know, If you test with default settings, vulkan is disabled by default. On top of that, laptops use the default choice between graphics card providers - my intel in this case.
So after generating a brand new config, setting Vulkan back to On and setting the graphics card to Nvidia, this is what it looks like -a shrunken screen with the 'ghost' image frozen in on it. But you can see at the top it says "Fatal Error", caused by me entering the error_fatal command.
-
-
- Posts: 3819
- Joined: Sun Aug 07, 2011 4:32 am
Re: Lockups occasionally occurring when rendering new things
Yes, I didn’t describe what I meant by default settings. Options you changes are OK, but could you please check with IWAD only, with and without map loaded?
-
- Posts: 8197
- Joined: Sun Jan 28, 2007 3:55 pm
- Preferred Pronouns: He/Him
- Location: QZDoom Maintenance Team
Re: Lockups occasionally occurring when rendering new things
Same results there too. Last frame frozen in place.