ZDoom

by **drfrag** » Sat Sep 05, 2020 3:54 am

vsonnier wrote:o be confirmed, but it could explain the RAM/ VRAM inflation.

No, it also happens with the test mod the OP attached and it doesn't contain hires textures (but i haven't confirmed it myself). There must be a problem in the new upscaling code.
About Vulkan yes it's ready to use but due to the out of vram issue it's not recommended to use it with HQ resize modes (BTW i never use those modes myself and they've introduced a lot of problems specially in the software renderer).

by **theleo_ua** » Sat Sep 05, 2020 2:58 am

vsonnier wrote: Give a try to reduce gl_texture_hqresize_maxinputsize to see if affects RAM consumption.

Tried with gl_texture_hqresize_maxinputsize=256 and even gl_texture_hqresize_maxinputsize=32, and lags + ram consumption was the same. I think that something is broken in the code, so gl_texture_hqresize_maxinputsize is ignored and textures are resized always (so taking too much ram)

Also I noticed next in my config:
gl_texture_hqresize_maxinputsize=512
gl_texture_hqresize_mt_height=4
gl_texture_hqresize_mt_width=16

I wonder, what hqresize_mt_width mean and why it 16

vsonnier wrote: You can also get a hint in Vulkan VRAM consumption by using 'vk_memstats' on the console.

Thanks, will use it when vulkan will work with my config and mods

vsonnier wrote: * There is a CVAR option controlling the max texture size to be resized (no need to resize already high enough resolution) : gl_texture_hqresize_maxinputsize = 512 (by default)
which means the texture is NOT resized if height * width > gl_texture_hqresize_maxinputsize * gl_texture_hqresize_maxinputsize

Interesting idea, but I think it should be expanded with something like this:

Add next options:

1) HQ rescale all textures
values: on, off, mod defined
("on" means, that next options (from 2 to 7) will be ignored)

2) dont HQ rescale textures in HIRES subfolder (on, off)

3) dont HQ rescale textures with more than X pixels count (on, off)
pixels count (integer)

4) dont HQ rescale textures with more than Y width (on, off)
width (integer)

5) dont HQ rescale textures with more than Z height (on, off)
height (integer)

6) dont HQ rescale textures, which are mentioned in texture black list file in the mod (on, off)

7) HQ rescale only the textures, which are mentioned in texture white list file in the mod (on, off)

By saying "textures" I mean textures, sprites, patches etc (at least which are located in HIRES folders/subfolders)

_mental_ wrote:
theleo_ua wrote:2) If I change to vulkan instead of OpenGL (I mean future (with full vulkan support without critical issues) poilshed versions, not current), will it take less RAM?
Please stop posting nonsense about major issues with Vulkan renderer. The only important thing it doesn't do at the moment is graceful handling of out-of-VRAM condition. Apparently, you encountered it with all these high resolution textures.

So Vulkan is ready to use now (if not taking to account memory issues you explained)? Then sorry, somehow I missed this

by **_mental_** » Sat Sep 05, 2020 2:41 am

theleo_ua wrote:2) If I change to vulkan instead of OpenGL (I mean future (with full vulkan support without critical issues) poilshed versions, not current), will it take less RAM?

Please stop posting nonsense about major issues with Vulkan renderer. The only important thing it doesn't do at the moment is graceful handling of out-of-VRAM condition. Apparently, you encountered it with all these high resolution textures.

by **vsonnier** » Sat Sep 05, 2020 12:52 am

I tried to understand, why new code should take 9x more RAM, but didn't get it. But found next:

I didn't find the reference, but in the new code ALL (* not quite, see below) textures are upscaled, while in the previous versions the additionnal texture pack textures were never resized because considered already "high resolution". To be confirmed, but it could explain the RAM/ VRAM inflation.
Indeed if I use my favorite texture pack in 4.5 the textures are clearly affected by the option "Resize textures" with xBRZ 2x depending on ON or OFF while it was never the case in (some, which one ?) past version.

* There is a CVAR option controlling the max texture size to be resized (no need to resize already high enough resolution) : gl_texture_hqresize_maxinputsize = 512 (by default)
which means the texture is NOT resized if height * width > gl_texture_hqresize_maxinputsize * gl_texture_hqresize_maxinputsize.

Give a try to reduce gl_texture_hqresize_maxinputsize to see if affects RAM consumption.

You can also get a hint in Vulkan VRAM consumption by using 'vk_memstats' on the console.

by **theleo_ua** » Fri Sep 04, 2020 5:25 pm

drfrag wrote:Seems upscaled textures now take up much more ram. Certainly there was a recent major refactor of the upscaling code.
https://github.com/coelckers/gzdoom/com ... 9797c9c32c

I tried to understand, why new code should take 9x more RAM, but didn't get it. But found next:

Still not solved: Material layers need explicit control, not only for scaling but also for filtering.

1) Does this mean, that after solving this "Material layers need explicit control also for filtering" issue, RAM usage will come back to 4.3.3 low consuming style?

2) If I change to vulkan instead of OpenGL (I mean future (with full vulkan support without critical issues) poilshed versions, not current), will it take less RAM?

P.S. I tried to change OpenGL to Vulkan in GZDoom settings (for 4.5pre), and GZDoom crashes during loading of map01:

Execution could not continue.

Could not submit command buffer: device lost

Here is crash report: https://u.pcloud.link/publink/show?code ... Vd2yizh4dk

by **drfrag** » Fri Sep 04, 2020 1:12 pm

Seems upscaled textures now take up much more ram. Certainly there was a recent major refactor of the upscaling code.
https://github.com/coelckers/gzdoom/com ... 9797c9c32c

by **theleo_ua** » Fri Sep 04, 2020 10:26 am

_mental_ wrote:Regarding lags, could you please compare both versions with the following stats enabled in console?
Code: Select all
stat rendertimes
stat renderstats
stat think

4.5 pre with lags: https://www.youtube.com/watch?v=cA7BYf7v9WM

4.3.3 without lags: https://www.youtube.com/watch?v=hVVpSpfy5r4

NOTE1: It was very lucky try with 4.5 pre, and only 25% of lags appeared (and some long lags, for example before map01 exit, suddenly don't appear), but anyway, there is a lot of difference between 4.5 and 4.3.3 even in these videos (for example map02 exit, map02 lags on the same places near golden doors etc.). Maybe this is because I often clicked PAUSE to explain something. I can create another videos with more and longer lags, if required

NOTE2: I forgot to say in the end of second video about checking RAM usage. But when I watched those videos after recording, I noticed, that memory usage difference is too much - 3.5GB (4.5pre) vs 400MB (4.3.3), so please pay attention to this

NOTE3: youtube provides low bitrate for 720p quality, so if you dont like blurry artifacts, please change youtube player settings to 1440p (or 1080, if 1440 still not available), because for 1440 youtube give enough bitrate for same video quality, as in my HDD during recording

by **_mental_** » Fri Sep 04, 2020 8:35 am

Regarding lags, could you please compare both versions with the following stats enabled in console?

Code: Select all

stat rendertimes
stat renderstats
stat think

by **theleo_ua** » Fri Sep 04, 2020 8:19 am

drfrag wrote:You said there was a big difference for both versions.

So - I tried my heavy hhmix mod on 4.3.3 and 4.5 pre with same INI from testmod (from 4.4.2) on map01 and map02 from hexen.wad iwad. And there is huge difference in lags and memory usage. No any noticeable lags in 4.3.3, and low ram usage (500 MB for map01 and 700 MB for map02+map01), but a lot of very noticeable and "almost unplayable" lags in 4.5 pre and high ram usage (1GB for map01 and 3 GB for map02+map01)

If you want, I can create video comparison with my voice comments, where it lags (with displaying gzdoom memory usage in realtime), and if you consider this video useful for further tests and explorations, I can upload mod (if required)

by **theleo_ua** » Fri Sep 04, 2020 7:44 am

drfrag wrote:No, i mean just use the same ini with your settings for both 4.4.0 and 4.5pre. You said there was a big difference for both versions.

if you mean these my quote:

NOTE: the reason I created this report and thinking it's a memory leak, because I have another heavy mod, and played it before 4.4.2 without much lags for loading graphics during game. But when I tried 4.4.2, I noticed big amount of lags (I can create and upload video comparisons, if you want). Trying to figure out, why those lags appeared, I noticed next:

then I mean 4.3.3 (I never tried 4.4.0 with my heavy mod). Here are examples of my last 9 tries with my heavy (hhmix) mod: https://www.youtube.com/playlist?list=P ... lymer=true (first 2 videos in playlist are played in 4.2.4, and last 7 videos are in 4.3.3). There are no any often lags like "1 lag per 10 seconds" and no any longtime lags like "freezing game for 10 seconds" (during all 9 longtime videos)

But if I will try the same mod now (with 4.4.2), even in easiest map01 from hexen.wad iwad, there will be often lags like "1 lag per 10 seconds" and also longtime lags like "freezing game for 10 seconds" (I can record a video example, even in 4.5.pre, if required)

So - here are the test results for 4.3.3, 4.4.0 and 4.5.pre (both with HQ/cache and without) with same INI from 4.4.2: https://u.pcloud.link/publink/show?code ... L4k7G60hbX (tests for attached 30MB testmod, not for heavy hhmix of course)

by **drfrag** » Fri Sep 04, 2020 6:54 am

No, i mean just use the same ini with your settings for both 4.4.0 and 4.5pre. You said there was a big difference for both versions.

by **theleo_ua** » Fri Sep 04, 2020 6:00 am

drfrag wrote:I've tried it myself, 309 MB for a recent master and 239 MB for the old lzdoom branch. Not much of a difference with this mod. 893 MB without precaching and 1060 MB with it seems there's more initial overhead (but i've seen a higher difference with other mods such as PB). I can't test with 4x becouse i can't run anything using more than 2 GB.
I've just ran the mod without your inis, let's see if someone else can reproduce the issue but i'm not sure you're using the same settings. Why don't you try with the same ini for both 4.4.0 and 4.5pre?
I forgot the file was for 3.3.2 even when the folder is named 3.3.2, i thought i was expected to try 4.4.0 now.

I'm ready to exactly do what you want, but I need to exactly understand the steps, which you want. Let me try to guess:

1) Take gzdoom.ini from 4.4.2 from testmod (but turn off HQresize and texture preaching)
2) Use ini from step 1 and try gzdoom 3.3.2 with it
3) Use ini from step 1 and try gzdoom 3.7.2 with it
4) Use ini from step 1 and try gzdoom 4.0.0 with it
5) Use ini from step 1 and try gzdoom 4.4.2 with it
6) Use ini from step 1 and try gzdoom 4.5.pre with it

Those 6 steps are exactly what you want me to do?

by **drfrag** » Fri Sep 04, 2020 5:47 am

I've tried it myself, 309 MB for a recent master and 239 MB for the old lzdoom branch. Not much of a difference with this mod. 893 MB without precaching and 1060 MB with it seems there's more initial overhead (but i've seen a higher difference with other mods such as PB). I can't test with 4x becouse i can't run anything using more than 2 GB.
I've just ran the mod without your inis, let's see if someone else can reproduce the issue but i'm not sure you're using the same settings. Why don't you try with the same ini for both 4.4.0 and 4.5pre?
I forgot the file was for 3.3.2 even when the folder is named 3.3.2, i thought i was expected to try 4.4.0 now.

by **theleo_ua** » Fri Sep 04, 2020 5:16 am

drfrag wrote: Also texture precaching is different between 3.3.2 and 4.4.2. If there's a higher ram usage without precaching and hqresize that's a different story.

Tried 3.3.2, 4.4.2 and 4.5pre-127-ge740e00e0 without precaching and hqresize - here are the results: https://u.pcloud.link/publink/show?code ... 7OEBV9JBwy

Should I try with 3.7.0 and 4.0.0 ?

by **theleo_ua** » Fri Sep 04, 2020 5:03 am

drfrag wrote:Those ini files you sent are different, you're using gl_texture_hqresizemode=4 and gl_texture_hqresizemulti=4 for the 8 GB folder and not for the 5 GB folder.
So they are not the same settings and it's logical that the engine uses more ram with hqresize enabled. Besides those two engine versions (4.4.x) are very similar

3.3.2 engine doesn't have 2 options (one for filter and another for value) - it has single option (values are XBRZ2x, XBRZ4x etc)

4.4.2 engine has 2 options (one for filter and another for value) - it has 2 options (filter type, values are XBRZ HQx etc, and value, values are 2x, 4x)

That's why my config for 3.3.2 doesnt have gl_texture_hqresizemode and gl_texture_hqresizemulti. But - if you use my 3.3.2 config for 3.7.2 or 4.0.0. or even 4.4.2, you will see, that gzdoom will add gl_texture_hqresizemode=4 and gl_texture_hqresizemulti=4 - exactly with values 4 and 4. So adding options gl_texture_hqresizemode=4 and gl_texture_hqresizemulti=4 to 3.3.2 ini makes no sense, because engine doesn't know them

Anyway, 3.7.2 and 4.0.0 with gl_texture_hqresizemode=4 and gl_texture_hqresizemulti=4 in INI, has same 5GB ram usage.

If you want, I can add gl_texture_hqresizemode=4 and gl_texture_hqresizemulti=4 to 3.3.2 INI and try 3.3.2 again, but I bet result will be the same (5GB)

drfrag wrote:The increase in ram usage when loading maps doesn't mean there's a memory leak, there's a garbage collector and that ram should be freed later

ok, thanks

drfrag wrote:Also texture precaching is different between 3.3.2 and 4.4.2. If there's a higher ram usage without precaching and hqresize that's a different story.

ok, I will try with "precache = off" and "hqresize = off", and will tell the results

ZDoom

Possible memory leaks in 4.4.2

Post a reply

Expand view Topic review: Possible memory leaks in 4.4.2

Re: Possible memory leaks in 4.4.2

Re: Possible memory leaks in 4.4.2

Re: Possible memory leaks in 4.4.2

Re: Possible memory leaks in 4.4.2

Re: Possible memory leaks in 4.4.2

Re: Possible memory leaks in 4.4.2

Re: Possible memory leaks in 4.4.2

Re: Possible memory leaks in 4.4.2

Re: Possible memory leaks in 4.4.2

Re: Possible memory leaks in 4.4.2

Re: Possible memory leaks in 4.4.2

Re: Possible memory leaks in 4.4.2

Re: Possible memory leaks in 4.4.2

Re: Possible memory leaks in 4.4.2

Re: Possible memory leaks in 4.4.2