[Fixed] Crash in 4.7pre when precaching model textures

Bugs that have been investigated and resolved somehow.

Moderator: GZDoom Developers

Re: Crash in 4.7pre when precaching model textures

Postby 0mnicydle » Sun Jun 27, 2021 11:25 am

Call Stack from Release build w/ debugging enabled (I think :P ):

Spoiler:
User avatar
0mnicydle
 
Joined: 27 Jun 2013

Re: Crash in 4.7pre when precaching model textures

Postby Graf Zahl » Sun Jun 27, 2021 12:07 pm

_mental_ wrote:It’s mostly useless. Strange that it crashes in sndfile though.
Please load symbols for system libraries like ntdll. It's in context menu of callstack window.

EDIT: To enable debug information in Release build, add /Zi to CMAKE_C_FLAGS_RELEASE and CMAKE_CXX_FLAGS_RELEASE CMake variables, and /DEBUG to CMAKE_EXE_LINKER_FLAGS_RELEASE variable.
The same can be done for ZMusic. The third variable should be CMAKE_SHARED_LINKER_FLAGS.



Or use RelWithDebInfo. Its compiler settings are identical to Release, aside from generating debug info.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
 
Joined: 19 Jul 2003
Location: Germany

Re: Crash in 4.7pre when precaching model textures

Postby 0mnicydle » Sun Jun 27, 2021 12:44 pm

Graf Zahl wrote:Or use RelWithDebInfo.


Oh duh, I can't believe I didn't even notice that... Coulda saved me a few minutes trying to figure out how to do what mental told me. Oh well...

I've given up trying to get a crash in the debug version, it's just not happening. I also downloaded a fresh zip of dev BoA just to make sure there wasn't any corruption issue like Rachael suggested.

I've switched over to RelWithDebInfo going forward. I also realized you can continue in the debugger after a breakpoint and the call stack gets updated, so I'm not sure if I was supposed to paste just the first break point as I did in my previous post, or if I am supposed to let it continue looking for something in particular... On some launches it brings me to different points first, so I'm not sure if there's something more useful I could be posting... For instance this time I got:

Spoiler:


On the next run the first point was something more like the first one I posted:

Spoiler:


I'm guessing this all might not be very useful if no one else is going to be able to reproduce it, but I do encourage anyone that is interested to try it in RelWithDebInfo instead of Debug, if they haven't already.

Files are Wolfendoom-master folder from git, dummy ipk3 with iwadinfo, and my dev BoA mapinfo from the first post.

Debugging command arguments: -file wolfendoom-master wolfendoom-master\tools\launcher\distribution\boa_dt.pk3 mapinfo.txt
User avatar
0mnicydle
 
Joined: 27 Jun 2013

Re: Crash in 4.7pre when precaching model textures

Postby 0mnicydle » Sun Jun 27, 2021 1:02 pm

Also I probably shouldn't even speculate, but I was wondering if the reason the debug build isn't crashing for me is precisely because it's slower, and there's some sort of race condition? And this is why sometimes it works even in the release version for me with the config in the most likely to crash state, just by dumb random chance... and loading doa_bt.pk3 or not also impacts how fast something is getting done and influences if the crash conditions are met...

Anyway, like I said, I probably shouldn't speculate, sorry...
User avatar
0mnicydle
 
Joined: 27 Jun 2013

Re: Crash in 4.7pre when precaching model textures

Postby 0mnicydle » Sun Jun 27, 2021 10:33 pm

Sorry to keep bumping this thread but I just discovered something incredibly weird that may or may not be useful...

If I add "+map titlemap" to the command line, the crash frequency is reduced to the point where I haven't actually seen it happen yet after at least 10 launches. If nothing else, at least I now have a practical workaround for this strange issue...
User avatar
0mnicydle
 
Joined: 27 Jun 2013

Re: Crash in 4.7pre when precaching model textures

Postby _mental_ » Mon Jun 28, 2021 1:42 am

Debug information is not the only difference between Release and RelWithDebInfo. The former uses /Ob2 (compiler can inline everything except explicit noinlines) while the latter uses /Ob1 (which skips implicit candidates for inlining). If the given crash happens with RelWithDebInfo too, this doesn't mean that code generated by both configurations are identical. And as long as we don't know what causes the crash, I saw no reason to add another variable to the equation.

Regarding the problem itself, I still cannot reproduce it. Breakpoint on RtlpLogHeapFailure() function never triggered for me.
What's in the Output window when this happens? Some error information should be there.
_mental_
 
 
 
Joined: 07 Aug 2011

Re: Crash in 4.7pre when precaching model textures

Postby Graf Zahl » Mon Jun 28, 2021 2:23 am

In GZDoom this was explicilty changed to use the same flags for Release and RelWithDebInfo, because otherwise RelWithDebInfo would be useless, if it cannot be used to debug actual release builds. No idea what the CMake people were smoking here.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
 
Joined: 19 Jul 2003
Location: Germany

Re: Crash in 4.7pre when precaching model textures

Postby 0mnicydle » Mon Jun 28, 2021 6:41 am

OK then, back to Release w/ modified cmake variables then, yeah? New run, new call stack with something new I haven't seen yet lol:

Spoiler:


Also I assume when you say output window you mean in VS:

Spoiler:


A couple things have been holding true so far:

Never seen a crash in the debug build
Never seen a crash in the Release build with my worst case config if I add +map titlemap to the arguments

EDIT/ oops, made a typo (intermap/titlemap)
User avatar
0mnicydle
 
Joined: 27 Jun 2013

Re: Crash in 4.7pre when precaching model textures

Postby _mental_ » Tue Jun 29, 2021 1:11 am

Calls to FTextureManager::CheckForTexture() function here and here may create a new texture. However, the size of the hitlist is set once at the beginning of precaching process.
We need to grow the hitlist if we want to create new textures here, or pass TEXMAN_DontCreate if we don't need to do this. I cannot decide which one is right for precaching.
_mental_
 
 
 
Joined: 07 Aug 2011

Re: Crash in 4.7pre when precaching model textures

Postby 0mnicydle » Sat Jul 10, 2021 9:06 am

_mental_ wrote:Calls to FTextureManager::CheckForTexture() function here and here may create a new texture. However, the size of the hitlist is set once at the beginning of precaching process.
We need to grow the hitlist if we want to create new textures here, or pass TEXMAN_DontCreate if we don't need to do this. I cannot decide which one is right for precaching.


I wish I knew enough to make use of this info and experiment with it, but alas, I am but a simple user. I did however just want to say that I finally took some time to try this on my work machine just to make sure there wasn't something wrong with my home PC, and I was able to reproduce the issue pretty quickly. I'm at a loss for why no one else is seeing it. My work computer is a wildly different build than my home computer. The only thing they have in common besides Win 10 is they are both using NVIDIA cards.

Work specs: i7-8086K, 32GB RAM, GTX 1070

It looks like there has been a new release of BoA on Moddb: boa_c31.1.zip. Along with gzdoom-x64-g4.7.0pre-72-g250fac5b7 pulled straight from DRD extracted to an empty directory, I dropped in boa.ipk3, boa_dt.pk3, and my mapinfo.txt.

Command line: gzdoom -file boa_dt.pk3 mapinfo.txt

The mapinfo in my first post for the dev version is still valid for this "c31.1" release.

One thing I will say about doing this on my work PC, it seemed to crash only half as often as my home PC. Maybe the crash is more likely to happen the faster the PC? By the way, using the boa.exe packaged version of gzdoom, which looks to be 4.6 proper, I get no crashes on either machine.
User avatar
0mnicydle
 
Joined: 27 Jun 2013

Re: Crash in 4.7pre when precaching model textures

Postby _mental_ » Sat Jul 10, 2021 10:28 am

The crash doesn't depend on anything except some internals of memory management. Randomness in allocation makes its repro rate less than 100%.
At this point, any experiments are pointless as apparently we have a write to unallocated memory.

If you want to test, add | FTextureManager::TEXMAN_DontCreate right after | FTextureManager::TEXMAN_ReturnFirst in two lines I linked before. Actually, I'm pretty sure that doing this fixes the crash.
The problem is, I'm still not sure that this is the best way to fix it.
_mental_
 
 
 
Joined: 07 Aug 2011

Re: Crash in 4.7pre when precaching model textures

Postby 0mnicydle » Sat Jul 10, 2021 11:54 am

That did indeed stop the crashing, and I can see by checking VRAM usage that gl_precache and the manual model texture precaching are both still working, and of course I am still able to perceive the benefit in terms of reduced gameplay stalls.

Thanks! At least this is something simple enough that I can do it myself until such time as you decide what the best course of action is.
User avatar
0mnicydle
 
Joined: 27 Jun 2013

Re: Crash in 4.7pre when precaching model textures

Postby _mental_ » Mon Jul 12, 2021 4:30 am

_mental_
 
 
 
Joined: 07 Aug 2011

Re: Crash in 4.7pre when precaching model textures

Postby 0mnicydle » Mon Jul 12, 2021 8:11 am

Thank you so much for doing this, and for your patience. I really appreciate it!
User avatar
0mnicydle
 
Joined: 27 Jun 2013

Previous

Return to Closed Bugs

Who is online

Users browsing this forum: No registered users and 0 guests