Page 1 of 1

Extremely ultra rare VM abort address zero upon starting map

Posted: Mon Dec 10, 2018 4:02 am
by Nash
This one had me really baffled. I am getting extremely, EXTREMELY random "VM abort address zero" when the map in my project loads.

Here's the thing: There is nothing useful beyond that message. That's literally all it says. I don't even know why or where is it trying to read null.

There is no example file because I literally do not know how to reproduce it. As I said, it is ultra rare. Like 1/1000 chances of happening.

Things that other devs have suggested to me which did NOT work:

1) Attach debugger to process. The debugger doesn't catch VM aborts.
2) Repeatedly load the map, hoping to trigger it. I setup an Autohotkey script that spams a "map TestMap" keybind. I fired the script, left the game running while it automatically and repeatedly runs the keybind. After 30 minutes with no results, I gave up.

The ONLY lead that MIGHT be related so far is that it apparently "has a higher chance of happening if the map is loaded with +map on the command line" - in which case - this is a VERY hard sequence to repeat or automate due to the nature of having to: run and load the entire project with +map, quit GZDoom then repeating the process again.

Any help on how I can track this down? Again, really sorry for no example file but I can't even make one if I literally don't know why it happens - the error message contains nothing other than "VM abort address zero".

Re: Extremely ultra rare VM abort address zero upon starting

Posted: Mon Dec 10, 2018 4:36 am
by Player701
I've already reported the missing stack trace problem here - could be the same issue. The stack trace is missing if the map is loaded via a "+map" or "-warp" command line parameter. But it could be something different if you used the in-game console to load your map and didn't get a stack trace along with a VM abort error.

Also, the debugger does catch VM aborts if you know where to set a breakpoint. In fact, that's how I was able to determine the cause of the error in my code (back then I didn't know I'd get a proper stack trace if I loaded the map via the console). For example, this line in vmframe.cpp would suffice (I just searched the source code for "VM execution aborted"). Here is the call stack produced by the error in the example script from my bug report, where it is easy to see that the problem originated in a BeginPlay method:


Re: Extremely ultra rare VM abort address zero upon starting

Posted: Mon Dec 10, 2018 4:39 am
by Nash
Hmm, okay.

In that case, I already automated the process of running my entire project from the cmdline (with +map) and Alt + F4'ing out the game repeatedly.

After 10 minutes of letting the automation run, I still got no VM aborts. Kinda giving up now. -____-

Re: Extremely ultra rare VM abort address zero upon starting

Posted: Mon Dec 10, 2018 4:45 am
by Graf Zahl
You can let the debugger break on throwing exceptions, it's somewhere in the settings. Normally this is off because lots of software is using exceptions during normal program flow, making debugging hard. But for VM aborts there's only one or two centralized places that can do it where a breakpoint can be set.

Re: Extremely ultra rare VM abort address zero upon starting

Posted: Mon Dec 10, 2018 8:13 am
by _mental_
Did VM abort start to happen since JIT merged in? Let's say, did it work fine two weeks ago and earlier?

Re: Extremely ultra rare VM abort address zero upon starting

Posted: Mon Dec 10, 2018 9:46 am
by Nash
_mental_ wrote:Did VM abort start to happen since JIT merged in? Let's say, did it work fine two weeks ago and earlier?
Yes, only happened ever since JIT merge.

Re: Extremely ultra rare VM abort address zero upon starting

Posted: Tue Dec 11, 2018 3:43 am
by Nash
I think this can be closed for now. It's very hard to reproduce and hopefully with the recent VM stack trace fix, hopefully I can figure the problem out on my own.

Re: Extremely ultra rare VM abort address zero upon starting

Posted: Tue Dec 11, 2018 7:38 am
by Rachael
I'd rather not close it right now, unless it doesn't happen again for a couple weeks. If I remember correctly, we do at least have it printing a stack trace if you do +map now, so hopefully you can get something next time it happens.

Then again, it may be an issue that was inadvertently fixed with another fix. Hopefully we'll find out soon.

Re: Extremely ultra rare VM abort address zero upon starting

Posted: Fri Dec 14, 2018 5:40 am
by Nash
Here's what I've learned so far:

The VM abort happens because in extremely rare cases, it is trying to retrieve an instance of a static thinker but said reference is null.

So yes, this is a bug because prior JIT merge, this never happens. At all.

I can make a small example file, but, same as the original issue in the OP - getting it to reproduce consistently is next to impossible. It has literally only happened 3 times since my last post in this thread.

It's completely random. So far, all I've got is there is a WorldLoaded event handler that instantiates the static thinker.

Will see if I can get anything else out of this.