r3088 Crash when starting a game

[Enjay]

On starting a game of Doom2 with my build of r3088, the game crashes as the level starts. The compile did not report any errors.

Seeing as how the binary doesn't work, I haven't put it on to the DRD SVN page but I have uploaded the map file to the server:

http://svn.drdteam.org/zdoom/zdoom-r3088.map.gz

If you need the binary, let me know.

[edit] GZdoom r1156, based off of r3088, does not crash. [/edit]

[Enjay]

Same with r3089

http://svn.drdteam.org/zdoom/zdoom-r3089.map.gz

[NeuralStunner]

Does r3087 work properly?

[Gez]

I don't get crashes with my own build.

[Enjay]

Strange. I have now downloaded and compiled all revisions from 3085 to 3089 and they all crash for me. The compiling process seems to be going properly - there are no warnings - but the game crashes when I try to start a level. This is even more worrying because I can download and play r3085 (and other revisions) from the DRD SVN page and it works - and it was me who compiled them!

I have cleaned and rebuilt the solution a number of times. I have done a completely fresh check out and build. Neither worked.

I have made a debug build but when it crashed everything froze and it was difficult to get back to the Visual C++ screen, but I managed. I also ran the minidump file from a crash report through the debugger and both the minidump and the debug build ended up pointing at the same location:

---

I have recently formatted my computer and reinstalled everything so there could be something going wrong with my compiling environment. However, I can see absolutely nothing to give me a clue as to what it is. Zdoom is apparently compiling correctly, but it crashes. GZdoom compiles properly and does not crash. I can't think of anything that I did differently when setting up my compiling environment.

Any ideas?

[Graf Zahl]

The crash appears to happen in some assembly code so I guess something went wrong there during your reinstall.

[Enjay]

Any clues as to how I can track that down or why GZdoom works when Zdoom doesn't?

[Gez]

It's a render function, and presumably you use GZDoom in OpenGL mode...

[Enjay]

It's a render function, and presumably you use GZDoom in OpenGL mode...

Indeed, and I can't even remember how to get GZdoom to run in software mode to check.

However, finding out that the problem was in assembly code gave me the clue that I needed. When I installed my compiling environment, I downloaded the windows install version of nasm (nasm-2.09.04-installer.exe) so I unzipped the old copy of nasm from my backup zip (nasm 2.01) and now my builds work.

So, for what ever reason, Zdoom does not seem to be compatible with nasm-2.09.04. I don't know if this is something that needs to be looked at or not, but it is what was at the root of my problem.

[edit spelling]

[Gez]

The version I use is 2.07; so presumably the problem appeared either in 2.08 or 2.09.

If anybody feels like trying with 2.08: http://www.nasm.us/pub/nasm/releasebuilds/2.08/win32/

[Graf Zahl]

Maybe they changed something that doesn't work with ZDoom's self-modifying code...

[Enjay]

OK, I checked with 2.08.01 and then rechecked with 2.09.04 (this time, just the zipped version rather than the windows exe installer).

2.08.01 did not crash but, like before 2.09.04 did. So, there is indeed some incompatibility between Zdoom and nasm 2.09.04.

[randi]

The culprit is this change in 2.09:

Make -Ox the default optimization level.

There are some four-byte values that get self-modified, and they were optimized into byte values, so the self-modification ended up trashing the following instructions.

[Demolisher]

Damn, thanks, this was the reason I couldn't test my patches.