[BUG][ANY GZDOOM >= 4.12.0][7z/pk7] Doom 2: PsychoPhobia mod crash gzdoom on start

[Premo36]

Hello everyone,
first time reporting a bug on the forum. Let me know if I've missed anything.

Issue
I'm having an issue playing the mod for Doom2 "PsychoPhobia" (v2.25) on modern versions of GZDoom.

Gzdoom versions till 4.11.3a (included) loaded the mod files without issues.

Starting from 4.12.0 (included), gzdoom crash on start with the following error

Code: Select all

Code: C0000005 (Access Violation - tried to read address 0000000000000008)

I've tried the following versions:
Note: In this post due to the 3 file limit, I just attached 4.12.0, 4.14.2 and 4.15pre-407-g9b44f2315 crash reports, as they seemed the more relevant versions. Let me know if you need also the other ones.

• 4.11.3a -> Working, so no crash report
• 4.12.0 -> First version to crash
• 4.12.1
• 4.12.2
• 4.13.0
• 4.14.2 -> Latest stable
• 4.15pre-407-g9b44f2315 -> Latest build at the time I'm writing this post

I did all the test runs on my Windows 10 laptop, because I could use the .bat provided by the mod for loading, but the same issue appears also under my PopOS 22.04 desktop.
I did re-download both mod and gzdoom to exclude this issue was due some kind of file corruption / gzdoom misconfiguration.

One weird thing about PsychoPhobia files, is that while all files have a .pk3 extension, actually some of them are 7z archive, so the correct extension should be .pk7 or .7z.
Tried to rename those file with the more appropriate .pk7 extension, but it didn't make any difference.

Workaround
I tried to repackage the files from the 7z archive to a real .pk3 by extracting the content with 7zip to a folder, and then re-compressing it in a .zip file, then I changed the extension to .pk3, and it worked! After this file format change I was able to play PsychoPhobia also with version 4.14.2. (On both Windows and Linux)

Experimenting
Since the repackaging was successful, I tried repackage the mod once again, but in a fresh 7z archive (with a .pk7 extension), thinking that maybe the original archive had some issues. The new one still crashed.
Making a plain 7z file, with like just a DECORATE lump did not crash.
So it seems that is something related to gzdoom 7z handling, but ONLY in particular cases, like this one.

Potential "suspect"
Looking for clues in the changelog for the first version that crashed, 4.12.0 (https://zdoom.org/wiki/GZDoom_version_history#4.12.0), this line caught my attention:

Update LZMA SDK to 23.01

as if I'm not mistaken is the LZMA SDK is used integrate 7z compression into third-party softwares. Could be this update the root cause of this issue?

How to reproduce the issue
The mod files to be loaded are (the batch in the mod will do it for you):

---

1. PsychoPhobia.pk3
2. PsychoPhobia_3d_ammo.pk3
3. PsychoPhobia_3d_decorate.pk3
4. PsychoPhobia_3d_gibs.pk3
5. PsychoPhobia_3d_nature.pk3
6. PsychoPhobia_3d_pickups.pk3
7. PsychoPhobia_3d_projectiles.pk3
8. PsychoPhobia_3d_weapons.pk3
9. PsychoPhobia_HiResFont.pk3
10. PsychoPhobia_ShellsStay_3d.pk3

Step-by-step bug reproduction (Windows 10)

---

1. Download Doom2: PsychoPhobia v.2.25 from https://www.moddb.com/mods/psychophobia ... obia-v-220 (Apologies for the huge file size, I wasn't able to reproduce the issue with a smaller example...).
2. Extract the .RAR file content in a dedicate folder. Eg, let's assume the folder is C:/PsychoPhobia_V_2_25/.
3. Navigate to C:/PsychoPhobia_V_2_25/GZDoom_1_5_06/.
4. Inside that folder you will find the old gzdoom version the mod was shipped with. Delete everything inside the folder except the PsychoPhobia, which contains the batch file to run the mod.
5. Download any version of gzdoom after 4.12.0 (included). Eg: 4.14.2.
6. Extract the content inside C:/PsychoPhobia_V_2_25/GZDoom_1_5_06/, this way we are replacing the original gzdoom version with the new one, so we can leverage the original batch for -file loading
7. Copy your copy of doom2.wad inside C:/PsychoPhobia_V_2_25/GZDoom_1_5_06/, to ensure is picked up by gzdoom.
8. Navigate to C:/PsychoPhobia_V_2_25/GZDoom_1_5_06/PsychoPhobia, inside you will find a list of batch files, double click on PsychoPhobia_3D Shells stay forever.bat
9. After a few seconds gzdoom 4.12.2 will start, select doom 2 in the gzdoom iwad selector and then it will start to load. It will crash after a few seconds, showing the crash dump pop-up window.

Thank you! Let me know if you need more info / the other versions crash reports.

[Rachael]

The commit you mentioned came from a PR: https://github.com/ZDoom/gzdoom/pull/2172

Unfortunately reverting this pull request in order to test if it is indeed the culprit is not easy, since several file system refactors happened on top of it (and to be quite honest, I think they are more likely to cause this problem than this PR, because I did a post-mortem review of that PR and it looked very simple and contained to me, so I would be surprised if it is causing the actual problem).

[drfrag]

Hey wenas. It's not that PR since it runs with LZDoom 4.11.4 and it has that PR as you can see checking the tag, i merged until i considered master stable. I edited the PsychoPhobia_3d batch file to run it (loading all files there of course) but there's a non fatal script error, something about unrecognized string "rotate" in modeldef. However i haven't tried later versions and that mod it's a bit of a mess since it has a lot of folders and files. I've downloaded this file and it's v2.25: https://www.moddb.com/mods/psychophobia ... obia-v-220

[Premo36]

The commit you mentioned came from a PR: https://github.com/ZDoom/gzdoom/pull/2172

Unfortunately reverting this pull request in order to test if it is indeed the culprit is not easy, since several file system refactors happened on top of it (and to be quite honest, I think they are more likely to cause this problem than this PR, because I did a post-mortem review of that PR and it looked very simple and contained to me, so I would be surprised if it is causing the actual problem).

Ah bummer... thought I had find a trail

Hey wenas. It's not that PR since it runs with LZDoom 4.11.4 and it has that PR as you can see checking the tag, i merged until i considered master stable. I edited the PsychoPhobia_3d batch file to run it (loading all files there of course) but there's a non fatal script error, something about unrecognized string "rotate" in modeldef. However i haven't tried later versions and that mod it's a bit of a mess since it has a lot of folders and files. I've downloaded this file and it's v2.25: https://www.moddb.com/mods/psychophobia ... obia-v-220

As far as I'm aware the latest official version for the mod is 2.25 (OT: I'm working on an unofficial 2.26 patch, which is a small pk3 you load after everything, which included per maps fixing + zscript event handler to "inject" compatibility fixes for modern gzdoom / rebalancing without having to touch the original mod files, is not yet public and I didn't include it in my tests to not add even more noise), the one you linked is correct, should be the same I linked if I didn't messed up something in my original post, I'll double check.
There are a few non fatal error / warnings, but those have been there since a long time, the mod is very old and gone through a lot of revision/update (including the infamous switch from 2D to 3D, which ultimately killed the mod... see 0xCobalt comment, which is one of the original autors https://www.moddb.com/mods/psychophobia#8080162).

It seems to happens just with this mod and only if left in the original "fake .pk3" (which is a 7z archive), as if i repackage it in zip format, making it a true .pk3 it works. There is indeed a lot of "noise" in the mod, given it is multiple files each one with a fairly complex tree...
I'll try to reproduce the bug in a clean and minimal example / slim down the problematic file list...
I'll also try to look for other 7z compressed mod and see if I have the same issue running them

Thank you both for the help!

[drfrag]

I tried with 4.14 and it certainly crashes but it has to be something else. Y wenas.

[Premo36]

I did a little more tinkering with the mod files, and I think I've found a small reproducible example , I'm attaching it to the thread, is PsychiPhobia_3d_gibs.pk3, which is one of the mod file that causes the crash.

Here everything I found:
I checked via powershell the file header to see the true format of the mod files, 7z is for 7zip archive, pk is for zip ones.

Code: Select all

> Get-Content -Path .\PsychoPhobia.pk3 -TotalCount 1
7z¼¯'∟♥8BæÞG£ÿ  'm¾:')↑JB►Re¹-Æ ......................

> Get-Content -Path .\PsychoPhobia_HiResFont.pk3 -TotalCount 1
PK♥♦¶ù{)=V↓M▼¶♦Ú¶

With that I was able to see which file was a true .pk3 and which one should have been a .pk7

• PyschoPhobia.pk3 -> is a 7z archive with wrong extension
• PsychoPhobia_3d_ammo.pk3 -> is a 7z archive with wrong extension
• PsychoPhobia_3d_decorate.pk3 -> is a 7z archive with wrong extension
• PsychoPhobia_3d_gibs.pk3 -> is a 7z archive 7z archive with wrong extension
• PsychoPhobia_3d_nature.pk3 -> is a 7z archive with wrong extension
• PsychoPhobia_3d_pickups.pk3 -> is a 7z archive 7z archive with wrong extension
• PsychoPhobia_3d_projectiles.pk3 -> is a 7z archive 7z archive with wrong extension
• PsychoPhobia_3d_weapons.pk3 -> is a 7z archive 7z archive with wrong extension
• PsychoPhobia_HiResFont.pk3 -> correct, is a zip archive
• PsychoPhobia_NoBloodTrails.pk3 -> correct, is a zip archive
• PsychoPhobia_NoPuffSparks.pk3 -> correct, is a zip archive
• PsychoPhobia_NoSFX.pk3 -> correct, is a zip archive
• PsychoPhobia_NoSmoke.pk3 -> correct, is a zip archive
• PsychoPhobia_ShellsStay.pk3 -> correct, is a zip archive
• PsychoPhobia_ShellsStay_3d.pk3 -> correct, is a zip archive
• BrutalCombat.pk3 -> is a 7z archive with wrong extension

Then tried launching one by one the file, of course most of them will halt with a fatal error, due to missing dependencies from the main PsychoPhobia file, but I wanted to see which one will make gzdoom crash with the access violation error.

• PyschoPhobia.pk3 (7z)-> Crash with access violation error message (can't upload crash report, is half a megabyte )
• PsychoPhobia_3d_ammo.pk3 (7z) -> Fatal Error due to missing dependencies, but no crash
• PsychoPhobia_3d_decorate.pk3 (7z) -> Fatal Error due to missing dependencies, but no crash
• PsychoPhobia_3d_gibs.pk3 (7z) -> Crash with access violation error message (attached both file and crash report to this post)
• PsychoPhobia_3d_nature.pk3 (7z) -> Fatal Error due to missing dependencies, but no crash
• PsychoPhobia_3d_pickups.pk3 (7z) -> Fatal Error due to missing dependencies, but no crash
• PsychoPhobia_3d_projectiles.pk3 (7z)-> Fatal Error due to missing dependencies, but no crash
• PsychoPhobia_3d_weapons.pk3 (7z) -> Fatal Error due to missing dependencies, but no crash
• PsychoPhobia_HiResFont.pk3 (zip)-> Starts Successfully
• PsychoPhobia_NoBloodTrails.pk3 (zip) -> Starts Successfully
• PsychoPhobia_NoPuffSparks.pk3 (zip) -> Starts Successfully
• PsychoPhobia_NoSFX.pk3 (zip) -> Fatal Error due to missing dependencies, but no crash
• PsychoPhobia_NoSmoke.pk3 (zip) -> Starts Successfully
• PsychoPhobia_ShellsStay.pk3 (zip) -> Fatal Error due to missing dependencies, but no crash
• PsychoPhobia_ShellsStay_3d.pk3 (zip) -> Fatal Error due to missing dependencies, but no crash
• BrutalCombat.pk3 (7z) -> Fatal Error due to missing dependencies, but no crash

The Fatal Error for missing dependencies goes away if load the fixed copy of PsychoPhobia.pk3 (The one that is a true .zip)

So this narrows the file that generate the issue just to PyschoPhobia.pk3 (7z) and PsychoPhobia_3d_gibs.pk3 (7z)
The first one does not help much being the biggest of them all, but PsychoPhobia_3d_gibs.pk3 is interesting, because is mostly empty, made of one empty folder and 3 blank files:

Code: Select all

3dmodels/ 
Decorate.c
glDefs.c
ModelDef.c

Thanks again everyone

[drfrag]

Those are malformed pk7 files, PyschoPhobia.pk3 even has a rar archive inside. May be it has some empty folder of file inside too. I cannot track the changes, before it worked but loading was slow. Here's the backtrace:

Code: Select all

>	lzdoom.exe!FileSys::FileSystem::ReadFile(int lump, void * dest) Line 1232	C++
 	lzdoom.exe!FScanner::OpenLumpNum(int lump) Line 206	C++
 	lzdoom.exe!FScanner::FScanner(int lumpnum, TMap<FName,FScanner::Symbol,THashTraits<FName>,TValueTraits<FScanner::Symbol>> * extsymbols) Line 116	C++
 	lzdoom.exe!ParseAllDecorate() Line 1357	C++
 	lzdoom.exe!LoadActors() Line 462	C++
 	lzdoom.exe!PClassActor::StaticInit() Line 401	C++
 	lzdoom.exe!D_InitGame(const FIWADInfo * iwad_info, std::vector<std::string,std::allocator<std::string>> & allwads, std::vector<std::string,std::allocator<std::string>> & pwads) Line 3354	C++
 	lzdoom.exe!D_DoomMain_Internal() Line 3766	C++
 	lzdoom.exe!GameMain() Line 3817	C++
 	lzdoom.exe!DoMain(HINSTANCE__ * hInstance) Line 267	C++
 	lzdoom.exe!wWinMain(HINSTANCE__ * hInstance, HINSTANCE__ * nothing, wchar_t * cmdline, int nCmdShow) Line 557	C++
 	[External Code]	

And the source in filesystem.cpp:

Code: Select all

void FileSystem::ReadFile (int lump, void *dest)
{
	auto lumpr = OpenFileReader (lump);
->	auto size = lumpr.GetLength ();
	auto numread = lumpr.Read (dest, size);

	if (numread != size)
	{
		throw FileSystemException("W_ReadFile: only read %td of %td on '%s'\n",
			numread, size, FileInfo[lump].LongName);
	}
}

Seems it's crashing getting the size of the lump, may be i could check if it's empty. But anyway i'm afraid you'll need to reupload the files.

[Premo36]

Yes! Empty files are the problem!
There is one empty file in PsychoPhobia.pk3, which is [DooM].gl.c. If deleted the mod starts but halt with a fatal error due to that file being referenced somewhere... even if empty (I guess is some leftover code) but is not crashing anymore. If I open it and just add // and nothing else it works! Even keeping the 7z format!

Let's go in order, first i looked for the rar file, it looks like there are 3 .rar in PsychoPhobia.pk3, weird... maybe it was the original source file for player SFX by the name / path of the files.

Code: Select all

$ find . -type f -name '*.rar'
./PsychoPhobia/3dmodels/player/Grunt/_sound.rar
./PsychoPhobia/3dmodels/player/Caleb/_sounds.rar
./PsychoPhobia/Sounds/Player/Grunt_female/_Grunt_female.rar

As for the empty files, I've found an empty file and an empty folder

Code: Select all

$ find .  -empty
./PsychoPhobia/Actors/Effect/[DooM].gl.c
./PsychoPhobia/Flats/DooM_Replacer

I left everything as is (so the empty folder DooM_Replacer and the 3 rar files have not been removed or altered in any way) but I filled with an empty comment the file [DooM].gl.c, and it worked!

So I think adding a null/empty check as you suggested should fix the issue! (Or anything that can detect if a lump have a size of 0 bytes I think...)

But I've got one question, why does it happen only if the mod is packaged in a 7z archive? Because if I repack the mod, by extracting the files to a folder, and then compressing the folder to a .zip, it works with 4.14.2 without changing anything in the content of the archive.

Thanks!

P.S. I've attached two sample 7z file (I couldn't attach file with a .pk7 extension). Both of them just have 1 lump DECORATE.
In Empty.7z the lump is empty and will crash gzdoom. In Not_Empty.7z the lump have a small comment, it will start.

[drfrag]

Wenas. There were a lot of changes in the filesystem around december 2023, i don't know exactly why it happens but it's fixed at least in LZDoom. I added a null pointer check and soon i should make a PR for GZDoom. BTW with softpoly it already crashed in old versions probably some model in nature but just guessing.
https://github.com/drfrag666/lzdoom/com ... 6822b6bbb7

[Premo36]

Thanks drfrag!

As for the softpoly bug, I gave it a try with lzdoom 4.11.4 launching PsychoPhobia w\ PsychoPhobia_3D Shells stay forever.bat ,using the original, unpatched PsychoPhobia.pk3, and indeed while it boots up, after a bit of playing it crash straight to desktop, didn't even show the usual crash report pop-up.
Also performance were really degraded, like 15 times worse then opengl ES, plus when shooting it took up to 7000ms for a single frame . I didn't notice any difference using the converted .zip version of PsychoPhobia.pk3 or the one with the comment inside [DooM].gl.c.

Anyway I don't think is a big issue, this mod always have been picky with the rendering settings. I think that using original gzdoom version it shipped with It was locked with opengl hardware accelerated rendering and In modern gzdoom version you can play around with the render settings, but anything that is not "hardware accelerated" (no matter the backend) will result in a glitchy experience, where no model (including the player weapons) are rendered and most of the decals are replaced by a black square. Considering how weird and full of random stuff are the mod files I'm already more then happy that we managed to find what was the source of the crash

[drfrag]

I just added a null pointer check, i've made the PR.
The softpoly crash is no big deal but i'd like to fix it, unfortunately i dunno how. Softpoly already crashed on debug builds even with just doom 2 so i have to use RelWithDebInfo.
https://github.com/ZDoom/gzdoom/pull/3262

[Premo36]

I can confirm this is fixed in gzdoom dev build pre-423-ge4a77cf8b