Problem with removing unused ZDoom textures with Slade

[IgnyteZero]

So my crappy little megawad is nearing it's completion with 13 levels, 2 secret levels and 1 dm level. Naturally I am thinking of sharing it with the world, as it does little to have it just sitting ignored on my harddrive. No matter where I end up uploading it (another issue I have to look into soon enough) I figured it was time to trim down my WAD-file as uncompressed it sits at around 100 megabytes, and I know there's quite a few unused textures from the packs I've used for it.

However, as the title of this thread says, when trying to remove them via Slades function to do so, it breaks my wad completely, turning it into an unplayable state unable to run. And trying to handpick them for removal is a no-no. That's just an impossible task that'd take years, so I am kind of dependent on having some automated way of removing them.

Now, the wall textures in my WAD are all, except flats, sorted as Zdoom textures in the file. I've used the Archive/Maintenance option "Remove unused ZDoom Textures" (as naturally the option "Remove Unused Textures" does nothing). When doing so it says there are around 1600 unused textures and over 100 unused flats. Now, while I know there probably would be some hundred unused textures that number just seem to be approximately around ALL custom textures added in the file. And if I let Slade remove them and load it into either UDB or GZDoom I get quite a handful of random errors, among other claiming to lack resources connected to custom monsters, weapons etcetera. So that function does not seem to work.

I've tried following this guide on Doomworld https://www.doomworld.com/forum/topic/1 ... your-maps/, but to the same disastrous result.

Is there any tool out there or method that would be more efficient on this or am I just screwed in this regard?

[Enjay]

I'm not sure how many tools deal with cleaning up ZDoom textures. There are a few options for cleaning up the old format of TEXTURE1 and PNAMES textures, but ZDoom textures being newer and port specific, have fewer options. Slade may be it, as far as dedicated tools go.

As a clumsy, more involved way of doing things, it is possible to use TexSpy to get a list of textures that your map uses and send that list to a text file.
Then you could extract all your ZDoom textures into a folder on your HD, remove them from your project file (working on a copy for safety) and convert the text file into a batch file that copies all the used texture names to a second folder on your HD. Then you could re-insert those textures, and only those textures, back into your project file. I have done that kind of thing before. Like I said, it's clumsy and you have to be confident with batch files and file operations, but it can work.

That being said, if deleting the textures is causing other conflicts, I wonder what's going on there? Like, do you maybe have sprites in the wrong namespace (i.e. the textures namespace) or something? If not, then Slade may have a problem.

It's also worth flagging up that maps use several textures that are not on the walls at the outset. e.g. if you have a switch on a wall, one state for the switch (e.g. "Off") will be in the map, but the other "On" texture isn't there until you flip the switch. Similarly, any animated textures will only have one frame of the animation on the wall in the map, and the subsequent frames appear when the animation gets cycled at runtime. If you are changing any textures via ACS during play, that too means that there could be some textures that are not saved in the map file, but will appear later on during play. I'm not sure how clever Slade is at catching these issues, but the batch file method I described knows nothing about them. So, it's something to watch out for regardless of what method you use.

[IgnyteZero]

Thanks for the speedy reply

As a clumsy, more involved way of doing things, it is possible to use TexSpy to get a list of textures that your map uses and send that list to a text file.
Then you could extract all your ZDoom textures into a folder on your HD, remove them from your project file (working on a copy for safety) and convert the text file into a batch file that copies all the used texture names to a second folder on your HD. Then you could re-insert those textures, and only those textures, back into your project file. I have done that kind of thing before. Like I said, it's clumsy and you have to be confident with batch files and file operations, but it can work.

Well, that I am not. My knowledge of programming stops with old C64 Basic (and some html) more or less. My edits to Zscripts etc for my wad has just been by pure deduction. But I guess a tool like that could still be useful and make a pure manual editing out of unused textures work. Tiresome and a large task indeed, but better than to pointlessly keeping it all in there needlessly.

So where can I find TexSpy?

That being said, if deleting the textures is causing other conflicts, I wonder what's going on there? Like, do you maybe have sprites in the wrong namespace (i.e. the textures namespace) or something? If not, then Slade may have a problem.

That might be possible, but in such cases that would probably have been done by the author of said resource, as I've simply added them to my WAD as is and only added Doom numbers and fixed errors/customised stuff with the scripts to make them work. But from my limited knowledge it at all at least looks like it should be correct. I'd share the file to show you, but I don't want to risk having it public before I'm confident I'm fairly done with the playtesting of it.

[IgnyteZero]

I found out ONE thing with SLADE though, it's does one VERY weird thing with it's purging of unused ZDoom textures. First it goes through all the Textures used as expected, it just seem to do a waaay too wide stroke when deciding what's unused. Waaay too wide. Even if I try to exclude animated textures it also deletes used textures. However, what it does next is really baffling, it goes on to check for flats, but it's not flats it's trying to purge, but all lumps, scripts and graphics that are not maps.

Now my personal uneducated guess is that the problem is that Slade tries to treat the .wad as if it was a .pk3. Can't swear on it, but it's the vibe I get. And not much I can do about. (Yeah, I know you're going to argue I could make it into a .pk3 but, that's also way too much work at this point).

Still haven't figured out what TexSpy is though. Any clarification on that? I googled a lot, but to no success. The closest I came to find was an article on something called WadSpy which seemed to be some old defunct mod for early UDB versions.

Deleted som 6 megs of uncompressed textures from the wad by hand though. Things I hundred percent KNEW I never used. But I'm sure there's couldbe around to up to 5 or 10 more megs worth of textures to be removed, but that's way too much trial and error for my taste so I guess I'll just have to find a way to distribute it as is whether I like it or not. I know Idgames isn't too fond of surplus unused textures for things though, from what I've read.

[Enjay]

I've always found converting a WAD to a PK3 reasonably straight forward - particularly if the WAD is well organised (e.g. just select everything between the first and last sprite and export to a folder called Sprites) but I understand not wanting to do it if you don't have to.

As for Slade's internal workings, sorry, I don't know enough to comment on how it does things, but what you are describing it doing does seem unusual.

TexSpy is one of MTrop's utils. You can read about them here:

https://www.doomworld.com/forum/topic/6 ... s-for-you/

[IgnyteZero]

I've always found converting a WAD to a PK3 reasonably straight forward - particularly if the WAD is well organised (e.g. just select everything between the first and last sprite and export to a folder called Sprites) but I understand not wanting to do it if you don't have to.

As for Slade's internal workings, sorry, I don't know enough to comment on how it does things, but what you are describing it doing does seem unusual.

Yeah, it's more of a principle based on nostalgia. Emotion trumps rationality in this case. And there's nothing to say it would matter for Slade in the end anyway after all.

TexSpy is one of MTrop's utils. You can read about them here:

https://www.doomworld.com/forum/topic/6 ... s-for-you/

Thank you. I'll definitely be looking into that!

[Enjay]

Fair enough on the WAD versus PK3 thing.

If you do decide to use TexSpy, here's the command line that I used for it: texspy map01.wad >TexMap01.txt
That redirects the output from texspy to a text file (in this case called TexMap01.txt). from there, you can open the text file and see all the textures that are used in the map.
It gives a nice simple list. For example, here's Doom2 map01.

Code: Select all

Opening file map01.wad...
    Opening map MAP01...
    Format is DOOM...
        Reading SIDEDEFS...
        Reading SECTORS...
-TEXTURE
ASHWALL3
BIGDOOR1
BRNSMAL1
BRONZE1
BRONZE2
BROVINE2
BROWN1
BROWN96
BROWNGRN
BROWNHUG
CEMENT9
COMPSPAN
COMPWERD
DOORTRAK
EXITDOOR
EXITSIGN
LITE5
MIDBARS3
MODWALL2
MODWALL3
PIPE2
PIPE4
SKY1
SPACEW3
SPACEW4
STEP2
STEP6
STONE7
SUPPORT2
SW1BRCOM
SW1BRNGN
SW1COMP
SW2BRCOM
TEKGREN1
TEKGREN2
TEKGREN3
TEKGREN4
TEKGREN5
TEKWALL4
TEKWALL6
-FLAT
CEIL1_2
CEIL1_3
CEIL3_1
CEIL3_3
CEIL5_1
CEIL5_2
FLAT10
FLAT18
FLAT1_1
FLAT1_2
FLAT2
FLAT20
FLAT23
FLAT4
FLAT5
FLAT5_5
FLOOR0_1
FLOOR0_2
FLOOR1_1
FLOOR3_3
FLOOR4_1
FLOOR5_3
FLOOR5_4
FLOOR7_1
FWATER4
F_SKY1
GRASS1
GRNLITE1
MFLR8_3
RROCK09
TLITE6_4

So, you can see that the textures are just listed by name, one on each line. So, it's quite easy to turn that into a batch file that can manipulate files with the same names.

The only problem I have encountered with it is that very big maps give an out of memory error and you can't get any output from them.

[IgnyteZero]

Thanks! That's nice. I'll see if I can make something with that. The newer maps are a bit big though... like 2-4 times the size of a normal classic Doom-map. We'll see.

Still don't know how to do batchfiles that could help me. But it might help me assemble what textures I'm using on all maps, and manually pull those out, purge the textures in the wad, and put those pulled back. If it works.

[Enjay]

If you are at all familiar with the command line interface, then batch files are pretty straight forward. They usually mostly just contain the kinds of commands that you can type at the CLI to manipulate files.

So, with the above example, once you have a text file listing all the textures in your map, you could extract all the textures from your WAD into a folder somewhere, and create a second empty folder somewhere and then edit the text file so it looks something like this:

Code: Select all

copy ASHWALL3.png c:\usedtex
copy BIGDOOR1.png c:\usedtex
copy BRNSMAL1.png c:\usedtex
copy BRONZE1.png c:\usedtex
copy BRONZE2.png c:\usedtex
copy BROVINE2.png c:\usedtex
copy BROWN1.png c:\usedtex
copy BROWN96.png c:\usedtex
...
...
...

(Where c:\usedtex is the path of the empty folder you created and .png is the extension that your extracted files have)

Rename the file so that it has the extension BAT instead of txt,
make sure it is in the same folder as your extracted textures
double click it and it will execute all the instructions in it - i.e. copy the textures.

Now you will have a folder that only has textures that were listed as being in the map(s), so you can delete all the textures in your WAD, and then re-insert the ones from the folder that only contains used textures. After that, add back in any animation and switch frames that were not listed as being in the map (oh, and possibly skies) and you should be good to go.

It's not as quick as having a tool that just does it for you but, given that Slade doesn't seem to be doing what you want, this should work.

[edit] 2-4 times the size of a classic map probably won't be a problem. The first map I found difficulties was the first ZDoom Community maps, and it is significantly bigger than a typical map. So, it really should only be the very, very big many, many linedef maps that cause a problem. It'll be interesting to hear how you get on if you give it a go though.

[IgnyteZero]

If you are at all familiar with the command line interface, then batch files are pretty straight forward. They usually mostly just contain the kinds of commands that you can type at the CLI to manipulate files.

So, with the above example, once you have a text file listing all the textures in your map, you could extract all the textures from your WAD into a folder somewhere, and create a second empty folder somewhere and then edit the text file so it looks something like this:

Code: Select all

copy ASHWALL3.png c:\usedtex
copy BIGDOOR1.png c:\usedtex
copy BRNSMAL1.png c:\usedtex
copy BRONZE1.png c:\usedtex
copy BRONZE2.png c:\usedtex
copy BROVINE2.png c:\usedtex
copy BROWN1.png c:\usedtex
copy BROWN96.png c:\usedtex
...
...
...

(Where c:\usedtex is the path of the empty folder you created)

Rename the file so that it has the extension BAT instead of txt,
make sure it is in the same folder as your extracted textures
double click it and it will execute all the instructions in it - i.e. copy the textures.

Now you will have a folder that only has textures that were listed as being in the map(s), so you can delete all the textures in your WAD, and then re-insert the ones from the folder that only contains used textures. After that, add back in any animation and switch frames that were not listed as being in the map (oh, and possibly skies) and you should be good to go.

It's not as quick as having a tool that just does it for you but, given that Slade doesn't seem to be doing what you want, this should work.

[edit] 2-4 times the size of a classic map probably won't be a problem. The first map I found difficulties was the first ZDoom Community maps, and it is significantly bigger than a typical map. So, it really should only be the very, very big many, many linedef maps that cause a problem.

I guess that's easy enough ofc.

Sadly however TexSpy didn't like my wad it seems.

Code: Select all

PS C:\texspy> c:\texspy\texspy.exe VACANTBARON_RC3.wad
Opening file VACANTBARON_RC3.wad...
    Opening map VACANT01...
    Format is UDMF...
Exception in thread "main" net.mtrop.doom.map.udmf.UDMFParseException: (UDMFLexer) Line 4, Token "864.0": Expected valid value.
        at net.mtrop.doom.map.udmf.UDMFReader$UParser.read(Unknown Source)
        at net.mtrop.doom.map.udmf.UDMFReader$UParser.<init>(Unknown Source)
        at net.mtrop.doom.map.udmf.UDMFReader.readData(Unknown Source)
        at net.mtrop.doom.map.udmf.UDMFReader.readData(Unknown Source)
        at net.mtrop.utility.doom.texspy.TextureSpy.inspectMap(Unknown Source)
        at net.mtrop.utility.doom.texspy.TextureSpy.inspectWAD(Unknown Source)
        at net.mtrop.utility.doom.texspy.TextureSpy.processWAD(Unknown Source)
        at net.mtrop.utility.doom.texspy.TextureSpy.execute(Unknown Source)
        at net.mtrop.utility.doom.texspy.TextureSpy.execute(Unknown Source)
        at com.blackrook.utility.Utility.go(Utility.java:83)
        at net.mtrop.utility.doom.texspy.Main.main(Unknown Source)

[Enjay]

Oh, that's weird. I've used TexSpy on literally hundreds of UDMF maps and I haven't come across that before. Obviously there is something about your map that it doesn't like but I have no idea what it would be. I take it that the map gets a clean bill of health in the Ultimate Doom Builder error checker?

[IgnyteZero]

Oh, that's weird. I've used TexSpy on literally hundreds of UDMF maps and I haven't come across that before. Obviously there is something about your map that it doesn't like but I have no idea what it would be. I take it that the map gets a clean bill of health in the Ultimate Doom Builder error checker?

Yes it does. They load in fine without any errors at all. So I wouldn't know what.

I haven't made a map analysis and cleaned them up as of recent changes, but I can't see how any misplaced textures or missing textures (some who are deliberate actually) would affect anything.

I could send you a link of the current version if you want to look at it, but I feel I am already taking up a lot of your time with this issue.

[Enjay]

If you want to DM me a link to the file, I'll happily take a look. Obviously, no guarantees that I'll be able to fix anything, but I can certainly look and maybe find something.

[IgnyteZero]

If you want to DM me a link to the file, I'll happily take a look. Obviously, no guarantees that I'll be able to fix anything, but I can certainly look and maybe find something.

Great. I'll send you a DM with a link in a little while. Just going through all levels and doublechecking things first.

[Enjay]

Just for the sake of the public thread - the problem was in fact that some of the maps in the WAD were too big for texspy. Details resolved via DM.

It's a real shame that texspy has that limit - especially with maps getting even bigger these days. I'm sure I did mention the problem with big maps to MTrop years ago. It looks like it was never fixed though.