Recomendations for map inspection software?

Discuss anything ZDoom-related that doesn't fall into one of the other categories.

Recomendations for map inspection software?

Postby Enjay » Sun Oct 13, 2019 3:44 am

What I'm looking for are tools that can read WAD/PK3 files, interrogate the maps therein and give me some kind of searchable output to tell me what is present on a map by map basis.

What I mean by that is - I run the tool on a particular map set and I could, say, get a text file liting all the texture names used on a map by map basis, or perhaps a list of ednums used on (again) a map by map basis.

With ZDoomHexen format maps, I can use DeePSea, but I'm pretty much leaving that format behind now. In the past, If I have been altering resources, I have found it very useful to be able to search through a map set to find which maps contain the altered resource. Then I can open those maps, and only those maps, in the map editor and check whether I am happy with the result without having to manually search through every possible map in a set "just in case".

Here's an example output for part of doom.wad from DeePsea that shows the kind of thing I mean:

Spoiler:


So, if I have altered STARTAN3, for example, I can just load the above up in a text editor, search for STARTAN3 and then only bother opening the maps that use that texture. It's not perfect, of course, it won't catch textures that change during play (via ACS etc) but it does what I need 99% of the time.

DeePsea can't do the above for EdNums, but I have a separate tool called EdNumExplorer that does something similar to the above, but for map things. However, it does not work on UDMF maps.

So, what - if any - tools are good for doing this with UDMF maps?

Thanks
User avatar
Enjay
Everyone is a moon, and has a dark side which he never shows to anybody. Twain
 
 
 
Joined: 15 Jul 2003
Location: Scotland

Re: Recomendations for map inspection software?

Postby DOTHEDA » Tue Oct 15, 2019 9:20 am

Can you tell me what EdNums are? I'm also not sure if you also want to be able to see what assets have changed from the base IWAD or just list what assets are used in the map (or both). The reason I ask is because I can make this program for you (if you didn't already find a solution,) I already have part of the program made as I wanted to see if I could do it and I think I can, once I know what EdNums are at least.
User avatar
DOTHEDA
 
Joined: 10 Oct 2019
Github ID: DOTHEDA
Operating System: Other Linux 64-bit
OS Test Version: No (Using Stable Public Version)

Re: Recomendations for map inspection software?

Postby Enjay » Tue Oct 15, 2019 3:53 pm

EdNums = Editor Numbers - the numbers given to things; used to place an actor in the map.

e.g. Zombieman has EdNum 3004, Imps 3001, Soul Sphere 2013, Super Shotgun 82 etc etc.

I'm not trying to compare maps (e.g. original maps versus IWAD or whatever). I'm just trying to find out if particular resources are used on certain maps.

As examples:
Imagine if I have a particular texture that I'm not happy with, so I decide to improve it. However, doing so means it looks a bit different to what it did before and therefore it may now not look quite right in some places in my maps. So, I'd need to have a quick look at it in game/in an editor to see if the new version looks right. I know that I have used the original texture in several maps in a megawad, but I'm not sure exactly which maps. I don't want to search through every map, so I use a quick inspection tool that tells me that texture is on maps 02, 07, 12... So, all I have to do is look at those particular maps to check the new texture and I can safely ignore the other maps.

In the same megawad, I have placed my own DECORATE monster. I'm basically happy with it, but I feel, just occasionally, I want to use a tougher version, so I make a slightly more aggressive one with more hit points. I don't want to replace it randomly, I want to find every instance of the monster's usage and evaluate for myself whether it should be the easy version or the tough version that appears this time. Again, I know the monster is used in several maps in the megawad, but I'm not sure which ones. So, again, I'm looking for a tool that tells me which maps to look at and which maps to ignore; this time it would do it by searching for the EdNum of the monster.

So, I want to be able to point a tool at a multi-map WAD/PK3/Directory, have the tool(s) interrogate any maps that it finds and report back which textures/EdNums are used on which maps.


I know it probably sounds like a minor-use niche requirement, but I have needed to do this many, many times over the years in my old Doom and Hexen format maps. Now that I have (finally) moved over to UDMF, I don't have any tools that can do the above for that map format. So, any help would be greatly appreciated. :)
User avatar
Enjay
Everyone is a moon, and has a dark side which he never shows to anybody. Twain
 
 
 
Joined: 15 Jul 2003
Location: Scotland

Re: Recomendations for map inspection software?

Postby Kappes Buur » Wed Oct 16, 2019 2:03 am

Maybe MTrop's Doom Utilities
User avatar
Kappes Buur
 
 
 
Joined: 17 Jul 2003
Location: British Columbia, Canada

Re: Recomendations for map inspection software?

Postby Enjay » Wed Oct 16, 2019 12:01 pm

Thanks for the suggestion. However, I just tried ThingSpy on a UDMF map and
Code: Select allExpand view
thingspy map17u.wad -s 2011
Opening file map17u.wad...
    Opening map MAP17...
    Format is UDMF...
Exception in thread "main" net.mtrop.doom.map.udmf.UDMFParseException: (UDMFLexer) Line 3192, Token "1": Expected "}" to terminate object.
        at net.mtrop.doom.map.udmf.UDMFReader$UParser.read(Unknown Source)
        at net.mtrop.doom.map.udmf.UDMFReader$UParser.<init>(Unknown Source)
        at net.mtrop.doom.map.udmf.UDMFReader.readData(Unknown Source)
        at net.mtrop.doom.map.udmf.UDMFReader.readData(Unknown Source)
        at net.mtrop.utility.doom.thingspy.ThingSpy.inspectMap(Unknown Source)
        at net.mtrop.utility.doom.thingspy.ThingSpy.inspectWAD(Unknown Source)
        at net.mtrop.utility.doom.thingspy.ThingSpy.processWAD(Unknown Source)
        at net.mtrop.utility.doom.thingspy.ThingSpy.execute(Unknown Source)
        at net.mtrop.utility.doom.thingspy.ThingSpy.execute(Unknown Source)
        at com.blackrook.utility.Utility.go(Utility.java:83)
        at net.mtrop.utility.doom.thingspy.Main.main(Unknown Source)


The same map in Hexen format works just fine:
Code: Select allExpand view
thingspy map17h.wad -s 2011
Opening file map17h.wad...
    Opening map MAP17...
    Format is HEXEN...
        Reading THINGS...
MAP17

:?

[edit] Same result with TexSpy
Shame, because these tools are meant to do exactly what I wanted.
[/edit]
[edit2]
Same result with the jar versions. :( [/edit2]
User avatar
Enjay
Everyone is a moon, and has a dark side which he never shows to anybody. Twain
 
 
 
Joined: 15 Jul 2003
Location: Scotland

Re: Recomendations for map inspection software?

Postby DOTHEDA » Wed Oct 16, 2019 4:24 pm

https://github.com/DOTHEDA/ednim/releases

Sorry it took me so long to make. I will probably be asleep soon so if it doesn't work, you have issues with it or it has bugs I will get to them tomorrow.
You use this program by typing:
Code: Select allExpand view
ednim [-t/-e] [-s] wadname.wad


As an example, you want to search;
Textures:
Code: Select allExpand view
ednim -t -s="startan2" MAP01.wad

or EdNums:
Code: Select allExpand view
ednim -e -s=321 MAP01.wad

You can also list them without a search:
Code: Select allExpand view
ednim -e MAP01.wad
ednim -t MAP01.wad

DO NOT put spaces between the '-s', the '=' and the search string. This is a limitation of the library I'm using and I will fix it at some point.

It does not support PK3's yet as I don't like using libraries if I don't have to and I need to learn how zip files work to program that functionality in.

It might be confusing to use so sorry about that and I'm not sure if it will please you entirely but I'm very much so open to suggestions/critiscism.
[edit]Oh ya, it also only supports UDMF right now, but I do intend to add support for standard Doom/Hexen (once I learn how those lumps work).[/edit]
User avatar
DOTHEDA
 
Joined: 10 Oct 2019
Github ID: DOTHEDA
Operating System: Other Linux 64-bit
OS Test Version: No (Using Stable Public Version)

Re: Recomendations for map inspection software?

Postby Enjay » Tue Nov 12, 2019 12:40 pm

:oops:
I'm sorry. For some reason I didn't spot this reply until now. I've just tried the tool and, unfortunately, I'm getting mixed results with it.

Some maps seem to be doing exactly what they should - I can get a list of all textures (or EdNums), or search for specific textures (or EdNums) and get the results I expected. Other maps give me feedback like:

Code: Select allExpand view
C:\Users\enjay\Desktop\EdNimTest>ednim -t map01.wad
fatal.nim(39)            sysFatal
Error: unhandled exception: over- or underflow [OverflowError]


All maps tested have been in UDMF format and get a clean bill of health from the GZDB error checker. The same thing happens with a texture check and an ednum check. In this particular case, MAP01 is bigger and more complicated than MAP02, but both maps are within UDMF spec.

The only other (much more minor) problem is that the feedback when no textures of a specified type are found is a bit misleading IMO.

Code: Select allExpand view
C:\Users\enjay\Desktop\EdNimTest>ednim -t -s="STARTAN2" MAP02.wad
-- MAP02 --
no textures found in this map (invalid wad?)


The "invalid wad?" message part always appears regardless of whether the tool has a good reason to suspect the wad's validity or not. The above was generated when searching for STARTAN2 (which it correctly could not find in this map) but it can read the wad OK because searching for other textures that are in the map gives a valid result.
User avatar
Enjay
Everyone is a moon, and has a dark side which he never shows to anybody. Twain
 
 
 
Joined: 15 Jul 2003
Location: Scotland

Re: Recomendations for map inspection software?

Postby bLUEbYTE » Mon Nov 18, 2019 3:30 am

Hi Enjay,
I know next to nothing about Doom editing, but I think I can help.

Seeing as how UDMF maps are just text, you can make use of the traditional unix tools to do things like the ones you mentioned. I started my exporting the map out of the wad/pk3 into
Code: Select allExpand view
TEXTMAP.txt
using SLADE. Thanks to wadext and unzip, this archive extraction can be easily automated as well. In fact, the whole workflow can be scripted - let me know how you like this approach and we'll take it from there. The unix toolset is amazingly flexible overall.

To demonstrate (Using https://www.doomworld.com/idgames/level ... u/udmforig):

Search for and list the textures used in the map:
Code: Select allExpand view
$ grep texture TEXTMAP.txt
texturefloor = "FLOOR4_8";
textureceiling = "CEIL3_5";
texturefloor = "FLOOR4_8";
textureceiling = "FLAT20";
texturefloor = "FLOOR4_8";
textureceiling = "CEIL3_5";
texturemiddle = "BROWN1";
texturetop = "BIGDOOR2";
texturemiddle = "BROWN1";
texturemiddle = "BROWN1";
texturemiddle = "BROWN1";
texturemiddle = "BROWN1";
texturemiddle = "DOORTRAK";
texturemiddle = "DOORTRAK";
texturetop = "BIGDOOR2";
texturemiddle = "STARTAN1";
texturemiddle = "STARTAN1";
texturemiddle = "STARTAN1";
texturemiddle = "SW1STRTN";
texturemiddle = "STARTAN1";
texturemiddle = "STARTAN1";
texturemiddle = "STARTAN1";


Like above, but only list unique entries:
Code: Select allExpand view
$ grep texture TEXTMAP.txt  | sort | uniq
textureceiling = "CEIL3_5";
textureceiling = "FLAT20";
texturefloor = "FLOOR4_8";
texturemiddle = "BROWN1";
texturemiddle = "DOORTRAK";
texturemiddle = "STARTAN1";
texturemiddle = "SW1STRTN";
texturetop = "BIGDOOR2";


Same thing, but only list the unique texture names used in the map:
Code: Select allExpand view
$ grep texture TEXTMAP.txt | cut -d '"' -f2 | sort | uniq
BIGDOOR2
BROWN1
CEIL3_5
DOORTRAK
FLAT20
FLOOR4_8
STARTAN1
SW1STRTN


Count the number of unique textures used:
Code: Select allExpand view
$ grep texture TEXTMAP.txt  | sort | uniq | wc -l
8


Count the number of imps in every file in current directory, with filename (only one file in this example, but imagine each map in a separate .txt):
Code: Select allExpand view
$ grep -Hc 'type = 3001' *
TEXTMAP.txt:3


What OS are you on? On Windows, you can install Windows Subsystem for Linux (google a guide). On Mac & Linux, well, you already have what you need. Let me know if you like this approach and I'll walk you through or possibly develop scripts for you.
bLUEbYTE
 
Joined: 15 Nov 2019
Location: Australia
Graphics Processor: Intel with Vulkan Support

Re: Recomendations for map inspection software?

Postby Enjay » Mon Nov 18, 2019 2:30 pm

I'm on Windows (10) and being able to get text outputs (particularly useful to me would be the unique ones but all are potentially useful). I have to admit that it sounds like you are proposing installing something quite hefty (i.e. the subsystem) versus what I plan to use it for though - i.e. to run some relatively simple scripted text extractions and consolidations.
User avatar
Enjay
Everyone is a moon, and has a dark side which he never shows to anybody. Twain
 
 
 
Joined: 15 Jul 2003
Location: Scotland

Re: Recomendations for map inspection software?

Postby m8f » Mon Nov 18, 2019 8:57 pm

For trying out Unix tools at Windows, I think "Git for Windows" is the easiest way to get in: https://gitforwindows.org/
Don't mind if you don't need Git itself. This package contains Unix command line interpreter (bash) and all the things like grep, uniq, sort, wc. The installer is about 46 MB. The installation is easy, just like any other windows program. This thing won't interfere with your OS. After the installation, you will be able to launch command line with Unix tools just by selecting an item in directory context menu.
git.png
User avatar
m8f
dreamer
 
 
 
Joined: 29 Dec 2017
Location: Siberia (UTC+7)
Discord: m8f#0629
Github ID: mmaulwurff

Re: Recomendations for map inspection software?

Postby bLUEbYTE » Tue Nov 19, 2019 3:58 am

Yep, good call m8f. Another similarly light option is Cygwin, but it is kinda convoluted to pick packages to install, etc. Git for Windows should do it nicely for sure.
bLUEbYTE
 
Joined: 15 Nov 2019
Location: Australia
Graphics Processor: Intel with Vulkan Support

Re: Recomendations for map inspection software?

Postby DOTHEDA » Wed Jan 08, 2020 5:41 pm

I'm so sorry for vanishing for so long, I had a rut to get through, but I finally made a (hopefully) functional program for your use case. Please tell me if there are any issues, I'll try to be more attentive to this thread.

Edit:
I actually updated ednim with a from-scratch re-write, so it should function as intended this time around. Use '-e #' to search for ednums and '-t STRING' to search for textures. Syntax would be;
'.\ednim.exe -e STARTAN2 DOOM2.WAD'
for example.
Again, sorry this took so long to make.

Download:
https://github.com/DOTHEDA/ednim/releases
User avatar
DOTHEDA
 
Joined: 10 Oct 2019
Github ID: DOTHEDA
Operating System: Other Linux 64-bit
OS Test Version: No (Using Stable Public Version)


Return to General

Who is online

Users browsing this forum: Google [Bot], Yandex [Bot], Zackdoessomething and 8 guests