Page 1 of 5

Zip Support

PostPosted: Tue Aug 12, 2003 8:09 am
by Fredrik
I don't see any catches with using zips in addition to WADs. It's obviously better in every way for the person on the editing side. Along with support for loading PNGs (or is it in yet? I might have missed it), no one should ever again have to face the horror of lump management in any editor :)

The issue of lump order and duplicate names can be resolved using subdirectories 99% of the time, and the other 1% are hacks that aren't necessary if using ZDoom (or in the worst case problems that could be solved by placing a WAD containing the ordered lumps inside the zip).

You know you want it!

PostPosted: Tue Aug 12, 2003 9:45 am
by randomlag
Simple put, I disagree.

PostPosted: Tue Aug 12, 2003 10:00 am
by Sphagne
How about asking the author of excelent 7z program to add support for wad files?

PostPosted: Tue Aug 12, 2003 10:52 am
by Fredrik
randomlag wrote:Simple put, I disagree.

Why?

PostPosted: Tue Aug 12, 2003 11:07 am
by boris
I think it would be great if ZDoom could load .wads stored in .zips. That would save A LOT of space!

PostPosted: Tue Aug 12, 2003 11:38 am
by Fredrik
boris wrote:I think it would be great if ZDoom could load .wads stored in .zips. That would save A LOT of space!

That's great too, but I'm mainly thinking of editing ease. Keeping a directory structure and just doing a drag & drop onto a zip is incredibly much faster and easier than fumbling around with a lump managing editor.

PostPosted: Tue Aug 12, 2003 3:50 pm
by Enjay
Graphics offsets are the first problem that springs to my mind. Zip's simply don't have that, therefore there would have to be some sort of new control lump for them. Or perhaps you could convert them to their lump format and then insert them in the Zip, but that seems to defeat the purpose.

Bottom line, I don't find managing lumps in a WAD any more difficult than using Winzip anyway. In fact, sometimes I find it easier. Mind you, I probably use DeePsea more than winzip so I guess I'm more familiar with it. :D

PostPosted: Tue Aug 12, 2003 6:49 pm
by Fredrik
There are numerous ways to deal with offsets. A control lump would work. There's also the (though weird) possibility of using prefixes or suffixes, e.g. possa1_25_33.png. You could also choose to have a WAD within the zip for those sprites.

And you shouldn't compare Deepsea with Winzip. The very point is that you'd do the management in your operating system's file manager - and you are more familiar with that than you are with Deepsea, right? :) - and then just use a compression utility to compile the zip.

PostPosted: Wed Aug 13, 2003 2:21 am
by GooberMan
Using zips for WAD management is a good idea. DEUTex outputs the different types of data in to different directories (sprites, patches, etc), so setting up a directory structure similar to that in a ZIP file would probably be a good way to get around markers in the WAD format like P_START and S_START etc.

PostPosted: Wed Aug 13, 2003 2:55 am
by Hirogen2
Oh we just need a brand new editor.
And if I may propose, add gzip/bzip2 too. (To make it a .wad.gz which --
combines keeping the offsets plus compression. Only meaningful after the wad is
*COMPLETE*)

PostPosted: Wed Aug 13, 2003 4:05 am
by Enjay
Fredrik wrote:And you shouldn't compare Deepsea with Winzip. The very point is that you'd do the management in your operating system's file manager - and you are more familiar with that than you are with Deepsea, right? :) - and then just use a compression utility to compile the zip.


True enough, but that's basically what I do. I organise my files in directories then use a "zip" tool to put them all into one container. It's just that in this case the container is a WAD and has no compression. I'll concede that there is an extra step in doing it the way I do because with zipping things up, I would drop the folders onto the zip to preserve the paths (which is the suggestion for identifying flats/texture/sprites etc - yes?), but with a lump management tool, you need to go into the directory and select all files.

I don't see any problem with sticking the various lumps between F_, P_, S_ markers provided you have a good util to work with - it's dead easy if you do. I'll agree, there is room for people to make a mistake and people have to learn that part of lump management rather than just using their (presumably) existing knowledge of path structures. There is the possibility of errors with zips too. A number of times with a zip, I've put everything into an archive using paths, then modified a file, dropped it into the zip and it has ended up outwith the path structure contained in the zip. From experience, moving an errant sprite from outside the S_ markers in a WAD (again, with a good tool) is easier than moving a file around the path sructure inside a zip.

With Zdoom, however, in future there may be no need to have flats/textures/patches individually identified a lot of the time. Zdoom will soon have a release version that supports "any graphic anywhere" so all you have to do in most cases is include all your new graphics as patches and use them as and where you see fit. No need for paths, or even markers in a WAD if you want to be untidy about it.

Not wanting to get bogged down with the one issue, as this is a bigger question than just this, but sprite offsets are still a stumbling block IMO. The ways suggested: a control file, incorporating it into the name etc would increase the time involved getting sprites to reasonable offsets hugely. I'm not sure how many lump tools actually do this, but when you import a projectile or monster/decoration sprite using DeePsea, it analyses the dimensions of the graphic and imports it at reasonable offsets. eg 500 sprites all offset in one go automatically without the user even noticing the time it took. If I had to type a control lump, or individually name the sprites it would take ages.

Once the sprites are in a WAD, you may still want to tweak their position. DeePsea, XWE and Wintex all provide similar tools for doing this. Sprites are shown on screen and can be moved around in single pixel increments, quickly checking how they line up with the next sprite by simply clicking on the next one in the list and going back and forward between them. I'm not aware of any Zip tool that allows you to preview graphics simply by highlighting them, much less see how they would line up in a game with the next graphic in a series. So, with the zip method, I'd have to go into game, try and spot how well my sprites were lining up, try and work out which ones were the problem, remember them, quit, go to my control lump/file name, re type the offset, then go back into the game to see if what I had done looked OK.

I guess someone could write a tool that auto generates whichever control method was opted for. This tool could perhaps even allow interactive adjustment of sprite offsets in a zip and modify the control method appropriately. However, that's getting into a new, specific Zdoom-zip management tool which pretty much defeats the purpose of what this thread is about.

There is also another issue, which is a pretty big one IMO. No wad editing tools support zips. "So what?" You might be saying - the OS does it for you. But, for example, how do you propose an editor would allow you to preview a wall or flat graphic if it doesn't suport the format? You'd end up putting texture name onto side defs "blind" or doing something odd like browsing your texture directory using PSP and switching backwards and forwards to your editor to type in texture names (no selecting from drop down lists either). Or perhaps putting everything into a WAD for editing purposes, loading that into an editor and using it whilst you make the level, then compiling a zip for the distribution version. In other words, any editing tools would really have to be updated to support zips or lumps/graphics in a seperate directory before this would be a practical option. Already I see things getting messy with that kind of support and plenty of tools aren't updated anymore either.


I think really what I am saying is that the WAD format is a format designed for the job. Zips aren't, and do not natively offer all the reqirements of lump storeage for a Doom game. To make them provide those functions would mean the need for additional conventions, options and lumps for people to learn and for updated/new tools to support them. These would just be un-necessary work arounds IMO, to provide something the WAD format already does.

PostPosted: Wed Aug 13, 2003 7:15 am
by GooberMan
No editor supports zips YET. If the feature is introduced, editors that still get updated would no doubt be modified accordingly, which means you'd still be able to use DeePsea to play around with sprite offsets.

Supporting ZIPs would seem to make more sense to me with the any graphic any where idea than hacking around in an archive format that was designed for explicit segregation of the data. New lumps, the directory structure idea I mentioned above, and even the old P_ S_ and F_ lumps aren't necessary if all graphics are treated equal.

PostPosted: Wed Aug 13, 2003 9:52 am
by randomlag
The reason zips, etc makes NO sense is that all it does is INCREASE the difficulty in editing. I liked ENJAY's "container" word. That says it all.

Think about it. Now only do you have to create the right resource, but then all the xtra controls and then you have to ZIP the damn thing. Sounds like about 3X the amount of work to me, plus a PIA to "fix".

More steps and more confusion - No thanks :o

PostPosted: Wed Aug 13, 2003 3:10 pm
by HotWax
I have mixed feelings on this. On the one hand, I don't see a great need for a new format, although I disagree that it might not be at least a little easier for some people, in addition to the obvious fact that the files would be compressed and thus take less space on one's hard drive.

That said, I think everyone who is against the idea is looking at it in completely the wrong way. S_START? P_START? WHY? Just so that we maintain the zips to look like a wad structure? Foolish and unnecessary. If support is added for a new format, that new format could easily encorporate a completely new structure. There'd be no backwards compatibility concerns because no other port can read ZIP files. We don't have to conform to anything here.

Maps could go in their own subfolder within the ZIP. The subfolder name would be used as the lump name. Example:

.\MAP01\THINGS
.\MAP01\LINEDEFS
.\MAP01\SIDEDEFS
.\MAP01\VERTEXES
.\MAP01\SEGS
.\MAP01\SSECTORS
.\MAP01\NODES
.\MAP01\SECTORS
.\MAP01\REJECT
.\MAP01\BLOCKMAP
.\MAP01\BEHAVIOR
.\MAP02\THINGS
.\MAP02\LINEDEFS
...etc...

This avoids the obvious problem of duplicate filenames.

Sprites, textures, and flats could similarly be placed in appropriate folders. The names can be whatever we want. (Remember, no backwards compatibility to worry about)

.\SPRITES\CHGGA0
.\SPRITES\CHGGB0
.\SPRITES\CHGFA0
.\SPRITES\CHGFB0
.\SPRITES\SAWGA0
...etc...

Textures and flats would be handled in a similar fashion. With the move to universally accepted graphics on floors and walls, it may not even be necessary to separate them into their appropriate categories at all. (The only exception being two resources of different type with the same name, in which case you could optionally differentiate with a simple extension. MYPIC.SPR, MYPIC.FLT, etc.)

Most resources could simply be dumped in the main ZIP folder and would work exactly as they do now once extracted. Implementing this should not be that difficult.

As far as offsets are concerned, a simple text file inserted as a lump would suffice. DEUTEX uses such a file (WADINFO.TXT) to, among other things, specify the necessary offsets. I would disagree with saying that writing such a file would be any more difficult than entering the numbers any other way. Future tools could even incorporate making the file automatically into what they already do.

Animations would need special handling since ZIP files do not have an order. We already have an ANIMDEFS lump to handle this. Files made in ZIP format would require this file to be present if animations are to be used. Again, this is not a problem with older material because it won't use the format anyway.

Hopefully I've made it clear why adding ZIP support shouldn't be too hard to do. I do need to restate though that I don't see this as a huge deal. Most WAD sizes are not that much to deal with, although there are some who are using older systems with limited space that it might benefit. If there is enough call for this system, I see no reason why it can't be done. The question that remains then is if there is enough demand to consider it a priority.

[EDIT]It also occurs to me that it'd be even easier to use the existing output of most editors when you make a new map. That is, they typically output a WAD with a single "MAP01" header and the lumps listed after. Being able to rename this WAD to whatever you want you map to be called. (i.e. OUTPOST.LVL) and then insert it into a ZIP at the root level *would* be as easy or easier for most people than using a WAD editting tool to merge the wads together. If this system is used either with or in place of the above subdirectory system, ZDoom would simply ignore the "MAP01" header inside the file and use the filename as the map's lumpname.

That way you could either do this...

.\MYMAP\THINGS
...etc...

or this:

.\MYMAP.LVL
(in PWAD format containing a map header (ignored) followed by all map lumps for that level)

Either way, the level would be loaded in ZDoom by typing "map mymap".

PostPosted: Wed Aug 13, 2003 3:32 pm
by Enjay
I think from my perspective, the anser to this is "no need".

Yes, I agree the zip format is convenient, universally understood (unless you are my friend Russ who has never understood them) and can do a great deal of what is needed. If a game is written with them in mind (eg Q3) then the engine natively supports the format and the zip isn't asked to do anything the format isn't supposed to do.

However, Doom was written with WADs in mind, not zips. WADs and the lumps withing them already do the job required. There have been a number of sensible suggestions in this thread as to how the data required by (Z)doom could be incorporated into a zip. The problem is, it's a case of reinventing the wheel. Why bother trying to come up with ways to do things that are already handled quite happily by the WAD format?

The only advantage I can see to using the Zip format is the file size thing, and to be fair, that's pretty irrelevant with most of us having multi gigabyte HD's these days.

I just can't see the point of creating a new system with new conventions, requiring new tools (and thereby making many people's favourites obsolete) when a system exists that does the job perfectly already, especially when moving to a new system confers no real advantage over the old one.