Zip Support

[randomlag]

You don't get it, do you?

No Fred, it's you that doesn't get it. You keep repeating the same stuff over and over and refuse to think for a second that maybe there are some flaws. Just because YOU haven't run into it doesn't mean it doesn't happen.

You ran on an on about how C/C++ is so bad and this interpretive stuff was so cool. You still believe that - but you are wrong. Both are a force fit for an application they were not made for.

The entire point is that editors for Zips ALREADY EXIST. *cough* *cough*

Your cold is fogging your brain.

Zips are designed to be container files. There's no force-fitting.

So? It's a force fit for the game and for editing. I'd much rather see an extended sub-directory. Think outside of the box, not just what can be forced into an existing box.

[Fredrik]

There is no reason why we can't just write a really comprehensive tutorial to manipulate a WAD. In my eyes, this is the END of the discussion.

This is not about tutorials. I already know how to manipulate every single aspect of a WAD file down to the bit level. It's a matter of features that WADs lack.

[Mighty Duck X-treme]

*DELETED*

[Fredrik]

You keep repeating the same stuff over and over and refuse to think for a second that maybe there are some flaws. Just because YOU haven't run into it doesn't mean it doesn't happen.

I keep repeating it because you're not listening. There is just one flaw - lack of internal order, but it's easily worked around by using subdirectories (which is a better solution than order).

You ran on an on about how C/C++ is so bad and this interpretive stuff was so cool. You still believe that - but you are wrong. Both are a force fit for an application they were not made for.

That has nothing WHATSOEVER to do with this discussion. But I'm right, and if you want to argue that, we can take it via private messages, by e-mail or chat or some other means.

[wildweasel]

There is no reason why we can't just write a really comprehensive tutorial to manipulate a WAD. In my eyes, this is the END of the discussion.

This is not about tutorials. I already know how to manipulate every single aspect of a WAD file down to the bit level. It's a matter of features that WADs lack.

The only feature I can think that WAD lacks is compression. The way to fix that is to have ZDoom read from a ZIP file that contains the WAD inside of it, and then use the WADs in the ZIP in the game.

[Fredrik]

The only feature I can think that WAD lacks is compression. The way to fix that is to have ZDoom read from a ZIP file that contains the WAD inside of it, and then use the WADs in the ZIP in the game.

Read the thread. These features over WADs (and perhaps some others that I missed) have been pointed out:

• Compression
• Long entry names
• Directory structure
• Type identification of lumps
• Much wider support
• Easier to work with (perhaps not for you but for many others!)

[wildweasel]

* Compression
Easily fixed by my solution!
* Long entry names
I can see that people would like to label their lumps more clearly, but that's not a good single reason to use ZIP over WAD...
* Directory Structure
Yes, THIS would work. OR we could implement some kind of marker system for every type of lump (M_START, SN_START, etc).
* Type identification of lumps
Hmm. No comment.
* Much wider support
Why? I'm fine with Wintex and XWE. If there's really some reason people would rather use WinZIP or PowerArchiver or something, they can go right ahead.
* Easier to work with
This depends entirely on who's using it!

[GooberMan]

Cut the crap. I wasn't quoting and what you are doing is distorting the simplest of concepts. If you don't see quotes, it doesn't take much smarts to realize it's not quoting. Or are we rewriting the dictionary too.

Wow, that's great, I say that I used the wrong word, then you go and say the exact same thing. <sarcasm>OMG I must bow down to your superior intellect.</sarcasm> Seems you're still reading what you want instead of what's there...

LIST the steps. If it's the same amount of steps then you have gained nothing.

I've gained some disk space from using ZIPS.

I am quite willing to think for a different perspective. Mine is quite more flexible and adaptive than zips. Rather than sticking stubbornly to "zips", how about you thinking from a different perspective. There are many other ways to make WADs more flexible. I just listed an obvious one. But there are many others.

And yes, your idea about a directory lump would make the WAD format more flexible. But I'd say it's more work that supporting zips (as my code example demonstrated) - directory lump has long filenames that refer to 8 byte filenames, which means you'd have to come up with a proper naming convention for the filenames on creation of the WAD. Oh, and just so we're clear with your Quake .PAK reference, Quake 3's .PK3 format are (wait for it) ZIPs...

Lots of things are easy to do. Mine suggestion is even easier. And there's very little to new to learn conceptually. All you need to provide is a set of functions that act as a common interface for WADs and extended WADs. See how easy that is to write

But why treat them as extended WADs if all the extended information is is a lump in a normal WAD? A WAD is a WAD, is it not? The standard code implementation of a WAD should cover all possible aspects of a WAD, there's absolutely no need to write a whole new set of functions to handle them differently and provide a common interface.

Both you and Fredrik run on the premise that if you say something it must be true. It's NOT.

Yes, and you claim that ZIPs would be too hard to implement. I don't know if you've looked at the code I wrote, but it sure as hell proves that everything you say isn't true (even though you seem to believe it is).

Remember, it's you that said DeePsea was just like DEU. Probably still want to insist that is true too. I wouldn't put too much faith in your logic

Good to see you haven't changed, makes it easier to prove a point if one can predict your responses every single time
I drew comparisons between the interfaces and came up with quite a lot of similarities which I documented in detail, but you seemed to think that there's not a single similarity except for the fact that it's a level editor. It speaks volumes about your skills as a programmer and a designer if you really couldn't see any truth in the document I wrote. But why use examples from two years ago that have nothing to do with the discussion when I can use examples from this discussion to prove my point?

Rather than making your own logical arguments you'd prefer to make an attempt to devalue both mine and Fredrik's arguments by bring up totally unrelated arguments that you also think you're 100% right about. SCO is using a similar tactic in it's current war with IBM by stating that the GPL violates copyright law and therefore is null and void. Slashdot referred to South Park's Chewbacca Defense to display how stupid that tactic is.

"The reason zips, etc makes NO sense is that all it does is INCREASE the difficulty in editing" - opening a WAD management program like DeePsea and inserting files in to WADs in there is less difficult than mass selecting files, right click, and add to ZIP? I'd have thought your reliance on the mouse as an efficient and quicker input device for menial tasks in DeePsea would have influenced your opinion on this matter, but I guess it isn't the case. Sphagne does have a good idea with making a WAD manager that looks like a standard archiving program, so perhaps if that were around it would be about the same amount of difficulty. And, of course, as I've been saying all along, if ZDoom supports ZIPs then the editors will have to aswell so there's no reason you can't still use DeePsea or XWE or whatever to manage your WADs/ZIPs.

"Force fitting something else (zips) to do this job is about as far away from this "designed to do" concept as it can get." Yet you want to force fit a directory structure into a WAD by treating it as a lump. The WAD format was designed to explicitly segreate the data used by making assumptions about filenames and by using lump markers to signify that a certain type of resource followed. It wasn't designed to have long filenames and a directory structure.

"And you just showed there is no need for a zip since you are already working with a directory structure." Which gets totally thrown away when all the data gets put in to a WAD. With ZIP, however, the structure is preserved.

"You are merely making up your own definition of elegant." The definition of elegant is "Characterized by or exhibiting refined, tasteful beauty of manner, form, or style" according to dictionary.com, and that is the definition myself and Fredrik have been using. Tell me, what's your definition of elegant?

"It's a force fit for the game and for editing." Really? So ZIPs and WADs aren't both containers for data? ZIPs and WADs can't conceptually be interchanged at will? Should I refer to my code example again?

[GooberMan]

I guess I should point out quite a big blaring logic fact in the whole "you're assuming newbies know ZIP when they might not" argument. Currently ZDoom is stored in .CAB format (supported by WinZip), non-beta versions (and even the earlier 1.23 version) were stored in ZIPs. To even run ZDoom, one needs to be familiar with the process of working with ZIP/CABs through an archiving program such as WinZip or the build in support in XP. I think it's a fair assumption to assume newbies are familiar with some ZIP fundamentals, don't you?

[Hirogen2]

>* Long entry names
>I can see that people would like to label their lumps more clearly, but that's not a good

I do, in part. Having multiple lumps called "THINGS" is not really a wise thing. Directory stucture "map01/things" would be fine, but THAT requires long names, at least 16.

>* Type identification of lumps
>Hmm. No comment.
Hmm possibly. ZDoom already does in part: WAVE, OGG/MP3,etc (FMOD)

[Fredrik]

* Compression
Easily fixed by my solution!

Your solution requires the implementation of Zip support without making use of the primary benefits.

* Directory Structure
Yes, THIS would work. OR we could implement some kind of marker system for every type of lump (M_START, SN_START, etc).

The marker system is really just a workaround for not having a directory system. It's silly and it should not be extended. Thankfully, ZDoom is (unlike Boom) heading towards obsoleting it by unifying graphics resources.

* Much wider support
Why? I'm fine with Wintex and XWE. If there's really some reason people would rather use WinZIP or PowerArchiver or something, they can go right ahead.

Both WinTex and XWE have limited feature sets. They do not work on all operating systems.

[Hirogen2]

>>Why? I'm fine with Wintex and XWE. If there's really some reason people would rather use WinZIP or PowerArchiver or something, they can go right ahead.

>Both WinTex and XWE have limited feature sets. They do not work on all operating systems.

Point for ZIP.

[HotWax]

HotWax: There's no need to keep map lumps separate...

Don't make me hit you.

I simply disagree.

We get that already. Your arguments have all been stated. Why can't you just drop it?

Everything stated is NEW and just as much work.

ZIP files have been around for a LONG time, are supported by all OS'es, and have an already well-established base of time-tested applications supporting them. How is that new?

Why not just make a wad browser that can go inside the wads by a click or "Enter", justlike 7zip or NC that lets you enter zip files and threat them as directories and copy files into or from them.

Why not, instead of treating WADs as if they have a directory structure, which they don't, you use an already established format to give them a *real* directory structure?

What if we just design a complete new file format (and an editor on top), which handles all these issues?

ZIP already supports the issues discussed and has already been invented.

There is nothing wrong with the WAD format, only the editors that work with it.

I don't think anyone is saying there's something inherently wrong with the WAD format. As Randy points out, it did exactly what it was designed to do. But then, so did Doom. And yet, jumping, free-looking, built-in Dehacked support, colored lighting, underwater effects, ACS scripting support, slopes, and more have been added since. If Doom did what it was designed to do, then why the need for the new features? Maybe to add more functionality? Or make things easier for people? ZIP support would do both of those things.

If you wanted to expand the PWAD format -AND- still be compatible with old tools, it's pretty simple. What you do is create a directory subset with the PWAD. It will look like a giant LUMP to all the old tools, but to newer tools they could have ALL the things you mentioned above. Not unlike the way you stuck the fonts into one lump.

Why? The ZIP format has a REAL directory structure without having to do any kind of cheap hacking. Maintaining compatibility with old editors isn't an issue, as the WAD support will remain untouched. What you're doing here could theoretically break older tools, AND if you wanted to do more with it later, you'd always have the issue of backwards-compatibility hanging over your head, something that has already caused Randy headaches with Dehacked support. ZIP format support would leave WAD files alone, and yet add a more powerful and easier-to-use format for people who choose to use it.

..not a disastrous mess the will escape the installation capability of most users.

This is one thing that has been niggling in the back of my mind since this discussion started. Since ZIP is a popular format for distribution, people *may* get confused about whether they're supposed to extract the file, or just dump it in their ZDoom folder.

As I see it, there are two solutions:

1) Allow files containing a single WAD-format file to be used in ZDoom without modification. In other words, if Joe uses the WAD format, zips it up for distribution, and then Paul downloads it, Paul can just drop it in the ZDoom folder and use it even though it wasn't intended to be used like that. This has the advantage of saving Paul a little space, too.

2) Use a different extension, similar to the way Quake 3 uses PK3 files. When I download a Q3 map, sometimes they come in ZIPs and sometimes in PK3s. Even though both files are the exact same format (ZIP), I know simply by looking at the extension what I'm supposed to do with it.

I think the first method would be a more elegant solution, and would allow for compressing all of those old WAD files you've got hanging around. You could even ZIP multiple files you typically use together (TNTRECL.WAD and TNTRECL.BEX, for example) and load them as a single file. How's that for convenience, RL?

Anything that compresses has a much higher chance of failure.

Any file's chance of failure is directly related to the competance of the tool writing said file. While there are limited WAD-editting tools and most have at least a few bugs, the sheer number and popularity of ZIP tools ensures that they are better tested and that people have more options to use when creating them.

3. Since I'd normally be working with a directory structure outside the zip, damage to the file would only require me to re-pack.

You just admitted the problem. And you just showed there is no need for a zip since you are already working with a directory structure.

No he didn't. You're embodying the typical stereotypical lawyer who asks the question "How long have you been cheating on your wife?" and then claims the person admitted guilt if they answer it in kind. You said the file could be corrupt, he is saying if that should happen all he has to do is repack it. Don't put words in his mouth.

Your next sentence is even more clueless. He's already working with a directory structure so he doesn't need a ZIP? What kind of stupidity is that? The directory is GONE the moment the files are placed into WAD format.

You can't just up with an idea and refuse to consider the bad aspects or other methods that yield better solutions for the work put out. IOW, I don't mind change, but let's make it something that is an extension that fits the general philosopy and intent of PWADs.

PWADs are a container file, just like PIGs, DATs, REZs, PK3s, MPQs, FARs, WARs, and yes, ZIPs. They all serve the same purpose, to enclose multiple files into a single, larger file. ZIPs also have the added functionality of compression, and unlike some of the others above have a built-in directory structure. Why reinvent these features and expand the WAD format, possibly breaking backwards-compatibility in the process, when everything we need is already in the ZIP structure?

I am quite willing to think for a different perspective.

That's RL for you... so totally clueless about his own shortcomings.

Lots of things are easy to do. Mine suggestion is even easier. And there's very little to new to learn conceptually. All you need to provide is a set of functions that act as a common interface for WADs and extended WADs.

Ease is relative. Not everyone in the world thinks or works like you do. Also, you forgot to mention that in addition to writing a new interface to support both WADs and extended WADs, you'd have to actually *invent* the extended WAD format, while keeping in mind compatibility with outdated WAD editors.

Both you and Fredrik run on the premise that if you say something it must be true.

Coming from you, that's hilarious.

Remember, it's you that said DeePsea was just like DEU.

The line above is the only reason RL is still arguing. I knew there must be some reason he doesn't like Goober. Now I know why.

ZIP support is not needed. There is no reason why we can't just write a really comprehensive tutorial to manipulate a WAD. In my eyes, this is the END of the discussion.

Good, then go away.

So? It's a force fit for the game and for editing. I'd much rather see an extended sub-directory. Think outside of the box, not just what can be forced into an existing box.

He IS thinking outside the box! He's talking about moving away from the outdated WAD format using an already-existing format that already handles what it is we need. You, on the other hand, are talking about trying to redecorate the old box while keeping its sides firmly in place so that other programs still recognize it.

The only feature I can think that WAD lacks is compression. The way to fix that is to have ZDoom read from a ZIP file that contains the WAD inside of it, and then use the WADs in the ZIP in the game.

I thought the discussion was over.

* Compression
Easily fixed by my solution!

So you want to support ZIP format, but not actually do anything with it? Good idea.

* Long entry names
I can see that people would like to label their lumps more clearly, but that's not a good single reason to use ZIP over WAD...

Yeah, but it is 1 reason. The 2nd is compression. Keep going.

* Directory Structure
Yes, THIS would work. OR we could implement some kind of marker system for every type of lump (M_START, SN_START, etc).

Or we could leave the WAD format alone so we don't break every editor on the planet, and use an already-existing format (ZIP) that already has everything we need. That's the 3rd reason.

* Type identification of lumps
Hmm. No comment.

Good call. 4th reason.

* Much wider support
Why? I'm fine with Wintex and XWE. If there's really some reason people would rather use WinZIP or PowerArchiver or something, they can go right ahead.

Wintex and XWE have no support outside Windows. 5th reason.

* Easier to work with
This depends entirely on who's using it!

You didn't quote his entire line, because if you had, you'd look like a jackass responding with that. Now you're thinking like RL. 6th reason.

Well, that about wraps up your post, WW. Good points all, I'm sure.

Seems you're still reading what you want instead of what's there...

Story of his life.

Quake 3's .PK3 format are (wait for it) ZIPs...

I wonder why id chose to go with an already-established format rather than using the one that "works as intended." Clearly they haven't been listening to RL.

It speaks volumes about your skills as a programmer and a designer if you really couldn't see any truth in the document I wrote.

His Peril-Sensitive Sunglasses won't let him.

But why use examples from two years ago that have nothing to do with the discussion when I can use examples from this discussion to prove my point?

Because he *can't* use examples from this discussion that proves his point. He's at a disadvantage, and as is typical for him will latch onto anything, related or not, that he thinks he can win at.

I do, in part. Having multiple lumps called "THINGS" is not really a wise thing. Directory stucture "map01/things" would be fine, but THAT requires long names, at least 16.

As I have stated many, many times, you could just slap the container WAD in the ZIP as a whole rather than having each individual lump as its own file in the ZIP. This would eliminate the duplicate-name problem.

Hmm possibly. ZDoom already does in part: WAVE, OGG/MP3,etc (FMOD)

Wrong. When a song is played, either by MAPINFO, from the console, or from an ACS script, the lumpname is used. ZDoom attempts to open it and determine what kind of lump it is. If it can't, or it's not a valid type, you get an error. Otherwise it plays. Prior to that, ZDoom has no idea what the lump does because it doesn't check. Other types, such as flats, textures, etc, have to be loaded in at startup so ZDoom doesn't have the luxury of waiting until their called on and checking the type at that time. To alleviate the situation, fake directories using _START and _END tags are placed in the WAD files. You wouldn't even have to use directories if a simple extension was allowed that told ZDoom at a glance what type of file it was dealing with.

And the award for most quotes in a single post goes to HotWax...

[Graf Zahl]

As I have stated many, many times, you could just slap the container WAD in the ZIP as a whole rather than having each individual lump as its own file in the ZIP. This would eliminate the duplicate-name problem.

Unfortunately that won't work well because it is not that easy to access the lumps in the compressed WAD. It's certainly possible but there might be some unacceptable performance loss.

The only way I can see a ZIP file work is to use a directory structure similar to the internal structure of a WAD but with a few enhancements to separate the different file types. A file of a certain type would have to be in its specific directory. The whole thing still has to be compatible with the current WAD management so many of the really cool things (like long file names or complex directory trees) cannot be done anyway. There are too many places (especially texture names) which will be forever limited to 8 characters.

[Fredrik]

Unfortunately that won't work well because it is not that easy to access the lumps in the compressed WAD. It's certainly possible but there might be some unacceptable performance loss.

It'll work for small WADs though if you load the entire WAD - so no problem keeping a level in its own WAD inside the Zip instead of extracting the lumps.

There are too many places (especially texture names) which will be forever limited to 8 characters.

Sprites, sounds, music, miscellaneous graphics and maps can have long names though.