A PSA on Filesizes.

Handy guides on how to do things, written by users for users.

Moderators: GZDoom Developers, Raze Developers

Forum rules
Please don't start threads here asking for help. This forum is not for requesting guides, only for posting them. If you need help, the Editing forum is for you.
User avatar
Jaxxoon R
Posts: 772
Joined: Sun May 04, 2014 7:22 pm

A PSA on Filesizes.

Post by Jaxxoon R »

Hello Zdoom, It's recently occurred to me how gigantic the size of mods have become. Not just in ambitiousness, content, or variety, but also in filesize. Nobody likes to wait ten minutes for Doomseeker to grind through a download for a version of a mod that'll only be made obsolete in a couple weeks.

The worst part is that most of the time, this is easily avoidable with just a little extra effort. Here and now, I'm gonna show you how to correct this.

(Remember that even if you don't care about us poor American scrubs with our slow, monopolized internet connections, a lower filesize will definitely make your mod more appealing for people to download if it isn't shit.)

IMAGES

Now, say we've just got this sweet texture we want to use for whatever reason. In this case, it's two planet-sized robots ready to duke it out to the death in the clouds:
Image
Now this'll look great as a part of my new 27-layer skybox in my latest UDMF masterpiece! Except...
Image
1.11 Megabytes. Even for an image of this size that's way too much. This will hurt not only download speeds, but also performance. As it will take up a lot more than 1 MB of ram while in action, this will only get worse and worse as you add in the other 57 skyboxes, Curly Brace, DecorACS monster abominations, and 32 hours of WAV music you've got planned.

But this can be rectified by getting to know a few gems of a program your lord and masters for the moment:
Image
Pngquant, a badass PNG compression tool; Paint.NET, a GIMP/Photoshop clone with an interface that's actually usable by humans; and Audacity, a radical sound editor. Praise them. Or they'll f*ck you up.

Now it's very simple to just try and lower the resolution of the image. That's a great idea, but say you don't want to do that for whatever reason. If it's not anything with transparency you'll probably want to use JPEGs:
Image
The lovely thing about JPEGs is that they allow for great compression at a negligible loss of quality for things like photographs, desktop backgrounds, and stupidly high-res textures. It's better to tweak the quality just right for your image in particular. And remember: they also tend to work best with large images, as that's when the artifacting is the least noticeable.

But say you're wanting to compress a sprite, or any other image that requires transparency? Say that for some reason you want the maximum quality versus size in a palleted image? Well, pngquant may just be your savior:
Image
A curious beast, pngquant will devour images and spit out heavily optimized versions. One cool thing about pngquant, though, is that it supports alpha channels. Just drag and drop into either the 256 color or automatic color batch (I use automatic because I'm lazy) and...
Image
Huh... Well... It's done something weird here. Looks like it made two pictures, but what's the difference? Well, the fs8 version weighs in at 628 KB and is dithered extensively, while the or8 version weighs in at a whole whopping 330 Kb and is slightly posturized. But which one to chose? Well, at a glance they all kind of look very similar:
Image
(Which one is which? lol)
So I guess just choose the smallest one. This gets even more indifferent at lower resolutions, there you'll just have to pick the best-looking one, and I guess it's the same here, too. Seriously though, it's probably best if you chose sane texture sizes instead, like consistently things that are Doom-sized, just use Paint.NET's resize tool for that.

AUDIO

But what about your hours of audio? Your band's excellent new album must not go without excellent distribution, how do you make sure all the sad little Americans and Australians with their horrible monopolized internet get your mod that you worked so hard on? I mean, take a look at this!
Image
Good lord, how will we ever fix this mess? Oh wait, I guess this is where we use Audacity.

The nice thing about Audacity is that it allows you to do general sound editing, so with it you can make sound effects that are unique to your mod, or you can lower or increase the volume of a song as needed. But in this case, all we need to do is compress our song.
Image
Here we can do things like set the project rate to 44100 HZ (only X Men will notice a quality increase above this) and export it.
Image
Don't save it is MP3 because that is what filthy neanderthals do. Save it as OGG instead.
Image
It's best to export at a quality level of 5 or lower (0 or 1 for SFX) because it allows for a far higher quality than MP3 at a significantly lower filesize. Another reason to use OGG is that it also decompresses much faster ingame. And in this instance our final filesize is is only 3.1 MB even at the highest sane quality level. Technically we could reach sizes of 700 KB just by sacrificing (quite) a bit of audio fidelity.
Image
Well, a silly man would tell you to do all this manually, but once again good ol' Audacity has a few neat tricks that'll make your day much easier.

There's this thing called chains and a tutorial for it on the Audacity Wiki. But that is complicated and I am a simpleton. So I just load multiple files into the window, do whatever to them, and hit "export multiple". This gives me better control of the project rate and what effects I want to apply and does about the same thing in the end. Here's a picture of Audacity exporting:
Image
Hey look.

But say you don't have your own band to make songs you can use, say you have no musical talent whatsoever and don't want to use royalty-free music, or music from other artists that could run into copyright problems?

There's the 'technically not legal but still tolerated by the corporations and the /idgames archive' art of taking music from other video games. Well, if you're using anything from a game that uses any form of synthesis (whether it be FM, chiptune, or sample sequencing) it's definitely better to use rips of the songs; there's a reason most early games were as tiny as they are so make the most of it.

These rips can be found in many places, but the big thing is that they are emulated by Zdoom and so can be handled like any other music file. This is most convenient.

TEXTURES

So now that you are compressing, I bet that you've come to a confusing thing, say you want to cheaply combine two textures, just layer one on top of the other. So you've already got both textures in your mod, what now, do you just combine them with Paint.NET?
Image
No! Zdoom and all it's babby ports (Yes, even Zandronum surprisingly!) have something that does generally the same thing, it's called TEXTURES.

Now a TEXTURES lump is a bit of a bitch to make, just make a new entry and name it "TEXTURES.TXT". Now select the entry, click on the "View as Text", then open the "Text Language" dropdown menu and select ZDoom Textures. Now just put two forward slashes (//) without the commas, and then save the entry. If you did it right, it should look like this:
Image
If it doesn't look like this, you did it wrong so go back and do it right.

Open up the thing. Now I could go into extreme detail about how to do a TEXTURES, but there's probably 37,000 different tutorials out there, and It really is quite simple. Just make a new texture, and add the patches from your mod file like this:
Image
You might as well call me Vincent Van Gogh because I'm craaaaaazy talented.

CONCLUSION

Well, I hope this guide serves to help someone somewhere, and that I didn't accidentally do something illegal in the process of making it. Remember everyone: I only had the best intentions! If you have any suggestions of things to add or touch up, just scream at me below. Although, I guess I could leave you all with a tip:

We are living in the year 2015, so please, please, please, do yourself a favor and use PK3 files built with 'store' or 0% compression. It's much neater, lowers loading times, and is probably a better choice for large projects. Just look at Ao Oni Doom, under the hood that thing's a goddamn mess of a WAD file and is an example of exactly what you shouldn't do (Did I mention that it's also 96 MB?).

And here are some important links to check out for all the things I found:

http://www.getpaint.net/
http://pngquant.org/
http://audacity.sourceforge.net/
http://manual.audacityteam.org/o/man/ch ... ation.html

http://project2612.org/
http://snesmusic.org/v2/
http://www.zophar.net/music.html
Last edited by Jaxxoon R on Sun Jan 11, 2015 7:38 pm, edited 9 times in total.
User avatar
Enjay
 
 
Posts: 26534
Joined: Tue Jul 15, 2003 4:58 pm
Location: Scotland
Contact:

Re: A PSA on Filesizes.

Post by Enjay »

Just getting people to zip up their WADs before sticking them on dropbox is a major battle these days, never mind compressing individual lumps within a project. :?
User avatar
Ed the Bat
Posts: 3060
Joined: Thu May 03, 2012 1:18 pm
Graphics Processor: nVidia with Vulkan support
Location: Maryland, US
Contact:

Re: A PSA on Filesizes.

Post by Ed the Bat »

Audacity can batch-convert wavs to oggs with just a few clicks. I'm a huge fan. I do it all the time. As for .pk3 format... it's got its advantages, no question, but I don't get the point of double-zipping projects (such as putting a .pk3 into a .7z and putting that up for download); it's like shipping a box inside of another box so I just have more things to open when I receive it. At least putting a .wad into a .zip or something for distribution (as in how Doomworld /idgames does) means I can just execute it immediately upon download without having to unpack it.
User avatar
Enjay
 
 
Posts: 26534
Joined: Tue Jul 15, 2003 4:58 pm
Location: Scotland
Contact:

Re: A PSA on Filesizes.

Post by Enjay »

Ed the Bat wrote:but I don't get the point of double-zipping projects (such as putting a .pk3 into a .7z and putting that up for download)
I think it's usually done because it allows you to have myfile.pk3 and myfile.txt inside the zip (or 7z). This, in turn, means it complies with /idgames requirements.

Unfortunately, it does indeed mean that you can't just load the distribution package and have it work, like you can with a WAD in a ZIP. The PK3 has to be extracted from the ZIP first. For that reason, I too find that method of distribution a little annoying.
Nevander
Posts: 2254
Joined: Mon Jan 06, 2014 11:32 pm

Re: A PSA on Filesizes.

Post by Nevander »

I've used the OGG format for my own music packs I've made for personal use and love the file size it has. Even at 0 quality for music, it still sounds good in-game when there are other sounds going on. I couldn't tell any difference between a 320kbps MP3. I don't do much texture work, so I wouldn't need the PNG thing, but it looks great to keep file sizes down. IMO, Doom and its mods is all about the small file sizes.
User avatar
leileilol
Posts: 4449
Joined: Sun May 30, 2004 10:16 am
Preferred Pronouns: She/Her
Location: GNU/Hell

Re: A PSA on Filesizes.

Post by leileilol »

I still don't understand the whole 'download audacity to save a mp3 or ogg' when one could just download the small lameenc+lamedrop or oggenc+oggdrop programs to do that with a lot more ease where batch conversions are done with selecting and then drag-and-dropping.


Also consider the memory use of the textures - that 1mb png's not going to be 1mb when loaded.



also a little random compression tip - build your PK3 with 'store' or 0% compression and then just ZIP your final archive to take advantage of a solid archive (for distribution) without having Zdoom deal with a solid archive (and long load times). I might start building OA3 this way (which, by the way as of now, thanks to careful asset control and skeletal model formats, is 23 fricking megabytes).

just because it's 2015 with cheap multi-terabyte harddrives doesn't mean you are free to pig out and be incompetent with asset control. This is something I try to argue against other FOSS gamedevs too, but alas, the 'submit a patch' fallacy... even when I was transcribing game music to the "outdated" MOD format with better-than-overcompensating-it-files accuracy I rarely ever exceeded 100KB in size for each song. Compressing all of them actually makes a well nicely sized 1.4mb zip which is almost as small as that 1920x1080 1mb png image and as big as your typical 1 minute MP3 song, and smaller than every unresized, bilinear filtered PNG GZDoom screenshot posted here....INDIVIDUALLY.
User avatar
Jaxxoon R
Posts: 772
Joined: Sun May 04, 2014 7:22 pm

Re: A PSA on Filesizes.

Post by Jaxxoon R »

Uh... I know, right? XD

Anyways yeah, I added a few new sections on video game music and TEXTURES lumps, and even added some stuff at your guys' suggestion.
User avatar
Ed the Bat
Posts: 3060
Joined: Thu May 03, 2012 1:18 pm
Graphics Processor: nVidia with Vulkan support
Location: Maryland, US
Contact:

Re: A PSA on Filesizes.

Post by Ed the Bat »

Batch conversion with Audacity is even easier without chains. Just drag-and-drop all the files in, select Export Multiple, and bam.
Don't mind me, just being illiterate again. Never skip your medication, kids!
Last edited by Ed the Bat on Sun Jan 11, 2015 1:01 am, edited 1 time in total.
User avatar
Jaxxoon R
Posts: 772
Joined: Sun May 04, 2014 7:22 pm

Re: A PSA on Filesizes.

Post by Jaxxoon R »

"However, that is complicated and I am a simpleton. So I just load multiple files into the window, do whatever to them, and hit "export multiple"."

Look who is the bat now, bat man!
User avatar
Ed the Bat
Posts: 3060
Joined: Thu May 03, 2012 1:18 pm
Graphics Processor: nVidia with Vulkan support
Location: Maryland, US
Contact:

Re: A PSA on Filesizes.

Post by Ed the Bat »

I am not a smart man. I think I may have been once, but my memory has gone since then, as well. My apologies; as you were. :P

I get a bit confused when reading documents that have lots of pictures in them. I believe the technical term for my condition is "stupidity".
User avatar
Kinsie
Posts: 7401
Joined: Fri Oct 22, 2004 9:22 am
Graphics Processor: nVidia with Vulkan support
Location: MAP33
Contact:

Re: A PSA on Filesizes.

Post by Kinsie »

A PSA on writing tutorials: If you aren't funny, don't try to be, or the result will be insufferable and painful to read.
User avatar
Jaxxoon R
Posts: 772
Joined: Sun May 04, 2014 7:22 pm

Re: A PSA on Filesizes.

Post by Jaxxoon R »

Now you know how my hard drive feels.

But you know to be perfectly honest I had stayed up about 24 hours before writing this and it was basically my thought process at the time. After an actual night of sleep the only thing amusing about this to me is how stupid it is.

But you know what, since you feel so strongly about this I'll rewrite this whole thing right here.
Nevander
Posts: 2254
Joined: Mon Jan 06, 2014 11:32 pm

Re: A PSA on Filesizes.

Post by Nevander »

What's in the HOLY SHIT folder? :)
User avatar
Jaxxoon R
Posts: 772
Joined: Sun May 04, 2014 7:22 pm

Re: A PSA on Filesizes.

Post by Jaxxoon R »

Image
Snoic.
User avatar
Enjay
 
 
Posts: 26534
Joined: Tue Jul 15, 2003 4:58 pm
Location: Scotland
Contact:

Re: A PSA on Filesizes.

Post by Enjay »

And there was me thinking that the folder contained an ancient relic handed down through the centuries and eventually ending up on the HD of a ZDoom player.

The One True Holy Shit of Our Lord originated in the Middle-East, confirmed by micro analysis of pollen grains found on the Shit, and it has been carbon dated to around 25 A.D. It spent over a thousand years at various Holy sites along the Jordan river but it was captured during the second crusade by Flemish foot soldiers and transported from the Holy Land to Europe under armed guard. It spent some time being toured around Europe, spending over 100 years in Paris alone, and was put on display in most of the major European countries. The Holy Shit was ordered to be shipped to a permanent home in Rome by Pope Eugene IV in 1443 but it seems to have taken a very circuitous route and it is unclear if the Holy Shit ever actually made it to Rome. There are records from both before and after this date detailing the Holy Shit's stay in various abbeys, cathedrals, monasteries, churches and some tales of it spending time in less holy locations such as castles, palaces, administrative centres and even one, possibly apocryphal, story of it spending over a decade under a bed in a brothel in Budapest. The exact path and time-scale taken, or even if the Holy Shit ever actually reached Rome is unclear because there are a number of gaps in the historical record, notably and most recently during WWII and the years immediately following the conflict. There are unsubstantiated claims that it was captured by the Nazis but also that it was protected from capture by a baker based in Milan who is said to have encased it in bread and hidden it in full view by displaying the loaf in his shop.

Since the war there have been statements and counter statements from various sources who claim to have the One True Holy Shit of Our Lord in their possession. Most of these claims can be dismissed easily because even cursory investigation shows the items to be medieval fakes produced by monks at the time when trade in holy relics was at its peak. However, scientists, historians and the official Papal record all agree that the One True Holy Shit of Our Lord now resides on the hard drive of Jaxxoon R.
Post Reply

Return to “Tutorials”