DOOM-Crusher : Crush your PK3/WAD/PNG/JPG files!

Software projects like source ports (3DGE, Eternity, etc), launchers like ZDL, and other useful utilities belong in this forum.
Forum rules
The Projects forums are ONLY for YOUR PROJECTS! If you are asking questions about a project, either find that project's thread, or start a thread in the General section instead.

Got a cool project idea but nothing else? Put it in the project ideas thread instead!

Projects for any Doom-based engine (especially 3DGE) are perfectly acceptable here too.

Please read the full rules for more details.

Re: DOOM-Crusher : Crush your PK3/WAD/PNG/JPG files!

Postby Kroc » Fri Jan 13, 2017 1:50 pm

New version released: v1.1

  • Skip files that have already been crushed before (a cache of file hashes is kept in `bin\hashes.txt`) -- incrementally crush your files without having to waste time on files that had been crushed before!
  • WADPTR's sidedef packing is disabled to prevent glitches in maps
  • Added `/NOPK3` and `/NOWAD` options to ignore these types
  • Added `/ZSTORE` option to repack the PK3s without compression: Whilst the PK3 file will be larger than before, it will boot faster. If you are compressing a number of PK3s together, then using /ZSTORE on them might drastically improve the final size of .7Z and .RAR archives when using a very large dictionary size (256 MB or more)
  • Fixes for file size reduction reporting
The added file caching to skip previously crushed files should make this release much more practical for use in a development project. Whilst the gains might be minimal in some cases, you won't have to think about what files you've crushed or not since last time.

Instructions: https://github.com/Kroc/DOOM-Crusher
Download: https://github.com/Kroc/DOOM-Crusher/releases

I'm working towards adding logging so that it's safer to throw entire directories full of PK3/WADs at DOOM-Crusher and you can be assured that nothing breaks afterwards. :)
Kroc
 
Joined: 02 Oct 2016

Re: DOOM-Crusher : Crush your PK3/WAD/PNG/JPG files!

Postby Gez » Fri Jan 13, 2017 6:28 pm

Enjay wrote:One thing does concern me from the docs though. I'm not sure how the map sidedef packing is done but if it is the kind of sidedef packing where linedefs which share identical siddefs are given the same sidedef references I have experienced problems with that in the past (although, being perfectly honest, I forget what the problems were).

A side effect of sidedef packing IIRC is that if you have scrolling effect on a line, it will affect the other lines that use the same sidedef (and that's not necessarily desired). And still IIRC, it can even be exploited deliberately by using several lines scrolling the same to get a faster scrolling effect, because each line will cumulatively apply.
Gez
 
 
 
Joined: 06 Jul 2007

Re: DOOM-Crusher : Crush your PK3/WAD/PNG/JPG files!

Postby Kroc » Fri Jan 13, 2017 8:02 pm

I disabled sidedef packing in this latest release due to the potential of it causing glitches in maps; only image and deflate optimisations are done.
Kroc
 
Joined: 02 Oct 2016

Re: DOOM-Crusher : Crush your PK3/WAD/PNG/JPG files!

Postby Kroc » Sat Jan 21, 2017 4:48 pm

New version released: v1.2

First, my apologies for v1.1 being completely broken; I hadn't touched the code for a while and forgot how terribly fragile batch scripts can be.

  • Fixed options not applying!
  • Many major fixes and improvements to caching:
    • If skipping PNG/JPG files, allow a PK3/WAD to be added to the cache if does not contain any such files
    • Separate hashes into different buckets for file-type, this resolves the issue `/ZSTORE` not repacking PK3s that had been maximally compressed and cached
    • A "cache" folder is now used. Your existing "hashes.txt" will be invalid due to critical bugs in v1.1, sorry :(
  • Fixes and improvements to filesize and percentage reporting
  • Added basic logging (a replica of what appears on screen), see "bin\log.txt", more advanced logging is planned which will capture full error details
  • Signigicantly faster WAD processing (lumpmod modified to identify PNG/JPG lumps, with thanks to _mental_)
  • Faster PK3 processing

Instructions: https://github.com/Kroc/DOOM-Crusher
Download: https://github.com/Kroc/DOOM-Crusher/releases
Kroc
 
Joined: 02 Oct 2016

Re: DOOM-Crusher : Crush your PK3/WAD/PNG/JPG files!

Postby Graf Zahl » Sat Jan 21, 2017 6:14 pm

Gez wrote:A side effect of sidedef packing IIRC is that if you have scrolling effect on a line, it will affect the other lines that use the same sidedef (and that's not necessarily desired). And still IIRC, it can even be exploited deliberately by using several lines scrolling the same to get a faster scrolling effect, because each line will cumulatively apply.



In ZDoom sidedef packing is irrelevant because the engine requires unique sidedefs in the game and unpacks them again.
For other engines a good sidedef compressor ignores sidedefs with attached specials.
User avatar
Graf Zahl
Lead GZDoom Developer
 
Joined: 19 Jul 2003
Location: Germany

Re: DOOM-Crusher : Crush your PK3/WAD/PNG/JPG files!

Postby Kroc » Fri Jan 27, 2017 5:26 pm

More fixes.

DOOM-Crusher v1.2.1:

Known issue: Above approximately 30 MB the percentage calculations are completely wrong; this is a limit of Windows' Batch processor and I'll fix it in the next release or so -- it has no effect on the actual functionality of DOOM-Crusher.

  • Fixed hash check failing
  • If JPG/PNG/WAD optimisation encounters an error, a parent WAD/PK3 will not be added to the cache. This is so that it will be retried in the future until there are no more errors
  • Command line options given displayed in the header
Instructions: https://github.com/Kroc/DOOM-Crusher
Download: https://github.com/Kroc/DOOM-Crusher/releases
Kroc
 
Joined: 02 Oct 2016

Previous

Return to Software and Ports

Who is online

Users browsing this forum: No registered users and 1 guest