Relighting v4.0165b - blurry shadows w/ rlassets

[Dan_The_Noob]

any luck seeing if there's a way to angle the sprites before blurring so the feet look more sharp than the head?

[Hey Doomer_]

any luck seeing if there's a way to angle the sprites before blurring so the feet look more sharp than the head?

I haven't looked at that possibility in depth, but it's a cool idea. I'm adding code to read from zip (pk3) and/or WAD files; that's easy going. The idea is to pass the source(s) on Python's command line to create assets, since I can't possibly create them for everything. There is sprite name overlap between games and mods, for one thing. My guess is the limit for indexed sprites is 32768 (this returns as an integer), so indexed assets can be quite large.

v4.0165b will have a minimal hd_crosswalk.zs file that always returns nothing, since there will be no assets. A user will have to download Python, Node, the script, and install dependencies (a few steps that need to be done only once). The script is run, creates the sprite assets and new hd_crosswalk.zs file, and this is all copied into Relighting with the new crosswalk copied over the old. This means the mod can be customized however a user likes. It's a pretty cool effect but very likely not for everyone. Some don't even like shadows. Also I prefer not to release existing assets, altered or not (non-shareware assets are problematic at least), as part of Relighting.

It's a pain for a non-programmer to install this stuff, but it's straightforward (with instructions) and no programming is required. It's just a lot of command line stuff. Once everything is installed creating new assets is easy and takes a few minutes. It's possible to add as many assets as one likes and name folders differently in Relighting; the #include crosswalk file controls everything. (I don't believe conditional includes are possible in ZScript.)

I have looked at the pillow library, and there's quite a few options for image manipulation. I'll check out how images can be blurred, although Gaussian blur seems to affect the entire image.

[Hey Doomer_]

Progress

The basic code in the script is done with zipfile doing the heavy pk3 lifting for Python. Testing with Brutal Doom.

I've no idea why, but many of the PNG files are not valid images according to PIL. I also can't read any lumps indicating offsets and so have to use the "monster" parameter for pngOffsets. I'll have to investigate this further although I have an idea of what the problem is. Brutal Doom is... interesting.

ZScript seems to have a line number limit. I tried combining the crosswalk and indexing scripts into one and the engine threw an error. It didn't recognize the file as a script at all (neither did Slade, which is interesting.) I have made some changes and separated these files again. Another reason to keep these assets small and run when needed, but I don't know what the practical limits of a script file is.

Ultimately Brutal Doom fails because many sprites in the archive are missing rotations. I don't check for that and don't plan to, but I suppose a BD fan could delete all the single sprites and references in the crosswalk files. Still testing.

Finally it is possible to blur part of an image, although I'll have to see how that looks; the effect may be jarring.

Release "soon."

[Hey Doomer_]

More Progress

Boring progress report

Might be of interest to programmers... although maybe all of this is already common knowledge.


From looking at Slade and pngOffsets.js GZDoom extracts x,y offsets from a custom "grAb" chunk of a PNG. This can be extracted from Brutal Doom (and other zipped) sprites with the following Node javascript (the only dependency to install is png-chunks-extract):

Code: Select all

const extract = require('png-chunks-extract');
const fs = require('fs')
const process = require('process')

try {
    const buffer = fs.readFileSync(process.argv[2]);
    const chunks = extract(buffer);
    const x_offset = chunks.find(chunk => chunk.name == 'grAb').data[3];
    const y_offset = chunks.find(chunk => chunk.name == 'grAb').data[7];
    
    console.log(`x_offset ${x_offset} y_offset ${y_offset}`);
}
catch (error) {
    console.log(error);
}

Chunks are returned as a key:value pair array, with the value being a byte array.

Here's the Python snippet that saves an extracted PNG file to a temp folder and passes the filepath to the above:

Code: Select all

                zip.extract(s, 'temp')
                p = subprocess.Popen(['node.exe', 'c:\save\doom\python\get-offset.js', '{0}\{1}'.format('temp',s)], stdout=subprocess.PIPE)
                print(p.stdout.read())

Piped stdout from Python looks like this:

Code: Select all

b'x_offset 110 y_offset 11\n'

Once these are read as PIL images and the images are transformed in some way these offsets must be added back in, a reverse process of creating a grAb chunk with offsets and encoding it.

Currently still POC but much closer to release. It's certainly possible to create blurry shadows for everything from the looks of it. Still haven't looked at how blurry, but there are many possibilities.

[Hey Doomer_]

Progress

Works great with Brutal Doom, although missing frames reported by GZDoom have to be commented out in the BlurredAssets class.

Here are BD blurry shadow examples:

E1M3
MAP01

The Python script removes assets from a zip (pk3) file and saves these in a temp folder. This reliably separates the basename from the zip path. This isn't necessary with wad files, as those assets can be loaded into memory. Then the sprite base name and offsets are stored. For Doom 2 this is around 3950 files, although it includes HUD sprites. For BD this is around 4980 files, and this does not include HUD sprites for testing. Many BD sprites have corrupted PNG headers or missing frames; I assume those with missing frames aren't used anyway. Once unique swap base names for sprites are found, the images are blurred and saved to a sprites folder. This takes around 5 minutes or so.

Interestingly, a single class works to index sprites:

Code: Select all

class Blurred_Assets: Actor{
   States
   {
   Spawn:
       Y1CB A 1;
       E2CB A 1;
       B2DG A 1;
       L3DG A 1;
       P3EI A 1;
       R3EZ A 1;
       R4EZ A 1;
       UHF1 A 1;
       GHF2 A 1;
       X762 A 1;
       PCNB A 1;
       ...

This now works with any combination of wads and pk3 files from any game passed on the command line to the Python script so long as engine limits aren't exceeded.

I'll check out image options next. Release "soon."

[Dan_The_Noob]

so wait, is this an external process? like do you run it to generate an addon?

[Hey Doomer_]

so wait, is this an external process? like do you run it to generate an addon?

Yes. ZScript can't generate assets.

Update:

Now that POC is done I'm working on replacing Node dependencies with pypng (Python library) to handle PNG chunks. That means one will only need Python, omgifol, and pypng. It should also generate assets much faster.

Code: Select all

            img = png.Reader(filename='{0}\{1}'.format('temp',s))
            while True:
                try:
                    chunk = img.chunk()
                    if str(chunk[0], 'utf-8') == 'grAb':
                        print(chunk[1][3], chunk[1][7])
                        break
                except Exception as e:
                    print(e)
                    break

From BD images:

Code: Select all

27 68
27 68
ChunkError: Checksum error in grAb chunk: 0x529F621E != 0xB501D011.
ChunkError: Checksum error in grAb chunk: 0x97B88E98 != 0x70263C97.
ChunkError: Checksum error in grAb chunk: 0xC13C944B != 0x26A22644.
19 49
ChunkError: Checksum error in grAb chunk: 0x4BE1E1C0 != 0xAC7F53CF.
ChunkError: Checksum error in grAb chunk: 0x153F18CE != 0xF2A1AAC1.
ChunkError: Checksum error in grAb chunk: 0x153F18CE != 0xF2A1AAC1.
ChunkError: Checksum error in grAb chunk: 0xA71FC4DE != 0x408176D1.
ChunkError: Checksum error in grAb chunk: 0xEC4132A9 != 0x0BDF80A6.
20 59
20 41
21 45
23 39
24 11

My guess is many of the BD images were manipulated by buggy software that didn't take into account checksums, but I've got no idea.

[Hey Doomer_]

Progress Report

The pypng library is much faster than subprocessing to Node. Doom2 sprite assets are processed in 5.8 seconds.

Extraction of x_offset, y_offset (the IHDR checksum appears to be corrupted in some of the BD images)

Code: Select all

        sucase = s.upper()
        if not re.match('[A-Z0-9]{6}', sucase[:6]):
            print('{0} not a valid frame, ignored'.format(sucase))
            continue
        if fname is None:
            img = png.Reader(filename='{0}\{1}'.format('temp',s))
            while True:
                try:
                    chunk = img.chunk(lenient=True)
                    if str(chunk[0], 'utf-8') == 'grAb':
                        offsets.append({
                            "sprite" : sucase,
                            "x" : chunk[1][3],
                            "y" : chunk[1][7]
                        })
                        break
                except Exception as e:
                    print(sucase, str(chunk[0], 'utf-8'), e)
                    break
        else:
            offsets.append({
                "sprite" : sucase,
                "x" : wad.sprites[s].x_offset,
                "y" : wad.sprites[s].y_offset,
            })

If fname (the was file) is None, then assets were extracted already from a zipfile and saved to .\temp. The argument lenient=True raises warnings and not exceptions just to be safe. So far I've recovered offsets from all sprites, which is to be expected if there is a grAb chunk.

Insertion since the grAb chunk is a custom chunk that is erased when PIL opens the image

Code: Select all

            img_blurred = img.filter(ImageFilter.GaussianBlur(radius=1))
            f = 'sprites\{0}'.format(pngname.replace(pngname[:4], sswap[sbase.index(pngname[:4])]))
            print(s, f)
            img_blurred.save(f)

            # insert grAb chunk
            grAb = [b'grAb', struct.pack(">II", abs(x_offset), abs(y_offset))]
            reader = png.Reader(f)
            chunks = [*reader.chunks()]
            chunks.insert(1, grAb)
            with open(f, 'wb') as file:
                png.write_chunks(file, chunks)

There is a request to update PIL to accommodate this, but the code is trivial. It took some time to work through this, since I saw a number of approaches that don't work. Mine is simple enough and may be useful to someone. I'm sure I'll find more bugs in the Python script. Still testing.

The good news is this is far faster and simpler to install and run. The next Relighting version will include a folder with this source code (a tiny file) and instructions. The user will have to copy this to a folder, install Python, and install two Python modules. The script is run from the command line like so:

Code: Select all

python rlassets.py {wadfile/pk3}

After that the sprites folder is copied into Relighting, and the ZScript files crosswalk.zs and initspites.zs are copied to the root. It is an extra step or two, and it all depends on how one feels about blurry shadows in the game. As a plus it works with Voxel Doom, since those voxels are linked to sprite indices. Might be worth six seconds' worth of trouble.

[Hey Doomer_]

Progress - Close to Release

Testing is going well. I have not added checks for missing rotations, so that will need to be adjusted. Heretic, for example, has a sprite with missing rotations. At present these sprites need to commented out in initsprites.zs.

Here is an example of Heretic blurred shadows.

It is also feasible to code Python to update Relighting, fully automating this modification. I'm not sure I'll do that. I have added a command line parameter to change the SD of the Gaussian blur (float default is 1); lower values make this less blurry. Overall this is fast; Doom and Heretic assets take around 6 seconds to create, and Brutal Doom around 30. Note that wads and pk3s can be combined in the same run.

Other than these asset modifications this is a maintenance release. I was going to add a modification to make dynamic lighting on light sources optional, but this is already the case (set size to 0). I'll look at other tweaks.

Release in the next few days.

[Hey Doomer_]

Posted v4.0165b

Code: Select all

v4.0165b

8/19 - fixed DIV/0 error
8/20 - added distance check to lights/shadows beyond FOV
8/21 - added length check to texture lights
8/22 - fixed bug in flashlight scale
8/22 - added rlassets to create blurry shadows

Relighting-wise this is a maintenance release without new features. I have fixed a few bugs here and there, including one that affected how flashlights interact with shadow casters. The big addition is rlassets, which allows a user to create blurred shadows of varying degrees to add to this mod. This means mods including 3d models (including Voxel Doom) can have sprite shadows.

Interesting problem and my solution seems to test OK. Many thanks for your indulgence on this and let me know what you think!

[BROS_ETT_311]

Not sure if you check your other threads, but I posted about an issue with the Window mod. Sorry to bring it up here, just wasn't sure if you still had access to those topics since it looks to be on a slightly different account.

[Hey Doomer_]

Not sure if you check your other threads, but I posted about an issue with the Window mod. Sorry to bring it up here, just wasn't sure if you still had access to those topics since it looks to be on a slightly different account.

Oh I see it! I'll have to look at that. It's certainly possible if I remember correctly.

[Hey Doomer_]

Pattern recognition

This isn't something that ZScript can do, but it is something that OpenCV in Python can easily do with contours. Simple example:

Code: Select all

if not os.path.exists('patches'):
    os.mkdir('patches')
if not os.path.exists('contours'):
    os.mkdir('contours')

wad = WAD('DOOM.WAD')
pat = wad.patches

for pname in pat:
    img = pat[pname].to_Image('RGBA')
    img.save('patches\{0}.png'.format(pname))

    im = cv.imread('patches\{0}.png'.format(pname))
    imgray = cv.cvtColor(im, cv.COLOR_BGR2GRAY)
    ret, thresh = cv.threshold(imgray, 20, 255, 0) # 127
    contours, hierarchy = cv.findContours(thresh,cv.RETR_TREE,cv.CHAIN_APPROX_SIMPLE)

    cv.drawContours(im, contours, -1, (0,255,0), 1)
    cv.imwrite('contours\{0}.png'.format(pname), im)

The above creates a folder with green contours drawn on Doom patches. Example:

W13_A.png

What I've been able to do with Citrix - an app that is only an image to the OS - is compare a stored contour to a screen-to-contour image and determine a probability that one is contained in the other regardless of what the user is doing in Citrix. I was interested in knowing an app or page without searching for unique images on the screen, which is time consuming and somewhat error-prone. This allowed me to automate Citrix.

Thus with Relighting it's feasible to examine any image in a wad or pk3 and determine if it contains a light source with reasonable accuracy using this method. As I recall there wasn't much code involved with my Citrix method and it was extremely fast.

I'll have to look into this more, but that's the idea.

[cortes2k]

Posted v4.0165b

Code: Select all

v4.0165b

8/19 - fixed DIV/0 error
8/20 - added distance check to lights/shadows beyond FOV
8/21 - added length check to texture lights
8/22 - fixed bug in flashlight scale
8/22 - added rlassets to create blurry shadows

Relighting-wise this is a maintenance release without new features. I have fixed a few bugs here and there, including one that affected how flashlights interact with shadow casters. The big addition is rlassets, which allows a user to create blurred shadows of varying degrees to add to this mod. This means mods including 3d models (including Voxel Doom) can have sprite shadows.

Interesting problem and my solution seems to test OK. Many thanks for your indulgence on this and let me know what you think!

Hi dude following your great job.

I have noticed you add lights like in E1M1 first door and E1M2 but \I have nocided too, you removed the lights in places like grid of lights at E1M2 and E1M3 at the radiation pool... specialy at the E1M3 pool light creates great shadows looking great. Why you removed taht feature? or is my bad? do I have to change settings with the new version?
I leve here creen captures.
https://imgur.com/a/15FKJ8z

Cheers

[Hey Doomer_]

Hi dude following your great job.

I have noticed you add lights like in E1M1 first door and E1M2 but \I have nocided too, you removed the lights in places like grid of lights at E1M2 and E1M3 at the radiation pool... specialy at the E1M3 pool light creates great shadows looking great. Why you removed taht feature? or is my bad? do I have to change settings with the new version?
I leve here creen captures.
https://imgur.com/a/15FKJ8z

Cheers

It could be this line in hd_relighting_EventHandler.zs (around 1190):

Code: Select all

	if (lin.delta.length() < max && lin.delta.length() <= 64) ///////// test

This adds texture lights to shorter lines, and it's been an attempt to add light where there should be light. This doesn't work well, unfortunately, a reason I'm looking at a more elegant solution using Python. You can try changing the line to this:

Code: Select all

	if (lin.delta.length() < max) // && lin.delta.length() <= 64) ///////// test