Page 2 of 3

Re: [WIP] Neural Network Upscaled Flats/Patches

PostPosted: Tue Sep 03, 2019 11:57 am
by Scuba Steve
Some textures tile vertically so it would be necessary to do it for the top and bottom as well.

Check the Neural upscale thread at Doomworld... I can't find it at the moment because I'm at work, but the author, hidfan, talks about his method for making the effective masks.

I don't think there's any real need to upscale a project that is not an official id Software release. I think the primary goal should be compatibility with all official releases. After that, community members can do their own upscales if they so choose.

Re: [WIP] Neural Network Upscaled Flats/Patches

PostPosted: Thu Sep 05, 2019 4:46 pm
by Phredreeke
Most tiles are turning out alright. This is one where I had to manually edit two together to tile properly (and will have to separate again for the mod)

https://imgsli.com/NjIzMw

I don't think I've ever finished either of the episodes so I should probably give them a go to get a better context

Re: [WIP] Neural Network Upscaled Flats/Patches

PostPosted: Thu Sep 05, 2019 9:02 pm
by Scuba Steve
At first, I looked at some of your work and thought to myself, "Man, these are REALLY ugly? We can't use these upscales, they look like somebody applied the plastic wrap filter to everything in Paint Shop Pro!"

Then I went into the Plutonia iwad and realized, "Holy shit, the stock Final Doom textures are REALLY ugly... it's not the upscaling, it's the awful SOURCE material!" I took a crack at using hidfan's technique for making the alpha masks. Here's the grate (which is ugly as all hell...)



I'm not sure the exact technique you're using, but I know one that was really successful before was upscaling at 8x, downsampling to 4x using bicubic, then downsampling a final time to 2x using nearest neighbor to produce some of the noise inherent in Doom's artwork.

Re: [WIP] Neural Network Upscaled Flats/Patches

PostPosted: Tue Sep 10, 2019 8:24 am
by Darkcrafter
I don't know why you guys think grate is ugly, it's resampling, it's not the source! It really can be upscaled nicely with paint, just draw one fragment and copypaste it so it fill all the texture, all this gimmicks come from your laziness. You will invest a lot of time in searching of a mysterious AI, opacity filtering tricks or that crappy plastic wrap filter that is supposed to do all the job for you, there is no that single button yet, the technology isn't perfect yet so you still have to do most of the job by hand.

Re: [WIP] Neural Network Upscaled Flats/Patches

PostPosted: Tue Sep 10, 2019 8:49 am
by Scuba Steve
Durr maybe that's why the upscaling is merely the first step in producing an accurate higher resolution recreation of the original artwork? What do you think we're doing, running this shit through an algorithm, dusting off our hands and saying "a job well done!"?

The real magic happens after this; I spent roughly an hour cleaning up the alpha matte and the final texture looks pretty reasonable. I can't simply "copy and paste it from paint" because the original texture is not a copy&paste job. The point is to make it as accurate as possible to the original while still being twice the resolution. Until now, remaking the texture by hand was the only way to introduce new data to a larger image, but with machine learning, they can produce impressive results to use as a baseline for constructing a really accurate upscale.

In the last week I've logged probably 48 hours of work cleaning up textures, eliminating seams from repeating edges, and trying to decipher some seriously hideous artwork (specifically the godawful DOB textures) and making it look better, while still retaining the original style.

So yes, duh, we've been doing more than pressing a "single button."

Re: [WIP] Neural Network Upscaled Flats/Patches

PostPosted: Tue Sep 10, 2019 9:39 am
by Phredreeke
What do you think we're doing, running this shit through an algorithm, dusting off our hands and saying "a job well done!"?


...that kinda describes my part in it TBH.

Re: [WIP] Neural Network Upscaled Flats/Patches

PostPosted: Tue Sep 10, 2019 10:29 am
by Scuba Steve
Phredreeke wrote:
What do you think we're doing, running this shit through an algorithm, dusting off our hands and saying "a job well done!"?


...that kinda describes my part in it TBH.


Which is an important first step in order for me to clean them up. Had you not done them, I doubt they would have ever been completed.

Re: [WIP] Neural Network Upscaled Flats/Patches

PostPosted: Tue Sep 10, 2019 11:11 am
by Darkcrafter
Oh well then I deserve a rotten egg thrown my way lol. By the way GigaPixel isn't the only solution on the market, check this thing out: https://letsenhance.io/

Here is some comparison:

Gigapixel 4 with separate alpha channel processing
https://drive.google.com/open?id=1KnQnj ... ofzja-825d

Photoshop 4x upscale - boxy + diffusion filter tricks
https://drive.google.com/open?id=1VW4QF ... oWLveUf2o5

Boxy 4X:
https://drive.google.com/open?id=1B0Ln9 ... _1s03th_0q

Deleted my previous post since imgur fucked up the images :bang:

Re: [WIP] Neural Network Upscaled Flats/Patches

PostPosted: Tue Sep 10, 2019 1:29 pm
by Scuba Steve
Some of the best results came from using Nvidia's Super Resolution tool. Unfortunately, nobody has access to it anymore, so we're using a smattering of other tools to accomplish something similar. Fortunately, Final Doom's textures are so ugly, a lower quality upscale is hardly noticeable. I'm also using some of the post processing techniques from hidfan's upscale project to improve the work here, as well as incorporating art from the other project to repair some of the work Phredreeke did.

Re: [WIP] Neural Network Upscaled Flats/Patches

PostPosted: Tue Sep 10, 2019 1:59 pm
by Phredreeke
To clarify what I'm doing, I use four ESRGAN models and a suite of PSP scripts.

Dedither (alsa) as a preprocessor for Box filter model (buildist)
Manga109attempt (kingdomakrillic) preprocessed with a custom PSP script
Those two are combined with another PSP script, then I use the Desharpen (loinne) model on the result

There are a ton of different models available, making the different combinations practically infinite.

Source for ESRGAN models: https://upscale.wiki/wiki/Model_Database

Re: [WIP] Neural Network Upscaled Flats/Patches

PostPosted: Tue Sep 10, 2019 2:46 pm
by Darkcrafter
There is a trick of adding some gaussian blur by setting the radius at around 0.3-0.5 pixels maybe even fading the effect a bit so the source picture becomes a bit blurry but that can help to remedy gigapixel's love for producing ringing where it's not neeeded to be. I also try to avoid using 4 and 6x upscaling modes, and only 2x seems to work better.

Re: [WIP] Neural Network Upscaled Flats/Patches

PostPosted: Tue Sep 10, 2019 2:51 pm
by Darkcrafter
Phredreeke wrote:To clarify what I'm doing, I use four ESRGAN models and a suite of PSP scripts.

Dedither (alsa) as a preprocessor for Box filter model (buildist)
Manga109attempt (kingdomakrillic) preprocessed with a custom PSP script
Those two are combined with another PSP script, then I use the Desharpen (loinne) model on the result

There are a ton of different models available, making the different combinations practically infinite.

Source for ESRGAN models: https://upscale.wiki/wiki/Model_Database


Seems like it's a much better solution out there but I just don't want to pollute my system with python :|

Would be nice to see how well all of these models perform.

BTW, there is a trick of adding some gaussian blur by setting the radius at around 0.3-0.5 pixels maybe even fading the effect a bit so the source picture becomes a bit blurry but that can help to remedy gigapixel's love for producing ringing where it's not neeeded to be. I also try to avoid using 4 and 6x upscaling modes, and only 2x seems to work better.

Re: [WIP] Neural Network Upscaled Flats/Patches

PostPosted: Tue Sep 10, 2019 4:38 pm
by Phredreeke
First of all, scale is hardcoded into each ESRGAN model. Dedither and Desharpen are 1x scale (in other words, they leave the image the same size), most upscaling models are 4x although I've seen one as high as 16x.

I took a few textures and upscaled them with four different models. For the box model I included a separate upscale where the image was pre-processed with the Dedither model. The Manga109 model I used was interpolated with the PSNR model, which in effect attenuates it (I felt the raw Manga109 had too much noise)

https://imgur.com/a/pc1Kdje

I like the Fatality model, but it drifts the color tones away from the original. Rebout kills too much detail IMO. I feel pre-processed with the dedither model to be necessary for the box model, without it the resulting image looks very harsh and artificial (the box model wont interpolate with the other models for reasons too complicated to go into here). Manga109 is what I've been mostly been using for my previous upscales. I felt the best results were achieved by dedither+box model and manga109 model, so I blended those two for these upscales.

Re: [WIP] Neural Network Upscaled Flats/Patches

PostPosted: Wed Sep 11, 2019 1:38 am
by Darkcrafter
Thanks for the examples. I think I like the very first one "Box", but try blurring an image before you process it, now it all happens like software treats the image like if sharpness was all 100% everywhere so lots of ringing and other artifacts occur. If there would be a possibility to train the network exclusively by doom, that would be even more interesting to see how it performs. Also, if I'm not mistaken, ESRGAN is getting old as it's 2 years since it came out, there was a massive challenge so even more advanced alghorithms appeared recently.

Re: [WIP] Neural Network Upscaled Flats/Patches

PostPosted: Wed Sep 18, 2019 10:45 am
by Phredreeke
Part of this project has been integrated in the other neural upscale pack

Now, I am considering two different directions of taking this project. The first one is going one step further and making a 4x upscale. The other is upscaling the one remaining official 2.5D Doom game...