[WIP] Neural Network Upscaled Flats/Patches

For high-res texture/sprite projects, sprite-fix patches, music add-ons, music randomizers, and other graphic/sound-only projects.
Forum rules
The Projects forums are only for projects. If you are asking questions about a project, either find that project's thread, or start a thread in the General section instead.

Got a cool project idea but nothing else? Put it in the project ideas thread instead!

Projects for any Doom-based engine (especially 3DGE) are perfectly acceptable here too.

Please read the full rules for more details.
Scuba Steve
Posts: 1059
Joined: Sat Mar 27, 2004 8:56 pm

Re: [WIP] Neural Network Upscaled Flats/Patches

Post by Scuba Steve »

Some textures tile vertically so it would be necessary to do it for the top and bottom as well.

Check the Neural upscale thread at Doomworld... I can't find it at the moment because I'm at work, but the author, hidfan, talks about his method for making the effective masks.

I don't think there's any real need to upscale a project that is not an official id Software release. I think the primary goal should be compatibility with all official releases. After that, community members can do their own upscales if they so choose.
User avatar
Phredreeke
Posts: 293
Joined: Tue Apr 10, 2018 8:14 am

Re: [WIP] Neural Network Upscaled Flats/Patches

Post by Phredreeke »

Most tiles are turning out alright. This is one where I had to manually edit two together to tile properly (and will have to separate again for the mod)

https://imgsli.com/NjIzMw

I don't think I've ever finished either of the episodes so I should probably give them a go to get a better context
Scuba Steve
Posts: 1059
Joined: Sat Mar 27, 2004 8:56 pm

Re: [WIP] Neural Network Upscaled Flats/Patches

Post by Scuba Steve »

At first, I looked at some of your work and thought to myself, "Man, these are REALLY ugly? We can't use these upscales, they look like somebody applied the plastic wrap filter to everything in Paint Shop Pro!"

Then I went into the Plutonia iwad and realized, "Holy shit, the stock Final Doom textures are REALLY ugly... it's not the upscaling, it's the awful SOURCE material!" I took a crack at using hidfan's technique for making the alpha masks. Here's the grate (which is ugly as all hell...)



I'm not sure the exact technique you're using, but I know one that was really successful before was upscaling at 8x, downsampling to 4x using bicubic, then downsampling a final time to 2x using nearest neighbor to produce some of the noise inherent in Doom's artwork.
User avatar
Darkcrafter
Posts: 562
Joined: Sat Sep 23, 2017 8:42 am
Operating System Version (Optional): Windows 10
Graphics Processor: nVidia with Vulkan support

Re: [WIP] Neural Network Upscaled Flats/Patches

Post by Darkcrafter »

I don't know why you guys think grate is ugly, it's resampling, it's not the source! It really can be upscaled nicely with paint, just draw one fragment and copypaste it so it fill all the texture, all this gimmicks come from your laziness. You will invest a lot of time in searching of a mysterious AI, opacity filtering tricks or that crappy plastic wrap filter that is supposed to do all the job for you, there is no that single button yet, the technology isn't perfect yet so you still have to do most of the job by hand.
Scuba Steve
Posts: 1059
Joined: Sat Mar 27, 2004 8:56 pm

Re: [WIP] Neural Network Upscaled Flats/Patches

Post by Scuba Steve »

Durr maybe that's why the upscaling is merely the first step in producing an accurate higher resolution recreation of the original artwork? What do you think we're doing, running this shit through an algorithm, dusting off our hands and saying "a job well done!"?

The real magic happens after this; I spent roughly an hour cleaning up the alpha matte and the final texture looks pretty reasonable. I can't simply "copy and paste it from paint" because the original texture is not a copy&paste job. The point is to make it as accurate as possible to the original while still being twice the resolution. Until now, remaking the texture by hand was the only way to introduce new data to a larger image, but with machine learning, they can produce impressive results to use as a baseline for constructing a really accurate upscale.

In the last week I've logged probably 48 hours of work cleaning up textures, eliminating seams from repeating edges, and trying to decipher some seriously hideous artwork (specifically the godawful DOB textures) and making it look better, while still retaining the original style.

So yes, duh, we've been doing more than pressing a "single button."
User avatar
Phredreeke
Posts: 293
Joined: Tue Apr 10, 2018 8:14 am

Re: [WIP] Neural Network Upscaled Flats/Patches

Post by Phredreeke »

What do you think we're doing, running this shit through an algorithm, dusting off our hands and saying "a job well done!"?
...that kinda describes my part in it TBH.
Scuba Steve
Posts: 1059
Joined: Sat Mar 27, 2004 8:56 pm

Re: [WIP] Neural Network Upscaled Flats/Patches

Post by Scuba Steve »

Phredreeke wrote:
What do you think we're doing, running this shit through an algorithm, dusting off our hands and saying "a job well done!"?
...that kinda describes my part in it TBH.
Which is an important first step in order for me to clean them up. Had you not done them, I doubt they would have ever been completed.
User avatar
Darkcrafter
Posts: 562
Joined: Sat Sep 23, 2017 8:42 am
Operating System Version (Optional): Windows 10
Graphics Processor: nVidia with Vulkan support

Re: [WIP] Neural Network Upscaled Flats/Patches

Post by Darkcrafter »

Oh well then I deserve a rotten egg thrown my way lol. By the way GigaPixel isn't the only solution on the market, check this thing out: https://letsenhance.io/

Here is some comparison:

Gigapixel 4 with separate alpha channel processing
https://drive.google.com/open?id=1KnQnj ... ofzja-825d

Photoshop 4x upscale - boxy + diffusion filter tricks
https://drive.google.com/open?id=1VW4QF ... oWLveUf2o5

Boxy 4X:
https://drive.google.com/open?id=1B0Ln9 ... _1s03th_0q

Deleted my previous post since imgur fucked up the images :bang:
Scuba Steve
Posts: 1059
Joined: Sat Mar 27, 2004 8:56 pm

Re: [WIP] Neural Network Upscaled Flats/Patches

Post by Scuba Steve »

Some of the best results came from using Nvidia's Super Resolution tool. Unfortunately, nobody has access to it anymore, so we're using a smattering of other tools to accomplish something similar. Fortunately, Final Doom's textures are so ugly, a lower quality upscale is hardly noticeable. I'm also using some of the post processing techniques from hidfan's upscale project to improve the work here, as well as incorporating art from the other project to repair some of the work Phredreeke did.
User avatar
Phredreeke
Posts: 293
Joined: Tue Apr 10, 2018 8:14 am

Re: [WIP] Neural Network Upscaled Flats/Patches

Post by Phredreeke »

To clarify what I'm doing, I use four ESRGAN models and a suite of PSP scripts.

Dedither (alsa) as a preprocessor for Box filter model (buildist)
Manga109attempt (kingdomakrillic) preprocessed with a custom PSP script
Those two are combined with another PSP script, then I use the Desharpen (loinne) model on the result

There are a ton of different models available, making the different combinations practically infinite.

Source for ESRGAN models: https://upscale.wiki/wiki/Model_Database
User avatar
Darkcrafter
Posts: 562
Joined: Sat Sep 23, 2017 8:42 am
Operating System Version (Optional): Windows 10
Graphics Processor: nVidia with Vulkan support

Re: [WIP] Neural Network Upscaled Flats/Patches

Post by Darkcrafter »

There is a trick of adding some gaussian blur by setting the radius at around 0.3-0.5 pixels maybe even fading the effect a bit so the source picture becomes a bit blurry but that can help to remedy gigapixel's love for producing ringing where it's not neeeded to be. I also try to avoid using 4 and 6x upscaling modes, and only 2x seems to work better.
User avatar
Darkcrafter
Posts: 562
Joined: Sat Sep 23, 2017 8:42 am
Operating System Version (Optional): Windows 10
Graphics Processor: nVidia with Vulkan support

Re: [WIP] Neural Network Upscaled Flats/Patches

Post by Darkcrafter »

Phredreeke wrote:To clarify what I'm doing, I use four ESRGAN models and a suite of PSP scripts.

Dedither (alsa) as a preprocessor for Box filter model (buildist)
Manga109attempt (kingdomakrillic) preprocessed with a custom PSP script
Those two are combined with another PSP script, then I use the Desharpen (loinne) model on the result

There are a ton of different models available, making the different combinations practically infinite.

Source for ESRGAN models: https://upscale.wiki/wiki/Model_Database
Seems like it's a much better solution out there but I just don't want to pollute my system with python :|

Would be nice to see how well all of these models perform.

BTW, there is a trick of adding some gaussian blur by setting the radius at around 0.3-0.5 pixels maybe even fading the effect a bit so the source picture becomes a bit blurry but that can help to remedy gigapixel's love for producing ringing where it's not neeeded to be. I also try to avoid using 4 and 6x upscaling modes, and only 2x seems to work better.
User avatar
Phredreeke
Posts: 293
Joined: Tue Apr 10, 2018 8:14 am

Re: [WIP] Neural Network Upscaled Flats/Patches

Post by Phredreeke »

First of all, scale is hardcoded into each ESRGAN model. Dedither and Desharpen are 1x scale (in other words, they leave the image the same size), most upscaling models are 4x although I've seen one as high as 16x.

I took a few textures and upscaled them with four different models. For the box model I included a separate upscale where the image was pre-processed with the Dedither model. The Manga109 model I used was interpolated with the PSNR model, which in effect attenuates it (I felt the raw Manga109 had too much noise)

https://imgur.com/a/pc1Kdje

I like the Fatality model, but it drifts the color tones away from the original. Rebout kills too much detail IMO. I feel pre-processed with the dedither model to be necessary for the box model, without it the resulting image looks very harsh and artificial (the box model wont interpolate with the other models for reasons too complicated to go into here). Manga109 is what I've been mostly been using for my previous upscales. I felt the best results were achieved by dedither+box model and manga109 model, so I blended those two for these upscales.
User avatar
Darkcrafter
Posts: 562
Joined: Sat Sep 23, 2017 8:42 am
Operating System Version (Optional): Windows 10
Graphics Processor: nVidia with Vulkan support

Re: [WIP] Neural Network Upscaled Flats/Patches

Post by Darkcrafter »

Thanks for the examples. I think I like the very first one "Box", but try blurring an image before you process it, now it all happens like software treats the image like if sharpness was all 100% everywhere so lots of ringing and other artifacts occur. If there would be a possibility to train the network exclusively by doom, that would be even more interesting to see how it performs. Also, if I'm not mistaken, ESRGAN is getting old as it's 2 years since it came out, there was a massive challenge so even more advanced alghorithms appeared recently.
User avatar
Phredreeke
Posts: 293
Joined: Tue Apr 10, 2018 8:14 am

Re: [WIP] Neural Network Upscaled Flats/Patches

Post by Phredreeke »

Part of this project has been integrated in the other neural upscale pack

Now, I am considering two different directions of taking this project. The first one is going one step further and making a 4x upscale. The other is upscaling the one remaining official 2.5D Doom game...
Post Reply

Return to “Graphic/Audio Patches”