Page 1 of 5

[gizdoom] Lazy palette shader

PostPosted: Sat Aug 20, 2016 2:18 am
by leileilol
Because it's the current zdoom noveltrendy and is the new volumetric light linedef.

This is the lazy unproper way of postprocessing the view to a lookup table palette (A more proper way would be indexing every texel and literally colormapping in a shader with LUTs, but...... that calls for more in-depth engine hackery). This is more of a proof-of-concept.

Replaces tonemap shaders, and as a result, there are multiple palettes to look up to:

Unsharted 2 = Doom
Bagel Dawson = Heretic
ReignHard = Strife
Liner = Hexen

Recommended you play this in Software light mode, nearest texture filter and in a low res for best intendedeffect

TELECHARGER:
lei-glsl-palettev2.pk3
(5.73 KiB) Downloaded 263 times


GReeTZ:
- just VileRancour for refactoring my old palette shader for dosbox from a long time ago. this is a porting from hlsl as well
- myself for even getting it to work with 256 colors

BUICKS:
- the flash quad gets caught up in it instead of blending after
- it's slow and not a lookup texture is used.
- it might not even compile on your lower-end card's drivers.

SCREEN SHOOT'S:
thisisnotsoftware.png


FACK:
Q: why whats the point just use zdoom
A: FACK YOU MAN

Spoiler: older than u think

Re: [gizdoom] Lazy palette shader

PostPosted: Sat Aug 20, 2016 4:46 am
by torridgristle
It gave me the error "Compile Shader 'shaders/glsl/tonemap.fp': 0(1064) : error C7549: OpenGL does not allow C style initializers" so I looked it up online and found what might have been a solution.

The solution proposed was to change const vec3 rgbi_palette[256] = { to vec3 rgbi_palette[256] = vec3[256]( and also remove the final comma in the array. I don't know if that comma matters and because of what happened when I ran it I'm too scared to test it to find out.

I didn't get an error from GZDoom when I ran it that time, instead my entire laptop locked up for several minutes until the screen went black and a Windows error sound played. I had to hard reboot. It was a terrible experience.

I guess this just won't happen for me until we can access an image from fragment shaders for a proper LUT in GZDoom. It really is a cool concept though.

Re: [gizdoom] Lazy palette shader

PostPosted: Sat Aug 20, 2016 9:22 am
by dpJudas
According to that same page the following syntax should have a lot better performance: uniform vec3 rgbi_palette[256] = vec3[](

However, what is really pulling out teeth is the way the shader brute forces the color->palette lookup. A real testament to just how insanely fast modern GPUs are - doing what id software had to precalculate on every single pixel on the screen in real time. :D

The solution is to do what zdoom does itself for colors: a rgb555 to palette index LUT (https://github.com/rheit/zdoom/blob/master/src/v_video.h#L443). To build it, loot this from zdoom: https://github.com/rheit/zdoom/blob/master/src/v_video.cpp#L657 where ColorMatcher.Pick is something like nearest_rgbi. Shader part then becomes something ala:

Code: Select allExpand view
ivec3 c = ivec3(clamp(color.rgb, vec(0.0), vec(1.0)) * 255.0 + 0.5);
int index = ((c.r >> 3) * 32 + (c.g >> 3)) * 32 + (c.b >> 3);
FragColor = RGB32k[index];

Btw., very cool to see both this shader and the RetroShader! Had a feeling the community would go wild with such stuff. :D

Re: [gizdoom] Lazy palette shader

PostPosted: Sat Aug 20, 2016 10:25 am
by Nash
Banded true colour shading in the OpenGL renderer? My dreams are finally coming true. :') This should be a stock GZDoom optional feature, TBH. Get pull requesting!!!

Because it's the current zdoom noveltrendy and is the new volumetric light linedef.


cool grafic's and shader's fx (C) leilielol DON"T NOT STEAL:!!!!!

EDIT:

File in OP does not work. Tried both torridgristle's and dpJudas' code, both do not work. Commands at the end of the arrays have been removed. Additioanlly, changing ( to { also doesn't work. Error is that the shader does not compile.

I'm on an NVidia GTX 960 with the latest drivers if that's relevant.

Re: [gizdoom] Lazy palette shader

PostPosted: Sat Aug 20, 2016 5:58 pm
by dpJudas
Here's the adjusted version that I used on my computer (Nvidia 980).

Re: [gizdoom] Lazy palette shader

PostPosted: Sat Aug 20, 2016 6:07 pm
by leileilol
dpJudas wrote:a rgb555

not precise enough. It's 18-bit (rgb666) or nothing

Re: [gizdoom] Lazy palette shader

PostPosted: Sat Aug 20, 2016 6:10 pm
by dpJudas
Haha, point taken. :D

Re: [gizdoom] Lazy palette shader

PostPosted: Sat Aug 20, 2016 6:17 pm
by torridgristle
dpJudas wrote:Here's the adjusted version that I used on my computer (Nvidia 980).

This one doesn't give me an error and looked good but wow does it not run well at all. I imagine that's to be expected though.

Edit: Black appearing in what should be white.
Image

Re: [gizdoom] Lazy palette shader

PostPosted: Sun Aug 21, 2016 1:03 am
by Nash
dpJudas' fixed version runs for me too.

Oh WOW, MAP25 actually looks ugly, just like in the software renderer again. :O This is just blowing my mind. Never expected to see a banded lighting shader for GZDoom this soon. =P

Also LMAO 30 FPS

My only "complaint" (if you could even call it that) so far is that even the colours on your weapon sprite is slightly affected, but I guess you can't really control this because this is processing the entire screen...

Re: [gizdoom] Lazy palette shader

PostPosted: Sun Aug 21, 2016 1:33 pm
by Rachael
torridgristle wrote:Edit: Black appearing in what should be white.
Image

What mod is that?

I think I know what is going on here, the "white" is simply too white, and floating point pixel values are being passed to the shader, so obviously they are allowed to exceed the maximum.

Look for this line in tonemap.fp:
Code: Select allExpand view
  float min_dst = 2.0;


Change it to something arbitrary, like
Code: Select allExpand view
  float min_dst = 1000.0;


If you're comfortable with shader programming, you can instead put caps on the incoming color vectors and that will be less glitchy but it will take a few more lines of code. This solution is only a hack. Alternatively, you can also do this:

In this loop:
Code: Select allExpand view
  for (int i=0; i<256; i++) {
    dst = distance(original, rgbi_palette[i]);
    if (dst < min_dst) {
      min_dst = dst;
      idx = i;
    }
  }

Change this code:
Code: Select allExpand view
    if (dst < min_dst) {

to
Code: Select allExpand view
    if (dst < min_dst || i == 0) {

This will force a distance check no matter what at the beginning of the loop, allowing really really bright whites to pass through the loop without problems.

Only one of these "fixes" is required. The last one might be the best one.

Re: [gizdoom] Lazy palette shader

PostPosted: Sun Aug 21, 2016 1:37 pm
by torridgristle
Eruanna wrote:What mod is that?


It's D4D, and your change fixed it.

Re: [gizdoom] Lazy palette shader

PostPosted: Sun Aug 21, 2016 1:40 pm
by Rachael
torridgristle wrote:It's D4D, and your change fixed it.

Great. :) It's exactly what I suspected, then.

Here's the "fixed" version for anyone else then. I also cleaned up the initialization code since it was not needed anymore.

Re: [gizdoom] Lazy palette shader

PostPosted: Mon Aug 22, 2016 2:49 am
by Nash
Thank you for the fixes Eruanna! And of course thanks to leileilol and dpJudas too for the earlier iterations. =) This is so fun to play with even just for the novelty and despite the 30 FPS cap :S

Re: [gizdoom] Lazy palette shader

PostPosted: Mon Aug 22, 2016 3:58 am
by Nash
Update: it seems that after just playing normally for some time, the framerate suddenly goes to 60 and stays smooth indefinitely. However, in that same play session, when I turn off the tonemap feature then back on, the game will go back to the usual 30 FPS and I can't seem to find a way to make it magically go back to 60 FPS.

I've only managed to make this happen once and I don't know how to reproduce it. Thought it was worth mentioning that the game CAN get smooth with this shader loaded, and it's probably worth investigating?

(This happened to me with Eruanna's version)

Re: [gizdoom] Lazy palette shader

PostPosted: Mon Aug 22, 2016 5:48 am
by Rachael
What happens if you try setting vid_vsync to false and set vid_maxfps to 61?

Sounds like something is taking longer than exactly 1/60th of a second but not long enough to take 1/30th of a second - but whatever it is it's triggering the vsync wait anyway, which causes your FPS to drop to 30.

EDIT: Cancel that - vid_maxfps seems to be broken at the moment, reporting as a bug.