WIP- Colour Map Shader

[Pixel Eater]

I'm stuck on this project and hope someone can help me. I am using Old Video Shader as a template and replacing it's guts with a modified copy of the Palette shader code. The aim is to replace the brightness values from the Palette shader with the information from the true colour render.
I can get it to run but it's tinting everything based on the lower left corner. Even when I comment out my portion of the code it behaves similarly, only darker. It does seem to be slightly palette aware though.

This link downloads what I have so far but with my lines disabled, so it should behave the same as the built in Palette shader (it doesn't):

colourmap_shader_broken.pk3

(1.99 KiB) Downloaded 112 times

gldefs.txt:

---

HardwareShader PostProcess scene
{
Name "colourmapshader"
Shader "shaders/PostProcess/ColourMap.fp" 330
// Uniform float timer
}

shaders/PostProcess/ColourMap.fp:

---

uniform sampler2D ExposureTexture;
uniform sampler2D PaletteLUT;

vec3 Linear(vec3 c)
{
//c = max(c, vec3(0.0));
//return pow(c, 2.2);
return c * c; // cheaper, but assuming gamma of 2.0 instead of 2.2
}

vec3 Tonemap(vec3 color)
{
ivec3 c = ivec3(clamp(color.rgb, vec3(0.0), vec3(1.0)) * 63.0 + 0.5);
int index = (c.r * 64 + c.g) * 64 + c.b;
int tx = index % 512;
int ty = index / 512;
return texelFetch(PaletteLUT, ivec2(tx, ty), 0).rgb;
}

void main()
{
vec3 color = texture(InputTexture, TexCoord).rgb;

// float truecolourlum = (color.r + color.g + color.b) / 3.0;
// float lum = 1.0;

float exposureAdjustment = texture(ExposureTexture, vec2(0.5)).x;
color = color * exposureAdjustment;
color = Linear(color); // needed because gzdoom's scene texture is not linear at the moment
color = Tonemap(color);

// float palettelum = (color.r + color.g + color.b) / 3.0;
// if (palettelum > 0.0)
// lum = 1.0 / palettelum * truecolourlum;
// color *= lum;
FragColor = vec4(color, 1.0);
}

[Rachael]

First and foremost, the palette table requires support code from the C++ side - I have no doubt it can be done shader side but it will require inputting the palette data in some way and then it's a lot of extra work to make it work.

Secondly, can you use some screenshots to explain what you mean? I am sorry but reading what you are trying to do is just very confusing.

[Pixel Eater]

Sorry for the confusion! If you think about it as using the Color/Luminosity blend modes in Photoshop between the Palette shader and the dry output does that make more sense? In doing that it should only utilize the colour map part of the Palette shader's effect.
Notice the colour banding is removed from the last pane:

Won't the background C++ stuff already be in place for the current Palette shader? This is only a tweak on the output of that...

[Rachael]

Okay, I see what you are doing now. You're applying True-Color's luminosity to Paletted's Saturation/Hue.

One of the bigger problems you have is the PaletteLUT table (is a texture) is not populated until the first time you type "gl_tonemap 5" - I suppose I could add in an argument for custom shaders to force it to be populated when the shader is activated.

What other problems are you having besides this?

[Pixel Eater]

Okay, I see what you are doing now. You're applying True-Color's luminosity to Paletted's Saturation/Hue.

That's better than I could ever put it

One of the bigger problems you have is the PaletteLUT table (is a texture) is not populated until the first time you type "gl_tonemap 5" - I suppose I could add in an argument for custom shaders to force it to be populated when the shader is activated.

That sounds like it would be helpful. Does that override custom palettes?

What other problems are you having besides this?

This is what I get when I omit my blending code (for a basic Palette shader):


And this is when I include it:


The colour of the tint changes depending on what occupies the lower left corner of the screen.

[Rachael]

It's quite likely that PaletteLUT is not actually being passed to the shader - this will require further modification on the C++ side.

[Pixel Eater]

Thanks Rachael. I will submit a feature request then
*Dun!

[Pixel Eater]

I've had an idea: can I embed a custom image into the .pk3 file to use instead of PaletteLUT? I have an extended colour map which I designed and proposed in this feature request that I could reuse for this project. If I could, it would mean I would no longer be overlaying a 24bit image with an 8bit colour map

Actually it would be cool if I could store different versions and be able to select them from the menu. Sounds like another feature request

[Rachael]

You can do it but keep in mind that PaletteLUT is made a certain way and is nowhere near the same format as a real COLORMAP.

[Pixel Eater]

I'm guessing PaletteLUT is generated on startup because I can't find it in gzdoom.pk3 or by searching this site. Is there a way to export it while Doom is running, maybe even by drawing it to screen and taking a screenshot? At that point, will embedding it into the .pk3 file's root directory cause the line "uniform sampler2D PaletteLUT;" to work correctly or is it more complicated than that?

[Rachael]

There is no way to export it - it is dynamically generated. It is also referenced by pointer, not by name, so you cannot access it anyhow.

It is possible, however, to create a program that's embedded into GZDoom's source code and create a PNG of it after it's generated. Graf could probably easily do it if he had the time and desire, I could do it with a great deal more difficulty but eventually I'd figure it out, but unfortunately I have a lot of projects as it is that I've been putting off that I really need to get to.

[Pixel Eater]

No worries and thank you for all your help, I appreciate it

I might go for plan B and request an additional option to go alongside Palette Exponent and Order that blends the luminosity back in. I had hoped to play with perceptually weighting the luminance or matching the visibility of software Doom but I won't be able to experiment with those in that situation. Which is why I wanted to have it as a 'standalone' working prototype

[Pixel Eater]

I've had an idea: can I embed a custom image into the .pk3 file to use instead of PaletteLUT?

You can do it but keep in mind that PaletteLUT is made a certain way and is nowhere near the same format as a real COLORMAP.

Could you please expand on how that is achieved? I have created a custom LUT file (hopefully in the right configuration ) and placed it in a .pk3 file, called it by typing "uniform sampler2D *filename*;" and then mapped it to the screen. All I get is a regular view of E1M1. Does the LUT file need to be in a certain directory?

[dpJudas]

The filename is the name of a lump. I can't remember exactly which naming rules it uses - you might have to type the full path of the file within the .pk3 - i.e. with a slash in front or maybe place it in a subfolder so that there is a path component in the filename.

[Pixel Eater]

Hmm, I've just tried every permutation I can think of: "./graphics/", "/graphics/", using the .png file extension, enclosing it in quotation marks and all again within gldefs instead of the shader.

Here's the LUT I've made, does it look to be the correct format?: