Request: Help with modernizing Elementalism's shaders

[Bauul]

Hello lovely people,

With the RC1 release of Elementalism, it's been pointed out that the shaders I have been using are out of date, to the extent that they might not even work properly with 4.8 when it releases.

I know basically nothing about writing shaders, and had cobbled most of these together from bits and pieces I found online back in 2018. Apparently these use some rather old approaches that are not future proof, but I don't know what exactly. So I was hoping that a kind soul here might have a few minutes to look through these and update them where anything is out of date?

The shaders are:

https://pastebin.com/iM5q4hLs - a sort of bump mappy, parallax-mappy thingy for icky flesh textures
https://pastebin.com/yjpfmMJq - an environment-mapping warpy shader for water textures
https://pastebin.com/UAEbGZSj - a simple warping shader I use on some flesh-sprites
https://pastebin.com/91bNHMY9 and https://pastebin.com/r7mDCtGR - two halves of a screenspace blur shader, used to accompany a couple of big explosions
https://pastebin.com/wTZQ0PqT and https://pastebin.com/dHTDxtKz - two halves of an screenspace warping shader, used when the player goes underwater

All of these currently work in 4.7.1, and I don't want the end result changed at all. I would just forever be grateful if anyone was able to review the syntax and post any modernization changes that would need to be made to make the future proof.

Thank you in advance to anyone who is able to help!

[Rachael]

That is not correct at all. The shaders will work perfectly fine in 4.8, I don't know where you got that information but it is absolutely dead false. The shaders you have will not stop working.

The thing that has changed with post-process shaders is how they are changed in ZScript. Material shaders (i.e. object/texture shaders) are unchanged. However if you have tried out the OpenGL ES backend at some point they might accidentally be disabled. (They can be reenabled again by typing gl_customshader true in the console)

Instead of: playermobj.player.shader.SetEnabled("ShaderName") for example - change the playermobj.player.shader part to ppshader. So - ppShader.SetEnabled("ShaderName") Also this means the effects are global and affect every player. Uniforms undergo a similar change, with the exact same syntactic adjustment.

The player.shader.command functions in ZScript will also continue to work as long as your ZScript is set to an older version. Once you use 4.8 however, then the deprecation takes hold and you will have to use the new method.

[Bauul]

That's fantastic news, thanks for reviewing them Rachael! That's a big relief!

To confirm it was Graf who suggested the shaders needed updating, he said:

"A technical note: While looking at the shaders I noticed that there's still several that use the broken and deprecated ProcessMaterial function instead of the future-proof SetupMaterial replacement. I strongly recommend to rewrite those shaders to the new interface to ensure that they continue working in the future. ProcessMaterial has a serious design flaw that is not fixable, so it should be restricted to old mods that are done and not updated anymore."

I could find at least one of the above shaders (the first one) that uses "ProcessMaterial" (to the extent those words appear in the shader - I don't know what they do exactly). There was also one shader, the second one in the list above, that someone told me a few weeks ago was using outdated methods and needed to be updated. Additionally at least one shader (not one above, I've since gotten rid of it) was non-functional in a 4.8 dev build, so in light if all this feedback I thought it was safer to just post all of them and hopefully someone could review and let me know what needed updating.

So do I not need to worry about ProcessMaterial? That's fantastic news if so! And the others look good too? That's a great weight off my shoulders!

I did look at the ZScript you mentioned. I do have a couple of post-process shaders being called through it, specifically:

Code: Select all • Expand view

class WaterHandler : StaticEventHandler {
   override void RenderOverlay(RenderEvent e) {
      // set the player's timer up correctly (more-than-1-tick precision)
      PlayerInfo p = players[consoleplayer];
      Shader.SetUniform1f(p, "watershader", "timer", gametic + e.FracTic);
      
      if (p.mo.waterlevel >= 3) {
         Shader.SetEnabled(p, "watershader", true);
         Shader.SetEnabled(p, "waterzoomshader", true);
         double effectSize = CVar.GetCVar("uw_effectsize", p).GetFloat();
         Shader.SetUniform1f(p, "watershader", "waterFactor", effectSize);
         Shader.SetUniform1f(p, "waterzoomshader", "zoomFactor", 1 - (effectSize * 2));
      }
      else {
         Shader.SetEnabled(p, "watershader", false);
         Shader.SetEnabled(p, "waterzoomshader", false);
      }
   }
}

and

Code: Select all • Expand view

class RoarController {
   static void setRoarSize(Actor mo, int size) {
      if (mo != NULL && mo.player != NULL) {
         if (size != 0) {
            Shader.setEnabled(mo.player, "Roar_RadialBlurShader", true);
            Shader.setEnabled(mo.player, "Roar_ZoomShader", true);
         }
         else {
            Shader.setEnabled(mo.player, "Roar_RadialBlurShader", false);
            Shader.setEnabled(mo.player, "Roar_ZoomShader", false);
         }
         
         Shader.setUniform1i(mo.player, "Roar_RadialBlurShader", "blurSize", size);
         Shader.setUniform1i(mo.player, "Roar_ZoomShader", "blurSize", size);
      }
   }
}

Are these using the old method you outlined above? I can't see "player.shader" anywhere but I appreciate it's not always a 1-1 in terms of the coding.

Thanks again for your help!

[Rachael]

Oh, I see - well if Graf says ProcessMaterial is being deprecated, then yeah, it does need to be updated. I'll take a look when I get a chance to see if I can update it as per his specs that he gave you.

[Nash]

Converting ProcessMaterial to SetupMaterial should be very easy. It's just a matter of changing the function signature (returning void instead of Material), and then assigning the values of each member of the material inside the function itself.

Here's an example I posted in #materials in the Discord server.

Code: Select all • Expand view


void SetupMaterial(inout Material mat)
{
    vec2 texCoord = vTexCoord.st;
    mat.Base = getTexel(texCoord);
    mat.Normal = ApplyNormalMap(texCoord);
#if defined(SPECULAR)
    mat.Specular = texture(speculartexture, texCoord).rgb;
    mat.Glossiness = uSpecularMaterial.x;
    mat.SpecularLevel = uSpecularMaterial.y;
#endif
#if defined(PBR)
    mat.Metallic = texture(metallictexture, texCoord).r;
    mat.Roughness = texture(roughnesstexture, texCoord).r;
    mat.AO = texture(aotexture, texCoord).r;
#endif
    mat.Bright = texture(brighttexture, texCoord);
}
 

This is a base template custom shader that does nothing (in other words, it's exactly what GZDoom is doing by default). It's a good starting point for a custom material shader. What you do from here on is up to you.

It should be self explanatory and straight-forward. Do whatever you want with mat's members to manipulate the various material parameters

Finally, here's some documentation by dpJudas from back in the day:

Code: Select all • Expand view


    SPECULAR will be defined if the shader is being compiled for a specular material.
    PBR will be defined for PBR materials.
    BRIGHTMAP will also be defined if it is using a brightmap texture.

    The following textures will be defined depending on the type of material:
        tex
        brighttexture
        speculartexture
        metallictexture
        roughnesstexture
        aotexture

    The helper functions getTexel and ApplyNormalMap can also be used.

    The material struct looks like this:

    struct Material
    {
        vec4 Base;
        vec4 Bright;
        vec3 Normal;
        vec3 Specular;
        float Glossiness;
        float SpecularLevel;
        float Metallic;
        float Roughness;
        float AO;
    };
 

[Nash]

Code: Select all • Expand view

//===========================================================================
//
// BEFORE
//
//===========================================================================

Material ProcessMaterial()
{
    mat3 tbn = GetTBN();
    vec2 texCoord = ParallaxMap(tbn);

    Material material;
    material.Base = getTexel(texCoord);
    material.Normal = GetBumpedNormal(tbn, texCoord);
#if defined(SPECULAR)
    material.Specular = texture(speculartexture, texCoord).rgb;
    material.Glossiness = uSpecularMaterial.x;
    material.SpecularLevel = uSpecularMaterial.y;
#endif
#if defined(PBR)
    material.Metallic = texture(metallictexture, texCoord).r;
    material.Roughness = texture(roughnesstexture, texCoord).r;
    material.AO = texture(aotexture, texCoord).r;
#endif
#if defined(BRIGHTMAP)
    material.Bright = texture(brighttexture, texCoord);
#endif
    return material;
}

//===========================================================================
//
// AFTER
//
//===========================================================================

void SetupMaterial(inout Material mat)
{
    mat3 tbn = GetTBN();
    vec2 texCoord = ParallaxMap(tbn);

    mat.Base = getTexel(texCoord);
    mat.Normal = GetBumpedNormal(tbn, texCoord);
#if defined(SPECULAR)
    mat.Specular = texture(speculartexture, texCoord).rgb;
    mat.Glossiness = uSpecularMaterial.x;
    mat.SpecularLevel = uSpecularMaterial.y;
#endif
#if defined(PBR)
    mat.Metallic = texture(metallictexture, texCoord).r;
    mat.Roughness = texture(roughnesstexture, texCoord).r;
    mat.AO = texture(aotexture, texCoord).r;
#endif
#if defined(BRIGHTMAP)
    mat.Bright = texture(brighttexture, texCoord);
#endif
}
 

Before/after conversion. Very simple. I apologize in advance though that I can't commit to doing the entire thing, but hopefully this serves to demonstrate how it can be done by anyone regardless of experience. :)

[SanyaWaffles]

Instead of: playermobj.player.shader.SetEnabled("ShaderName") for example - change the playermobj.player.shader part to ppshader. So - ppShader.SetEnabled("ShaderName") Also this means the effects are global and affect every player.

What if you want a postprocessing shader to only affect one player? Some post-processing shaders should not affect all players, like colorblind shaders. Seems kind of counter intuitive for things like that.

[Rachael]

The old way of handling that was not quite the proper way. All it did was check to see if the calling player number was the same as consoleplayer. A more preferred way to do it would have been to create uniform copies for each player actor - that way the correct uniform set can be activated when F12 spy-camming to each one in a multiplayer game - but that is not how it works currently.

If you really want you can make a wrapper function (incidentally, doing what the deprecated support functions in the player info class do), which adds that same check - but - consider it unsupported and likely to be problematic.

It might be safe to export the playerpawn's spytarget as a native pointer so that the preferred method can be done manually on the zscript side. From my observations that is sent over network games anyhow.

[Bauul]

Code: Select all • Expand view

//===========================================================================
//
// BEFORE
//
//===========================================================================

Material ProcessMaterial()
{
    mat3 tbn = GetTBN();
    vec2 texCoord = ParallaxMap(tbn);

    Material material;
    material.Base = getTexel(texCoord);
    material.Normal = GetBumpedNormal(tbn, texCoord);
#if defined(SPECULAR)
    material.Specular = texture(speculartexture, texCoord).rgb;
    material.Glossiness = uSpecularMaterial.x;
    material.SpecularLevel = uSpecularMaterial.y;
#endif
#if defined(PBR)
    material.Metallic = texture(metallictexture, texCoord).r;
    material.Roughness = texture(roughnesstexture, texCoord).r;
    material.AO = texture(aotexture, texCoord).r;
#endif
#if defined(BRIGHTMAP)
    material.Bright = texture(brighttexture, texCoord);
#endif
    return material;
}

//===========================================================================
//
// AFTER
//
//===========================================================================

void ProcessMaterial(inout Material mat)
{
    mat3 tbn = GetTBN();
    vec2 texCoord = ParallaxMap(tbn);

    mat.Base = getTexel(texCoord);
    mat.Normal = GetBumpedNormal(tbn, texCoord);
#if defined(SPECULAR)
    mat.Specular = texture(speculartexture, texCoord).rgb;
    mat.Glossiness = uSpecularMaterial.x;
    mat.SpecularLevel = uSpecularMaterial.y;
#endif
#if defined(PBR)
    mat.Metallic = texture(metallictexture, texCoord).r;
    mat.Roughness = texture(roughnesstexture, texCoord).r;
    mat.AO = texture(aotexture, texCoord).r;
#endif
#if defined(BRIGHTMAP)
    mat.Bright = texture(brighttexture, texCoord);
#endif
}
 

Before/after conversion. Very simple. I apologize in advance though that I can't commit to doing the entire thing, but hopefully this serves to demonstrate how it can be done by anyone regardless of experience.

Thanks for outlining the required changes Nash, this was incredibly helpful for someone with very little coding skill. I needed to swap out "void ProcessMaterial" for "void SetupMaterial" in the first line of your fixed version, but otherwise it works like a charm.

Looking through the rest of the shaders I think that was the only instance where ProcessMaterial appeared, so unless there's any other instances of older syntax in the above shaders (which based on the discussion so far it doesn't seem there is) I think we're all set!

Thanks for everyone's help, it's immensely appreciated!

[Enjay]

Sorry to piggyback onto the thread with my own question, but it seems relevant/related.

hopefully this serves to demonstrate how it can be done by anyone regardless of experience.

There is a saying that I am fond of: if you make something idiot proof, then someone will just invent a better idiot.

I am that idiot.

The following shader uses ProcessMaterial but all of my attempts to change it result in failed compilations. Can anyone figure out what should be changed to modernise it?

Code: Select all • Expand view

mat3 GetTBN();
uniform float timer;
vec3 GetBumpedNormal(mat3 tbn, vec2 texcoord);

Material ProcessMaterial()
{
    mat3 tbn = GetTBN();
    vec2 texCoord = vTexCoord.st;

    Material material;
    vec4 addEnv = texture(glowscan, ((pixelpos.xyz).xy + (timer*30)) * 0.01);
    
    material.Base = getTexel(texCoord);
    material.Base.r += addEnv.r;
    material.Base.g += addEnv.g;
    material.Base.b += addEnv.b;
    
    material.Normal = GetBumpedNormal(tbn, texCoord);
    
    
#if defined(SPECULAR)
    material.Specular = texture(speculartexture, texCoord).rgb;
    material.Glossiness = uSpecularMaterial.x;
    material.SpecularLevel = uSpecularMaterial.y;
#endif
#if defined(PBR)
    material.Metallic = texture(metallictexture, texCoord).r;
    material.Roughness = texture(roughnesstexture, texCoord).r;
    material.AO = texture(aotexture, texCoord).r;
#endif
    material.Bright = addEnv;
    return material;

}

mat3 GetTBN()
{
    vec3 n = normalize(vWorldNormal.xyz);
    vec3 p = pixelpos.xyz;
    vec2 uv = vTexCoord.st;

    vec3 dp1 = dFdx(p);
    vec3 dp2 = dFdy(p);
    vec2 duv1 = dFdx(uv);
    vec2 duv2 = dFdy(uv);

    vec3 dp2perp = cross(n, dp2);
    vec3 dp1perp = cross(dp1, n);
    vec3 t = dp2perp * duv1.x + dp1perp * duv2.x;
    vec3 b = dp2perp * duv1.y + dp1perp * duv2.y;

    float invmax = inversesqrt(max(dot(t,t), dot(b,b)));
    return mat3(t * invmax, b * invmax, n);
}

vec3 GetBumpedNormal(mat3 tbn, vec2 texcoord)
{
#if defined(NORMALMAP)
    vec3 map = texture(normaltexture, texcoord).xyz;
    map = map * 255./127. - 128./127.;
    map.xy *= vec2(0.5, -0.5);
    return normalize(tbn * map);
#else
    return normalize(vWorldNormal.xyz);
#endif
}

[Nash]

I needed to swap out "void ProcessMaterial" for "void SetupMaterial" in the first line of your fixed version

Woops! That was a copy/paste error on my part. It indeed should have been "void SetupMaterial". I've edited the original post.

@Enjay:

This should be all that's needed:

Code: Select all • Expand view


void SetupMaterial(inout Material mat)
{
    mat3 tbn = GetTBN();
    vec2 texCoord = vTexCoord.st;

    vec4 addEnv = texture(glowscan, ((pixelpos.xyz).xy + (timer*30)) * 0.01);

    mat.Base = getTexel(texCoord);
    mat.Base.r += addEnv.r;
    mat.Base.g += addEnv.g;
    mat.Base.b += addEnv.b;

    mat.Normal = GetBumpedNormal(tbn, texCoord);


#if defined(SPECULAR)
    mat.Specular = texture(speculartexture, texCoord).rgb;
    mat.Glossiness = uSpecularmat.x;
    mat.SpecularLevel = uSpecularmat.y;
#endif
#if defined(PBR)
    mat.Metallic = texture(metallictexture, texCoord).r;
    mat.Roughness = texture(roughnesstexture, texCoord).r;
    mat.AO = texture(aotexture, texCoord).r;
#endif
    mat.Bright = addEnv;
}
 

Here's a comparison shot, for learning purposes. :D

[Enjay]

Thanks, but I guess I must be doing something wrong:

Code: Select all • Expand view

**** DIED WITH FATAL ERROR:
Shader 'glowscan' could not be compiled:
ERROR: 0:31: 'material' : undeclared identifier
ERROR: 0:31: 'return' : void function cannot return a value
ERROR: 0:31: '' : compilation terminated
ERROR: 3 compilation errors.  No code generated.

I'm pretty sure that my file looks like your screenshot.

edit: I might have figured the material one out. Shaders are case sensitive, right?, and there is Material and material in there.

Making them agree now gives me:

Code: Select all • Expand view

**** DIED WITH FATAL ERROR:
Shader 'glowscan' could not be compiled:
ERROR: 0:31: '' :  syntax error, unexpected SEMICOLON, expecting LEFT_PAREN
ERROR: 1 compilation errors.  No code generated.

I've been spoiled by GZDoom's very clear error messages with line numbers. Got to look for that semicolon...

[Nash]

Enjay: sorry, again, copy/paste error strikes again, I've edited the post (and the screenshot).

I forgot to remove the "return" command in the SetupMaterial version.

And yes, GLSL is absolutely case sensitive.

[Enjay]

No worries, help is always appreciated and your fixed code works just fine now. Thank you kindly.