[4.6pre-229] Player color not changing using Vulkan

[grahf78]

I don't know since when this bug has been present, but I just noticed it using the latest devbuild (aaf8323eb294b1a5ab258b729df714488be11c2a).

Using Vulkan, if you go to Player Setup and change colors, they don't get reflected in the player preview box, instead it uses the last color that was applied to the sprite (caching related?).

Steps to reproduce:

- Use the Vulkan backend.
- Go to Player Setup, and change the player color.

[Rachael]

I will investigate this further later if I have time, but I had to stop temporarily. So far in my investigation though, I've found that VkRenderState::ApplyMaterial() is not called anywhere near the same way that FGLRenderState::ApplyMaterial(FMaterial *mat, int clampmode, int translation, int overrideshader) is - which has actual parameters in its function prototype. Which is relevant because the GL version has this line:

Code: Select all

	lastTranslation = translation;

Hopefully I can do more with this later. Not even sure if I am hitting the right function here - but the Vulkan version of that function is quite a bit more simplified.

[Rachael]

Well turns out I can't fix this one on my own, I don't know enough about what is going on here. I attempted this to fix it:

---

Code: Select all

diff --git a/src/common/rendering/vulkan/renderer/vk_renderstate.cpp b/src/common/rendering/vulkan/renderer/vk_renderstate.cpp
index b933046e30..944e5a895f 100644
--- a/src/common/rendering/vulkan/renderer/vk_renderstate.cpp
+++ b/src/common/rendering/vulkan/renderer/vk_renderstate.cpp
@@ -427,6 +427,16 @@ void VkRenderState::ApplyVertexBuffers()

 void VkRenderState::ApplyMaterial()
 {
+       if (mMaterial.mMaterial)
+       {
+               FMaterial* mat = mMaterial.mMaterial;
+               auto tex = mat->Source();
+               int translation = mMaterial.mTranslation;
+               int clampmode = mMaterial.mClampMode;
+               clampmode = tex->GetClampMode(clampmode);
+               if (mat != lastMaterial || lastClamp != clampmode || translation != lastTranslation)
+                       mMaterial.mChanged = true;
+       }
        if (mMaterial.mChanged)
        {
                auto fb = GetVulkanFrameBuffer();
diff --git a/src/common/rendering/vulkan/renderer/vk_renderstate.h b/src/common/rendering/vulkan/renderer/vk_renderstate.h
index d4df3a7747..bcf8cda662 100644
--- a/src/common/rendering/vulkan/renderer/vk_renderstate.h
+++ b/src/common/rendering/vulkan/renderer/vk_renderstate.h
@@ -119,6 +119,11 @@ protected:
                VkSampleCountFlagBits Samples = VK_SAMPLE_COUNT_1_BIT;
                int DrawBuffers = 1;
        } mRenderTarget;
+
+private:
+       int lastTranslation = 0;
+       FMaterial* lastMaterial = nullptr;
+       int lastClamp = 0;
 };

 class VkRenderStateMolten : public VkRenderState

but that did absolutely nothing to help with the situation.

[Lippeth]

I noticed that this seemed to be fixed in gzdoom-x64-g4.6pre-259-gfef835bca, but is misbehaving again in gzdoom-x64-g4.6pre-264-gbe4e19b07.

EDIT: I take it back, I was using OpenGL when it worked, and didn't notice the correlation.