Partially discolored or black sky

[Chris]

On Linux/AMD, the southern half of the sky has a purple or magenta hue. Actually some time back I noticed that the same half of the sky was a solid black, but I had just updated and thought it was more likely some subtle driver bug since there was no other reports of it. However, I recently (just yesterday) updated my system again, and the issue persisted. I updated GZDoom about an hour ago as of writing this (commit 1785788bdcb75dd072b3b56ff384b1838faa9c33), and now more often than not when I run it, half the sky has a purple hue, although once during recent testing it was black again. I've tried toggling various options like HDR, ambient occlusion, shadows, antialiasing, anisotropic filtering, texture filter modes, and nothing seems to have an effect. No errors or warnings in the console.

The sky looks perfectly fine with OpenGL.

Radeon RX 580, Mesa 21.3.7, kernel version 5.16.11.

If there's anything more to try or check, let me know.

[kinker31]

I've been having this issue as well over at my Fedora 35 partition, opened up a GitHub page for it: https://github.com/coelckers/gzdoom/issues/1593

I've got some screenshots of this bug in action as well, I've also had the sky show up as pitch-black as well, but I didn't get that screenshotted:

[Rachael]

Whatever this problem is, I suspect it is a newer version of Mesa and that eventually it will filter down to the rest of the distros very soon.

[Blzut3]

I can confirm this issue occurs on 4.8pre, but 4.7.1 seems to be fine. Don't have time to bisect at the moment.

[Blzut3]

Bisect is pointing to this commit for the sky half blacking out: https://github.com/coelckers/gzdoom/com ... 741577e76d

Possibly turned purple with this merge: https://github.com/coelckers/gzdoom/com ... b90399b22e

More generally it broke on the lightmap branch since that merge is when it broke for master.

[dpJudas]

Unfortunately I don't have the hardware where this happens. If someone could try install the vulkan sdk and then run the game with vk_debug 1 set (cvar requires a restart after being set) then maybe the validation layer can reveal what is wrong here.

[Chris]

Unfortunately I don't have the hardware where this happens. If someone could try install the vulkan sdk and then run the game with vk_debug 1 set (cvar requires a restart after being set) then maybe the validation layer can reveal what is wrong here.

I'm not sure what I need to install for that. I installed vulkan-validationlayers, and ran with vk_debug set to 1, but I didn't see anything different in the console. $ vulkaninfo | grep validation also didn't return anything, though I'm not sure what to expect. Is there something I'm supposed to see if validation layers are in use?

[dpJudas]

I don't know what it takes to get the validation layers working on Linux, but essentially the VK_LAYER_KHRONOS_validation layer needs to be available (or VK_LAYER_LUNARG_standard_validation if you are using a build older than about 2 days ago).

You can identify that it is active by it writing out some performance warnings about unused in/out variables between the vertex and fragment shaders (when it loads a map). If it doesn't write out those warnings then it probably didn't find the layer.

I suppose the code in vk_device.cpp could be improved to write out a warning if vk_debug is set to active and it did not find the layer.

Edit: also note that right now master crashes due to an out of bounds indexing into a vector (introduced by 2be13e1b9f2735660fb147048f35d23628023214), so you probably want to test it on the commit just prior to that one.

[Blzut3]

Here's what I'm getting with vulkan-validationlayers installed with commit e619e40f9272d326e08d33efe8d75cf497a418c5:

Code: Select all

                 Validation Performance Warning: [ UNASSIGNED-CoreValidation-Shader-OutputNotConsumed ] Object 0: handle = 0xf400000000f4, type = VK_OBJECT_TYPE_SHADER_MODULE; | MessageID = 0x609a13b | Vertex attribute at location 3 not consumed by vertex shader
                 Validation Performance Warning: [ UNASSIGNED-CoreValidation-Shader-OutputNotConsumed ] Object 0: handle = 0xf400000000f4, type = VK_OBJECT_TYPE_SHADER_MODULE; | MessageID = 0x609a13b | Vertex attribute at location 4 not consumed by vertex shader
                 Validation Performance Warning: [ UNASSIGNED-CoreValidation-Shader-OutputNotConsumed ] Object 0: handle = 0xf400000000f4, type = VK_OBJECT_TYPE_SHADER_MODULE; | MessageID = 0x609a13b | Vertex attribute at location 5 not consumed by vertex shader
                 Validation Performance Warning: [ UNASSIGNED-CoreValidation-Shader-OutputNotConsumed ] Object 0: handle = 0xf400000000f4, type = VK_OBJECT_TYPE_SHADER_MODULE; | MessageID = 0x609a13b | Vertex attribute at location 6 not consumed by vertex shader
                 Validation Performance Warning: [ UNASSIGNED-CoreValidation-Shader-OutputNotConsumed ] Object 0: handle = 0xf400000000f4, type = VK_OBJECT_TYPE_SHADER_MODULE; | MessageID = 0x609a13b | vertex shader writes to output location 0.0 which is not consumed by fragment shader
                 Validation Performance Warning: [ UNASSIGNED-CoreValidation-Shader-OutputNotConsumed ] Object 0: handle = 0xf400000000f4, type = VK_OBJECT_TYPE_SHADER_MODULE; | MessageID = 0x609a13b | vertex shader writes to output location 1.0 which is not consumed by fragment shader
                 Validation Performance Warning: [ UNASSIGNED-CoreValidation-Shader-OutputNotConsumed ] Object 0: handle = 0xf400000000f4, type = VK_OBJECT_TYPE_SHADER_MODULE; | MessageID = 0x609a13b | vertex shader writes to output location 9.0 which is not consumed by fragment shader

[dpJudas]

Thanks Blzut3. So all validation checks pass. Since it the lightmap branch that started the bug it probably is related to sampling the 2D Array lightmap texture. Could you try comment out line 664 in main.fp and see if that helps?

[Blzut3]

The sky indeed is restored with that line commented out.

Edit: Also it seems the purple vs blacked out is entirely luck of the draw and they are the same bug.

Edit 2: And for whatever it's worth, the shader seems to be fine under Zink with RADV. Mesa 21.1's lavapipe also seems to be fine, so it could still be a RADV bug but it is interesting that Zink circumvents it if it is.

Edit 3: The bug appears in both Mesa's RADV and AMD's AMDVLK.

[dpJudas]

I finally figured out what is going on here.

The lightmap branch introduced a texture for the light map, but if there is no actual lightmap the vulkan backend creates a dummy lightmap texture as the pipeline needs one bound at all times regardless of whether the texture is actually used by the shader or not.

This dummy texture is created with undefined contents. On Nvidia's driver this creates a 1x1 texture with a black pixel. On AMD/Mesa it creates whatever is in memory. That's why the bug doesn't manifest itself on my computer.

Now for the actual bug in the code:

The sky is drawn using the skydome code. It only provides vertex attributes for the SIMPLE vertex shader (aPosition, aTexCoord, aColor). But the actual shader used is not a SIMPLE shader. Thus half the vertex attributes aren't assigned at all! Yay for using every exotic/bad idea feature in OpenGL.

Anyway, this results in main.fp's check for lightmaps being active to fail and thus it samples from the lightmap texture. Due to NV always clearing the texture that didn't have any visual effect at all on my computer, but on AMD hardware it does.

Time to implement glVertexAttrib in this OpenGL emulator...

[dpJudas]

And here's the pull request that fixes it.

[Rachael]

Thank you

[Blzut3]

Well... it's certainly different now but seems that's not quite the fix.