No graphics output on Linux, window never opens

[Chris]

Okay, this is interesting. Unfortunately, I'm still unable to replicate the issue you're having. Thankfully, native wayland with sdl2 (non compat) is not the default, so I don't think this will end up being an issue (though I might be wrong).

It does mean I won't be able to use HDR in GZDoom anymore, since X11 doesn't handle the necessary information. Not a great loss all things considered I suppose, but it is something. Though also not knowing the inner workings of SDL2 or XWayland, it's difficult to tell if it's not actually an issue with X11, or if it just happens to avoid what seems to be some kind of timing or race condition.

I'm wondering if this has something to do with your framerate? That specific render call never happens on my machine, due to the throttling that is implemented. If I disable the throttling, things still work as expected. I'm curious what would happen on your system if the throttling is disabled.

To do that you can change the if statement at `startscreen.cpp:668` from

Code: Select all

if (nowtime - screen->FrameTime > minwaittime || force)

to

Code: Select all

if (1)

With that, it freezes notably sooner, now after Texman.Init: Init texture manager.. The loading bar printed in the terminal also doesn't fill up (previously it would fill almost all the way up with, essentially, [=======.] before freezing, while now it stops right away with [........]).

Code: Select all

... everything above is basically the same as before ...
#12 0x0000555555d56d22 in VulkanSwapChain::QueuePresent (this=0x555557dcc480, imageIndex=<optimized out>, semaphore=<optimized out>) at /home/chris/projects/gzdoom/libraries/ZVulkan/src/vulkanswapchain.cpp:262
#13 0x000055555585e0dd in VkFramebufferManager::QueuePresent (this=<optimized out>) at /home/chris/projects/gzdoom/src/common/rendering/vulkan/textures/vk_framebuffer.cpp:78
#14 0x0000555555843c6a in VkCommandBufferManager::WaitForCommands (this=0x555557229540, finish=finish@entry=true, uploadOnly=uploadOnly@entry=false) at /usr/include/c++/14/bits/unique_ptr.h:193
#15 0x00005555558406f4 in VkCommandBufferManager::WaitForCommands (this=<optimized out>, finish=true) at /home/chris/projects/gzdoom/src/common/rendering/vulkan/system/vk_commandbuffer.h:22
#16 VulkanRenderDevice::Update (this=0x555558ca48b0) at /home/chris/projects/gzdoom/src/common/rendering/vulkan/system/vk_renderdevice.cpp:230
#17 0x0000555555ac04fe in FStartScreen::Render (this=this@entry=0x55555723d090, force=force@entry=false) at /home/chris/projects/gzdoom/src/common/startscreen/startscreen.cpp:705
#18 0x0000555555ac0645 in FStartScreen::Progress (this=0x55555723d090, advance=<optimized out>) at /home/chris/projects/gzdoom/src/common/startscreen/startscreen.cpp:652
#19 0x0000555555b04cef in FTextureManager::AddGroup (this=this@entry=0x555556cdac20 <TexMan>, wadnum=wadnum@entry=0, ns=ns@entry=1, usetype=usetype@entry=ETextureType::Sprite) at /home/chris/projects/gzdoom/src/common/textures/texturemanager.cpp:594
#20 0x0000555555b07a7b in FTextureManager::AddTexturesForWad (this=0x555556cdac20 <TexMan>, wadnum=0, build=...) at /home/chris/projects/gzdoom/src/common/textures/texturemanager.cpp:945
#21 0x0000555555b0865d in FTextureManager::AddTextures (this=this@entry=0x555556cdac20 <TexMan>, progressFunc_=progressFunc_@entry=0x55555587dde0 <_FUN()>, checkForHacks=checkForHacks@entry=0x55555587cc10 <CheckForHacks(BuildInfo&)>, customtexturehandler=0x555555a27710 <InitBuildTiles()>) at /home/chris/projects/gzdoom/src/common/textures/texturemanager.cpp:1227
#22 0x0000555555883e65 in D_InitGame (iwad_info=<optimized out>, iwad_info@entry=0x555557207ec0, allwads=std::vector of length 0, capacity 0, pwads=std::vector of length 0, capacity 0) at /home/chris/projects/gzdoom/src/d_main.cpp:3399
#23 0x0000555555886414 in D_DoomMain_Internal () at /home/chris/projects/gzdoom/src/d_main.cpp:3846
#24 GameMain () at /home/chris/projects/gzdoom/src/d_main.cpp:3927
#25 0x00005555556a6bd2 in main (argc=1, argv=0x7fffffffdb38) at /home/chris/projects/gzdoom/src/common/platform/posix/sdl/i_main.cpp:207

EDIT:
To make sure, I also ran it without VK_HDR_layer, and the freeze still happens exactly the same (verified in the backtrace that vkroots::wrap_QueuePresentKHR and HdrLayer::VkDeviceOverrides::QueuePresentKHR are no longer part of it, VulkanSwapChain::QueuePresent calls right to wsi_common_queue_present).

[phinet]

Thank you for testing that! I'm sorry I can't be of more help. I'm still completely incapable of reproducing this. I think this might be an issue specific to your machine, but I'll keep an eye out for similar reports.

I'm not sure if it's in the debian repos, but you might try switching from sdl2 to sd3+sd2-compat. That might fix the issue for you, but I'm really not sure. I'm also not sure if it;s a good idea to swap out the software you're using, for obvious reason.

[Chris]

Thank you for testing that! I'm sorry I can't be of more help. I'm still completely incapable of reproducing this. I think this might be an issue specific to your machine, but I'll keep an eye out for similar reports.

Maybe AMD/Mesa vs nVidia? lemming mentioned having nVidia using the proprietary driver, I have AMD with Mesa 25.0.7 (latest in Debian unstable). What do you have? Since it seems the issue is trying to present the swapchain prior to the window being shown, causing the driver to wait for something that never happens, a couple possible fixes would be to not try presenting during startup prior to the window being shown, or show the window earlier when it can start presenting to the window. Though I would need to take some time to learn GZDoom's startup code to attempt doing something like that, nor do I understand what the original issue was that lead to the instigating commit and how that solved whatever that problem was (all it seemed to do before then is show the same window at a earlier point, which is essentially what's needed here, and I don't understand the relation to "phantom window freezing desktop" that the change is meant to handle).

I'm not sure if it's in the debian repos, but you might try switching from sdl2 to sd3+sd2-compat. That might fix the issue for you, but I'm really not sure. I'm also not sure if it;s a good idea to swap out the software you're using, for obvious reason.

On Debian, you can actually have libsdl2 and libsdl2-compat installed together. libsdl2-compat installs the alternate lib into some out of the way location, that you can opt into using with LD_PRELOAD. A separate package, libsdl2-compat-shim, installs that alternate lib into the normal library path where it's used automatically.

In either case, sdl2-compat unfortunately doesn't fix the issue with Wayland native. It still freezes during init when trying to display startup progress.

EDIT:
Indeed, it seems setmodeneeded is being set too late, well after FStartScreen::Render is getting called, so if the display throttling allows a frame to be rendered before setmodeneeded is set to true after checking the network game status, it freezes. If instead setmodeneeded is set after StartScreen is created and before it starts calling FStartScreen::Render, it works:

Code: Select all

diff --git a/src/d_main.cpp b/src/d_main.cpp
index df64adb84..84b5535e3 100644
--- a/src/d_main.cpp
+++ b/src/d_main.cpp
@@ -3353,7 +3353,11 @@ static int D_InitGame(const FIWADInfo* iwad_info, std::vector<std::string>& allw
        FBaseCVar::EnableCallbacks ();
 
        StartScreen = nostartscreen? nullptr : GetGameStartScreen(per_shader_progress > 0 ? max_progress * 10 / 9 : max_progress + 3);
-       if (StartScreen != nullptr) StartScreen->Render();
+       if (StartScreen != nullptr)
+       {
+               setmodeneeded = true;
+               StartScreen->Render();
+       }
        
        // +compatmode cannot be used on the command line, so use this as a substitute
        auto compatmodeval = Args->CheckValue("-compatmode");

I have no idea whatsoever if this is a proper fix, I don't know why setmodeneeded was set so late to begin with, or if the later instance being set true is still needed with the earlier one, or if there's any other conditions where setmodeneeded should or shouldn't be set, or if it may reintroduce the issue the instigating patch tried to handle, but this fixes the freeze for me.

Also another thing that seems odd to me is with commit 2e8fb069f30e2427d0810e43fb5944873e7833a9. It's calling screen->ToggleFullscreen and V_OutputResized(screen->GetWidth(), screen->GetHeight()); between screen->BeginFrame();/twod->Begin(screen->GetWidth(), screen->GetHeight()); and twod->End();/screen->Update();. I'm surprised this doesn't cause a problem, toggling fullscreen to show the window, and possibly resize the video output, after the screen frame has begun and the "2d" texture rendering started with the pre-fullscreen size, potentially causing it to go out of sync if going fullscreen caused the framebuffer to change size. Would it make more sense to do that earlier? Like this:

Code: Select all

diff --git a/src/common/startscreen/startscreen.cpp b/src/common/startscreen/startscreen.cpp
index 8800fb0fa..56895ae3e 100644
--- a/src/common/startscreen/startscreen.cpp
+++ b/src/common/startscreen/startscreen.cpp
@@ -667,6 +667,13 @@ void FStartScreen::Render(bool force)
        // Do not refresh too often. This function gets called a lot more frequently than the screen can update.
        if (nowtime - screen->FrameTime > minwaittime || force)
        {
+               if (setmodeneeded)
+               {
+                       setmodeneeded = false;
+                       screen->ToggleFullscreen(vid_fullscreen);
+                       V_OutputResized(screen->GetWidth(), screen->GetHeight());
+               }
+
                screen->FrameTime = nowtime;
                screen->BeginFrame();
                twod->ClearClipRect();
@@ -694,13 +701,6 @@ void FStartScreen::Render(bool force)
                        DrawTexture(twod, StartupTexture, 0, 0, DTA_VirtualWidthF, displaywidth, DTA_VirtualHeightF, displayheight, TAG_END);
                }
 
-               if (setmodeneeded)
-               {
-                       setmodeneeded = false;
-                       screen->ToggleFullscreen(vid_fullscreen);
-                       V_OutputResized(screen->GetWidth(), screen->GetHeight());
-               }
-
                twod->End();
                screen->Update();
                twod->OnFrameDone();

Though again, I have no idea what the implications are of this change. Why it was originally put where it is, and if doing it earlier may be a problem. In either case, it Works For Me(tm).

[phinet]

Maybe AMD/Mesa vs nVidia? lemming mentioned having nVidia using the proprietary driver, I have AMD with Mesa 25.0.7 (latest in Debian unstable). What do you have?

AMD on both of my computers. Need to get an nvidia card for testing at some point...

Code: Select all

Device: AMD Radeon 780M Graphics (radeonsi, phoenix, LLVM 20.1.8, DRM 3.63, 6.15.10-200.fc42.x86_64) (0x15bf)
Version: 25.1.7

and 

Device: AMD Radeon RX 7900 XTX (radeonsi, navi31, LLVM 20.1.8, DRM 3.63, 6.15.10-200.fc42.x86_64) (0x744c)            
Version: 25.1.7

I have no idea whatsoever if this is a proper fix, I don't know why setmodeneeded was set so late to begin with, or if the later instance being set true is still needed with the earlier one, or if there's any other conditions where setmodeneeded should or shouldn't be set, or if it may reintroduce the issue the instigating patch tried to handle, but this fixes the freeze for me.

Glad you have a fix! I will test if that breaks what the original patch was fixing.

Originally, raspberry pi's pixel de and some x11 other compositors would "freeze" (create a non-interactive, transparent fullscreen window) when the system tried to open a window before the network wait was done. Instead, now all windows are launched hidden, then setmodeneeded is used to show the window when it is ready. I'm worried that moving it earlier will cause the same issue, but I will certainly test that and get back to you tomorrow.

Also another thing that seems odd to me is with commit 2e8fb069f30e2427d0810e43fb5944873e7833a9.

You're absolutley right. That's an oversight on my end. I'm also surprised it isn't causing issues. Will fix

[phinet]

Okay, some testing your fix works perfectly on all of my systems, including the pi that started this all. Gonna open a PR, but I'd like to credit you. What name + email can I put into the commit metadata? Feel free to dm.

Edit:
If you're not comfortable sharing stuff like that, I can also credit as Chris <chris@forum.zdoom.org>

[Chris]

Okay, some testing your fix works perfectly on all of my systems, including the pi that started this all. Gonna open a PR, but I'd like to credit you. What name + email can I put into the commit metadata? Feel free to dm.

I'm kcat on github, I've contributed to the GZDoom repo a few times before, so I presume just @'ing me or something should be fine.

[lemming]

Wait, you're the OpenAL-Soft/libopenal1 maintainer? Awesome.