While SDL seems to work okay on Weston (the reference Wayland compositor), it does seem to have some issues.
Test environment:
Spoiler:
System: Ubuntu 20.04 (aarch64) on Raspberry Pi 4, installed from Raspberry Pi Ubuntu MATE image
CPU: Broadcom BCM2711B0 quad-core A72 (ARMv8-A) 64-bit @ 1.5GHz
GPU: V3D provided from above SoC
OpenGL version (forced): GL_VERSION 3.3 (Core Profile) using a self-compiled Mesa 21.1.0-devel (git-205e737f51)
GL Shading Language version: 3.30
Desktop: Weston with default shell (*no* Xwayland loaded), loaded directly using terminal tty1 (bypassing X11)
It works okay for fullscreen, but in windowed mode the window simply disappears. vid_preferbackend 0 (opengl) works okay, vid_preferbackend 2 (softpoly) simply crashes.
I'm sticking this in tech issues for now, not sure if the bug is in GZDoom itself or in SDL. It appears that the SDL version being compiled is 2.0.0 (what is available in Ubuntu's apt repo).
SDL on Ubuntu 20.04 should be 2.0.10. I would definitely build the latest SDL and check that first. Also check setting vid_sdl_render_driver to software.
But I definitely wouldn't rule out a bug in the use of SDL_Renderer for SoftPoly. On Linux the only value proposition is that with the software render driver it does provide a pure software code path, but on systems with OpenGL or Vulkan available it's slower even for throwing the Carmack renderer to the screen (even when not using the software render driver) so I'd be surprised if it gets a whole lot of testing.
I recently tested GZDoom on Softpoly backend on a Amlogic S912 device running Manjaro ARM 64-bit and it doesn't seem to crash. Is it some sort of CPU/RAM problem exclusive to the Raspberry Pis?
Possibly, I don't know. I know Pi's don't typically use the most up-to-date ARM chips but I also wouldn't expect it to fail on something this badly either. Even if so - I would have thought maybe the compiler would catch it.
Do also try with -march=armv8.2-a if compiling with -march=armv8-a doesn't fix the Softpoly crash when you finally set up your system. There was a couple of memory model changes introduced in the ARMv8.2-A revision.
I tested all the rendering backends (vid_preferbackend 0–3, restart after each change).
In windowed mode the window's position is half outside the screen, it cannot be moved but it can be resized. The window does not have a title bar. Despite these issues in the windowed mode the game itself works.
In full screen mode everything works fine.
Also I did not experience any crashes during testing.
