Metal renderer

[johncurley]

Hi, I have a little experience with older graphics API's like opengL. As a macOS fan and big fan of total conversion mods for GZDoom such as the Ashes series, I'd like to help write a native metal renderer for macOS. I don't have a lot of metal experience but I am asking Ai for help and implementing each function based on the Vulkan renderer and then hand writing and tuning what I can. I was wondering if there was any interest in this as building the engine with MoltenVK seems to crash on my Intel Mac. Eventually I'd like to help maintain if I can the renderer and the engine on macOS given the huge porting effort to Cocoa. I can't promise anything at this stage but I'd like to help try at least.

[Redneckerz]

Hi, I have a little experience with older graphics API's like opengL. As a macOS fan and big fan of total conversion mods for GZDoom such as the Ashes series, I'd like to help write a native metal renderer for macOS.

That is cool.

But...

I don't have a lot of metal experience but I am asking Ai for help and implementing each function based on the Vulkan renderer and then hand writing and tuning what I can.

That's a red flag for me. Writing a renderer is no sinecure and AI isn't a shortcut to what essentially boils down to hard math. Writing a renderer isn't a one snap easy thing, and without wanting to ruin your dreams, but either learn the hard maths or wait for someone with a proper hand in this kind of thing to take a shot at it.

[johncurley]

I'm not really sure what you mean by "hard math". Most calculations are just linear equations through matrix math, other mathematical concepts are for things like texture conversion(understanding the architecture endian or intrinsic math) and the only really "hard math" is in advanced rendering like lighting which requires calculus level algorithms(though most are calculated already and the code can be used). I'm mainly using Ai to scaffold a lot of the Apple specific code that I'm not familiar with and also to help with GLSL->SPIR-V->MSL conversion at this stage.

[Graf Zahl]

Are you sure this will make the game run better?
One important issue is that none of the main developers here own a Mac. For example, the entire new startup popup doesn't work there yet bacause ZWidgets does not have a Mac backend.

Creating a Metal backend is also about maintaining it later - it is pointless to add this but then be stuck with it because nobody can make updates when new features are needed.
Will you be able to maintain that code once it is done? If your answer is "no", let's better stop right here.

[johncurley]

Hi Graf Zahl,

To answer your questions, I do believe creating a native backend for MacOS would be worthwhile. Apple silicon has tile based renderers which Metal is designed to take advantage of. In terms of maintenance it's the easiest modern framework to use vs Vulkan and DirectX 12. I also believe that given the price of Apple silicon Macs having gone down, it's an affordable way of people getting into gaming especially given the price increases of NVIDIA products recently.

I do have some renderer experience though to be honest I'm quite green on Apple platforms. I also only own an older MacBook Air at this stage with an Intel processor though in the very near future I plan to get Apple silicon and I've also had some experience with that platform. Though it also means we'll be stuck on Metal 2 functionality for now which is perfectly congruent with the other renderer features at this stage. I'm currently working on translation of the shaders to msl from glsl and hopefully once that milestone is accomplished then it wouldn't be too difficult to get the rest of the renderer working. Though for Mac that might mean a few extra dependencies but they can be easily documented and accounted for on the platform.

In terms of maintenance, I do plan to stick exclusively with Macs for my desktops and also development work so I wouldn't abandon the platform. However like probably everyone here we all have lives but I'd do my best to update and fix bugs/add features if possible. I'll create the renderer and ZWidget backend anyway on my fork and you can decide if you want to use it.

[Redneckerz]

I'm not really sure what you mean by "hard math". Most calculations are just linear equations through matrix math, other mathematical concepts are for things like texture conversion(understanding the architecture endian or intrinsic math) and the only really "hard math" is in advanced rendering like lighting which requires calculus level algorithms(though most are calculated already and the code can be used). I'm mainly using Ai to scaffold a lot of the Apple specific code that I'm not familiar with and also to help with GLSL->SPIR-V->MSL conversion at this stage.

Hard math means actually the maths behind things considering you feel the need to use AI to help you instead of learning the entire thing yourself.

But judging by this and your other response i also feel you are overly ambitious and won't accept anything other than ''i can make it happen'' so... you make it sound like you already have a renderer ready to go, but you don't.

Why don't you start smaller or show something you have made yet? Why instantly go for the moon without building the rocket first?

[Rachael]

Math actually isn't a strong requirement for maintaining rendering backends unless you're writing a software renderer - most of the math is already done in the HWRenderer (which to avoid confusion, is the first renderer point in GZDoom before the actual GL/Vulkan commands - it was split in 4.0.0 - the hwrenderer handles the BSP walk and the sprite positioning before the GL/Vulkan backends which handle the command interface to the GPU drivers).

[johncurley]

Redneckerz wrote:
> [quote=johncurley post_id=1263225 time=1760137394 user_id=41920]
> I'm not really sure what you mean by "hard math". Most calculations are
> just linear equations through matrix math, other mathematical concepts are for things
> like texture conversion(understanding the architecture endian or intrinsic math) and
> the only really "hard math" is in advanced rendering like lighting which
> requires calculus level algorithms(though most are calculated already and the code
> can be used). I'm mainly using Ai to scaffold a lot of the Apple specific code that
> I'm not familiar with and also to help with GLSL->SPIR-V->MSL conversion at
> this stage.
> [/quote]
> Hard math means actually the maths behind things considering you feel the need to use
> AI to help you instead of learning the entire thing yourself.
>
> But judging by this and your other response i also feel you are overly ambitious and
> won't accept anything other than ''i can make it happen'' so... you make it sound
> like you already have a renderer ready to go, but you don't.
>
> Why don't you start smaller or show something you have made yet? Why instantly go for
> the moon without building the rocket first?

Yes you are right and that's good advice. I am sorry to have come across as arrogant, I mostly was excited and enthusiastic about development for the Apple silicon platform and it's potentialities. I completely understand the sentiment, particularly as most game developers don't see macOS as a first class platform and Apple in the past never helped that image.

In terms of knowledge and what I used Ai for, that at this point is mainly objective-c code stub code required to hook metal-cpp into the cocoa framework and translation of engine methods into objective-c. I am lacking knowledge and skill in this area so will start looking at both that and native metal code to understand it better. I have reasonable knowledge of C++ and I have worked on renderers with OpenGL, though mostly fixed pipeline code which is ancient. I understand the concepts of things such as UBO's, though I know modern graphics pipelines can get very complex even for the best of developers and maintaining multiple backends could easily develop into a nightmare.

In terms of what I have developed and starting small, once I get a handle on integrating .mm files into the C++ code I do have a native ZWidget that I could send as a pull request once it's stable. I'm not trying to stick anything I wrote down anyone's throats though or come across as a nuisance given the already large workloads of the developers though if they need a macOS developer I'm willing to help.

[MartinHowe]

Hard math means actually the maths behind things considering you feel the need to use AI to help you instead of learning the entire thing yourself.

But judging by this and your other response i also feel you are overly ambitious and won't accept anything other than ''i can make it happen'' so... you make it sound like you already have a renderer ready to go, but you don't.

Why don't you start smaller or show something you have made yet? Why instantly go for the moon without building the rocket first?

I don't see anything in the OP's statements as arrogant. If he wants to do this let him. He clearly has a game plan in mind. I will judge him by the final results. He makes it clear the AI is for something other than the core rendering, so it's no different than me writing 10000 lines of elite level ZScript but hiring Nash or Neoworm to do the spriting, which is something I regard as just a means to an end.

More importantly, if he's not accepting anything but "I can make it happen", then clearly (as mentioned) he has determination, perseverance, and a roadmap in mind and is likely to get there eventually, though maybe not as quickly as he might like. That isn't arrogance, it's being sure with reason of one's ground.

I speak from experience regarding what those three qualities can achieve, as anyone who knows what a "morph style flag" is will recognise.

[Chris]

Wouldn't it be better to either help improve MoltenVk (a Vulkan -> Metal wrapper), or alternatively adjust GZDoom's use of Vulkan to work better with MoltenVk/Metal's quirks? Improving MoltenVk would benefit more than just GZDoom, and maintaining a few MoltenVk-specific code paths on top of the more widely usable cross-platform Vulkan renderer would probably be a lot easier than maintaining a whole separate renderer that can only be used on a single platform.

[dpJudas]

If you got a working Cocoa backend for ZWidget feel free to PR it. You can already use ZWidget on macOS via its SDL 2 backend, but a native Cocoa one would be preferred once it reaches a similar level of maturity. Even without a Metal renderer it would be useful for GZDoom as it would allow the transition from platform specific launchers to be completed (*).

Regarding writing a Metal renderer, note that porting over the GLSL by hand will only get you partially there as lots of mods come with their own shaders that are also written in GLSL. With the Vulkan backend the MoltenVK layer converts it (GLSL -> SPIRV -> Metal IR whatever that's called), so if you're planning on full support you'd have to find a way to deal with the GLSL coming from mods.

Having worked with Direct3D 9, 11, 12, OpenGL and Vulkan on various projects, it has been my experience that there are no obvious performance benefits to be gained by using the newer ones. You can, theoretically, get better performance with D3D12/Vulkan/Metal, but it depends heavily on re-architecting the entire rendering subsystem to build up command buffers on multiple threads. Simply converting existing commands from an older API to a newer one generally won't do anything. Half the times I've done it I've actually lost performance compared to the older API. Driver heuristics in the older APIs do a lot of stuff that newer APIs simply don't offer (you have to do it yourself), which means stuff was using multiple threads with OpenGL and now it all runs on a single thread on Vulkan and so on. In any case, good luck!

*) Can it already do it with the SDL backend? Don't have an Apple hardware anymore so I didn't try it to find out.

[Graf Zahl]

Having worked with Direct3D 9, 11, 12, OpenGL and Vulkan on various projects, it has been my experience that there are no obvious performance benefits to be gained by using the newer ones. You can, theoretically, get better performance with D3D12/Vulkan/Metal, but it depends heavily on re-architecting the entire rendering subsystem to build up command buffers on multiple threads. Simply converting existing commands from an older API to a newer one generally won't do anything. Half the times I've done it I've actually lost performance compared to the older API. Driver heuristics in the older APIs do a lot of stuff that newer APIs simply don't offer (you have to do it yourself), which means stuff was using multiple threads with OpenGL and now it all runs on a single thread on Vulkan and so on. In any case, good luck!

The real problem is not that the new APIs may be faster but that the old ones are falling out of support. OpenGL never worked well on Macs, AMD and Intel - and Vulkan on Windows also never really delivered on its promises because again driver manufacturers do not care.
TBH, I'd rather invest my time into transitioning GZDoom entirely to SDL3 and using its 3D rendering interface which should map to what each platform supports best. I wouldn't want to write that middleware myself, actually.

[dpJudas]

The real problem is not that the new APIs may be faster but that the old ones are falling out of support. OpenGL never worked well on Macs, AMD and Intel - and Vulkan on Windows also never really delivered on its promises because again driver manufacturers do not care.
TBH, I'd rather invest my time into transitioning GZDoom entirely to SDL3 and using its 3D rendering interface which should map to what each platform supports best. I wouldn't want to write that middleware myself, actually.

I don't have any data to back it up, but my impression has actually been that OpenGL has become rather reliable to use on all platforms except Apple. Probably because Intel switched to using some Mesa solution and their old hardware got so old nobody uses the drivers anymore, solving the original problems we had with the Intel GLSL compiler and so on. That said, you're of course right that its only a matter of time where it be like using Direct3D 9 today. You can, but nothing new ever got added so you're out of luck the moment you want anything that wasn't already there.

I have never looked at SDL3's abstraction so I can't really comment on it. I did however get endless messages on Discord about how amazing WebGPU was and when I finally then took a look at it I was greeted by something that couldn't utilize the RT cores of the GPU. In other words, at least 7 years behind what D3D12 and Vulkan can do. They wanted me use a completely new shading language and their setup stuff was almost as complicated as what Vulkan itself uses. It DID have the advantage of at least validating the setup between shader and pipeline, and they did deal with the image transition insanity from Vulkan, but that was about it. Is SDL 3 better? I have no idea, but if I were to try port GZDoom over to it I'd at least first figure out how they imagine shaders to be compiled. For WebGPU the deal basically was that you had to write your own transpiler or drop support for all current material shaders out in the wild.

[johncurley]

The first time I used GZDoom was on Linux and I remember that Ashes 2063 had quite atmospheric dark areas. When I played the game with the latest stable version on Mac, I thought maybe a change in monitor as I was using the laptop screen was causing issues with the gamma or the monitor was a TN panel and just couldn't get dark enough. However it turns out I managed to get the native version compiling and what a difference it makes! Color profiles and gamma curves seem to be translated accurately over, giving much darker and accurate rendering on macOS given the native window system.

I made some bug fixes to the codebase and it's put me into namespace scoping hell with ZWidget. But once I get it going I'll push my changes and people can see for themselves. I don't know much about SDL3 but I know that while Vulkan has tile rendering support being on mobile GPU's, it's not as tightly integrated as what Metal is on Mac. I'm also having some strange issues like the window system is trying to double load between ZWidget and the native cocoa backend. But I'll have to wait and see until I get everything namespaced properly due to forward declarations and variable name ambiguity.

[dpJudas]

OK I took a quick look at SDL3 since I got curious to the answers to my own questions and here's a few things I found:

• Shader language depends on the backend you're targeting (SPIRV for Vulkan, MSL for Metal, DXIL for Direct3D)
• GPU buffers can't be mapped
• Vertex, fragment and compute shaders are the only shader types supported
• No raytracing and acceleration structure support
• Descriptor sets must be ordered a certain way (set 0 is for vertex shader, set 1 is for fragment shader)
• No triangle fan support

So that means it can't do the persistent mapping current OpenGL and Vulkan backends are doing.
It can't do the raytracing dynlights VKDoom did.
It needs a workaround similar to the one I did for MoltenVK to draw the triangle fans.
It needs the GLSL compiler that the vulkan backend also uses to produce SPIRV, but now the sets are restricted to a certain format (doable).
The GLSL solution only works for when SDL3 targets Vulkan (and actually probably Direct3D 12 in the future as Microsoft is moving D3D12 over to SPIRV).