Asynchronous Vulkan Shader/Pipeline Generation & Portable Cache

[Graf Zahl]

I mostly agree, that's why I pointed to the 3 dynamic_state extensions. At least the first one seems to be core in 1.3

Sorry, forgot to reply on this on specifically (why not use VK_EXT_extended_dynamic_state). In this particular case that would mean VKRenderState would have to issue different commands depending on what render path is active. Doable, sure, but also what exactly are the performance implications of using the extended dynamic state?

Part of the rationale behind creating these huge pipeline state objects in vulkan was that the old model for DX11/OpenGL left too much performance on the floor. It got so bad the drivers begun to create worker threads building pipelines behind the back of the application and a bunch of heuristics deciding when it was worth it and when it wasn't. The more the pipeline is set into "DX11 mode", the further you go back to that world. One really wonderful thing about Vulkan is how changing the pipeline is super fast, easy and predictable compared to OpenGL. I'd personally therefore much prefer an alternative approach, if possible. Especially since most of the state there isn't something GZDoom really does a lot of it - most of it is used for the portal stuff in a limited set of fixed configurations.

I know what the rationale was, and I also know that in particular for NVidia it crippled their hardware because these items are all hardware flags. I think AMD's first GCN generation was the main culprit for this design choice because it implemented too much state as shader instructions but from what I read AMD backpedaled from this design by now for the precise reason that it turned out to be the wrong way of doing things due to the explosion in shader code that needs to be created. If you ask me it's a typical case of micro-optimization gone wrong and missing the forest for the trees, i.e. seeing a minuscule performance improvement and hunting for it while totally neglecting the overhead this induced elsewhere.

Long story short: Since this was made core I think we can assume that the cost of switching (and maintaining) pipelines is far higher than any perceived boost in shader performance.

As for VK_EXT_graphics_pipeline_library, look at the list of contributors. It's not just NVidia, but also AMD, ARM and many game developers. It is clear this is a high-in-demand feature that will get more widespread support because the relevant people need it. Of course the docs cannot give an estimate how hard it will be to implement.

That's also one of the reasons I was surprised that Nvidia was the only vendor with a driver supporting the extensions (added back in the spring). Maybe the others are just being slow, I don't know.

NVidia has always been the first with releasing drivers with new extensions, so nothing new here. AMD has always taken their time with these things, even in OpenGL land.

[dpJudas]

Long story short: Since this was made core I think we can assume that the cost of switching (and maintaining) pipelines is far higher than any perceived boost in shader performance.

I guess that makes sense. One thing is interestingly absent from the dynamic state: blend modes. Seems this particular thing is something they really don't want to be dynamic. Too bad for that is the part that creates the most permutations for us.

[Graf Zahl]

Yeah, I noticed the same. Which makes it all the more annoying that this has never been made a shader feature, because custom blend modes would be cool.
This most likely means that at least one manufacturer still has some dependencies in there - and my guess would be AMD again.

I still think we haven't seen the last of it here. The idea behind PSOs was certainly valid but the way it was implemented turned out not to be what game designers need.
I actually wouldn't be surprised if the end result is the return of much of the API flexibility that OpenGL used to have, but with far better user side control. From what I read shader permutation is currently one of the biggest challenges 3D programmers are facing so chances are high that work is being done here to improve things.

One thing we may do ourselves is adding some pipeline diagnostics, i.e. a CCMD that prints out all currently active pipelines, just to get an idea which ones are being created and maybe making some targeted optimizations to get those pre-initialized.

[0mnicydle]

Whatever path ends up being taken in the future, I just want to say thank you for turning your attention to this issue. I'm just happy it's "on the radar" now.

Also I just want to say that if there's a mitigation, but it would be a ton of work and it still won't address permutations, it might not be worth the effort. If I knew that all I had to do was get through that initial flurry of stutters on the first level of something, I might not have ever complained about this. But in BoA for instance, it seems every level has at least one more shader related stutter at some point, and this is the issue I find most concerning. I verify this by immediately quitting, clearing standby memory, and relaunching the game to see if the stutter reappears. If it doesn't, I feel I can safely say it's shader related as if it was an asset or entity related stutter it would have come back, especially after clearing standby memory. And sometimes I even go the extra mile and clear the shader cache and see if that particular stutter returns in the same exact spot, and every time I have gone this extra length it has proven true. I don't know, but I strongly suspect these ongoing stutters are permutation related just from paying attention to what's in the scene when it happens; I think it's often related to transparent things, like every transparent sprite/texture needs its own permutation or something (or maybe one for every new alpha value)? Just wild speculation on my part here, sorry...

Now also just let me say that last night I came across something interesting... apparently there is a contingent of people using DXVK (on Windows) in those DX games that didn't go through "heroics" to eliminate shader related stutter, because it uses async shader compile. So I'm wondering, could async shader compile be applied selectively, or is it all or nothing? Just wondering if you could use async on all the low hanging fruit to avoid the significant time investment that was among the earliest objections to the idea, and if that would be able to cover these permutations? I feel like of all things, a transparency effect not being present for a frame or two would be among the least noticeable things, but then again maybe in reality what would actually happen is the the object just wouldn't be visible at all? I don't know... All I know is for myself, I would much prefer "pop-in" to stutter. When Rage first released and it was capped at 60fps and had texture pop-in as a result of strictly budgeted streaming operations, I remember much weeping and gnashing of teeth about it, but I thought it was absolutely glorious compared to the non-stop stutter fest of the typical Unreal engine game at the time (and sadly mostly to this day).

[Chris]

That's also one of the reasons I was surprised that Nvidia was the only vendor with a driver supporting the extensions (added back in the spring). Maybe the others are just being slow, I don't know.

FWIW, it is available on my RX 580 on Linux, using the open source drivers and MESA 22.3:

Code: Select all

$ vulkaninfo | grep library
        VK_KHR_pipeline_library                     : extension revision 1
        VK_EXT_graphics_pipeline_library             : extension revision 1
        VK_KHR_pipeline_library                      : extension revision 1