GZDoom renderer questions

Discuss anything ZDoom-related that doesn't fall into one of the other categories.

Re: GZDoom renderer questions

Postby Redneckerz » Mon Nov 16, 2020 4:27 pm

Graf Zahl wrote:You cannot multithread Doom's game code or the BSP traversal. What I did was the only feasible way to parallelize parts of the render process in OpenGL.

Rum and Raisn Doom suggest otherwise. - https://www.doomworld.com/forum/topic/1 ... go-brrrrr/
User avatar
Redneckerz
To it's ports i may have seen
Spotlight Team
 
Joined: 25 Nov 2019
Discord: Redneckerz#8399
Operating System: Windows Vista/7/2008 64-bit
Graphics Processor: nVidia (Legacy GZDoom)

Re: GZDoom renderer questions

Postby Graf Zahl » Mon Nov 16, 2020 4:42 pm

You cannot parallelize BSP traversal - what you can do is render multiple scenes in parallel. But since that' needs to be a screen space operation it cannot be done with a hardware renderer. If you try to do it in world space you inevitably run into concurrency issues when multiple slices try to access the same map element.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
 
Joined: 19 Jul 2003
Location: Germany

Re: GZDoom renderer questions

Postby SanyaWaffles » Tue Nov 17, 2020 4:05 am

It seems to me alot of people seem to think what works in a software renderer will work in a hardware renderer, and vice versa, when the two have very different ways of doing things. I've noticed this with following the development of Raze, it seems the software rendererer in ye old Build relied on a ton of tricks that simply do not work in a hardware renderer, not that it particularly helps because the software code is really hard to read at best. It also isn't 1:1 to Doom at all, due to Doom using BSPs, and Build using portals and non-euclidian geometry. It's why Raze just relies on a hardware renderer only. (not the only reason I'm sure).

Doom, even with it's original software code being a lot better organized, is a bit different than a hardware renderer, and both require different approaches with dealing with the BSP data and other stuff... thus parallelizing is only possible in other parts of the engine, depending on what renderer is in use.

I mean, GZDoom is an advanced source port, but it's still meant to be distinctly Doom, and Doom has some quirks that are becoming more and more apparent as we push the limit.

I'm not an expert programmer in any sense of the word, but from what I understand it seems that's the general reason why software and hardware renderers have different approaches, and may have different approaches to multithreading. Some parts might be able to be multithreaded and parallelized, but others might not be so easily done, lest stuff get out of sync. There might be some weird way of doing it, but it might be hacky.

What you explain seems to be that exact difference I'm talking about. It seems Rum and Raisin Doom is a software renderer, so it probably will do things differently than a hardware renderer. You can parallelize some parts of a software renderer, and some parts of a hardware renderer, but not necessarily stuff that both would share, like BSP traversal.

Also it doesn't help that some people absolutely insist upon using software as well, which... has a performance hit since it's done mostly on the CPU, hence the name. That'll be a bottleneck for sure. I've never been a purist for how hardware looks compared to software.
User avatar
SanyaWaffles
Wouldn't be an epic gamer if I didn't commit a few war crimes.
 
Joined: 25 Apr 2013
Location: Eastern Ohio
Discord: SanyaWaffles#5095
Twitch ID: sanyawaffles
Operating System: Windows 10/8.1/8/201x 64-bit
OS Test Version: No (Using Stable Public Version)
Graphics Processor: nVidia with Vulkan support

Re: GZDoom renderer questions

Postby Graf Zahl » Tue Nov 17, 2020 4:21 am

Yeah - and when you have a look at the multithreading in GZDoom's software renderer you'll notice that with increasing complexity of the map the performance advantage totally degrades. It only multithreads the actual drawing, but not the map processing which increases exponentially with complexity.

The hardware renderer's BSP traversal is as fast as it can get - the main bottleneck is not CPU performance anymore but cache performance. I've tried several optimizations that looked obvious but they brought nothing. The time got lost just the next time the resources in question were accessed. So the only way to make this faster would be to strip the game off some rarely used features and shrink the main data structures - and we all know how that would end.

And we're still at the point where you need insanely complex and wide open maps to run into these issues. In Frozen Time it is only the crossbeam bridges that make it slow. For testing purposes I once made a version of this map replacing all crossbeam effects with real 3D floors and that alone brought a > 20% performance increase in the most taxing spots.

All that, of course, doesn't mean anything with a mod that spawns thousands of debris actors flying around. That's something the Doom engine cannot handle, plain and simple.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
 
Joined: 19 Jul 2003
Location: Germany

Re: GZDoom renderer questions

Postby SanyaWaffles » Tue Nov 17, 2020 5:48 am

Graf Zahl wrote:All that, of course, doesn't mean anything with a mod that spawns thousands of debris actors flying around. That's something the Doom engine cannot handle, plain and simple.


This is probably why as nice as they are asthetically, I'm going to make the gibs and casings disappear after spawning and their initial effects are done in DD2. Scoot Hard DX all the gibs disappear, like they did in OG Rise of the Triad, and the performance in those two projects is night and day.

I know people probably will hate me for scaling back on the aesthetics, but I want my projects to perform nice. It does no good if my projects look nice but perform like ass.
User avatar
SanyaWaffles
Wouldn't be an epic gamer if I didn't commit a few war crimes.
 
Joined: 25 Apr 2013
Location: Eastern Ohio
Discord: SanyaWaffles#5095
Twitch ID: sanyawaffles
Operating System: Windows 10/8.1/8/201x 64-bit
OS Test Version: No (Using Stable Public Version)
Graphics Processor: nVidia with Vulkan support

Re: GZDoom renderer questions

Postby dpJudas » Tue Nov 17, 2020 9:49 am

Graf Zahl wrote:Yeah - and when you have a look at the multithreading in GZDoom's software renderer you'll notice that with increasing complexity of the map the performance advantage totally degrades. It only multithreads the actual drawing, but not the map processing which increases exponentially with complexity.

Only because I couldn't track down the bugs in the ZDoom renderer that appeared when you restricted the field of view. The feature (r_scene_multithreaded) did improve the overall performance.

I think it is worth mentioning that you can multithread the BSP traversal even for a hardware renderer by splitting it into multiple visible regions. However the catch is that it then must be followed up by a merging step for the (sub)sectors it found visible so that no sector is drawn multiple times.
dpJudas
 
 
 
Joined: 28 May 2016

Re: GZDoom renderer questions

Postby Graf Zahl » Tue Nov 17, 2020 11:07 am

That's where the concurrency issues I mentioned come in. Since overlaps between slices are unavoidable this will inevitably hit some synchronization object, which in a renderer is what we really do not want...

It's definitely not a low hanging fruit as the splitting between BSP traversal and object processing was.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
 
Joined: 19 Jul 2003
Location: Germany

Re: GZDoom renderer questions

Postby Darkcrafter » Tue Nov 17, 2020 3:05 pm

What if BSP traversal could be sped up by training a neural-network per map or just in general? Like adding probability of visibilty for each piece of BSP tree and varying it according to how a neural net would. Like splitting the whole map by a grid of some dimension, let's say 64x64 and recording that probability statistics for each of them individually and skip some amount of such "pieces"? Not sure if that was a smart thing I said :lol:

I also thought if BSP traversal and clipping results could be reused in the future frames if player doesn't move fast, or perhaps performing this operation not every frame? If player doesn't move I'd also say there is a reason to halt bsp traversal and clipping but of course that will bring some inconsistency and performance difference will be different when player moves or shifts its fov.
User avatar
Darkcrafter
 
Joined: 23 Sep 2017
Location: South Russia
Operating System: Windows 10/8.1/8/201x 64-bit
OS Test Version: No (Using Stable Public Version)
Graphics Processor: nVidia with Vulkan support

Re: GZDoom renderer questions

Postby Graf Zahl » Tue Nov 17, 2020 3:49 pm

Sorry, but that won't work. You are totally ignoring the fact that BSP checks are very, very fast - it only becomes a problem if you need to do too many of them. But then your neural network would probably collapse under the load of data and bring things down to a crawl.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
 
Joined: 19 Jul 2003
Location: Germany

Re: GZDoom renderer questions

Postby dpJudas » Tue Nov 17, 2020 3:58 pm

You don't have to synchronize read operations. r_scene_multithreaded only synchronized if a texture had not been loaded yet, and even then it only used a load mutex if it had already read a null pointer (the same optimization that C++ uses for calling constructors for static variables in a function). The output sector visibility merging doesn't have to be run until after the BSP traversal - that part naturally always will have to be serialized.
dpJudas
 
 
 
Joined: 28 May 2016

Re: GZDoom renderer questions

Postby Rachael » Tue Nov 17, 2020 4:45 pm

Darkcrafter wrote:What if BSP traversal could be sped up by training a neural-network per map or just in general? Like adding probability of visibilty for each piece of BSP tree and varying it according to how a neural net would. Like splitting the whole map by a grid of some dimension, let's say 64x64 and recording that probability statistics for each of them individually and skip some amount of such "pieces"? Not sure if that was a smart thing I said :lol:

I also thought if BSP traversal and clipping results could be reused in the future frames if player doesn't move fast, or perhaps performing this operation not every frame? If player doesn't move I'd also say there is a reason to halt bsp traversal and clipping but of course that will bring some inconsistency and performance difference will be different when player moves or shifts its fov.


If GZDoom is lagging, it's way more likely that you're overusing resources than it is something that can easily be fixed on the engine side. That's just the way it is.

You're going to have to scale back on the resource usage of your map or mod - that's just all there is to it. Thousands of maps and mods do perfectly well without persisting on this wild chase of a golden goose for engine-side performance. Use less linedefs in your map, shrink your scripts, and use less actors. Everyone else did it, you can too.

If you absolutely need detail, model the map in a 3D program like Blender and render it as a model in-game, while drawing an overly simplified physics shape of it as your only actual level geometry.
User avatar
Rachael
Webmaster
 
Joined: 13 Jan 2004
Discord: Rachael#3767
Twitch ID: madamerachelle
Github ID: madame-rachelle
Operating System: Windows 10/8.1/8/201x 64-bit
OS Test Version: No (Using Stable Public Version)
Graphics Processor: nVidia with Vulkan support

Re: GZDoom renderer questions

Postby Darkcrafter » Tue Nov 17, 2020 7:31 pm

Alright, thanks for your passion guys. What's regarding 3d modeling then I've been exploiting that trick of converting the map geometry to MD3 and it really adds performance if placed in the areas that are inaccessible by player and pretty much the decorative ones.
User avatar
Darkcrafter
 
Joined: 23 Sep 2017
Location: South Russia
Operating System: Windows 10/8.1/8/201x 64-bit
OS Test Version: No (Using Stable Public Version)
Graphics Processor: nVidia with Vulkan support

Re: GZDoom renderer questions

Postby Graf Zahl » Wed Nov 18, 2020 12:52 am

dpJudas wrote:You don't have to synchronize read operations. r_scene_multithreaded only synchronized if a texture had not been loaded yet, and even then it only used a load mutex if it had already read a null pointer (the same optimization that C++ uses for calling constructors for static variables in a function). The output sector visibility merging doesn't have to be run until after the BSP traversal - that part naturally always will have to be serialized.



Even if we could pull that off, there's one other bit:

Right now the BSP traversal already runs in parallel to the render data generation and with the exception of maps like Frozen Time with its overload of crossbeam bridges it is normally the render data generation step that takes more time, so splitting the BSP traversal into slices won't help that much unless you got 8 cores or more. We still have to consider 4 cores the norm - on Steam this config leads by a wide margin, followed by 6 and 2 cores. So the gains here are limited.

Right now, what may help a lot more would be to parallelize the actual render command generation on Vulkan. Unlike the BSP this could run in parallel to the other two threads and save a lot more time than parallellizing an existing task. On OpenGL this would be futile, of course.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
 
Joined: 19 Jul 2003
Location: Germany

Previous

Return to General

Who is online

Users browsing this forum: fluenCdoom and 3 guests