Some ideas on optimizing rendering of big maps

[Darkcrafter]

Well, I know for being some PITA in a while just for asking features but I though it might worth posting some ideas so keepers tell if they're good or not.

So, rendering big maps like these (please don't treat it like an ad!) is kind of tough for the engine.
So these are some ideas that could be taken into consideration:

1) Render skybox less often, e.g. with less FPS and with less resolution (Map16 has a really tough one!);
2) Divide the main camera rendering to close and far fields, with the latter being also rendered with less FPS and smaller resolution, albeit, upscaled back via box or linear filtering;
3) Render floor/ceilings reflections with lower resolution (GTA SA does this), maybe also with less FPS;
4) Not an optimization related one - render a low res and fps version from the player position in a 360 degree panorama fashion, so this could be used as a dynamic environment map for reflections on surfaces and 3D models like PBRs, AFAIK, isn't this what GTA 5 does?

[phantombeta]

GZDoom is CPU-bound, not GPU-bound, so these ideas would generally make performance worse instead. The "rendering at lower FPS" ones would also look particularly choppy and laggy.

1) Render skybox less often, e.g. with less FPS and with less resolution (Map16 has a really tough one!);

That'd look pretty bad for not much improvement in performance. The skybox would look extremely laggy when you turn.

2) Divide the main camera rendering to close and far fields, with the latter being also rendered with less FPS and smaller resolution, albeit, upscaled back via box or linear filtering;

That wouldn't help. You'd be doubling the number of drawcalls which would put more load on the CPU and further slow the game down. (GZDoom can't just "render at less FPS", the way it works, the whole scene has to be rendered every frame)

3) Render floor/ceilings reflections with lower resolution (GTA SA does this), maybe also with less FPS;

The cost of rendering reflections is largely drawcalls, not GPU power, so this wouldn't help much outside of good CPUs being used with anemic GPUs.

4) Not an optimization related one - render a low res and fps version from the player position in a 360 degree panorama fashion, so this could be used as a dynamic environment map for reflections on surfaces and 3D models like PBRs, AFAIK, isn't this what GTA 5 does?

That'd be quite a massive performance hit, because you'd need to render 4 to 6 different views (either a tetrahedral or a cube map) and each of them would need to do the whole rendering process from the beginning. It wouldn't work particularly well for anything that isn't basically centered around the player either, because the environment map would look more and more wrong the further away from the player an object is.
GTA 5 only does this for your car's reflections, everything else uses normal, non-realtime cubemaps.

[Darkcrafter]

Got it, thank you for such a thorough explanation!

[Darkcrafter]

So all this time I've been trying to modify lzdoom. I could install visual studio 2017, convert the source cmake project to sln, compile and run it nicely but I was eager to try modifying it. I don't code in C++ so I use ChatGPT4. Lately my attempts to debug crashing lzdoom succeeded finaly but usualy it won't matter since the results of such changes are utterly bad.

So I was wandering across the source codes and stumbled across an interesting couple of lines in p_tick.cpp:

Code: Select all

	// [RH] Frozen mode is only changed every 4 tics, to make it work with A_Tracer().
	if ((level.maptime & 3) == 0)
	{
		if (bglobal.changefreeze)
		{
			level.frozenstate ^= 2;
			bglobal.freeze ^= 1;
			bglobal.changefreeze = 0;
		}
	}

I gave these lines to ChatGPT and also gave it some of gl_bsp.cpp contents so that it could adapt them to make it work in the same fashion.
It gives the renderer a stroboscope like effect (I guess it's extremely unrecommended to epileptical people!).

Code: Select all

// 
//---------------------------------------------------------------------------
//
// Copyright(C) 2000-2016 Christoph Oelckers
// All rights reserved.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Lesser General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
// GNU Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public License
// along with this program.  If not, see http://www.gnu.org/licenses/
//
//--------------------------------------------------------------------------
//
/*
** gl_bsp.cpp
** Main rendering loop / BSP traversal / visibility clipping
**
**/

#include "p_lnspec.h"
#include "p_local.h"
#include "a_sharedglobal.h"
#include "g_levellocals.h"
#include "r_sky.h"
#include "p_effect.h"
#include "po_man.h"
#include "doomdata.h"
#include "g_levellocals.h"

#include "gl/renderer/gl_renderer.h"
#include "gl/data/gl_data.h"
#include "gl/data/gl_vertexbuffer.h"
#include "gl/scene/gl_scenedrawer.h"
#include "gl/scene/gl_portal.h"
#include "gl/scene/gl_wall.h"
#include "gl/utility/gl_clock.h"

EXTERN_CVAR(Bool, gl_render_segs)

CVAR(Bool, gl_render_things, true, 0)
CVAR(Bool, gl_render_walls, true, 0)
CVAR(Bool, gl_render_flats, true, 0)

void GLSceneDrawer::UnclipSubsector(subsector_t *sub)
{
	int count = sub->numlines;
	seg_t * seg = sub->firstline;

	while (count--)
	{
		angle_t startAngle = clipper.GetClipAngle(seg->v2);
		angle_t endAngle = clipper.GetClipAngle(seg->v1);

		// Back side, i.e. backface culling	- read: endAngle >= startAngle!
		if (startAngle-endAngle >= ANGLE_180)  
		{
			clipper.SafeRemoveClipRange(startAngle, endAngle);
			clipper.SetBlocked(false);
		}
		seg++;
	}
}

//==========================================================================
//
// R_AddLine
// Clips the given segment
// and adds any visible pieces to the line list.
//
//==========================================================================

CVAR(Float, gl_line_distance_cull, 8000.0, CVAR_ARCHIVE|CVAR_GLOBALCONFIG)

inline bool IsDistanceCulled(seg_t *line)
{
     if (line->linedef->flags & ML_NEVERCULL) // Keep linedefs with this flag
                                         // unculled under any circumstances
     {
       return false;
     }

	    double dist3 = gl_line_distance_cull * gl_line_distance_cull;
	    if (dist3 <= 0.0)
		    return false;

	    double dist1 = (line->v1->fPos() - r_viewpoint.Pos).LengthSquared();
	    double dist2 = (line->v2->fPos() - r_viewpoint.Pos).LengthSquared();
	    if ((dist1 > dist3) && (dist2 > dist3))
		    return true;
	    return false;
}

void GLSceneDrawer::AddLine(seg_t *seg, bool portalclip)
{
	// [RH] Frozen mode is only changed every 4 tics, to make it work with A_Tracer().
	// We now only execute freeze logic every four ticks. The rest of the time the map is "frozen"

	// This will check if we are *NOT* on a tick that is a multiple of 4.
	if ((level.maptime & 3) != 0)
	{
		// If not a multiple of 4, then skip the rest of the freezing logic
		return;
	}

	// If we made it here, it means maptime is on a tick that is a multiple of 4, and we can process the line

#ifdef _DEBUG
	if (seg->linedef->Index() == 38)
	{
		int a = 0;
	}
#endif

	sector_t * backsector = NULL;

	if (portalclip)
	{
		int clipres = GLRenderer->mClipPortal->ClipSeg(seg);
		if (clipres == GLPortal::PClip_InFront) return;
	}

	angle_t startAngle = clipper.GetClipAngle(seg->v2);
	angle_t endAngle = clipper.GetClipAngle(seg->v1);

	// Back side, i.e. backface culling	- read: endAngle >= startAngle!
	if (startAngle - endAngle < ANGLE_180)
	{
		return;
	}

	if (seg->sidedef == NULL)
	{
		if (!(currentsubsector->flags & SSECMF_DRAWN))
		{
			if (clipper.SafeCheckRange(startAngle, endAngle))
			{
				currentsubsector->flags |= SSECMF_DRAWN;
			}
		}
		return;
	}

	if (!clipper.SafeCheckRange(startAngle, endAngle))
	{
		return;
	}
	currentsubsector->flags |= SSECMF_DRAWN;

	uint8_t ispoly = uint8_t(seg->sidedef->Flags & WALLF_POLYOBJ);

	if (IsDistanceCulled(seg))
	{
		GLWall wall(this);
		wall.sub = currentsubsector;
		wall.Process(seg, seg->frontsector, seg->backsector, true);
		clipper.SafeAddClipRange(startAngle, endAngle);
		return;
	}

	if (!seg->backsector)
	{
		clipper.SafeAddClipRange(startAngle, endAngle);
	}
	else if (!ispoly)	// Two-sided polyobjects never obstruct the view
	{
		if (currentsector->sectornum == seg->backsector->sectornum)
		{
			if (!seg->linedef->isVisualPortal())
			{
				FTexture * tex = TexMan(seg->sidedef->GetTexture(side_t::mid));
				if (!tex || tex->UseType == ETextureType::Null)
				{
					// nothing to do here!
					seg->linedef->validcount = validcount;
					return;
				}
			}
			backsector = currentsector;
		}
		else
		{
			// clipping checks are only needed when the backsector is not the same as the front sector
			if (in_area == area_default) in_area = CheckViewArea(seg->v1, seg->v2, seg->frontsector, seg->backsector);

			backsector = gl_FakeFlat(seg->backsector, in_area, true);

			if (gl_CheckClip(seg->sidedef, currentsector, backsector))
			{
				clipper.SafeAddClipRange(startAngle, endAngle);
			}
		}
	}
	else
	{
		// Backsector for polyobj segs is always the containing sector itself
		backsector = currentsector;
	}

	seg->linedef->flags |= ML_MAPPED;

	if (ispoly || seg->linedef->validcount != validcount)
	{
		if (!ispoly) seg->linedef->validcount = validcount;

		if (gl_render_walls)
		{
			SetupWall.Clock();

			GLWall wall(this);
			wall.sub = currentsubsector;
			wall.Process(seg, currentsector, backsector);
			rendered_lines++;

			SetupWall.Unclock();
		}
	}
}

//==========================================================================
//
// R_Subsector
// Determine floor/ceiling planes.
// Add sprites of things in sector.
// Draw one or more line segments.
//
//==========================================================================

void GLSceneDrawer::PolySubsector(subsector_t * sub)
{
	int count = sub->numlines;
	seg_t * line = sub->firstline;

	while (count--)
	{
		if (line->linedef)
		{
			AddLine (line, GLRenderer->mClipPortal != NULL);
		}
		line++;
	}
}

//==========================================================================
//
// RenderBSPNode
// Renders all subsectors below a given node,
//  traversing subtree recursively.
// Just call with BSP root.
//
//==========================================================================

void GLSceneDrawer::RenderPolyBSPNode (void *node)
{
	while (!((size_t)node & 1))  // Keep going until found a subsector
	{
		node_t *bsp = (node_t *)node;

		// Decide which side the view point is on.
		int side = R_PointOnSide(viewx, viewy, bsp);

		// Recursively divide front space (toward the viewer).
		RenderPolyBSPNode (bsp->children[side]);

		// Possibly divide back space (away from the viewer).
		side ^= 1;

		// It is not necessary to use the slower precise version here
		if (!clipper.CheckBox(bsp->bbox[side]))
		{
			return;
		}

		node = bsp->children[side];
	}
	PolySubsector ((subsector_t *)((uint8_t *)node - 1));
}

//==========================================================================
//
// Unlilke the software renderer this function will only draw the walls,
// not the flats. Those are handled as a whole by the parent subsector.
//
//==========================================================================

void GLSceneDrawer::AddPolyobjs(subsector_t *sub)
{
	if (sub->BSP == NULL || sub->BSP->bDirty)
	{
		sub->BuildPolyBSP();
		for (unsigned i = 0; i < sub->BSP->Segs.Size(); i++)
		{
			sub->BSP->Segs[i].Subsector = sub;
			sub->BSP->Segs[i].PartnerSeg = NULL;
		}
	}
	if (sub->BSP->Nodes.Size() == 0)
	{
		PolySubsector(&sub->BSP->Subsectors[0]);
	}
	else
	{
		RenderPolyBSPNode(&sub->BSP->Nodes.Last());
	}
}


//==========================================================================
//
//
//
//==========================================================================

void GLSceneDrawer::AddLines(subsector_t * sub, sector_t * sector)
{
	currentsector = sector;
	currentsubsector = sub;

	ClipWall.Clock();
	if (sub->polys != NULL)
	{
		AddPolyobjs(sub);
	}
	else
	{
		int count = sub->numlines;
		seg_t * seg = sub->firstline;

		while (count--)
		{
			if (seg->linedef == NULL)
			{
				if (!(sub->flags & SSECMF_DRAWN)) AddLine (seg, GLRenderer->mClipPortal != NULL);
			}
			else if (!(seg->sidedef->Flags & WALLF_POLYOBJ)) 
			{
				AddLine (seg, GLRenderer->mClipPortal != NULL);
			}
			seg++;
		}
	}
	ClipWall.Unclock();
}

//==========================================================================
//
// Adds lines that lie directly on the portal boundary.
// Only two-sided lines will be handled here, and no polyobjects
//
//==========================================================================

inline bool PointOnLine(const DVector2 &pos, const line_t *line)
{
	double v = (pos.Y - line->v1->fY()) * line->Delta().X + (line->v1->fX() - pos.X) * line->Delta().Y;
	return fabs(v) <= EQUAL_EPSILON;
}

void GLSceneDrawer::AddSpecialPortalLines(subsector_t * sub, sector_t * sector, line_t *line)
{
	currentsector = sector;
	currentsubsector = sub;

	ClipWall.Clock();
	int count = sub->numlines;
	seg_t * seg = sub->firstline;

	while (count--)
	{
		if (seg->linedef != NULL && seg->PartnerSeg != NULL)
		{
			if (PointOnLine(seg->v1->fPos(), line) && PointOnLine(seg->v2->fPos(), line))
				AddLine(seg, false);
		}
		seg++;
	}
	ClipWall.Unclock();
}


//==========================================================================
//
// R_RenderThings
//
//==========================================================================

void GLSceneDrawer::RenderThings(subsector_t * sub, sector_t * sector)
{

	// [RH] Frozen mode is only changed every 4 tics, to make it work with A_Tracer().
// We now only execute freeze logic every four ticks. The rest of the time the map is "frozen"

// This will check if we are *NOT* on a tick that is a multiple of 4.
	if ((level.maptime & 3) != 0)
	{
		// If not a multiple of 4, then skip the rest of the freezing logic
		return;
	}

	// If we made it here, it means maptime is on a tick that is a multiple of 4, and we can process the line

	SetupSprite.Clock();
	sector_t * sec=sub->sector;
	// Handle all things in sector.
	for (auto p = sec->touching_renderthings; p != nullptr; p = p->m_snext)
	{
		auto thing = p->m_thing;
		if (thing->validcount == validcount) continue;
		thing->validcount = validcount;

		FIntCVar *cvar = thing->GetInfo()->distancecheck;
		if (cvar != NULL && *cvar >= 0)
		{
			double dist = (thing->Pos() - r_viewpoint.Pos).LengthSquared();
			double check = (double)**cvar;
			if (dist >= check * check)
			{
				continue;
			}
		}

		GLSprite sprite(this);
		sprite.Process(thing, sector, false);
	}
	
	for (msecnode_t *node = sec->sectorportal_thinglist; node; node = node->m_snext)
	{
		AActor *thing = node->m_thing;
		FIntCVar *cvar = thing->GetInfo()->distancecheck;
		if (cvar != NULL && *cvar >= 0)
		{
			double dist = (thing->Pos() - r_viewpoint.Pos).LengthSquared();
			double check = (double)**cvar;
			if (dist >= check * check)
			{
				continue;
			}
		}

		GLSprite sprite(this);
		sprite.Process(thing, sector, true);
	}
	SetupSprite.Unclock();
}


//==========================================================================
//
// R_Subsector
// Determine floor/ceiling planes.
// Add sprites of things in sector.
// Draw one or more line segments.
//
//==========================================================================

void GLSceneDrawer::DoSubsector(subsector_t * sub)
{
	unsigned int i;
	sector_t * sector;
	sector_t * fakesector;
	
#ifdef _DEBUG
	if (sub->sector->sectornum==931)
	{
		int a = 0;
	}
#endif

	sector=sub->sector;
	if (!sector) return;

	// If the mapsections differ this subsector can't possibly be visible from the current view point
	if (!(currentmapsection[sub->mapsection>>3] & (1 << (sub->mapsection & 7)))) return;
	if (sub->flags & SSECF_POLYORG) return;	// never render polyobject origin subsectors because their vertices no longer are where one may expect.

	if (gl_drawinfo->ss_renderflags[sub->Index()] & SSRF_SEEN)
	{
		// This means that we have reached a subsector in a portal that has been marked 'seen'
		// from the other side of the portal. This means we must clear the clipper for the
		// range this subsector spans before going on.
		UnclipSubsector(sub);
	}
	if (clipper.IsBlocked()) return;	// if we are inside a stacked sector portal which hasn't unclipped anything yet.

	fakesector=gl_FakeFlat(sector, in_area, false);

	if (GLRenderer->mClipPortal)
	{
		int clipres = GLRenderer->mClipPortal->ClipSubsector(sub);
		if (clipres == GLPortal::PClip_InFront)
		{
			line_t *line = GLRenderer->mClipPortal->ClipLine();
			// The subsector is out of range, but we still have to check lines that lie directly on the boundary and may expose their upper or lower parts.
			if (line) AddSpecialPortalLines(sub, fakesector, line);
			return;
		}
	}

	if (sector->validcount != validcount)
	{
		GLRenderer->mVBO->CheckUpdate(sector);
	}

	// [RH] Add particles
	//int shade = LIGHT2SHADE((floorlightlevel + ceilinglightlevel)/2 + r_actualextralight);
	if (gl_render_things)
	{
		SetupSprite.Clock();

		for (i = ParticlesInSubsec[sub->Index()]; i != NO_PARTICLE; i = Particles[i].snext)
		{
			GLSprite sprite(this);
			sprite.ProcessParticle(&Particles[i], fakesector);
		}
		SetupSprite.Unclock();
	}

	AddLines(sub, fakesector);

	// BSP is traversed by subsector.
	// A sector might have been split into several
	//	subsectors during BSP building.
	// Thus we check whether it was already added.
	if (sector->validcount != validcount)
	{
		// Well, now it will be done.
		sector->validcount = validcount;

		if (gl_render_things)
		{
			RenderThings(sub, fakesector);
		}
		sector->MoreFlags |= SECMF_DRAWN;
	}

	if (gl_render_flats)
	{

		// [RH] Frozen mode is only changed every 4 tics, to make it work with A_Tracer().
// We now only execute freeze logic every four ticks. The rest of the time the map is "frozen"

// This will check if we are *NOT* on a tick that is a multiple of 4.
		if ((level.maptime & 3) != 0)
		{
			// If not a multiple of 4, then skip the rest of the freezing logic
			return;
		}

		// If we made it here, it means maptime is on a tick that is a multiple of 4, and we can process the line

		// Subsectors with only 2 lines cannot have any area
		if (sub->numlines>2 || (sub->hacked&1)) 
		{
			// Exclude the case when it tries to render a sector with a heightsec
			// but undetermined heightsec state. This can only happen if the
			// subsector is obstructed but not excluded due to a large bounding box.
			// Due to the way a BSP works such a subsector can never be visible
			if (!sector->GetHeightSec() || in_area!=area_default)
			{
				if (sector != sub->render_sector)
				{
					sector = sub->render_sector;
					// the planes of this subsector are faked to belong to another sector
					// This means we need the heightsec parts and light info of the render sector, not the actual one.
					fakesector = gl_FakeFlat(sector, in_area, false);
				}

				uint8_t &srf = gl_drawinfo->sectorrenderflags[sub->render_sector->sectornum];
				if (!(srf & SSRF_PROCESSED))
				{
					srf |= SSRF_PROCESSED;

					SetupFlat.Clock();
					GLFlat flat(this);
					flat.ProcessSector(fakesector);
					SetupFlat.Unclock();
				}
				// mark subsector as processed - but mark for rendering only if it has an actual area.
				gl_drawinfo->ss_renderflags[sub->Index()] = 
					(sub->numlines > 2) ? SSRF_PROCESSED|SSRF_RENDERALL : SSRF_PROCESSED;
				if (sub->hacked & 1) gl_drawinfo->AddHackedSubsector(sub);

				FPortal *portal;

				portal = fakesector->GetGLPortal(sector_t::ceiling);
				if (portal != NULL)
				{
					GLSectorStackPortal *glportal = portal->GetRenderState();
					glportal->AddSubsector(sub);
				}

				portal = fakesector->GetGLPortal(sector_t::floor);
				if (portal != NULL)
				{
					GLSectorStackPortal *glportal = portal->GetRenderState();
					glportal->AddSubsector(sub);
				}
			}
		}
	}
}




//==========================================================================
//
// RenderBSPNode
// Renders all subsectors below a given node,
//  traversing subtree recursively.
// Just call with BSP root.
//
//==========================================================================

void GLSceneDrawer::RenderBSPNode (void *node)
{
	if (level.nodes.Size() == 0)
	{
		DoSubsector (&level.subsectors[0]);
		return;
	}
	while (!((size_t)node & 1))  // Keep going until found a subsector
	{
		node_t *bsp = (node_t *)node;

		// Decide which side the view point is on.
		int side = R_PointOnSide(viewx, viewy, bsp);

		// Recursively divide front space (toward the viewer).
		RenderBSPNode (bsp->children[side]);

		// Possibly divide back space (away from the viewer).
		side ^= 1;

		// It is not necessary to use the slower precise version here
		if (!clipper.CheckBox(bsp->bbox[side]))
		{
			if (!(gl_drawinfo->no_renderflags[bsp->Index()] & SSRF_SEEN))
				return;
		}

		node = bsp->children[side];
	}
	DoSubsector ((subsector_t *)((uint8_t *)node - 1));
}

For sure I tried modifying a numerous amount of rendering files also modifying others in order to make it compile like all gl_ files and my conclusion is that gl_bsp.cpp being the biggest bottleneck here as it creates a CPU overhead that gets larger with larger levels. If you apply the same "throttling" logic to gl_scene.cpp it's going to flicker too but it's not going to increase FPS.

So by modifying gl_bsp.cpp this way, the FPS increase is like from 3 to 4 times. I tried lots of ways to "interpolate" between traversed data but always failed, including modifications to gl_scene.cpp as it made the game extremely choppy. I guess the only option to go if I'd want to go that way further, is to create a BSP caching system. I think it's something that my current free GPT plan won't allow as that would require to scan many source files and modify just as much, then fix errors and debug and it's getting extremely tedious and long.


Here's the source that compiles and builds (based of LZDoom v3.87c)

What do you think?

[Graf Zahl]

That won't really help much because a) you are not saving time evenly across frames and b) you will use outdated clipping information for the intermediate frames which may result in render glitches.

The problems to be solved here are absolutely non-trivial, otherwise several people far more experienced than you wouldn't have produced any results so far.

[Darkcrafter]

That won't really help much because a) you are not saving time evenly across frames and b) you will use outdated clipping information for the intermediate frames which may result in render glitches.

The problems to be solved here are absolutely non-trivial, otherwise several people far more experienced than you wouldn't have produced any results so far.

Sure, not saying I'm the smartest. What I’ve done so far amounts to finding the source, compiling it, doing some basic studying and a little bit of editing. But even if those glitches would be considered "okayish" for low quality "preset" could it work out at least like "I did what I could to make it playable with some compromise as you deliberately chosen it and you were warned" in the end?

My mapset already includes a bat files based launcher that allows choosing graphics quality preset and I could provide the resulting build and sources along with it and that could help me to concentrate on mapping more.

[Darkcrafter]

I don't know if maybe somebody wanted to pluck me whether it's indeed true or not but seems like adding a validcount in some gl_scene.cpp functions "void GLSceneDrawer::RenderScene(int recursion)" and sector_t * GLSceneDrawer::RenderViewpoint (AActor * camera, GL_IRECT * bounds, float fov, float ratio, float fovratio, bool mainview, bool toscreen) - gives up to 43% FPS boost (R7 5700x, RTX3060, 28FPS vs 40FPS on Map13 beginning, with distance culling disabled)

Let's introduce that variable in gl_scenedrawer.h:

Code: Select all

#pragma once

#include "r_defs.h"
#include "m_fixed.h"
#include "gl_clipper.h"
#include "gl_portal.h"
#include "gl/renderer/gl_lightdata.h"
#include "gl/renderer/gl_renderer.h"

extern unsigned int gl_global_validcount;

Let's add a variable "unsigned int gl_global_validcount = 1;" in the beginning of gl_scene.cpp

Code: Select all

//==========================================================================
//
// CVARs
//
//==========================================================================
CVAR(Bool, gl_texture, true, 0)
CVAR(Bool, gl_no_skyclear, false, CVAR_ARCHIVE|CVAR_GLOBALCONFIG)
CVAR(Float, gl_mask_threshold, 0.5f,CVAR_ARCHIVE|CVAR_GLOBALCONFIG)
CVAR(Float, gl_mask_sprite_threshold, 0.5f,CVAR_ARCHIVE|CVAR_GLOBALCONFIG)
CVAR(Bool, gl_sort_textures, false, CVAR_ARCHIVE|CVAR_GLOBALCONFIG)

EXTERN_CVAR (Bool, cl_capfps)
EXTERN_CVAR (Bool, r_deathcamera)
EXTERN_CVAR (Float, underwater_fade_scalar)
EXTERN_CVAR (Float, r_visibility)
EXTERN_CVAR (Bool, gl_legacy_mode)
EXTERN_CVAR (Bool, r_drawvoxels)

extern bool NoInterpolateView;

area_t			in_area;
TArray<uint8_t> currentmapsection;
int camtexcount;

unsigned int gl_global_validcount = 1;

Then add it to one of the functions:

Code: Select all

//-----------------------------------------------------------------------------
//
// RenderScene
//
// Draws the current draw lists for the non GLSL renderer
//
//-----------------------------------------------------------------------------

void GLSceneDrawer::RenderScene(int recursion)
{
	gl_global_validcount++;

	RenderAll.Clock();

	glDepthMask(true);
	if (!gl_no_skyclear) GLPortal::RenderFirstSkyPortal(recursion);

	gl_RenderState.SetCameraPos(r_viewpoint.Pos.X, r_viewpoint.Pos.Y, r_viewpoint.Pos.Z);

	gl_RenderState.EnableFog(true);
	gl_RenderState.BlendFunc(GL_ONE,GL_ZERO);

	if (gl_sort_textures)
	{
		gl_drawinfo->drawlists[GLDL_PLAINWALLS].SortWalls();
		gl_drawinfo->drawlists[GLDL_PLAINFLATS].SortFlats();
		gl_drawinfo->drawlists[GLDL_MASKEDWALLS].SortWalls();
		gl_drawinfo->drawlists[GLDL_MASKEDFLATS].SortFlats();
		gl_drawinfo->drawlists[GLDL_MASKEDWALLSOFS].SortWalls();
	}

	// if we don't have a persistently mapped buffer, we have to process all the dynamic lights up front,
	// so that we don't have to do repeated map/unmap calls on the buffer.
	bool haslights = GLRenderer->mLightCount > 0 && FixedColormap == CM_DEFAULT && gl_lights;
	if (gl.lightmethod == LM_DEFERRED && haslights)
	{
		GLRenderer->mLights->Begin();
		gl_drawinfo->drawlists[GLDL_PLAINWALLS].DrawWalls(GLPASS_LIGHTSONLY);
		gl_drawinfo->drawlists[GLDL_PLAINFLATS].DrawFlats(GLPASS_LIGHTSONLY);
		gl_drawinfo->drawlists[GLDL_MASKEDWALLS].DrawWalls(GLPASS_LIGHTSONLY);
		gl_drawinfo->drawlists[GLDL_MASKEDFLATS].DrawFlats(GLPASS_LIGHTSONLY);
		gl_drawinfo->drawlists[GLDL_MASKEDWALLSOFS].DrawWalls(GLPASS_LIGHTSONLY);
		gl_drawinfo->drawlists[GLDL_TRANSLUCENTBORDER].Draw(GLPASS_LIGHTSONLY);
		gl_drawinfo->drawlists[GLDL_TRANSLUCENT].Draw(GLPASS_LIGHTSONLY, true);
		gl_drawinfo->drawlists[GLDL_MODELS].Draw(GLPASS_LIGHTSONLY);
		SetupWeaponLight();
		GLRenderer->mLights->Finish();
	}

	// Part 1: solid geometry. This is set up so that there are no transparent parts
	glDepthFunc(GL_LESS);
	gl_RenderState.AlphaFunc(GL_GEQUAL, 0.f);
	glDisable(GL_POLYGON_OFFSET_FILL);

	int pass;

	if (!haslights || gl.lightmethod == LM_DEFERRED)
	{
		pass = GLPASS_PLAIN;
	}
	else if (gl.lightmethod == LM_DIRECT)
	{
		pass = GLPASS_ALL;
	}
	else // GL 2.x legacy mode
	{
		// process everything that needs to handle textured dynamic lights.
		if (haslights) RenderMultipassStuff();

		// The remaining lists which are unaffected by dynamic lights are just processed as normal.
		pass = GLPASS_PLAIN;
	}

	gl_RenderState.EnableTexture(gl_texture);
	gl_RenderState.EnableBrightmap(true);
	gl_drawinfo->drawlists[GLDL_PLAINWALLS].DrawWalls(pass);
	gl_drawinfo->drawlists[GLDL_PLAINFLATS].DrawFlats(pass);


	// Part 2: masked geometry. This is set up so that only pixels with alpha>gl_mask_threshold will show
	if (!gl_texture) 
	{
		gl_RenderState.EnableTexture(true);
		gl_RenderState.SetTextureMode(TM_MASK);
	}
	gl_RenderState.AlphaFunc(GL_GEQUAL, gl_mask_threshold);
	gl_drawinfo->drawlists[GLDL_MASKEDWALLS].DrawWalls(pass);
	gl_drawinfo->drawlists[GLDL_MASKEDFLATS].DrawFlats(pass);

	// Part 3: masked geometry with polygon offset. This list is empty most of the time so only waste time on it when in use.
	if (gl_drawinfo->drawlists[GLDL_MASKEDWALLSOFS].Size() > 0)
	{
		glEnable(GL_POLYGON_OFFSET_FILL);
		glPolygonOffset(-1.0f, -128.0f);
		gl_drawinfo->drawlists[GLDL_MASKEDWALLSOFS].DrawWalls(pass);
		glDisable(GL_POLYGON_OFFSET_FILL);
		glPolygonOffset(0, 0);
	}

	gl_drawinfo->drawlists[GLDL_MODELS].Draw(pass);

	gl_RenderState.BlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);

	// Part 4: Draw decals (not a real pass)
	glDepthFunc(GL_LEQUAL);
	glEnable(GL_POLYGON_OFFSET_FILL);
	glPolygonOffset(-1.0f, -128.0f);
	glDepthMask(false);

	// this is the only geometry type on which decals can possibly appear
	gl_drawinfo->drawlists[GLDL_PLAINWALLS].DrawDecals();
	if (gl.legacyMode)
	{
		// also process the render lists with walls and dynamic lights
        gl_drawinfo->dldrawlists[GLLDL_WALLS_PLAIN].DrawDecals();
        gl_drawinfo->dldrawlists[GLLDL_WALLS_FOG].DrawDecals();
	}

	gl_RenderState.SetTextureMode(TM_MODULATE);

	glDepthMask(true);


	// Push bleeding floor/ceiling textures back a little in the z-buffer
	// so they don't interfere with overlapping mid textures.
	glPolygonOffset(1.0f, 128.0f);

	// Part 5: flood all the gaps with the back sector's flat texture
	// This will always be drawn like GLDL_PLAIN, depending on the fog settings
	
	glDepthMask(false);							// don't write to Z-buffer!
	gl_RenderState.EnableFog(true);
	gl_RenderState.AlphaFunc(GL_GEQUAL, 0.f);
	gl_RenderState.BlendFunc(GL_ONE,GL_ZERO);
	gl_drawinfo->DrawUnhandledMissingTextures();
	glDepthMask(true);

	glPolygonOffset(0.0f, 0.0f);
	glDisable(GL_POLYGON_OFFSET_FILL);
	RenderAll.Unclock();

}

... and add it to another function of gl_scene.cpp

Code: Select all

//-----------------------------------------------------------------------------
//
// Renders one viewpoint in a scene
//
//-----------------------------------------------------------------------------

sector_t * GLSceneDrawer::RenderViewpoint (AActor * camera, GL_IRECT * bounds, float fov, float ratio, float fovratio, bool mainview, bool toscreen)
{
	gl_global_validcount++;

	sector_t * lviewsector;
	GLRenderer->mSceneClearColor[0] = 0.0f;
	GLRenderer->mSceneClearColor[1] = 0.0f;
	GLRenderer->mSceneClearColor[2] = 0.0f;
	R_SetupFrame (r_viewpoint, r_viewwindow, camera);
	SetViewArea();

	GLRenderer->mGlobVis = R_GetGlobVis(r_viewwindow, r_visibility);

	// We have to scale the pitch to account for the pixel stretching, because the playsim doesn't know about this and treats it as 1:1.
	double radPitch = r_viewpoint.Angles.Pitch.Normalized180().Radians();
	double angx = cos(radPitch);
	double angy = sin(radPitch) * level.info->pixelstretch;
	double alen = sqrt(angx*angx + angy*angy);

	GLRenderer->mAngles.Pitch = (float)RAD2DEG(asin(angy / alen));
	GLRenderer->mAngles.Roll.Degrees = r_viewpoint.Angles.Roll.Degrees;

	// Scroll the sky
	GLRenderer->mSky1Pos = (double)fmod((double)screen->FrameTime * (double)level.skyspeed1, 1024.f) * 90./256.;
	GLRenderer->mSky2Pos = (double)fmod((double)screen->FrameTime * (double)level.skyspeed2, 1024.f) * 90./256.;



	if (camera->player && camera->player-players==consoleplayer &&
		((camera->player->cheats & CF_CHASECAM) || (r_deathcamera && camera->health <= 0)) && camera==camera->player->mo)
	{
		GLRenderer->mViewActor=NULL;
	}
	else
	{
		GLRenderer->mViewActor=camera;
	}

	// 'viewsector' will not survive the rendering so it cannot be used anymore below.
	lviewsector = r_viewpoint.sector;

	// Render (potentially) multiple views for stereo 3d
	float viewShift[3];
	const s3d::Stereo3DMode& stereo3dMode = mainview && toscreen? s3d::Stereo3DMode::getCurrentMode() : s3d::Stereo3DMode::getMonoMode();
	stereo3dMode.SetUp();
	for (int eye_ix = 0; eye_ix < stereo3dMode.eye_count(); ++eye_ix)
	{
		if (eye_ix > 0 && camera->player)
			SetFixedColormap(camera->player); // reiterate color map for each eye, so night vision goggles work in both eyes
		const s3d::EyePose * eye = stereo3dMode.getEyePose(eye_ix);
		eye->SetUp();
		GLRenderer->SetOutputViewport(bounds);
		Set3DViewport(mainview);
		GLRenderer->mDrawingScene2D = true;
		GLRenderer->mCurrentFoV = fov;
		// Stereo mode specific perspective projection
		SetProjection( eye->GetProjection(fov, ratio, fovratio) );
		// SetProjection(fov, ratio, fovratio);	// switch to perspective mode and set up clipper
		SetViewAngle(r_viewpoint.Angles.Yaw);
		// Stereo mode specific viewpoint adjustment - temporarily shifts global ViewPos
		eye->GetViewShift(GLRenderer->mAngles.Yaw.Degrees, viewShift);
		s3d::ScopedViewShifter viewShifter(viewShift);
		SetViewMatrix(r_viewpoint.Pos.X, r_viewpoint.Pos.Y, r_viewpoint.Pos.Z, false, false);
		gl_RenderState.ApplyMatrices();

		ProcessScene(toscreen);
		if (mainview)
		{
			if (FGLRenderBuffers::IsEnabled()) PostProcess.Clock();
			if (toscreen) EndDrawScene(lviewsector); // do not call this for camera textures.

			if (FGLRenderBuffers::IsEnabled())
			{
				GLRenderer->PostProcessScene(FixedColormap, [&]() { if (gl_bloom && FixedColormap == CM_DEFAULT) DrawEndScene2D(lviewsector); });
				PostProcess.Unclock();

				// This should be done after postprocessing, not before.
				GLRenderer->mBuffers->BindCurrentFB();
				glViewport(GLRenderer->mScreenViewport.left, GLRenderer->mScreenViewport.top, GLRenderer->mScreenViewport.width, GLRenderer->mScreenViewport.height);

				if (!toscreen)
				{
					gl_RenderState.mViewMatrix.loadIdentity();
					gl_RenderState.mProjectionMatrix.ortho(GLRenderer->mScreenViewport.left, GLRenderer->mScreenViewport.width, GLRenderer->mScreenViewport.height, GLRenderer->mScreenViewport.top, -1.0f, 1.0f);
					gl_RenderState.ApplyMatrices();
				}

				DrawBlend(lviewsector);
			}
		}
		GLRenderer->mDrawingScene2D = false;
		if (!stereo3dMode.IsMono() && FGLRenderBuffers::IsEnabled())
			GLRenderer->mBuffers->BlitToEyeTexture(eye_ix);
		eye->TearDown();
	}
	stereo3dMode.TearDown();

	interpolator.RestoreInterpolations ();
	return lviewsector;
}

I could use "validcount++" instead but I guess this way it's a bit better? Well, adding validcount optimization to other functions makes it slower, so just those two functions. So I'm really not sure what happens here

Here's the source that compiles and builds (based of LZDoom v3.87c)

[dpJudas]

Thank you for your AI contribution (I assume you didn't do this yourself as you mentioned AI). Unfortunately, adding a global integer that just keeps counting up without ever being used does in fact not improve the performance by 43%. It doesn't really actually do anything at all.

[Darkcrafter]

This must be a bug of some sort then cause when you start lzdoom you must disable distance culling, then restart a map an then indeed that FPS difference is there, on big maps like map13, map 17, at least on my end, well you can place validcount++ too.

[Rachael]

You know the old internet saying - "Screenshots or it didn't happen"

Preferably include some "benchmark" tests in that, both with pre and post changes. Use a savegame to ensure you are at the exact same location when doing these tests, and use the massacre cheat to ensure monsters are not going to tick up the thinker cost per frame.

[Darkcrafter]

Nah, I tested it again with MSI Afterburner overlay, comparing both versions, didn't happen indeed. It's like a FPS can slightly differ time to time, seems like a statistical error, sorry

[Darkcrafter]

So some other "strategies":
Tested:
1. Use distance flickering instead of distance culling - works but looks bad not just due to the fact it flickers but also the normal geometry adapts to that update frequency and game becomes a bit more choppier;

2. Along with gl_bsp.cpp update gl_scene.cpp functions with lower frequencies - reduces the overhead too but the whole render becomes choppier and makes some frames to overlap, requires a way more often update frequencies to reduce bad side effects;


Untested:
1. Implement distance flickering in gl_bsp.cpp and separate rendering of near and far planes in gl_scene.cpp by creating two renderers each for its distance, glue the distant flickering render and don't glue the near plane, then mix results in another third renderer.

2. Do not modify the gl_bsp.cpp and use its gl_distance_culling to cull out the distant geometry and later use gl_scene.bsp function to render a savegame screenshot (is there a way to update it?), then place a plane facing the camera, project that screenshot texture on it for the far distance.


Wanted:
1. Create a caching mechanism to eliminate flickering on throtlled bsp traversal. Introduce distance flickering to throttle only distant geometry to ensure smoother geometry changes in the near plane (doors, bars and etc). All of my attempts to do something similar here be it only inside gl_bsp.cpp or affecting other files failed as flickering was still there.

[Darkcrafter]

I'm pretty sure there's an alternative way to perform optimization without too much gzdoom source modifications involved. Map15 proved to work pretty well with that approach - dump the static map geometry into a set of 3D models a surface by surface per texture but this also can involve a process of baking sector light amount into them, so more models and textures as a result. Special effects sectors can either be done with it as the export function would detect such and create animated modeldef definitions for them. There are some downsides like it's going to get harder to change sector lighting parameters and impossible to illuminate by small radius dynamic lights. With some tradeoffs considered this can work for certain scenarios, like that Map15.

Then there comes an issue of model popping in and out at some spots and angles on the map which can be solved either by increasing render radius but lead to the massive slowdown which can be addressed with line distance culling (works pretty well) or by modifying gzdoom sources to make such models always rendered.

[Rachael]

No amount of "minor changes" is going to get around the weakness that is the BSP walk.

BSP certainly was a useful tool in its time and it enabled Doom to actually render complex level geometry by rendering only small parts of the level at once. But nowadays more geometry than ever is being ham-fisted into the screen at a given moment, which is forcing BSP to show its weaknesses, and how easily overwhelmed it is when it has to handle too much data, as is typical with many 2010's and 2020's "detailed" mapsets. To put it another way: it's forcing two dimensions into one, and walking the level along that single dimension, but taking forks and paths to different parts of that dimension along the way - and it does all of this recursively. The more subsectors and lines that it has to deal with, the worse it gets.

This is not a problem exclusive to GZDoom; all ports that are based on the original Doom source release that has not eliminated or worked around the original BSP-walk algorithm still suffers from these problems to varying extents - GZDoom is only one of the worst because GZDoom handles a lot more data than other ports do which causes more CPU cache misses and that causes performance problems. This is why upgrading to newer CPU's, even when their clock speed remains somewhat the same, is still beneficial.

There are ports that aim to eliminate this BSP walk in order to render complex levels much faster. One example is Helion - It works around the BSP walk completely, simply sending the entire level as-is to the GPU and letting the GPU handle clipping via what it already has always done to begin with - depth-testing each pixel. This works well for more powerful GPU's because pixel-fill-rate is generally pretty fast on most modern GPU's as long as the shaders are not too complex, so that allows Helion to work well in this scenario. As far as I understand this is also a future goal for VkDoom but it has not yet reached a point where it would be considered stable enough for general use as of yet.

This is, unfortunately, the only way you are ever going to find any "optimization". As I said, BSP worked well for much smaller sets of data - which is why the original Doom levels always rendered well even on quite modest hardware like 486's and early Pentiums. But nowadays mappers are using tens or even hundreds of thousands of linedefs as a weird flex, making fundamentally unplayable levels because they are "pretty", and forcing everyone else to deal with the problems that result from it. The problem isn't the source ports being pushed to their limits - the problem is the maps themselves are poorly optimized to take advantage of the BSP algorithms in order to render less of the levels at once in order to make every frame playable, and this is forcing source port authors to fundamentally rethink the BSP system as a whole, which at the end of the day is probably not such a bad thing but there's definitely growing pains to get to that point where it will be usable to the average everyday user.

[Darkcrafter]

That's why I told about a workaround that works and it does so even with the older hardware, the speedup is about 3-4 times. It's like a trick to circumvent the engine. Sure there are problems to solve but seems like it's worth it to me.

That's how Map15 was optimized:
1) Discriminate between static and dynamic geometry;
2) Export static map geometry to OBJ;
3) Process the resulting model (optional). I weld some vertices within some really small threshold in order to glue up duplicate polygons and make it more or less manifold. Then split the model to many other models, each per texture;
4) Export the resulting models, each per texture, I prefer MD3 as takes least space and probably resources to process but it may suffer from artifacts if the geometry was really sophisticated so MD3 is optional;
5) Create a decorate or zscript actor definitions list for the new models, I call them "geometry stacks";
6) Create a texture list that defines textures for the resulting 3D models but references to the original resources in order not to waste space and not to infringe copyrights;
7) Create a modeldef list;
8) Place the resulting models on the map (in the center of the map), reposition it and scale so that it repeats the map geometry as closely as possible;
9) Make the static map geometry invisible to the renderer: select your linedefs and remove all textures from them or replace them with "-" without double brackets;
10) Find the best balance between gl_linedistance_culling distance (I prefer 6000) and renderradius on the 3D models so that they wouldn't pop in and out (I prefer 8000);

And now I'd complicate the approach:
1) Discriminate between static and dynamic geometry;
2) Export static map geometry to OBJ per texture to make sure such things like 3D floors with transparent textures (metallic fences) don't split the walls where it connects them and doesn't drill holes in it;
3) Also process model to remove some dirt from it and probably subdivide geometry with pretty rough poly amount to ensure parallax mapping shader TBN doesn't get lost that much if applied;
4 to 10 - the same.

And here's how I'd like it to be done:
1) Discriminate between static and dynamic geometry;
2) Find out about possible changes in dynamic geometry, e.g. scan for how sectors and linedefs are affected in height, texturing and brightness, find out for triggers and scripts referencing them, then approximate the animation (trim their framecount);
3) Export static geometry per texture and per light (shaded according to the prefered sector lighting options perhaps);
4) Export dynamic geometry per texture and per state (possbile changes);
5) Create a decorate or zscript actor definitions list for the static and animated models - with fullbright flag as we have our lighting backed;
7) Create a modeldef list for static and animated models;
8) Create a switchdef list for animated textures;
9) Create an animdefs list for animated textures;
10) Make the map geometry invisible to the renderer: add a mapinfo CVAR "BigMapOptimization", introduce it to gl_bsp.cpp and gl_walls_draw.cpp, gl_flat.cpp (optional as they're not such as heavy burden to lift as walls) in case of usage - disable void GLSceneDrawer::AddLine(seg_t *seg, bool portalclip), void GLWall::RenderWall(int textured) and void GLFlat::Draw(int pass, bool trans). I'm not sure whether it's possible to keep traversal on skybox linedefs and flats (to allow for sky to draw) but it should be;
11) Implement "always drawn" flag for the actors.
12) Could be implemented in a doom editor as a separate process or in the source port?