Some way for sector skies to avoid being affected by SSAO

Moderator: GZDoom Developers

User avatar
Marisa the Magician
Posts: 3886
Joined: Fri Feb 08, 2008 9:15 am
Preferred Pronouns: She/Her
Operating System Version (Optional): (btw I use) Arch
Graphics Processor: nVidia with Vulkan support
Location: Vigo, Galicia

Some way for sector skies to avoid being affected by SSAO

Post by Marisa the Magician »

I'm getting frustrated with the amount of complaints I get about this. My usual suggestions to work around this (use GLDEFS skyboxes, or if you need a specific shape use a model with translucency and an alpha of 0.999999) are rejected because it's "too much work".

Can't we just get a MAPINFO flag that makes AO on portals not affect sky viewpoints?

Edit: I'm implementing this myself if necessary.

Edit 2: Wow this needed way less work than I imagined. The thing now is... should it be a flag to disable skybox ao or to enable it? Think I'll go with the latter seeing as the vast majority of mappers would probably want it that way.

The only catch with how I'm doing this is that stacked sectors report themselves as being skies, so they're also affected.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
Posts: 49177
Joined: Sat Jul 19, 2003 10:19 am
Location: Germany

Re: Some way for sector skies to avoid being affected by SSA

Post by Graf Zahl »

Stacked sectors report themselves as a different kind of portal, what did you check?
User avatar
Marisa the Magician
Posts: 3886
Joined: Fri Feb 08, 2008 9:15 am
Preferred Pronouns: She/Her
Operating System Version (Optional): (btw I use) Arch
Graphics Processor: nVidia with Vulkan support
Location: Vigo, Galicia

Re: Some way for sector skies to avoid being affected by SSA

Post by Marisa the Magician »

Just the IsSky() virtual.
User avatar
Tormentor667
Posts: 13548
Joined: Wed Jul 16, 2003 3:52 am

Re: Some way for sector skies to avoid being affected by SSA

Post by Tormentor667 »

Marisa Kirisame wrote:Edit 2: Wow this needed way less work than I imagined. The thing now is... should it be a flag to disable skybox ao or to enable it? Think I'll go with the latter seeing as the vast majority of mappers would probably want it that way.
As a lot of maps and mods already use sector skyboxes I’d suggest that as well.
User avatar
Marisa the Magician
Posts: 3886
Joined: Fri Feb 08, 2008 9:15 am
Preferred Pronouns: She/Her
Operating System Version (Optional): (btw I use) Arch
Graphics Processor: nVidia with Vulkan support
Location: Vigo, Galicia

Re: Some way for sector skies to avoid being affected by SSA

Post by Marisa the Magician »

The PR is live, moving this to code submissions.
Last edited by Rachael on Thu Dec 26, 2019 2:17 am, edited 1 time in total.
Reason: fixed link (it 404'd)
User avatar
Rachael
Posts: 13780
Joined: Tue Jan 13, 2004 1:31 pm
Preferred Pronouns: She/Her

Re: Some way for sector skies to avoid being affected by SSA

Post by Rachael »

Since it uses the IsSky() virtual to check, it also affects stacked sectors due to them technically reporting themselves as skies. I have not found a way around this.
This is the biggest issue I see with this, and probably why it was not done so quickly, previously. I don't know how involved it will be to check for a stacked sector, but this could be important.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
Posts: 49177
Joined: Sat Jul 19, 2003 10:19 am
Location: Germany

Re: Some way for sector skies to avoid being affected by SSA

Post by Graf Zahl »

You have to check the actual portal type which is unambiguous.

Code: Select all

//============================================================================
//
// All information about a sector plane portal
//
//============================================================================

enum
{
	PORTS_SKYVIEWPOINT = 0,		// a regular skybox
	PORTS_STACKEDSECTORTHING,	// stacked sectors with the thing method
	PORTS_PORTAL,				// stacked sectors with Sector_SetPortal
	PORTS_LINKEDPORTAL,			// linked portal (interactive)
	PORTS_PLANE,				// EE-style plane portal (not implemented in SW renderer)
	PORTS_HORIZON,				// EE-style horizon portal (not implemented in SW renderer)
};

enum
{
	PORTSF_SKYFLATONLY = 1,				// portal is only active on skyflatnum
	PORTSF_INSKYBOX = 2,				// to avoid recursion
};

struct FSectorPortal
{
	int mType; <------------- here
	int mFlags;
};
User avatar
Marisa the Magician
Posts: 3886
Joined: Fri Feb 08, 2008 9:15 am
Preferred Pronouns: She/Her
Operating System Version (Optional): (btw I use) Arch
Graphics Processor: nVidia with Vulkan support
Location: Vigo, Galicia

Re: Some way for sector skies to avoid being affected by SSA

Post by Marisa the Magician »

Found a way to access that, although it's not very pretty and I have no idea if what I'm doing is "correct", I've swapped the

Code: Select all

!di->mCurrentPortal->IsSky()
line with a

Code: Select all

!di->mCurrentPortal->lines[0].secportal->mType==PORTALTYPE_SKYBOX
Unfortunately this seems to be wrong because now ALL portals have no AO.

Edit: Wait, I've got a lot of stuff mixed up here.

Edit 2: Feck's sake, the solution was staring me in the face all this time. There's a GetName() function that will tell you what the portal is. I've replaced the line with:

Code: Select all

applySSAO = (strcmp(di->mCurrentPortal->GetName(),"Skybox") || di->Level->flags3&LEVEL3_SKYBOXAO);
Surprising that that function that mainly exists for debugging came in handy here.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
Posts: 49177
Joined: Sat Jul 19, 2003 10:19 am
Location: Germany

Re: Some way for sector skies to avoid being affected by SSA

Post by Graf Zahl »

Don't use that GetName function for any kind of decision making. Use the flags. The name is merely for printing diagnostics.
User avatar
Marisa the Magician
Posts: 3886
Joined: Fri Feb 08, 2008 9:15 am
Preferred Pronouns: She/Her
Operating System Version (Optional): (btw I use) Arch
Graphics Processor: nVidia with Vulkan support
Location: Vigo, Galicia

Re: Some way for sector skies to avoid being affected by SSA

Post by Marisa the Magician »

OK, but the alternative to that is horrendously ugly and involves casting void* to something else and crossing my fingers that it works.

I'm no C++ expert here, so this will take me a while to figure out.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
Posts: 49177
Joined: Sat Jul 19, 2003 10:19 am
Location: Germany

Re: Some way for sector skies to avoid being affected by SSA

Post by Graf Zahl »

What do you mean with casting void*?
User avatar
Marisa the Magician
Posts: 3886
Joined: Fri Feb 08, 2008 9:15 am
Preferred Pronouns: She/Her
Operating System Version (Optional): (btw I use) Arch
Graphics Processor: nVidia with Vulkan support
Location: Vigo, Galicia

Re: Some way for sector skies to avoid being affected by SSA

Post by Marisa the Magician »

I figured that the alternative here is to use the GetSource() function which returns void*. In the case of skybox portals it's a pointer to a FSectorPortal.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
Posts: 49177
Joined: Sat Jul 19, 2003 10:19 am
Location: Germany

Re: Some way for sector skies to avoid being affected by SSA

Post by Graf Zahl »

Ok. But I think this needs to be done differently anyway. The decision whether to apply SSAO must be done by the portal setup code in hw_walls.cpp/PutPortal, where you still have access to this stuff, not the actual portal renderer itself. Imagine you want to add some options later - it's going to be quite complicated as you already noticed that you do not have all info at hand anymore.
User avatar
Marisa the Magician
Posts: 3886
Joined: Fri Feb 08, 2008 9:15 am
Preferred Pronouns: She/Her
Operating System Version (Optional): (btw I use) Arch
Graphics Processor: nVidia with Vulkan support
Location: Vigo, Galicia

Re: Some way for sector skies to avoid being affected by SSA

Post by Marisa the Magician »

OK. Then I think I'll leave this work to someone else.
User avatar
Chris
Posts: 2954
Joined: Thu Jul 17, 2003 12:07 am
Graphics Processor: ATI/AMD with Vulkan/Metal Support

Re: Some way for sector skies to avoid being affected by SSA

Post by Chris »

Marisa Kirisame wrote:Found a way to access that, although it's not very pretty and I have no idea if what I'm doing is "correct", I've swapped the

Code: Select all

!di->mCurrentPortal->IsSky()
line with a

Code: Select all

!di->mCurrentPortal->lines[0].secportal->mType==PORTALTYPE_SKYBOX
Unfortunately this seems to be wrong because now ALL portals have no AO.
You left in the !, so it's checking if mType is false is equal to PORTALTYPE_SKYBOX.

Return to “Closed Feature Suggestions [GZDoom]”