[Added] Some way for sector skies to avoid being affected by SSAO

[Marisa Kirisame]

I'm getting frustrated with the amount of complaints I get about this. My usual suggestions to work around this (use GLDEFS skyboxes, or if you need a specific shape use a model with translucency and an alpha of 0.999999) are rejected because it's "too much work".

Can't we just get a MAPINFO flag that makes AO on portals not affect sky viewpoints?

Edit: I'm implementing this myself if necessary.

Edit 2: Wow this needed way less work than I imagined. The thing now is... should it be a flag to disable skybox ao or to enable it? Think I'll go with the latter seeing as the vast majority of mappers would probably want it that way.

The only catch with how I'm doing this is that stacked sectors report themselves as being skies, so they're also affected.

[Graf Zahl]

Stacked sectors report themselves as a different kind of portal, what did you check?

[Marisa Kirisame]

Just the IsSky() virtual.

[Tormentor667]

Edit 2: Wow this needed way less work than I imagined. The thing now is... should it be a flag to disable skybox ao or to enable it? Think I'll go with the latter seeing as the vast majority of mappers would probably want it that way.

As a lot of maps and mods already use sector skyboxes I’d suggest that as well.

[Marisa Kirisame]

The PR is live, moving this to code submissions.

[Rachael]

Since it uses the IsSky() virtual to check, it also affects stacked sectors due to them technically reporting themselves as skies. I have not found a way around this.

This is the biggest issue I see with this, and probably why it was not done so quickly, previously. I don't know how involved it will be to check for a stacked sector, but this could be important.

[Graf Zahl]

You have to check the actual portal type which is unambiguous.

Code: Select all • Expand view

//============================================================================
//
// All information about a sector plane portal
//
//============================================================================

enum
{
   PORTS_SKYVIEWPOINT = 0,      // a regular skybox
   PORTS_STACKEDSECTORTHING,   // stacked sectors with the thing method
   PORTS_PORTAL,            // stacked sectors with Sector_SetPortal
   PORTS_LINKEDPORTAL,         // linked portal (interactive)
   PORTS_PLANE,            // EE-style plane portal (not implemented in SW renderer)
   PORTS_HORIZON,            // EE-style horizon portal (not implemented in SW renderer)
};

enum
{
   PORTSF_SKYFLATONLY = 1,            // portal is only active on skyflatnum
   PORTSF_INSKYBOX = 2,            // to avoid recursion
};

struct FSectorPortal
{
   int mType; <------------- here
   int mFlags;
};


[Marisa Kirisame]

Found a way to access that, although it's not very pretty and I have no idea if what I'm doing is "correct", I've swapped the

Code: Select all • Expand view

!di->mCurrentPortal->IsSky()

line with a

Code: Select all • Expand view

!di->mCurrentPortal->lines[0].secportal->mType==PORTALTYPE_SKYBOX

Unfortunately this seems to be wrong because now ALL portals have no AO.

Edit: Wait, I've got a lot of stuff mixed up here.

Edit 2: Feck's sake, the solution was staring me in the face all this time. There's a GetName() function that will tell you what the portal is. I've replaced the line with:

Code: Select all • Expand view

applySSAO = (strcmp(di->mCurrentPortal->GetName(),"Skybox") || di->Level->flags3&LEVEL3_SKYBOXAO);


Surprising that that function that mainly exists for debugging came in handy here.

[Graf Zahl]

Don't use that GetName function for any kind of decision making. Use the flags. The name is merely for printing diagnostics.

[Marisa Kirisame]

OK, but the alternative to that is horrendously ugly and involves casting void* to something else and crossing my fingers that it works.

I'm no C++ expert here, so this will take me a while to figure out.

[Graf Zahl]

What do you mean with casting void*?

[Marisa Kirisame]

I figured that the alternative here is to use the GetSource() function which returns void*. In the case of skybox portals it's a pointer to a FSectorPortal.

[Graf Zahl]

Ok. But I think this needs to be done differently anyway. The decision whether to apply SSAO must be done by the portal setup code in hw_walls.cpp/PutPortal, where you still have access to this stuff, not the actual portal renderer itself. Imagine you want to add some options later - it's going to be quite complicated as you already noticed that you do not have all info at hand anymore.

[Marisa Kirisame]

OK. Then I think I'll leave this work to someone else.

[Chris]

Found a way to access that, although it's not very pretty and I have no idea if what I'm doing is "correct", I've swapped the

Code: Select all • Expand view

!di->mCurrentPortal->IsSky()

line with a

Code: Select all • Expand view

!di->mCurrentPortal->lines[0].secportal->mType==PORTALTYPE_SKYBOX

Unfortunately this seems to be wrong because now ALL portals have no AO.

You left in the !, so it's checking if mType is false is equal to PORTALTYPE_SKYBOX.