Ability to exclude a specific model from AO

[Enjay]

Would it be possible to add a flag to MODELDEF to exclude the model being defined from having ambient occlusion applied to it?

The reason I ask is that I have come across a couple of models where AO doesn't look good - and this is exacerbated if the model animates because the AO can flicker and change as the model moves - but a player might want AO to be on for most of their game.

For example, I have just been working on a model of a radar dish and as it pans left and right, the AO on its surfaces changes in a way that doesn't look good. If I switch off AO for the whole game, the model looks fine. The radar dish doesn't move around the map (pretty obviously), it just sits in place and pans left/right. So it's easy to set up the model to look good with respects to nearby lighting etc but the changing AO during animation doesn't look great.

I imagine that this suggestion would be useful for excluding specific models from AO (and where the model skin would have suitable shadows backed on to it anyway) for cases specifically like a large, mostly static, decoration of some sort - like the radar dish mentioned above.

I don't know if such a thing is even possible, but if it is, it would be nice to have. Thanks.

[Nash]

Instead of MODELDEFS however, I think it should be done at the texture level, because there are also situations where one might intentionally not want SSAO on certain kinds of level surfaces (like sectors meant to be light emitters or whatever).

Doing it on a texture basis would easily allow you to exempt certain model textures from receiving SSAO.

Alternatively, SSAO should take into account the brightmap to determine if it should be shaded or not.

[dpJudas]

This unfortunately not possible. SSAO works by looking at the depth buffer and the model has to be written into it. Thus the SSAO pass loses the ability to see what is behind the model it should be ignoring.

[Graf Zahl]

Would this be solvable by having another buffer that works like a stencil for the SSAO pass?

[dpJudas]

I suppose technically the main shader could write null normals into the normal gbuffer. The ssao pass already checks for that as its way to avoid shading portal entrances. It removes partly the artifact Enjay is talking about, but still leaves one as things near the ignored objects may appear brighter than they otherwise would have.

[Enjay]

Given that I was imagining this being used on a case-by-case basis for specific actors where no AO was preferable, then exactly how and where the model would be used would be part of the modders consideration on whether to disable AO for it or not. So that needn't necessarily be a problem. i.e. the modder could evaluate whether AO or no AO worked better for the particular circumstances where it was being considered.


Nash's point about textures is also something that would be interesting to look at - but I suspect it could be even more than that. For example, if I make a small recess in a wall with, say, LITE3 on the back wall and light it with light value 255, I would expect the LITE texture to look nice and bright, and indeed, all of the surfaces inside the recess to look bright. I literally cannot make it any brighter within normal Doom parameters. However, AO can make the texture looks far darker than intended - almost as if I had intended the lights to look like they were not on at all.

---

in both cases, the alcove is at brightness 255 and the gif is switching between AO=off and AO=high quality.


The second one in particular makes the I-want-them-to-look-bright LITE5 textures look like they are switched off.

Also, a couple of years ago, I posted this screenshot of pure white textures in a 255-bright alcove in GZDoom 3.5.0.

I'm not sure if things are more refined now, but this clearly shows that even with only 255 255 255 white on the texture and the sector being as bright as possible, it isn't possible to remove AO in particular situations where AO might be undesirable while keeping the generally desirable effect elsewhere.

I've tried using brightmaps and wall-specific brightness, but not come up with something that really helps.

I suspect that this is a different thing to what this particular feature suggestion is about though and, again, I have no idea how possible addressing this in some practical way would be.

[Graf Zahl]

You posted some good reasons why I do not use SSAO - I find the entire feature way too rough because it cannot interpret what a texture is supposed to represent.

[Enjay]

Generally, I personally quite like it - so I prefer it on rather than off but it certainly isn't perfect. Being able to exclude specific textures from it (either at a texture definition level somehow, or on a mapping line-by-line (and ceiling/floor) basis) would address the most common issues that I have with it. Again, I have no idea how possible that would be.

[phantombeta]

I imagine this can be done based on emissive maps. (i.e., what we have as brightmaps) It could probably also be done with some kind of material map that controls how much AO is allowed in that part of the texture.
Of course, this is usually not really a problem when the SSAO is applied properly (i.e., only applied to the ambient component of the light), but Doom maps generally mostly use ambient sector lighting, so that won't help here.

[axredneck]

Good idea. SSAO doesn't work properly for voxels anyway (because they have no normals, right?)