AO and models [3.6.0 and earlier]

[Enjay]

This may turn out just to be a problem with the model, but I wasn't really sure where else to post this because it could also be a bug.

I have noticed if I make a very simple model in Milkshape - just a simple box - and then use it in game, one side always looks weird. There is a bright strip down the middle and the shading to the side of this seems to be running the opposite way to what I'd expect (it gets darker as it approaches the middle - but maybe that is OK?). I've messed around with the normals on the model to no effect but it really is just a very simple default box as drawn by MilkShape and all sides are just made from 2 triangles. BTW, I've noticed this with a few boxes that I've made for various reasons and over the last few versions of GZDoom too.

Anyway, here's a screenshot of the attached map:


The map was actually made to compare the size of non-scaled model architecture versus 1:1.2 Doom pixels, so the crate on the left is not a model but made of a sector.

It's not map specific. You can warp to any map and "summon GreyCrate" to see the effect. Here's one on map12.

[Nash]

Inspected the MD3 from the PK3... normals look okay here, although it looks like every face is assigned to a single smoothing group. For cubes, I'd probably go for a faceted look:



Not sure how you'd achieve this with Milkshape, I imagine you'd select each side of the cube and assign each 6 of them a unique smoothing group (I could be wrong, haven't used MS3D in years)

Anyway, this is more of an art-related answer, as for the bug report, well, I don't have anything to contribute right now but maybe you'd like to test the MD3 after my normal fix and see if it looks any better?

[Nash]

Hey sorry for the double post.

It would seem that Milkshape is indeed exporting somewhat broken normals. Using NPherno's MD3 Compiler to fix the normals a different way - to achieve the same "single smoothing group" look (instead of faceted) - actually fixes the glitch. It looks correct in-game

(dpJudas normal viewer in-game when :P)

[Enjay]

Typed before seeing your post above...

Assigning different smoothing groups to the different cube faces does prevent this happening. Your model looks much better and I was able to do much the same in Milkshape. Unfortunately, my preferred method of making MD3s is to do the basic construction in Milkshape then convert to MD3 using Misfit Cubed and it seems that it's misfit that is putting everything into a single smoothing group, not Milkshape. That's frustrating because Misfit is the only program I have that can edit MD3 animations directly. Oh well, at least I know now and perhaps I can investigate how to address that further with the programs that I have.


Anyway, all that aside, I think I'm right in saying that AO in GZDoom still shouldn't be causing the effect on the model in my test file, given that there is nothing technically wrong, per se, with setting it up like that?

[Enjay]

So, based on your second post Nash, in what way are the Normals broken, and can I fix them?

[Nash]

Considering that I could use an external tool to fix the normals from the original model you provided, I'd conclude this as Misfit being the culprit for exporting broken normals on the models... so maybe there's no bug here? But let's wait to see what dpJudas says. :)

(With my current tool setup - Blender straight to MD3 - I am getting a 1:1 result from editor to GZDoom, no matter how complicated the smoothing groups are. Just some additional evidence that this might not be a bug :D)

So, based on your second post Nash, in what way are the Normals broken, and can I fix them?

Yeah it would seem so. I'm not sure what you can do with only Milkshape and Misfit. I have never used Misfit before so unfortunately I have zero advice here. What you CAN do is track down a copy of NPherno's MD3 Compiler. It's a very oooooold program with a weird learning curve so I suggest you read up a bit on how to navigate the UI first, then, with the knowledge on how to use the UI, what you would then do is basically:

1) Import the model into NMC
2) Be sure the object you want to fix is "Selected"
3) Click "Rebuild normals"

There will be 3 checkboxes. Only the first one is relevant, the bottom two should be ignored:
- Untick "ignore UV splits" for faceted look
- Tick it for the "smooth group" look

It's technically doing something more complicated than "faceted VS unfaceted" but let's just try to keep it simple here.

4) Export the object, overwrite the old file

[dpJudas]

Anyway, all that aside, I think I'm right in saying that AO in GZDoom still shouldn't be causing the effect on the model in my test file, given that there is nothing technically wrong, per se, with setting it up like that?

Technically, what is going on is that the SSAO pass is using smoothed face normals as input where it ideally would be using physical face normals.

The SSAO pass uses the normal to know in which hemisphere to look for occlusion and if the normal does not point straight away from a face it will see a bit of its own face. This causes slight self-shadowing which gets increasingly bad the further the smoothed normal deviates from the physical face normal.

This is basically a performance trade-off as getting the physical face normal requires an additonal array of normals for the mesh, or that the main.fp shader uses the derived functions to calculate it on the fly. Your box represents the worst case scenario because the normals there are really poor and not remotely close to the actual geometry being rendered. Generally you'll get better dynamic light if you use smoothing groups as it isn't only the SSAO pass that suffers.

[Enjay]

Thanks Nash.

I tracked down Npherno's. It seems reasonably straight forward to do this job. I also tried with another, more complex model with animations and so on and I was happy to see that it still worked and nothing was broken with it after using Npherno's to rebuild the normals. It's just a shame that Misfit seems to be causing this problem because, otherwise, it's a very useful program for quickly converting between MD2, MD3, MS3D and a whole bunch of other formats too. Unfortunately, this means adding an additional step to the production/conversion of any model. At least it's doable.


@dpJudas, thanks for the explanation. So I'm guessing things will be left "as is" on the GZDoom side? It seems as if, although my box isn't technically faulty, it's just not an ideal setup and, I'm guessing, it's something that there will be no particular desire, and there may even be a cost, to address engine-side?

[Graf Zahl]

Making SSAO not glitch is virtually impossible. I already disabled that feature entirely for playing because I found far too many maps with high detail geometry where the drawbacks far outweighed the benefits

[dpJudas]

So I'm guessing things will be left "as is" on the GZDoom side? It seems as if, although my box isn't technically faulty, it's just not an ideal setup and, I'm guessing, it's something that there will be no particular desire, and there may even be a cost, to address engine-side?

It can be addressed engine-side by relatively little code, but the catch is that it will affect performance. How much I'm not sure - personally I have a very high end GPU so I'm not really able to make a proper judgement whether it is worth the cost to address it.

However, the SSAO pass isn't the only thing affected. Try place an attenuated dynamic light next to the side where you see the SSAO pass self-shadow. Turn off SSAO and notice how the light might not look exactly how you'd expect - especially when compared to the other sides of the box. In contrast, if you do the same with Nash' box you'll see the dynamic light behaves uniformly the same way all around the box.

Smoothed normals are actually a visual error we like because it creates an illusion that the surface has more faces than it actually does, but like all illusions it breaks down if the illusion deviates enough from the truth. The SSAO pass may be the first one to break down and expose the problem with the normals, but other light calculations are affected as well. For that reason the ultra-smoothed box is probably not the best candidate to evaluate whether SSAO should be adjusted - it might very well be that for models with reasonably smoothed normals the self-shadowing is so minor it won't affect visuals.

[Graf Zahl]

I'd go by Nash's post saying that even as being smoothed, the model is not correct and just ignore the problem. This looks to me like the normals are actually facing somewhere into the wild.

[Enjay]

Thanks again. Today is certainly a school day for me. I'm finally learning and understanding more about normals - something that has been a bit of a black box (no pun intended) for me so far. I'll try the attenuated light test too.

Pretty sure a few of the models in my Genetech mod will be affected by this one way or another. I certainly made some simple cubes to be crates and so on, as well as some simple pickups etc. I'm pretty sure I noticed this problem the last time I played the mod.

[edit] Yup. [/edit]



I too have a pretty high end GPU (GTX1080) so I'd be unlikely to notice and assess any performance hit for you.

[Nash]

I suppose Misfit has been exporting broken normals all this while, but there was simply no way to notice this because the engine didn't display them in the past.

Now with more proper lighting calculations in the engine, it's inevitable that more care and thought needs to be taken when working with models. We actually have to pay attention to proper usage of smoothing groups now. =P What's that Spider-Man movie quote, "with great power comes great responsibility".... ;)

This might help a little in understanding the concept: http://wiki.polycount.com/wiki/Smoothing_Groups

Basically, now with proper lighting in the engine, it is important to manage which sets of polygons belong as a "group" to be smoothed together. Here's another thread where I talked about it briefly: viewtopic.php?p=1061628#p1061628 (tl;dr in the "final look" part, I have manually chosen which surfaces belong to which smoothing group, an artistic judgment on my part)

It gets even more technical when you start introducing normal maps (normal maps completely override the normals built from smoothing groups) but that's a topic for another time. =P

[dpJudas]

normal maps completely override the normals built from smoothing groups

Actually, they don't. The normal maps bends the smoothed normals. At least that is how it is implemented in GZDoom.

Edit: or do they? Haha, now you got me wondering..

Edit 2: they do bend the smoothing group normals, checked the code.