dpJudas wrote:
1) You may not always want this. Mesh editors like 3ds Max and Blender have smoothing groups that faces can belong to so that you can better control when smoothing happens. In other words: this isn't a global on/off toggle but something which requires map editor support.
That would be the least of all problems. Just add a new field to the linedef or sidedef definition in UDMF specifying that the normals are to be interpolated.
2) The ssao pass needs face normals, not smoothed normals. A smoothed normal causes self-shadowing. You can sometimes spot this on some md3 models today, but here it isn't that obvious because faces tend be relatively small there. If you start making large walls use smoothed normals it would be noticed very quickly. It is another step that requires additional work to support a feature like this.
That's indeed a problem. It could be solved by passing two normals to the shader but at some point the vertex buffer would become too large so that it negatively affects performance if there's too many elements per vertex.
By the way, your own image nicely illustrates #1: the smoothing didn't include the top face of the cylinder.
Indeed. This is something that simply cannot be done as some simply automatic step.
But all of this doesn't address the main issue here: This would only affect dynamic lights. All the sector based light is flat per linedef and since it's identical at both vertices there's nothing to interpolate. Another thing to be considered here is fake contrast which currently gets calculated on the CPU side. This also would have to be moved to the GPU - otherwise the feature would require to disable fake contrast entirely.
So all things considered: I doubt that this is going to work out in the confines of a Doom based engine.