ZDoom

I'd do the same as EDuke - preserve the behavior for classic maps but fix it for any new map format that doesn't need to preserve old bugs and add a flag to some definition file to allow specifying which behavior a map needs. That has generally always been the approach taken by ZDoom for things that aren't isolated edge cases.
2 is definitely unmanageable. It surely is the cleanest for the engine but requires active intervention for each map and there's simply too many of them.

From an inexperienced viewpoint, the second option sounds best. I imagine Maphacks could be used for this, similar to how Quakespasm ships a pak file that has offsets to correct z-fighting in the stock maps.

Well there are two choices

- Hack the hardware renderer to match classic.
- Let the hardware renderer wrap properly, and fix offsets in each map instead.

The first is much more manageable. But from what I can tell, EDuke32 went with a third option.

- Hack the hardware renderer to match classic, but disable the hack if later map version is detected (in which case the software renderer is fixed instead)

The only 'go' here is to retain the shader hack. To keep the backend code identical with GZDoom it will be added with a conditional #define so GZDoom can just ignore it - it has no use in Doom engine games, after all.

Not really surprised at all, and Gez's hypothesis sounds reasonable enough. What's the go here then? Fix it properly and hack the game code to do the right thing, or just preserve how it's been done as something unchangable?

I suppose the level designers, instead of flagging the issue when textures behaved weirdly, just assumed everything was working as intended and fiddled with the offsets until it looked right.

All Boom compatible engines implement proper texture wrapping in the software renderer - ZDoom was the last to get it but even that was a long time ago. For hardware renderers this isn't an issue, they can wrap NPOT textures natively. But from the looks Build not only never got fixed but apparently the games actively exploited the issue. Is anyone really surprised?

You've said it's essentially 'what Doom did', just curious if it's what GZDoom still does or is there a better way?

I think I know now how this thing works. I just needed to see this in a context I know better, so I hacked the feature into GZDoom and checked a level where I know that it has major alignment issues in strict vanilla compatible ports on an NPOT texture and also know how it lookes there. Turns out that this code is just doing mostly the same thing, i.e. it renders the texture normally until the first power of two beyond the texture's height is reached while offsetting the overflow part by one texel and then at that POT boundary wraps around to 0. It's essentially what Doom also did with non-POT textures, minus the tutti frutti effect that occured in the leftmost column.

This obviously means that there's no way to fix this on the engine side, the shader hack is really the only workable way to handlie this case.

I can see what this option does, but the one thing I wasn't able to derive from the observed scenes was how this works.
That grandfather clock is a good example. Where's offset 0, to what edge is the texture aligned and most importantly, why is the realignment done in the fragment shader and not up front in C++ code?

I got three examples
- dancing girl texture inside the projector in Hollywood Holocaust
- "You must be this tall" in House of Horrors in Blood
- the grandfather clock at the start of The Siege in Blood

Here's an example of the last one with and without npotwallmode correction enabled
[imgur]https://i.imgur.com/YU67pJM[/imgur]

It was also written for a different generation of graphics hardware. That clearly shows these days. Polymost is the child of hardware that wasn't good at performing coordinate transformation in hardware, so doing it all on the CPU didn't really matter.

The code probably hasn't changed at all since JFBuild3D which features Ken's original Polymost implementation. He has a penchant for numbers and a lot of his future works like Build 2, etc are all done on the CPU, so I imagine Polymost was his first exposure to hardware rendering.

This is solely about walls, there is no handling for flats anywhere - it's just that without the compensation the texture does not wrap correctly, which is what makes me think that they tiled a NPOT texture and somehow abused the side effects of it. But the entire math is so obtuse that I cannot make much sense of it. The code was optimized for "performance" in all the wrong ways, i.e. using math shortcuts where their gain is minimal and ignoring obvious routes of optimization at nearly every corner.

I know in the original, vanilla Duke/SW/etc (don't know about RR) it was not possible to use NP2 textures as a floor or ceiling for a sector, unless you wanted it to look really really bad. So that right there is already a feature enhancement if it works.

I am not sure about walls though. I think the original renderer expected it to be handled much the same way GZDoom already does.