Trace ignores 3D floors within line if line is skipped

[Marisa the Magician]

I think it's best if I put this on its own thread. Due to how FTraceInfo::LineCheck() works, if one were to skip over a line (e.g.: in a LineTracer callback) next to a sector with 3d floors, then this would also skip checking for any 3D floor edges overlapping the line itself.

At the moment, the way this "line check" works is as follows:

1. First checks if the floor in front of the line is hit.
2. If that fails, checks for ceiling hit.
3. If that one also fails, checks that either a line's lower or upper parts were hit, or it's an impassable line, or the line's flags match the mask passed to the Trace call.
4. If it "got past the line", meaning that it was passable AND didn't match the mask, THEN it checks for 3d floors.

Now, LineTracer passes 0xFFFFFFFF as the line flags mask, so ALL lines match, meaning that step 4 will never ever take place.

The most obvious solution I can think of is to force the check for 3D floors if the callback for the line itself returns skip or continue. I already attempted this myself, but I couldn't find the best way to restructure the code for this.

[Marisa the Magician]

I made a test map comparing the behaviour of the LineTracer class against my simpler Actor.LineTrace here.

Since Actor.LineTrace only stops at walls that have block hitscan/everything flags (by default) or are impassable, it can properly detect the 3D floor edge at height 80.

Meanwhile, the LineTracer subclass has been set up to skip everything until it hits a one-sided wall (as skipping those can cause a crash). You can see that on its way through ALL the lines, it never does detect any 3D floor edges.

I'm hoping this can be sorted out for 3.3.1, because it's kind of a big deal.

[Major Cooke]

Huh. I missed it, but ZZYZX's pull request said on the list that 3d floors aren't doable since they've not been exported yet.

[Marisa the Magician]

Actually that is unrelated. The trace will still report a line if a 3d floor edge is hit, or a plane if the floor/ceiling of said 3d floor was hit. 3D floors ARE doable, what isn't is access to the F3DFloor struct (which is set to the 3D floor itself that was hit in any of those conditions).

This problem is happening on the native C++ side, not on ZScript, and is unrelated to ZZYZX's code.

[_mental_]

A test map seems to be deleted. Could you please attach it to this topic?

[Marisa the Magician]

Oh, made a typo in the URL. It's fixed now.

[ZZYZX]

Supposedly fixed in this PR https://github.com/coelckers/gzdoom/pull/467
Supposedly didn't even break anything :Р

[Marisa the Magician]

You're my hero.

[Graf Zahl]

Ok, let's try this. BTW, what are your 'other ideas'?

[ZZYZX]

Unconditionally using callback for this case (i.e. run callback on main line, then on 3D floor if callback passed, and if there are any), and moving out part of CheckLine instead of using recursive call. I also didnt bother understanding what CheckLine does. much.