ZDoom

I can confirm everything looks okay now.

I believe this one is fixed by the new wallscan function.

It see nothing out of ordinary, thanks.

[quote="Edward-san"]... these posts should go to the right [url=http://forum.zdoom.org/viewtopic.php?f=2&t=50620]bug report[/url]. Any moderator available?[/quote]
A minor bout of thread-surgery later, I think I got it. I hope I didn't miss anything in the process.

... these posts should go to the right [url=http://forum.zdoom.org/viewtopic.php?f=2&t=50620]bug report[/url]. Any moderator available?

OK, further looking at this the problem seem be related to loss of precision when doing floating point math.

The 'bot' argument to wallscan_np2 is -0.0 and the partition value calculated becomes 0.0. The while (partition > bot) therefore runs one iteration too many. Probably will have to go through all the wallscan functions with a fine-comb to make sure all float to fixed conversions are correctly clamped and rounded.

Scratch that part about the 171 pixels. I forgot that its doing a stretch of the source texture. Either way, wallscan_np2 is probably the offending function.

According to my debugger the texture crashing for me right now is called DOOR3. It is 64x72 and it attempts to do a wallscan 'dc_count' run of 171 pixels.

If I understand the comments in the code right, wallscan is never supposed to do a run more than the height of the texture for non-power-of-two textures. I think that its the wallscan_np2 function that somehow fails to break up the run. That's the function I'm looking at right now. :)

Is it there a way to track down the involved texture?

	DWORD fracstep = dc_iscale;
	DWORD frac = dc_texturefrac;
	DWORD height = rw_pic->GetHeight();
	assert((frac >> vlinebits) < height);
	frac += dc_count * fracstep;
	assert((frac >> vlinebits) < height);

dc_texturefrac = xs_ToFixed(fracbits, MAX(dc_texturemid + iscale * (y1ve[0] - CenterY + 0.5), 0.0));

Yep, that one matches my call stack a lot better. I'm able to consistently trigger it by moving around in the first room of E1M1 if I add the following asserts to vlinec1:

[code]
DWORD fracstep = dc_iscale;
DWORD frac = dc_texturefrac;
DWORD height = rw_pic->GetHeight();
assert((frac >> vlinebits) < height);
frac += dc_count * fracstep;
assert((frac >> vlinebits) < height);
[/code]

I tried adding some clamping to the wallscan function ala this:

[code]
dc_texturefrac = xs_ToFixed(fracbits, MAX(dc_texturemid + iscale * (y1ve[0] - CenterY + 0.5), 0.0));
[/code]
This causes my second assert to fail instead of the first one (with a frac value of exactly height). It would seem that the height of the run is exactly one pixel too much in some condition I've not yet tracked down. I think the crash only shows itself if the texture height is a non power-of-two due to the vlinebits shifting.

I guess you meant [url=http://forum.zdoom.org/viewtopic.php?f=2&t=50620]here[/url].

Oops, probably the wrong thread to type this. But the crash is real. ;)

I think I may have managed to trigger this in my truecolor branch. At least I detected a buffer underflow as illustrated by the following debugger output:

[img]http://i.imgur.com/bJb5p3T.png[/img]

As you can see, dc_texturefrac somehow manages to have a negative value, which in turn causes it do a buffer overrun. Depending on where the column is in the texture it can cause zdoom to crash.

I admit the tag error seems weird, considering that when I run the normal zdoom program in debug mode, the value is not uninitialized and matches the data in the function FTagManager::AddLineID, so ignore that for now.

Now the only thing left is the buffer overflow thing, which seems to be reported by both the address sanitizer and valgrind. Just to clarify: I noticed this in the same period of time as the right edge error.

Nope, I noticed this also before...