Page 1 of 2

[4.1.3] WallSprites crash in software renderer

Posted: Fri Jul 19, 2019 12:23 am
by SanyaWaffles
https://www.dropbox.com/s/tbyujjqpw2sj5 ... .ipk3?dl=0

I was working on a project using wallsprites and I notice it crashes in software mode (and looks weird in true-color mode) but works on every other mode, including softpoly, true color softpoly and hardware accelerated.
Spoiler: Screenshots
Classic Software crashes with no crash dump generated (it doesn't crash gracefully).

Re: [4.1.3] WallSprites crash in software renderer

Posted: Fri Jul 19, 2019 12:30 pm
by Rachael
I am curious, are you using any texture resize mode? I wonder if it's related to this bug.

Re: [4.1.3] WallSprites crash in software renderer

Posted: Fri Jul 19, 2019 7:25 pm
by SanyaWaffles

Code: Select all

Texture Filter mode: Trilinear
Aniscotropic filter: 8x
Enable highres textures: Yes
High Qualitry Reszie mode: Off
High Quality Resize multiplier: Off

Resize textures: On
Resize sprites: On
Resize fonts: On
Precache GL texture: No
Sort draw lists by texture: No
Those are my settings.

Re: [4.1.3] WallSprites crash in software renderer

Posted: Fri Jul 19, 2019 7:32 pm
by Rachael
Okay, so it's not related. Thanks. :)

Re: [4.1.3] WallSprites crash in software renderer

Posted: Fri Sep 20, 2019 3:27 am
by drfrag
Bumping just to notice that the OP has deleted the sample, not that i'm surprised that wallsprites crash.

Re: [4.1.3] WallSprites crash in software renderer

Posted: Thu Oct 03, 2019 9:49 pm
by SanyaWaffles
Rachael did fix this just now, though there's still a problem with the renderer not grabbing the sprite properly.

Sorry about the file going away.

Re: [4.1.3] WallSprites crash in software renderer

Posted: Thu Oct 03, 2019 9:57 pm
by Rachael
The crash has fixed, but as Sanya said, the texture is not being retrieved properly. While the Y-coordinates work properly (from what I can see), the X coordinate is dependent on Z-depth and does not currently work.

Additionally, activating a texture resize mode will cause further problems with the texture.

As for the crash, it was caused by missing colormap data, and was fixed here.

Re: [4.1.3] WallSprites crash in software renderer

Posted: Fri Oct 04, 2019 4:32 am
by drfrag
I've just checked in master and for me it still crashes with render mode 0.

Code: Select all

 	gzdoom.exe!swrenderer::DrawColumnPalCommand::Execute(DrawerThread * thread) Line 908	C++
 	[Inline Frame] gzdoom.exe!DrawerCommandQueue::Push(const swrenderer::SpriteDrawerArgs &) Line 207	C++
 	gzdoom.exe!swrenderer::SWPalDrawers::DrawColumn(const swrenderer::SpriteDrawerArgs & args) Line 249	C++
 	gzdoom.exe!swrenderer::SpriteDrawerArgs::DrawMaskedColumn(swrenderer::RenderThread * thread, int x, int iscale, FSoftwareTexture * tex, int col, double spryscale, double sprtopscreen, bool sprflipvert, const short * mfloorclip, const short * mceilingclip, FRenderStyle style, bool unmasked) Line 127	C++
 	[Inline Frame] gzdoom.exe!swrenderer::RenderWallSprite::DrawColumn(swrenderer::RenderThread * style, swrenderer::SpriteDrawerArgs &) Line 258	C++
>	gzdoom.exe!swrenderer::RenderWallSprite::Render(swrenderer::RenderThread * thread, short * mfloorclip, short * mceilingclip, int __formal, int __formal, swrenderer::Fake3DTranslucent __formal) Line 242	C++
 	gzdoom.exe!swrenderer::VisibleSprite::Render(swrenderer::RenderThread * thread, swrenderer::Fake3DTranslucent clip3DFloor) Line 507	C++
 	gzdoom.exe!swrenderer::RenderTranslucentPass::DrawMaskedSingle(bool renew, swrenderer::Fake3DTranslucent clip3DFloor) Line 140	C++
 	gzdoom.exe!swrenderer::RenderTranslucentPass::Render() Line 190	C++
 	gzdoom.exe!swrenderer::RenderScene::RenderThreadSlice(swrenderer::RenderThread * thread) Line 297	C++
 	gzdoom.exe!swrenderer::RenderScene::RenderThreadSlices() Line 238	C++
 	gzdoom.exe!swrenderer::RenderScene::RenderActorView(AActor * actor, bool renderPlayerSprites, bool dontmaplines) Line 177	C++
 	gzdoom.exe!swrenderer::RenderScene::RenderView(player_t * player, DCanvas * target, void * videobuffer, int bufferpitch) Line 134	C++
 	gzdoom.exe!FSoftwareRenderer::RenderView(player_t * player, DCanvas * target, void * videobuffer, int bufferpitch) Line 201	C++
 	gzdoom.exe!SWSceneDrawer::RenderView(player_t * player) Line 113	C++
 	gzdoom.exe!OpenGLRenderer::FGLRenderer::RenderView(player_t * player) Line 236	C++
 	[External Code]	
 	[Inline Frame] gzdoom.exe!std::_Func_class<void>::operator()() Line 783	C++
 	[Inline Frame] gzdoom.exe!D_Render(std::function<void __cdecl(void)>) Line 382	C++
 	gzdoom.exe!D_Display() Line 783	C++
 	gzdoom.exe!D_DoomLoop() Line 1040	C++
 	gzdoom.exe!D_DoomMain() Line 2762	C++
 	gzdoom.exe!DoMain(HINSTANCE__ * hInstance) Line 972	C++
 	gzdoom.exe!wWinMain(HINSTANCE__ * hInstance, HINSTANCE__ * nothing, wchar_t * cmdline, int nCmdShow) Line 1304	C++

Re: [4.1.3] WallSprites crash in software renderer

Posted: Fri Oct 04, 2019 4:59 am
by Rachael
That particular call stack isn't being really helpful in this case.

Have you looked at it in the debugger? Are any of those variables null? That is the first thing I would check for.

Re: [4.1.3] WallSprites crash in software renderer

Posted: Fri Oct 04, 2019 5:53 am
by drfrag
Haven't looked into it yet.

Re: [4.1.3] WallSprites crash in software renderer

Posted: Fri Oct 04, 2019 11:35 am
by drfrag
In the last step colormap is NULL:

Code: Select all

>	gzdoom.exe!swrenderer::RenderWallSprite::Render(swrenderer::RenderThread * thread, short * mfloorclip, short * mceilingclip, int __formal, int __formal, swrenderer::Fake3DTranslucent __formal) Line 242	C++

+		WallSpriteTile	0x000001fb41e36480 {mTexture=0x000001fb2bc51500 {mImage=0x000001fb30051dc0 {BitDepth=8 '\b' ColorType=...} } ...}	FSoftwareTexture *
-		drawerargs	{dc_dest=0x000001fb3c29fb55 "\x4\x4\a\a\a\a\x6\x6\x6\x5\x6\x5\x6\x4\x6\x6\x6\x6\x5\x4\x4\b\b\b\x6\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\a\a\b\b\b\b\b\b\b\b\b\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\a\a\b\b\b\b\b\b\b\b\b\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\b\b\b\b\b\b\b\b\b\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\b\b\b\a\a\a\a\a\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\b\b\b\a\a\a\a\a\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a... ...}	swrenderer::SpriteDrawerArgs
-		swrenderer::DrawerArgs	{mBaseColormap=0x0000000000000000 <NULL> mLight=18.2693996 mShade=1376256 ...}	swrenderer::DrawerArgs
+		mBaseColormap	0x0000000000000000 <NULL>	FSWColormap *
		mLight	18.2693996	float
		mShade	1376256	int
+		mTranslation	0x0000000000000000 <NULL>	unsigned char *
+		dc_dest	0x000001fb3c29fb55 "\x4\x4\a\a\a\a\x6\x6\x6\x5\x6\x5\x6\x4\x6\x6\x6\x6\x5\x4\x4\b\b\b\x6\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\a\a\b\b\b\b\b\b\b\b\b\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\a\a\b\b\b\b\b\b\b\b\b\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\b\b\b\b\b\b\b\b\b\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\b\b\b\a\a\a\a\a\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\b\b\b\a\a\a\a\a\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a...	unsigned char *
		dc_dest_y	188	int
		dc_count	132	int
		dc_iscale	64079	int
		dc_texturefrac	2348	int
		dc_texturefracx	0	unsigned int
		dc_textureheight	129	unsigned int
+		dc_source	0x000001fb447bfd90 "ooooljjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjj@jjjjjjjjjj@gggggggggggggggggggg@jjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjggggooooljjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjj@jjjjjjjjjj@ggggggggggg...	const unsigned char *
+		dc_source2	0x0000000000000000 <NULL>	const unsigned char *
		drawer_needs_pal_input	false	bool
+		dc_srcblend	0x0000000000000000 {???}	unsigned int *
+		dc_destblend	0x0000000000000000 {???}	unsigned int *
		dc_srcalpha	65536	int
		dc_destalpha	0	int
		dc_x	237	int
		dc_yl	188	int
		dc_yh	319	int
		dc_color	0	int
		dc_color_bgra	0	unsigned int
		dc_srccolor	0	unsigned int
		dc_srccolor_bgra	0	unsigned int
		dynlightcolor	0	unsigned int
		colfunc	0x00007ff66b0a89e4 {gzdoom.exe![thunk]:swrenderer::SWPixelFormatDrawers::`vcall'{72,{flat}}' }'}	void(swrenderer::SWPixelFormatDrawers::*)(const swrenderer::SpriteDrawerArgs &)
+		dc_viewport	0x000001fb35cabdf0 {WorldToView={Matrix=0x000001fb35cabdf0 {1.55134330e-06, 7.105e-43#DEN, 3.78815912e-07, ...} } ...}	swrenderer::RenderViewport *
+		mceilingclip	0x000001fb35c5ddf8 {0}	short *
+		mfloorclip	0x000001fb35c58038 {613}	short *
		sprflipvert	false	bool
+		this	0x000001fb42b68ce0 {wallc={tleft={X=-61.9186058 Y=94.7692413 } tright={X=-67.9553375 Y=105.006615 } ...} ...}	swrenderer::RenderWallSprite *
+		thread	0x000001fb35c57f90 {Scene=0x000001fb314a6538 {dontmaplines=false clearcolor=0 Threads={ size=1 } ...} ...}	swrenderer::RenderThread *
+		translucentPass	0x000001fb34ca94a0 {Thread=0x000001fb35c57f90 {Scene=0x000001fb314a6538 {dontmaplines=false clearcolor=...} ...} ...}	swrenderer::RenderTranslucentPass *
+		walltexcoords	{VStep=0x00000049b70cf5c0 {0.000000000, 0.000000000, 0.000000000, 0.000000000, 0.000000000, 0.000000000, ...} ...}	swrenderer::ProjectedWallTexcoords
		x	237	int

>	gzdoom.exe!swrenderer::VisibleSprite::Render(swrenderer::RenderThread * thread, swrenderer::Fake3DTranslucent clip3DFloor) Line 507	C++

+		clip3DFloor	{clipBottom=false clipTop=false down2Up=false ...}	swrenderer::Fake3DTranslucent
+		clipbot	0x000001fb35c58038 {613}	short *
-		colormap	0x00007ff66b9545e8 {gzdoom.exe!FDynamicColormap NormalLight} {Maps=0x000001fb35533c30 "" Color={b=255 'ÿ' ...} ...}	FSWColormap *
+		Maps	0x000001fb35533c30 ""	unsigned char *
+		Color	{b=255 'ÿ' g=255 'ÿ' r=255 'ÿ' ...}	PalEntry
+		Fade	{b=0 '\0' g=0 '\0' r=0 '\0' ...}	PalEntry
		Desaturate	0	int
		minvoxely	0	int
+		this	0x000001fb42b68ce0 {wallc={tleft={X=-61.9186058 Y=94.7692413 } tright={X=-67.9553375 Y=105.006615 } ...} ...}	swrenderer::VisibleSprite * {swrenderer::RenderWallSprite}


Re: [4.1.3] WallSprites crash in software renderer

Posted: Fri Oct 04, 2019 11:44 am
by Rachael
In that case, then, check and make sure that all the information being passed to the drawer is correct. I would do a compare between the functions "RenderSprite::Project" and "RenderWallSprite::Project" - make sure both functions are functionally identical, other than the projection algorithms. In worst case scenario, simply disable wallsprites entirely and forward the function to RenderSprite::Project instead, at least to fix the crash until you can come up with a more comprehensive solution.

Re: [4.1.3] WallSprites crash in software renderer

Posted: Fri Oct 04, 2019 11:47 am
by drfrag
Just updated the previous post. To clarify for me it still crashes in master. That trace is from the latest master, is it crashing for you?

Re: [4.1.3] WallSprites crash in software renderer

Posted: Fri Oct 04, 2019 12:17 pm
by Rachael
No, it is not. What are you using to compile it? What are you using to test it?

Re: [4.1.3] WallSprites crash in software renderer

Posted: Fri Oct 04, 2019 12:43 pm
by drfrag
VS 2017 15.9.12 x64 RelWithDebInfo. navy3d.ipk3.