[4.1.3] WallSprites crash in software renderer

[SanyaWaffles]

https://www.dropbox.com/s/tbyujjqpw2sj5 ... .ipk3?dl=0

I was working on a project using wallsprites and I notice it crashes in software mode (and looks weird in true-color mode) but works on every other mode, including softpoly, true color softpoly and hardware accelerated.

---

Hardware:


True Color Softpoly


Softpoly


True Color Software

Classic Software crashes with no crash dump generated (it doesn't crash gracefully).

[Rachael]

I am curious, are you using any texture resize mode? I wonder if it's related to this bug.

[SanyaWaffles]

Code: Select all

Texture Filter mode: Trilinear
Aniscotropic filter: 8x
Enable highres textures: Yes
High Qualitry Reszie mode: Off
High Quality Resize multiplier: Off

Resize textures: On
Resize sprites: On
Resize fonts: On
Precache GL texture: No
Sort draw lists by texture: No

Those are my settings.

[Rachael]

Okay, so it's not related. Thanks.

[drfrag]

Bumping just to notice that the OP has deleted the sample, not that i'm surprised that wallsprites crash.

[SanyaWaffles]

Rachael did fix this just now, though there's still a problem with the renderer not grabbing the sprite properly.

Sorry about the file going away.

[Rachael]

The crash has fixed, but as Sanya said, the texture is not being retrieved properly. While the Y-coordinates work properly (from what I can see), the X coordinate is dependent on Z-depth and does not currently work.

Additionally, activating a texture resize mode will cause further problems with the texture.

As for the crash, it was caused by missing colormap data, and was fixed here.

[drfrag]

I've just checked in master and for me it still crashes with render mode 0.

Code: Select all

 	gzdoom.exe!swrenderer::DrawColumnPalCommand::Execute(DrawerThread * thread) Line 908	C++
 	[Inline Frame] gzdoom.exe!DrawerCommandQueue::Push(const swrenderer::SpriteDrawerArgs &) Line 207	C++
 	gzdoom.exe!swrenderer::SWPalDrawers::DrawColumn(const swrenderer::SpriteDrawerArgs & args) Line 249	C++
 	gzdoom.exe!swrenderer::SpriteDrawerArgs::DrawMaskedColumn(swrenderer::RenderThread * thread, int x, int iscale, FSoftwareTexture * tex, int col, double spryscale, double sprtopscreen, bool sprflipvert, const short * mfloorclip, const short * mceilingclip, FRenderStyle style, bool unmasked) Line 127	C++
 	[Inline Frame] gzdoom.exe!swrenderer::RenderWallSprite::DrawColumn(swrenderer::RenderThread * style, swrenderer::SpriteDrawerArgs &) Line 258	C++
>	gzdoom.exe!swrenderer::RenderWallSprite::Render(swrenderer::RenderThread * thread, short * mfloorclip, short * mceilingclip, int __formal, int __formal, swrenderer::Fake3DTranslucent __formal) Line 242	C++
 	gzdoom.exe!swrenderer::VisibleSprite::Render(swrenderer::RenderThread * thread, swrenderer::Fake3DTranslucent clip3DFloor) Line 507	C++
 	gzdoom.exe!swrenderer::RenderTranslucentPass::DrawMaskedSingle(bool renew, swrenderer::Fake3DTranslucent clip3DFloor) Line 140	C++
 	gzdoom.exe!swrenderer::RenderTranslucentPass::Render() Line 190	C++
 	gzdoom.exe!swrenderer::RenderScene::RenderThreadSlice(swrenderer::RenderThread * thread) Line 297	C++
 	gzdoom.exe!swrenderer::RenderScene::RenderThreadSlices() Line 238	C++
 	gzdoom.exe!swrenderer::RenderScene::RenderActorView(AActor * actor, bool renderPlayerSprites, bool dontmaplines) Line 177	C++
 	gzdoom.exe!swrenderer::RenderScene::RenderView(player_t * player, DCanvas * target, void * videobuffer, int bufferpitch) Line 134	C++
 	gzdoom.exe!FSoftwareRenderer::RenderView(player_t * player, DCanvas * target, void * videobuffer, int bufferpitch) Line 201	C++
 	gzdoom.exe!SWSceneDrawer::RenderView(player_t * player) Line 113	C++
 	gzdoom.exe!OpenGLRenderer::FGLRenderer::RenderView(player_t * player) Line 236	C++
 	[External Code]	
 	[Inline Frame] gzdoom.exe!std::_Func_class<void>::operator()() Line 783	C++
 	[Inline Frame] gzdoom.exe!D_Render(std::function<void __cdecl(void)>) Line 382	C++
 	gzdoom.exe!D_Display() Line 783	C++
 	gzdoom.exe!D_DoomLoop() Line 1040	C++
 	gzdoom.exe!D_DoomMain() Line 2762	C++
 	gzdoom.exe!DoMain(HINSTANCE__ * hInstance) Line 972	C++
 	gzdoom.exe!wWinMain(HINSTANCE__ * hInstance, HINSTANCE__ * nothing, wchar_t * cmdline, int nCmdShow) Line 1304	C++

[Rachael]

That particular call stack isn't being really helpful in this case.

Have you looked at it in the debugger? Are any of those variables null? That is the first thing I would check for.

[drfrag]

Haven't looked into it yet.

[drfrag]

In the last step colormap is NULL:

Code: Select all

>	gzdoom.exe!swrenderer::RenderWallSprite::Render(swrenderer::RenderThread * thread, short * mfloorclip, short * mceilingclip, int __formal, int __formal, swrenderer::Fake3DTranslucent __formal) Line 242	C++

+		WallSpriteTile	0x000001fb41e36480 {mTexture=0x000001fb2bc51500 {mImage=0x000001fb30051dc0 {BitDepth=8 '\b' ColorType=...} } ...}	FSoftwareTexture *
-		drawerargs	{dc_dest=0x000001fb3c29fb55 "\x4\x4\a\a\a\a\x6\x6\x6\x5\x6\x5\x6\x4\x6\x6\x6\x6\x5\x4\x4\b\b\b\x6\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\a\a\b\b\b\b\b\b\b\b\b\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\a\a\b\b\b\b\b\b\b\b\b\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\b\b\b\b\b\b\b\b\b\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\b\b\b\a\a\a\a\a\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\b\b\b\a\a\a\a\a\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a... ...}	swrenderer::SpriteDrawerArgs
-		swrenderer::DrawerArgs	{mBaseColormap=0x0000000000000000 <NULL> mLight=18.2693996 mShade=1376256 ...}	swrenderer::DrawerArgs
+		mBaseColormap	0x0000000000000000 <NULL>	FSWColormap *
		mLight	18.2693996	float
		mShade	1376256	int
+		mTranslation	0x0000000000000000 <NULL>	unsigned char *
+		dc_dest	0x000001fb3c29fb55 "\x4\x4\a\a\a\a\x6\x6\x6\x5\x6\x5\x6\x4\x6\x6\x6\x6\x5\x4\x4\b\b\b\x6\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\a\a\b\b\b\b\b\b\b\b\b\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\a\a\b\b\b\b\b\b\b\b\b\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\b\b\b\b\b\b\b\b\b\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\b\b\b\a\a\a\a\a\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\b\b\b\a\a\a\a\a\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a...	unsigned char *
		dc_dest_y	188	int
		dc_count	132	int
		dc_iscale	64079	int
		dc_texturefrac	2348	int
		dc_texturefracx	0	unsigned int
		dc_textureheight	129	unsigned int
+		dc_source	0x000001fb447bfd90 "ooooljjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjj@jjjjjjjjjj@gggggggggggggggggggg@jjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjggggooooljjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjj@jjjjjjjjjj@ggggggggggg...	const unsigned char *
+		dc_source2	0x0000000000000000 <NULL>	const unsigned char *
		drawer_needs_pal_input	false	bool
+		dc_srcblend	0x0000000000000000 {???}	unsigned int *
+		dc_destblend	0x0000000000000000 {???}	unsigned int *
		dc_srcalpha	65536	int
		dc_destalpha	0	int
		dc_x	237	int
		dc_yl	188	int
		dc_yh	319	int
		dc_color	0	int
		dc_color_bgra	0	unsigned int
		dc_srccolor	0	unsigned int
		dc_srccolor_bgra	0	unsigned int
		dynlightcolor	0	unsigned int
		colfunc	0x00007ff66b0a89e4 {gzdoom.exe![thunk]:swrenderer::SWPixelFormatDrawers::`vcall'{72,{flat}}' }'}	void(swrenderer::SWPixelFormatDrawers::*)(const swrenderer::SpriteDrawerArgs &)
+		dc_viewport	0x000001fb35cabdf0 {WorldToView={Matrix=0x000001fb35cabdf0 {1.55134330e-06, 7.105e-43#DEN, 3.78815912e-07, ...} } ...}	swrenderer::RenderViewport *
+		mceilingclip	0x000001fb35c5ddf8 {0}	short *
+		mfloorclip	0x000001fb35c58038 {613}	short *
		sprflipvert	false	bool
+		this	0x000001fb42b68ce0 {wallc={tleft={X=-61.9186058 Y=94.7692413 } tright={X=-67.9553375 Y=105.006615 } ...} ...}	swrenderer::RenderWallSprite *
+		thread	0x000001fb35c57f90 {Scene=0x000001fb314a6538 {dontmaplines=false clearcolor=0 Threads={ size=1 } ...} ...}	swrenderer::RenderThread *
+		translucentPass	0x000001fb34ca94a0 {Thread=0x000001fb35c57f90 {Scene=0x000001fb314a6538 {dontmaplines=false clearcolor=...} ...} ...}	swrenderer::RenderTranslucentPass *
+		walltexcoords	{VStep=0x00000049b70cf5c0 {0.000000000, 0.000000000, 0.000000000, 0.000000000, 0.000000000, 0.000000000, ...} ...}	swrenderer::ProjectedWallTexcoords
		x	237	int

>	gzdoom.exe!swrenderer::VisibleSprite::Render(swrenderer::RenderThread * thread, swrenderer::Fake3DTranslucent clip3DFloor) Line 507	C++

+		clip3DFloor	{clipBottom=false clipTop=false down2Up=false ...}	swrenderer::Fake3DTranslucent
+		clipbot	0x000001fb35c58038 {613}	short *
-		colormap	0x00007ff66b9545e8 {gzdoom.exe!FDynamicColormap NormalLight} {Maps=0x000001fb35533c30 "" Color={b=255 'ÿ' ...} ...}	FSWColormap *
+		Maps	0x000001fb35533c30 ""	unsigned char *
+		Color	{b=255 'ÿ' g=255 'ÿ' r=255 'ÿ' ...}	PalEntry
+		Fade	{b=0 '\0' g=0 '\0' r=0 '\0' ...}	PalEntry
		Desaturate	0	int
		minvoxely	0	int
+		this	0x000001fb42b68ce0 {wallc={tleft={X=-61.9186058 Y=94.7692413 } tright={X=-67.9553375 Y=105.006615 } ...} ...}	swrenderer::VisibleSprite * {swrenderer::RenderWallSprite}

[Rachael]

In that case, then, check and make sure that all the information being passed to the drawer is correct. I would do a compare between the functions "RenderSprite::Project" and "RenderWallSprite::Project" - make sure both functions are functionally identical, other than the projection algorithms. In worst case scenario, simply disable wallsprites entirely and forward the function to RenderSprite::Project instead, at least to fix the crash until you can come up with a more comprehensive solution.

[drfrag]

Just updated the previous post. To clarify for me it still crashes in master. That trace is from the latest master, is it crashing for you?

[Rachael]

No, it is not. What are you using to compile it? What are you using to test it?

[drfrag]

VS 2017 15.9.12 x64 RelWithDebInfo. navy3d.ipk3.