Page 1 of 2

[4.1.3] WallSprites crash in software renderer

PostPosted: Fri Jul 19, 2019 12:23 am
by SanyaWaffles
https://www.dropbox.com/s/tbyujjqpw2sj5q3/navy3d.ipk3?dl=0

I was working on a project using wallsprites and I notice it crashes in software mode (and looks weird in true-color mode) but works on every other mode, including softpoly, true color softpoly and hardware accelerated.

Spoiler: Screenshots


Classic Software crashes with no crash dump generated (it doesn't crash gracefully).

Re: [4.1.3] WallSprites crash in software renderer

PostPosted: Fri Jul 19, 2019 12:30 pm
by Rachael
I am curious, are you using any texture resize mode? I wonder if it's related to this bug.

Re: [4.1.3] WallSprites crash in software renderer

PostPosted: Fri Jul 19, 2019 7:25 pm
by SanyaWaffles
Code: Select allExpand view
Texture Filter mode: Trilinear
Aniscotropic filter: 8x
Enable highres textures: Yes
High Qualitry Reszie mode: Off
High Quality Resize multiplier: Off

Resize textures: On
Resize sprites: On
Resize fonts: On
Precache GL texture: No
Sort draw lists by texture: No


Those are my settings.

Re: [4.1.3] WallSprites crash in software renderer

PostPosted: Fri Jul 19, 2019 7:32 pm
by Rachael
Okay, so it's not related. Thanks. :)

Re: [4.1.3] WallSprites crash in software renderer

PostPosted: Fri Sep 20, 2019 3:27 am
by drfrag
Bumping just to notice that the OP has deleted the sample, not that i'm surprised that wallsprites crash.

Re: [4.1.3] WallSprites crash in software renderer

PostPosted: Thu Oct 03, 2019 9:49 pm
by SanyaWaffles
Rachael did fix this just now, though there's still a problem with the renderer not grabbing the sprite properly.

Sorry about the file going away.

Re: [4.1.3] WallSprites crash in software renderer

PostPosted: Thu Oct 03, 2019 9:57 pm
by Rachael
The crash has fixed, but as Sanya said, the texture is not being retrieved properly. While the Y-coordinates work properly (from what I can see), the X coordinate is dependent on Z-depth and does not currently work.

Additionally, activating a texture resize mode will cause further problems with the texture.

As for the crash, it was caused by missing colormap data, and was fixed here.

Re: [4.1.3] WallSprites crash in software renderer

PostPosted: Fri Oct 04, 2019 4:32 am
by drfrag
I've just checked in master and for me it still crashes with render mode 0.

Code: Select allExpand view
    gzdoom.exe!swrenderer::DrawColumnPalCommand::Execute(DrawerThread * thread) Line 908   C++
    [Inline Frame] gzdoom.exe!DrawerCommandQueue::Push(const swrenderer::SpriteDrawerArgs &) Line 207   C++
    gzdoom.exe!swrenderer::SWPalDrawers::DrawColumn(const swrenderer::SpriteDrawerArgs & args) Line 249   C++
    gzdoom.exe!swrenderer::SpriteDrawerArgs::DrawMaskedColumn(swrenderer::RenderThread * thread, int x, int iscale, FSoftwareTexture * tex, int col, double spryscale, double sprtopscreen, bool sprflipvert, const short * mfloorclip, const short * mceilingclip, FRenderStyle style, bool unmasked) Line 127   C++
    [Inline Frame] gzdoom.exe!swrenderer::RenderWallSprite::DrawColumn(swrenderer::RenderThread * style, swrenderer::SpriteDrawerArgs &) Line 258   C++
>   gzdoom.exe!swrenderer::RenderWallSprite::Render(swrenderer::RenderThread * thread, short * mfloorclip, short * mceilingclip, int __formal, int __formal, swrenderer::Fake3DTranslucent __formal) Line 242   C++
    gzdoom.exe!swrenderer::VisibleSprite::Render(swrenderer::RenderThread * thread, swrenderer::Fake3DTranslucent clip3DFloor) Line 507   C++
    gzdoom.exe!swrenderer::RenderTranslucentPass::DrawMaskedSingle(bool renew, swrenderer::Fake3DTranslucent clip3DFloor) Line 140   C++
    gzdoom.exe!swrenderer::RenderTranslucentPass::Render() Line 190   C++
    gzdoom.exe!swrenderer::RenderScene::RenderThreadSlice(swrenderer::RenderThread * thread) Line 297   C++
    gzdoom.exe!swrenderer::RenderScene::RenderThreadSlices() Line 238   C++
    gzdoom.exe!swrenderer::RenderScene::RenderActorView(AActor * actor, bool renderPlayerSprites, bool dontmaplines) Line 177   C++
    gzdoom.exe!swrenderer::RenderScene::RenderView(player_t * player, DCanvas * target, void * videobuffer, int bufferpitch) Line 134   C++
    gzdoom.exe!FSoftwareRenderer::RenderView(player_t * player, DCanvas * target, void * videobuffer, int bufferpitch) Line 201   C++
    gzdoom.exe!SWSceneDrawer::RenderView(player_t * player) Line 113   C++
    gzdoom.exe!OpenGLRenderer::FGLRenderer::RenderView(player_t * player) Line 236   C++
    [External Code]   
    [Inline Frame] gzdoom.exe!std::_Func_class<void>::operator()() Line 783   C++
    [Inline Frame] gzdoom.exe!D_Render(std::function<void __cdecl(void)>) Line 382   C++
    gzdoom.exe!D_Display() Line 783   C++
    gzdoom.exe!D_DoomLoop() Line 1040   C++
    gzdoom.exe!D_DoomMain() Line 2762   C++
    gzdoom.exe!DoMain(HINSTANCE__ * hInstance) Line 972   C++
    gzdoom.exe!wWinMain(HINSTANCE__ * hInstance, HINSTANCE__ * nothing, wchar_t * cmdline, int nCmdShow) Line 1304   C++

Re: [4.1.3] WallSprites crash in software renderer

PostPosted: Fri Oct 04, 2019 4:59 am
by Rachael
That particular call stack isn't being really helpful in this case.

Have you looked at it in the debugger? Are any of those variables null? That is the first thing I would check for.

Re: [4.1.3] WallSprites crash in software renderer

PostPosted: Fri Oct 04, 2019 5:53 am
by drfrag
Haven't looked into it yet.

Re: [4.1.3] WallSprites crash in software renderer

PostPosted: Fri Oct 04, 2019 11:35 am
by drfrag
In the last step colormap is NULL:

Code: Select allExpand view
>   gzdoom.exe!swrenderer::RenderWallSprite::Render(swrenderer::RenderThread * thread, short * mfloorclip, short * mceilingclip, int __formal, int __formal, swrenderer::Fake3DTranslucent __formal) Line 242   C++

+      WallSpriteTile   0x000001fb41e36480 {mTexture=0x000001fb2bc51500 {mImage=0x000001fb30051dc0 {BitDepth=8 '\b' ColorType=...} } ...}   FSoftwareTexture *
-      drawerargs   {dc_dest=0x000001fb3c29fb55 "\x4\x4\a\a\a\a\x6\x6\x6\x5\x6\x5\x6\x4\x6\x6\x6\x6\x5\x4\x4\b\b\b\x6\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\a\a\b\b\b\b\b\b\b\b\b\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\a\a\b\b\b\b\b\b\b\b\b\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\b\b\b\b\b\b\b\b\b\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\b\b\b\a\a\a\a\a\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\b\b\b\a\a\a\a\a\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a... ...}   swrenderer::SpriteDrawerArgs
-      swrenderer::DrawerArgs   {mBaseColormap=0x0000000000000000 <NULL> mLight=18.2693996 mShade=1376256 ...}   swrenderer::DrawerArgs
+      mBaseColormap   0x0000000000000000 <NULL>   FSWColormap *
      mLight   18.2693996   float
      mShade   1376256   int
+      mTranslation   0x0000000000000000 <NULL>   unsigned char *
+      dc_dest   0x000001fb3c29fb55 "\x4\x4\a\a\a\a\x6\x6\x6\x5\x6\x5\x6\x4\x6\x6\x6\x6\x5\x4\x4\b\b\b\x6\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\a\a\b\b\b\b\b\b\b\b\b\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\a\a\b\b\b\b\b\b\b\b\b\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\b\b\b\b\b\b\b\b\b\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\b\b\b\a\a\a\a\a\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\b\b\b\a\a\a\a\a\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\x6\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a\a...   unsigned char *
      dc_dest_y   188   int
      dc_count   132   int
      dc_iscale   64079   int
      dc_texturefrac   2348   int
      dc_texturefracx   0   unsigned int
      dc_textureheight   129   unsigned int
+      dc_source   0x000001fb447bfd90 "ooooljjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjj@jjjjjjjjjj@gggggggggggggggggggg@jjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjggggooooljjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjj@jjjjjjjjjj@ggggggggggg...   const unsigned char *
+      dc_source2   0x0000000000000000 <NULL>   const unsigned char *
      drawer_needs_pal_input   false   bool
+      dc_srcblend   0x0000000000000000 {???}   unsigned int *
+      dc_destblend   0x0000000000000000 {???}   unsigned int *
      dc_srcalpha   65536   int
      dc_destalpha   0   int
      dc_x   237   int
      dc_yl   188   int
      dc_yh   319   int
      dc_color   0   int
      dc_color_bgra   0   unsigned int
      dc_srccolor   0   unsigned int
      dc_srccolor_bgra   0   unsigned int
      dynlightcolor   0   unsigned int
      colfunc   0x00007ff66b0a89e4 {gzdoom.exe![thunk]:swrenderer::SWPixelFormatDrawers::`vcall'{72,{flat}}' }'}   void(swrenderer::SWPixelFormatDrawers::*)(const swrenderer::SpriteDrawerArgs &)
+      dc_viewport   0x000001fb35cabdf0 {WorldToView={Matrix=0x000001fb35cabdf0 {1.55134330e-06, 7.105e-43#DEN, 3.78815912e-07, ...} } ...}   swrenderer::RenderViewport *
+      mceilingclip   0x000001fb35c5ddf8 {0}   short *
+      mfloorclip   0x000001fb35c58038 {613}   short *
      sprflipvert   false   bool
+      this   0x000001fb42b68ce0 {wallc={tleft={X=-61.9186058 Y=94.7692413 } tright={X=-67.9553375 Y=105.006615 } ...} ...}   swrenderer::RenderWallSprite *
+      thread   0x000001fb35c57f90 {Scene=0x000001fb314a6538 {dontmaplines=false clearcolor=0 Threads={ size=1 } ...} ...}   swrenderer::RenderThread *
+      translucentPass   0x000001fb34ca94a0 {Thread=0x000001fb35c57f90 {Scene=0x000001fb314a6538 {dontmaplines=false clearcolor=...} ...} ...}   swrenderer::RenderTranslucentPass *
+      walltexcoords   {VStep=0x00000049b70cf5c0 {0.000000000, 0.000000000, 0.000000000, 0.000000000, 0.000000000, 0.000000000, ...} ...}   swrenderer::ProjectedWallTexcoords
      x   237   int

>   gzdoom.exe!swrenderer::VisibleSprite::Render(swrenderer::RenderThread * thread, swrenderer::Fake3DTranslucent clip3DFloor) Line 507   C++

+      clip3DFloor   {clipBottom=false clipTop=false down2Up=false ...}   swrenderer::Fake3DTranslucent
+      clipbot   0x000001fb35c58038 {613}   short *
-      colormap   0x00007ff66b9545e8 {gzdoom.exe!FDynamicColormap NormalLight} {Maps=0x000001fb35533c30 "" Color={b=255 'ÿ' ...} ...}   FSWColormap *
+      Maps   0x000001fb35533c30 ""   unsigned char *
+      Color   {b=255 'ÿ' g=255 'ÿ' r=255 'ÿ' ...}   PalEntry
+      Fade   {b=0 '\0' g=0 '\0' r=0 '\0' ...}   PalEntry
      Desaturate   0   int
      minvoxely   0   int
+      this   0x000001fb42b68ce0 {wallc={tleft={X=-61.9186058 Y=94.7692413 } tright={X=-67.9553375 Y=105.006615 } ...} ...}   swrenderer::VisibleSprite * {swrenderer::RenderWallSprite}


Re: [4.1.3] WallSprites crash in software renderer

PostPosted: Fri Oct 04, 2019 11:44 am
by Rachael
In that case, then, check and make sure that all the information being passed to the drawer is correct. I would do a compare between the functions "RenderSprite::Project" and "RenderWallSprite::Project" - make sure both functions are functionally identical, other than the projection algorithms. In worst case scenario, simply disable wallsprites entirely and forward the function to RenderSprite::Project instead, at least to fix the crash until you can come up with a more comprehensive solution.

Re: [4.1.3] WallSprites crash in software renderer

PostPosted: Fri Oct 04, 2019 11:47 am
by drfrag
Just updated the previous post. To clarify for me it still crashes in master. That trace is from the latest master, is it crashing for you?

Re: [4.1.3] WallSprites crash in software renderer

PostPosted: Fri Oct 04, 2019 12:17 pm
by Rachael
No, it is not. What are you using to compile it? What are you using to test it?

Re: [4.1.3] WallSprites crash in software renderer

PostPosted: Fri Oct 04, 2019 12:43 pm
by drfrag
VS 2017 15.9.12 x64 RelWithDebInfo. navy3d.ipk3.