Truecolor software rendering

[dpJudas]

There are mainly two advantages to premultiplied alpha. The first is the saved multiplication (probably not measurable on GPU nowadays, maybe a little bit in software). The second is that texture filtering works more correctly if it is premultiplied. When the alpha isn't premultiplied the color of a transparent pixel bleeds into the result when doing a linear sampling.

As example, lets say we have a red opaque and a blue transparent pixel next to each other: rgba(1,0,0,1) and rgba(0,0,1,0). With linear sampling halfway through (50% of each) we get:

Code: Select all

sampled.r = p0.r * 0.5 + p1.r * 0.5 = 0.5
sampled.g = p0.g * 0.5 + p1.g * 0.5 = 0
sampled.b = p0.b * 0.5 + p1.b * 0.5 = 0.5
sampled.a = p0.a * 0.5 + p1.a * 0.5 = 0.5

If the colors are premultiplied, the textures will contain rgba(1,0,0,0) and rgba(0,0,0,0). That will give this sampling:

Code: Select all

sampled.r = p0.r * 0.5 + p1.r * 0.5 = 0.5
sampled.g = p0.g * 0.5 + p1.g * 0.5 = 0
sampled.b = p0.b * 0.5 + p1.b * 0.5 = 0
sampled.a = p0.a * 0.5 + p1.a * 0.5 = 0.5

Which results in a clean blend to transparent without any artifacts. That is the main reason that premultiplied alpha is so widespread rather than speed.

Btw. I wasn't aware that zdoom sometimes discarded the alpha channel. Under what conditions does this happen - always with one sided walls? Maybe I just haven't seen a map yet where there was transparent pixels in such a one-sided wall. You're right that in such a situation premultiplied alpha wouldn't work.

[Graf Zahl]

Any texture being put on a one-sided wall, on the upper or lower part of a two-sided wall or a non-3D-floor flat will always be drawn without alpha blending. Even fully transparent parts will be rendered black.

[Graf Zahl]

As example, lets say we have a red opaque and a blue transparent pixel next to each other: rgba(1,0,0,1) and rgba(0,0,1,0). With linear sampling halfway through (50% of each) we get:

Code: Select all

sampled.r = p0.r * 0.5 + p1.r * 0.5 = 0.5
sampled.g = p0.g * 0.5 + p1.g * 0.5 = 0
sampled.b = p0.b * 0.5 + p1.b * 0.5 = 0.5
sampled.a = p0.a * 0.5 + p1.a * 0.5 = 0.5

If the colors are premultiplied, the textures will contain rgba(1,0,0,0) and rgba(0,0,0,0). That will give this sampling:

Code: Select all

sampled.r = p0.r * 0.5 + p1.r * 0.5 = 0.5
sampled.g = p0.g * 0.5 + p1.g * 0.5 = 0
sampled.b = p0.b * 0.5 + p1.b * 0.5 = 0
sampled.a = p0.a * 0.5 + p1.a * 0.5 = 0.5

Which results in a clean blend to transparent without any artifacts. That is the main reason that premultiplied alpha is so widespread rather than speed.

There's still artifacts, but less severe. In GZDoom I added a simple filter to fully get rid of them and give all fully transparent edge pixels the color of one of their neighboring pixels.

[dpJudas]

Okay since it sounds zdoom relies on this quite a bit I'll have to change the code to not use premultiplied alpha. I don't think it will affect the performance much. I just generally use premultiplied alpha unless there's good reasons not to. In this case there seems to be.

[Graf Zahl]

Maybe it makes sense to use

bool FTexture::SmoothEdges(unsigned char * buffer,int w, int h)

from GZDoom here, if you are doing some filtering, to reduce the artifacts.

[dpJudas]

Good idea. I do have a linear filter mode that can be toggled on where it would be useful.

[cortlong50]

have you ever felt dumb as shit reading something that went way over your head and so you just kinda exit the forum and hope nobody notices?

thats my cue.

[Edward-san]

I got some time to try the truecolor branch again, but I got this asan error whenever I opened doom2 map01:

Code: Select all

==27141==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7f8ed74dd2b0 at pc 0x000000f164e7 bp 0x7ffe61b43b70 sp 0x7ffe61b43b60
READ of size 4 at 0x7f8ed74dd2b0 thread T0
    #0 0xf164e6 in FTexture::GenerateBgraMipmaps()::Color4f::operator+(Color4f const&) const (/home/edward-san/zdoom/branch/truecolor/debug/gcc/zdoom+0xf164e6)
    #1 0xf17ab8 in FTexture::GenerateBgraMipmaps() /home/edward-san/zdoom/branch/truecolor/src/textures/texture.cpp:486
    #2 0xf161ce in FTexture::GenerateBgraFromBitmap(FBitmap const&) /home/edward-san/zdoom/branch/truecolor/src/textures/texture.cpp:376
    #3 0xf15363 in FTexture::GetPixelsBgra() /home/edward-san/zdoom/branch/truecolor/src/textures/texture.cpp:205
    #4 0x690391 in WallscanSampler::WallscanSampler(int, float, double, int, FTexture*, unsigned char const* (*)(FTexture*, int)) /home/edward-san/zdoom/branch/truecolor/src/r_segs.cpp:1162
    #5 0x693530 in wallscan_any(int, int, short*, short*, float*, int*, double, unsigned char const* (*)(FTexture*, int), void (*)(int, unsigned int (*&)(), void (*&)())) /home/edward-san/zdoom/branch/truecolor/src/r_segs.cpp:1427
    #6 0x694792 in wallscan(int, int, short*, short*, float*, int*, double, unsigned char const* (*)(FTexture*, int)) /home/edward-san/zdoom/branch/truecolor/src/r_segs.cpp:1528
    #7 0x695630 in call_wallscan /home/edward-san/zdoom/branch/truecolor/src/r_segs.cpp:1626
    #8 0x6979f8 in R_RenderSegLoop() /home/edward-san/zdoom/branch/truecolor/src/r_segs.cpp:1871
    #9 0x69f142 in R_StoreWallRange(int, int) /home/edward-san/zdoom/branch/truecolor/src/r_segs.cpp:2580
    #10 0x5c92e1 in R_ClipWallSegment(int, int, bool) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:180
    #11 0x5ccbb6 in R_AddLine(seg_t*) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:769
    #12 0x5d307c in R_Subsector(subsector_t*) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:1356
    #13 0x5d33bc in R_RenderBSPNode(void*) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:1396
    #14 0x5d332d in R_RenderBSPNode(void*) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:1387
    #15 0x5d332d in R_RenderBSPNode(void*) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:1387
    #16 0x5d332d in R_RenderBSPNode(void*) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:1387
    #17 0x5d332d in R_RenderBSPNode(void*) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:1387
    #18 0x5d332d in R_RenderBSPNode(void*) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:1387
    #19 0x5d332d in R_RenderBSPNode(void*) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:1387
    #20 0x674590 in R_RenderActorView(AActor*, bool) /home/edward-san/zdoom/branch/truecolor/src/r_main.cpp:900
    #21 0x5c487c in FSoftwareRenderer::RenderView(player_t*) /home/edward-san/zdoom/branch/truecolor/src/r_swrenderer.cpp:174
    #22 0x8dab32 in D_Display() /home/edward-san/zdoom/branch/truecolor/src/d_main.cpp:770
    #23 0x8dcadb in D_DoomLoop() /home/edward-san/zdoom/branch/truecolor/src/d_main.cpp:1013
    #24 0x8e3c91 in D_DoomMain() /home/edward-san/zdoom/branch/truecolor/src/d_main.cpp:2642
    #25 0x5b8ecd in main /home/edward-san/zdoom/branch/truecolor/src/posix/sdl/i_main.cpp:317
    #26 0x7f8ef3ae882f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
    #27 0x5aa0e8 in _start (/home/edward-san/zdoom/branch/truecolor/debug/gcc/zdoom+0x5aa0e8)

0x7f8ed74dd2b0 is located 0 bytes to the right of 174768-byte region [0x7f8ed74b2800,0x7f8ed74dd2b0)
allocated by thread T0 here:
    #0 0x7f8ef6e04532 in operator new(unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x99532)
    #1 0xf1bb72 in allocate /usr/include/c++/5/ext/new_allocator.h:104
    #2 0xf1ba96 in allocate /usr/include/c++/5/bits/alloc_traits.h:491
    #3 0xf1b9ef in _M_allocate /usr/include/c++/5/bits/stl_vector.h:170
    #4 0xf1b878 in _M_create_storage /usr/include/c++/5/bits/stl_vector.h:185
    #5 0xf1b612 in _Vector_base /usr/include/c++/5/bits/stl_vector.h:136
    #6 0xf1b453 in vector /usr/include/c++/5/bits/stl_vector.h:278
    #7 0xf16d8b in FTexture::GenerateBgraMipmaps() /home/edward-san/zdoom/branch/truecolor/src/textures/texture.cpp:419
    #8 0xf161ce in FTexture::GenerateBgraFromBitmap(FBitmap const&) /home/edward-san/zdoom/branch/truecolor/src/textures/texture.cpp:376
    #9 0xf15363 in FTexture::GetPixelsBgra() /home/edward-san/zdoom/branch/truecolor/src/textures/texture.cpp:205
    #10 0x690391 in WallscanSampler::WallscanSampler(int, float, double, int, FTexture*, unsigned char const* (*)(FTexture*, int)) /home/edward-san/zdoom/branch/truecolor/src/r_segs.cpp:1162
    #11 0x693530 in wallscan_any(int, int, short*, short*, float*, int*, double, unsigned char const* (*)(FTexture*, int), void (*)(int, unsigned int (*&)(), void (*&)())) /home/edward-san/zdoom/branch/truecolor/src/r_segs.cpp:1427
    #12 0x694792 in wallscan(int, int, short*, short*, float*, int*, double, unsigned char const* (*)(FTexture*, int)) /home/edward-san/zdoom/branch/truecolor/src/r_segs.cpp:1528
    #13 0x695630 in call_wallscan /home/edward-san/zdoom/branch/truecolor/src/r_segs.cpp:1626
    #14 0x6979f8 in R_RenderSegLoop() /home/edward-san/zdoom/branch/truecolor/src/r_segs.cpp:1871
    #15 0x69f142 in R_StoreWallRange(int, int) /home/edward-san/zdoom/branch/truecolor/src/r_segs.cpp:2580
    #16 0x5c92e1 in R_ClipWallSegment(int, int, bool) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:180
    #17 0x5ccbb6 in R_AddLine(seg_t*) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:769
    #18 0x5d307c in R_Subsector(subsector_t*) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:1356
    #19 0x5d33bc in R_RenderBSPNode(void*) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:1396
    #20 0x5d332d in R_RenderBSPNode(void*) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:1387
    #21 0x5d332d in R_RenderBSPNode(void*) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:1387
    #22 0x5d332d in R_RenderBSPNode(void*) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:1387
    #23 0x5d332d in R_RenderBSPNode(void*) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:1387
    #24 0x5d332d in R_RenderBSPNode(void*) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:1387
    #25 0x5d332d in R_RenderBSPNode(void*) /home/edward-san/zdoom/branch/truecolor/src/r_bsp.cpp:1387
    #26 0x674590 in R_RenderActorView(AActor*, bool) /home/edward-san/zdoom/branch/truecolor/src/r_main.cpp:900
    #27 0x5c487c in FSoftwareRenderer::RenderView(player_t*) /home/edward-san/zdoom/branch/truecolor/src/r_swrenderer.cpp:174
    #28 0x8dab32 in D_Display() /home/edward-san/zdoom/branch/truecolor/src/d_main.cpp:770
    #29 0x8dcadb in D_DoomLoop() /home/edward-san/zdoom/branch/truecolor/src/d_main.cpp:1013

SUMMARY: AddressSanitizer: heap-buffer-overflow ??:0 FTexture::GenerateBgraMipmaps()::Color4f::operator+(Color4f const&) const
Shadow bytes around the buggy address:
  0x0ff25ae93a00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ff25ae93a10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ff25ae93a20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ff25ae93a30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ff25ae93a40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0ff25ae93a50: 00 00 00 00 00 00[fa]fa fa fa fa fa fa fa fa fa
  0x0ff25ae93a60: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0ff25ae93a70: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0ff25ae93a80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0ff25ae93a90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0ff25ae93aa0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
==27141==ABORTING
Terminated sig=0x0d

[dpJudas]

have you ever felt dumb as shit reading something that went way over your head and so you just kinda exit the forum and hope nobody notices?

thats my cue.

I wouldn't feel bad it. There are other threads where I barely know what they are talking about either.

[dpJudas]

I got some time to try the truecolor branch again, but I got this asan error whenever I opened doom2 map01:

Buffer overflow in the mipmap generator code? Time for me to add some asserts! Thanks for making me aware of it.

[enderkevin13]

This sounds like it could be a great idea!

[dpJudas]

A quick update on this PR: I've completed the changes requested by Graf and fixed the buffer overrun identified by Edward-san.

[Major Cooke]

I gave this a try out and damn... This is actually pretty awesome how it turned out. This surely is one way of bridging the gap a little closer to GZDoom (even though it's still quite far off).

[Gamer With Dignity]

Can I expect this to be patched into a zdoom update in the near future?

[Rachael]

You ... really want to "expect" that?