[Added] Shadowmaps for GZDoom (can be tested in QZDoom now)

Moderator: GZDoom Developers

Shadowmaps for GZDoom (can be tested in QZDoom now)

Postby Rachael » Fri Mar 10, 2017 11:45 pm

(split)

However - on the OpenGL side, dpJudas went ahead and merged in his shadowmaps work to the mainline. The dev builds now include it, however it has to be turned on.

Currently, there is an issue where you will not be able to use them on Intel cards. (I'll probably end up adding a hack to make them usable if you have at least OpenGL 4.3, but disabling the check that Graf put in breaks start-up so that is not an option) (fixed: hack has been added in)

Otherwise, your card must support storage buffers (a feature that became core in 4.3). You must also specifically enable it in the menu (it's under dynamic lights).

This needs tested. Thoroughly, rigorously, and all that good stuff. :)
User avatar
Rachael
Admin
 
Joined: 13 Jan 2004
Discord: Rachael#3767
Twitch ID: madamerachelle
Github ID: madame-rachelle
Operating System: Debian-like Linux (Debian, Ubuntu, Mint, etc) 64-bit
OS Test Version: No (Using Stable Public Version)
Graphics Processor: nVidia with Vulkan support

Shadowmaps for GZDoom (can be tested in QZDoom now)

Postby ZZYZX » Sat Mar 11, 2017 1:43 am

Testing? I has testing!

1. Will shadowmaps ever be supported under GL4.3? (or I misread it and its working on GL3 as well, just not core?)
2. I hope shadowmaps are still under heavy development because otherwise this is a bug: http://i.imgur.com/838xL3W.png
3. are there any plans on making it work with midtextures and sprites? After all isn't that why the shadowmap and not stencil?

Bonus point: :bang: shadowmaps in GZDB :bang: :bang: :bang: (actually this is most likely not possible at all due to complete absence of any culling and resulting 0 fps, however I remember MaxED posting some weird screenshots...)
User avatar
ZZYZX
le chat du rabbin
 
 
 
Joined: 14 Oct 2012
Location: Ukraine
Discord: ZZYZX#1394
Github ID: jewalky

Re: QZDoom - ZDoom with True-Color (Version 1.2.2 released!)

Postby Rachael » Sat Mar 11, 2017 2:08 am

I can't answer the rest of your post except for one point here:

ZZYZX wrote:1. Will shadowmaps ever be supported under GL4.3? (or I misread it and its working on GL3 as well, just not core?)


The shadowmaps code checks for the "GL_ARB_buffer_storage" extension, and "GL_ARB_shader_storage_buffer_object" extension. If you have it, you can use it.

Here's the code that checks it and enables it: https://github.com/raa-eruanna/qzdoom/b ... #L230-L245
User avatar
Rachael
Admin
 
Joined: 13 Jan 2004
Discord: Rachael#3767
Twitch ID: madamerachelle
Github ID: madame-rachelle
Operating System: Debian-like Linux (Debian, Ubuntu, Mint, etc) 64-bit
OS Test Version: No (Using Stable Public Version)
Graphics Processor: nVidia with Vulkan support

Re: QZDoom - ZDoom with True-Color (Version 1.2.2 released!)

Postby Nash » Sat Mar 11, 2017 6:45 am

I would really like to see 2-sided lines get shadows too, eventually... it's weird putting a light on top of a table or drawer or something and only the side walls cast shadows, but not the surface itself... and also light bleeding through closed doors...

It's already amazing that it already works, but IMO it's just not quite "there" yet. :)
User avatar
Nash
AKA Nash Muhandes! Twitter/Facebook/Youtube: nashmuhandes
 
 
 
Joined: 27 Oct 2003
Location: Kuala Lumpur, Malaysia
Twitch ID: nashmuhandes
Github ID: nashmuhandes

Re: QZDoom - ZDoom with True-Color (Version 1.2.2 released!)

Postby dpJudas » Sat Mar 11, 2017 7:29 am

ZZYZX wrote:1. Will shadowmaps ever be supported under GL4.3? (or I misread it and its working on GL3 as well, just not core?)

The storage buffer (4.3) requirement is there because that is the way I upload the level collision data. While I could theoretically use a texture and texelFetch, or uniform buffers if the structure is small enough, the hardware required to run this acceptable will probably be OpenGL 4 compliant anyway.

2. I hope shadowmaps are still under heavy development because otherwise this is a bug

This has to do with the PCF filtering. I haven't noticed such stuff happening very often. You're welcome to see if you can improve the quality of the PCF filtering, but note that all games have trouble with their shadow maps if you look closely enough. There's also a performance vs quality aspect here. GTA SA wouldn't have shipped because sometimes the shadows go through buildings, BFBC2 have blinking shadows, and Skyrim have shadows going super blinking all over every time the sun moves (which happens every 30 seconds or something in that game).

3. are there any plans on making it work with midtextures and sprites? After all isn't that why the shadowmap and not stencil?

The code implements 1D shadow maps - not 2D shadow maps as you know it from most games. The point is to generate shadows from all light sources at real time, giving at least some amount of blocking of light without baking lightmaps.

The problem with real 2D shadow maps is that they are a lot more expensive to generate, which means you have to actively select which lights casts shadows and which does not. No modern engine I know of can do this automatically - they all rely on the modder/level designer taking an active role in selecting lights that casts shadows, and possibly also which geometry gets drawn into the shadow maps. Such a feature can only ever work on new maps. Note that one type of shadow map doesn't rule out the other - in the future it could be coded that a light is selected as a 2D shadow map caster.

Making sprites cast shadows is nasty due to portals and the fact it needs to collect which sprites are visible from an arbitrary "sun" light source. It might be easier to just code in a second sprite drawn the way Nash did it. Only for 3D models would it be worth the effort to try render it from a light.

Nash wrote:I would really like to see 2-sided lines get shadows too, eventually... it's weird putting a light on top of a table or drawer or something and only the side walls cast shadows, but not the surface itself... and also light bleeding through closed doors...

The closed doors thing is fixable, but my problem here is that I have to identify fast which 2-sided lines closed or opened. I need to take a closer look at how many lines there are in the most complex maps and see if I can brute force loop over them each frame or if that is just too costly.

As for the ceiling/floor part affecting the light, those are the breaks of working with a 1D shadow map.
dpJudas
 
 
 
Joined: 28 May 2016

Re: QZDoom - ZDoom with True-Color (Version 1.2.2 released!)

Postby Graf Zahl » Sat Mar 11, 2017 9:11 am

Rachael wrote:The shadowmaps code checks for the "GL_ARB_buffer_storage" extension, and "GL_ARB_shader_storage_buffer_object" extension. If you have it, you can use it.



... and this boils down to the same problem as the main shader: It works on AMD and NVidia if the extension is supported and on Intel if the driver supports 4.3.
And Apple users are screwed because Apple is too lazy to update their backend with modern features.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
 
Joined: 19 Jul 2003
Location: Germany

Re: QZDoom - ZDoom with True-Color (Version 1.2.2 released!)

Postby dpJudas » Sat Mar 11, 2017 9:32 am

I fear Apple wants us to rewrite GZDoom to use Metal.
dpJudas
 
 
 
Joined: 28 May 2016

Re: QZDoom - ZDoom with True-Color (Version 1.2.2 released!)

Postby Graf Zahl » Sat Mar 11, 2017 10:02 am

If they implemented Vulkan it would be good enough, but I am certainly not going to waste my time implementing a completely different backend for such a minority platform - which is all the more pointless because there's barely any Macs left to buy which contain something bettewr than a crappy Intel chipset, unless you pay 3x of what a good PC costs.

Sooner or later they will kill gaming on their platform anyway if they continue to focus on the profits and not the product.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
 
Joined: 19 Jul 2003
Location: Germany

Re: QZDoom - ZDoom with True-Color (Version 1.2.2 released!)

Postby Graf Zahl » Sat Mar 11, 2017 12:58 pm

So, shadowmaps...

I played a few maps that make good use of dynamic lighting, like Silent Steel and have to say, that it really improves things on those Vavoom maps that were made for a better lighting system. Not perfect but a definitive improvement. But I have to say, that on my Geforce 550Ti this really gives performance a beating - it's quite obvious that the card is getting a bit old by now - I think I'll have to upgrade this year...
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
 
Joined: 19 Jul 2003
Location: Germany

Re: QZDoom - ZDoom with True-Color (Version 1.2.2 released!)

Postby dpJudas » Sat Mar 11, 2017 1:12 pm

The performance drop was the primary reason why I made the cvar default to off. It is a little bit too heavy on system resources right now. I think I can still make it run better than it is doing right now, but I need a little break away from this code.
dpJudas
 
 
 
Joined: 28 May 2016

Re: QZDoom - ZDoom with True-Color (Version 1.2.2 released!)

Postby ZZYZX » Sat Mar 11, 2017 7:43 pm

dpJudas wrote:The problem with real 2D shadow maps is that they are a lot more expensive to generate, which means you have to actively select which lights casts shadows and which does not. No modern engine I know of can do this automatically - they all rely on the modder/level designer taking an active role in selecting lights that casts shadows, and possibly also which geometry gets drawn into the shadow maps. Such a feature can only ever work on new maps. Note that one type of shadow map doesn't rule out the other - in the future it could be coded that a light is selected as a 2D shadow map caster.

I'd prefer to select what lights have shadows the same way we can choose what lights are attenuated. All lights having shadows is not always good because it prevents unrealistic light usage (which is actually needed sometimes).
There can be a CVar like there was "make all lights additive", but "make all lights cast shadows", but not outright forcing it for all lights.
Also, if you use the more generic 2D shadowmap, it probably also means that it suddenly natively starts working on GL3 (GL2 even) (and also support OSX) and that midtextures can be done.

I mean, I'm sorry, I understand this is a brand new system that also allowed you to experiment with brand new 2017 technologies and I totally like the result from the technical side (do I understand it correctly that you are using the actual sector data there?), but can we pleeeeeease have something that's actually usable for most level designers and users in GZDoom, not scientific experiments?
Besides, judging from what was written above about lags in specific video cards, videocard requirements of this shadowmap technique are about ~75% of DOOM 2016 video card requirements.

Honestly would prefer this version to not be pulled into 2.4 like Graf apparently wants it judging by PRs in GZDoom's GitHub, because a) it's too greedy and b) it affects all dynamic lights.
Last edited by ZZYZX on Sun Mar 12, 2017 2:49 am, edited 1 time in total.
User avatar
ZZYZX
le chat du rabbin
 
 
 
Joined: 14 Oct 2012
Location: Ukraine
Discord: ZZYZX#1394
Github ID: jewalky

Re: QZDoom - ZDoom with True-Color (Version 1.2.2 released!)

Postby Nash » Sat Mar 11, 2017 10:08 pm

I generally agree with ZZYZX... as cool as the feature is, IMHO it's not quite ready yet to be unleashed to the wild and for the mass players. I'll have to keep saying again and again that is impressive as hell and blows my mind, but it still feels incomplete as a generic, "everyone can use it" feature.

As far as options, yes, it makes sense that sometimes, some lights should intentionally not cast shadows. I also think forcing shadows on all lights might not be the best idea.

dpJudas, I hope you'll take your time with it. :D I don't feel a rush to make shadow-enabled maps with it presently, I am also just toying with it and not making any serious projects with it, for now. Personally, from a level design POV, I feel that I can't use it seriously yet without 2-sided line support. And I am okay with the added workload to optimize the map by choosing carefully which light should cast shadows and which shouldn't (bonus points if GZDB 3D view could support previewing it for faster iteration time ;D).
User avatar
Nash
AKA Nash Muhandes! Twitter/Facebook/Youtube: nashmuhandes
 
 
 
Joined: 27 Oct 2003
Location: Kuala Lumpur, Malaysia
Twitch ID: nashmuhandes
Github ID: nashmuhandes

Re: QZDoom - ZDoom with True-Color (Version 1.2.2 released!)

Postby ZZYZX » Sat Mar 11, 2017 10:11 pm

Yes, there's much larger chance of getting previews in GZDB if only specific lights are shadowmapped.
The main reason for this is that if I ever add it, I'm not doing the geometry-based way and it'll be limited to 5-10 shadowmapped lights at any given time and with low buffer resolution. It can't be supported with ALL lights this way.
User avatar
ZZYZX
le chat du rabbin
 
 
 
Joined: 14 Oct 2012
Location: Ukraine
Discord: ZZYZX#1394
Github ID: jewalky

Re: QZDoom - ZDoom with True-Color (Version 1.2.2 released!)

Postby dpJudas » Sat Mar 11, 2017 10:49 pm

I'm sorry if this feature weren't what you guys were looking for. What I wanted was something that would allow me to play stuff like Doom 2's MAP05 and MAP06 without the dynamic lights flooding the scene with light. And I wanted to play with ray casting on the GPU. As far as I'm concerned, the feature does what I wanted. The only thing I'm a little bit unhappy about is that the pass is slightly more costly than it has to be, and that Apple's shitty OpenGL policy makes it not work on Mac.

I get that you'd much rather have me implement a full blown Unreal Engine, with a ray tracing engine generating baked lightmaps, light probes, shadow maps, casters and receivers. Hell, lets throw in point lights, directional lights, and spot lights while we're at it. And yes, fall back support back to OpenGL 2. But I'm sorry, I'm not going to code that.
dpJudas
 
 
 
Joined: 28 May 2016

Re: QZDoom - ZDoom with True-Color (Version 1.2.2 released!)

Postby ZZYZX » Sun Mar 12, 2017 2:27 am

dpJudas wrote:I get that you'd much rather have me implement a full blown Unreal Engine, with a ray tracing engine generating baked lightmaps, light probes, shadow maps, casters and receivers. Hell, lets throw in point lights, directional lights, and spot lights while we're at it. And yes, fall back support back to OpenGL 2. But I'm sorry, I'm not going to code that.

I think you are overreacting here. No one asked you to do #1, #2, #4, #5, #7 and #8. And #6 is already in :)
No one even asked you to do #3 (the shadow maps). I only said that they would be better than this.

It's not like I'm attacking you or something because you don't deliver the feature level I expect.

It's just that you essentially made something that has all the limitations of Doom3 stencil shadows (as in it doesn't work with non-solid geometry), and on top of that requires a 2013 videocard to not lag horribly.
(NVIDIA 550 = 2011, according to the wiki).
Additionally it doesn't work with height differences, even if we forget about midtextures, sprites, 3D floors and models.
I don't say that it's outright bad, but while it's nice and promising, its not even close to production quality to be included in GZDoom IMO. Sorry.

Again, here I'm assuming that if you merged it from your dev branch to the master branch and said "test it", then you think it's done enough...
If it's not, then I'm sorry for the reaction and you should have said that it's 10% done instead of "go and test it".
When you say "it's not done", you get "can't wait to see the complete thing" kind of responses. When you don't, expect to get "it's bad and doesn't have X, Y and Z".

Also: isn't classic shadowmapping dumb and very straightforward to implement?
Last edited by ZZYZX on Sun Mar 12, 2017 2:58 am, edited 5 times in total.
User avatar
ZZYZX
le chat du rabbin
 
 
 
Joined: 14 Oct 2012
Location: Ukraine
Discord: ZZYZX#1394
Github ID: jewalky

Next

Return to Closed Feature Suggestions

Who is online

Users browsing this forum: No registered users and 0 guests