SpriteShadow (Duke3D Shadows) v1.9

[Clownman]

This needs to be a GZDoom feature.

[ZZYZX]

Why not offset the shadow away from consoleplayer's camera? That's what I did and it somewhat works nice, although I'd still prefer to be able to offset it away from the view, like including portals, cameras and shit. But dpJudas disapproved.

[dpJudas]

I never disapproved more fancy shadows. I just didn't agree to coding it, that's all.

For what it is worth, I personally think it would be cool if this was a built-in feature, like Clownman suggests. There's a feature in 3dge that shows shadows under the actors that looks great.

While it could be generated by a 2D shadow map pass, the technical problem here is that it requires all sprites to be rendered into such a texture prior to the normal scene rendering. As far as I know, the way GL renderer does things, this is not known up front (Graf, please correct me if I'm wrong here). A second problem are the portals where each is drawn as a second scene in the scene, meaning either you need a new shadow map texture per portal or you need stencil clipping. On the plus side, it would allow 3D models to cast shadows, but once again, I'm not against such a feature - I just don't want to code it at this time.

[Rachael]

That's up to Graf. I am definitely willing to import this and add a CVAR for it.

[Nash]

Well, if it's going to be a built-in engine feature, it obviously needs better handling with regards to enforcing the shadows to be drawn behind the actor it's meant to "shadow". If it is done in-engine, there would obviously be better ways to do it, like pushing the shadow forward away from the camera. For stuff like that, doing it in C++ is just better equipped than user-script-fudging.

That's talking about SPRITE shadows.

What dpJudas talked about - however - REAL sprite and model shadowmaps - that sounds WAY cooler, but yeah, that's a different topic entirely. :D

Rachael - do you feel the code is engine-worthy, or should I clean it up a little more before it can be approved for PR submission? What else do you think needs to be done (besides CVar support)?

[Rachael]

True, but I think it's been pretty clearly shown here that some people would prefer fake shadows over no shadows at all.

As for the C++ stuff, I am not entirely sure about that since we're pretty much trying to get more of the engine OUT of C++ in the first place. At worst, it could probably use some additional C++ supports to make the implementation a little easier / less hacky. But again - it's all up to how Graf wants to handle it - if even at all. The plus side of this is it works out-of-box in the software renderer - so there is that.

[Nash]

For the Cvar, I think the option should be called "Monter shadows" - Off, Sprite shadows. Then, in future, if sprite shadowmaps get added, the Cvar can have a new "Shadowmaps" option added which will disable the Duke3D-like code path (because IMO using both sprite shadows and shadowmaps would look weird I think).

ALSO currently I haven't checked my code with monster models yet, I won't be surprised to find some weirdness (will it spawn a flattened model???). Need to try my WAD with a monster-model-heavy mod and see what happens...

[dpJudas]

I wouldn't implement this in the engine using what Nash's implementation is doing. Mostly because I'd want to draw all the shadow sprites first, then the sprites. I would implement that simply by iterating through the sprite list twice - and then have some flag indicating if a sprite should be excluded from drawing a shadow.

As for the sprite shadow itself, I'm not sure if I prefer the way 3dge does it here, or how Nash did it. The talk about using a true shadow map texture was just me replying to ZZYZX - because it sounds like he thinks I'm against such stuff. I'm not.

[Nash]

I wouldn't implement this in the engine using what Nash's implementation is doing. Mostly because I'd want to draw all the shadow sprites first, then the sprites. I would implement that simply by iterating through the sprite list twice - and then have some flag indicating if a sprite should be excluded from drawing a shadow.

Bingo. And this was exactly what I had in mind when I mentioned the C++ stuff in my other post - I just wasn't technically proficient enough to convey my thoughts into technically correct words. :)

And It's also for this reason which was why I was sorta "meh" on the idea of it being an engine feature, even waaaaay back when I first released this. This was more of a quick proof of concept (if anyone remembers that thread - this was made as a direct response to another user asking how to do it in DECORATE, to which I replied it's hard, but ZScript will make it easier... yadda yadda yadda)

And dpJudas - completely understood on what your reply really meant. :) It's still a cool idea though (shadowmapped sprites), even if it's just mere conversation at this point! :D

[Sleepyhead]

Apologies for my previous post; it appears QZDoom 1.3.0.1 crashes occasionally when saving for me even without DukeShadows enabled. I've uploaded the information and crash reports to the bug tracker. DukeShadows apparently isn't the source of the problem; sorry for bothering you.

[ZZYZX]

I never disapproved more fancy shadows. I just didn't agree to coding it, that's all.

I was referring to arbitrary per-viewpoint change scripting. Which you said is too deep in the renderer code.

[ZZYZX]

I wouldn't implement this in the engine using what Nash's implementation is doing. Mostly because I'd want to draw all the shadow sprites first, then the sprites. I would implement that simply by iterating through the sprite list twice - and then have some flag indicating if a sprite should be excluded from drawing a shadow.

Can we just have a flag/variable that tells that this actor should be drawn in another pass? RenderLayer 0-31 or something. This would decide the drawing order if two actors are at the exactly same Z coordinate (screen space).

So that regular actors would always be 16, and there will be space of 15 possible levels to the left and 15 possible levels to the right.
Why I'm saying this is that people might also want to for example display outline on a sprite, or flare, or something else that isn't actually a shadow. And having only two levels (and even then AUTOMAGICALLY controlled excluding modder from the process of deciding what's to draw as a shadow and where/how) means artificially capping the potential of this system if implemented. Plus there's this Nash's modular clothing that actually requires having actors on the front of the base one.

Also, 0-31 because then it's possible to make a bitfield for all things to be drawn and don't iterate the thing list if there are no things of specific pass. And maybe per-camera mask, Unity-like.

Note that this is not a feature request, I just outlined the way I see it and would like to know your opinion on that in terms of possibility to make this work with different GZDoom renderers.

[dpJudas]

For the GL renderer, I don't see any particular technical problems in allowing the modder to specify layers for the sprites. There is the catch that the more layers, the smarter it has to be about how it iterates through the list, but its a minor thing.

One thing that maybe could cause problems is that portals are rendered one at a time, so such a layer would only be effective within the portal. Thus an outline sprite that is meant to go in front of everything else would still be confined within the portal. For shadows this will only result in a minor rendering glitch (that will look like Nash' implementation right now) in the rare case of a sprite standing exactly half-way through the portal.

For the software renderer things get a little bit more nasty because masked walls aren't drawn in the opaque pass like they are done in GL mode. Layers get significantly more complicated because of that. For shadows I'd dodge that by ignoring the masked walls in the first sprite drawing pass, but once again if the feature is using multiple layers I'm not sure that will work for more advanced usage.

I think in the bigger scheme of things, it might be better to have something in the engine specifically made for outlines, and likewise for other things on the ground some kind of controllable decals on flats would be preferable. Actually thinking about it, if decals could have been programmatically placed on the floor, that would probably have fixed the depth sorting issue Nash was having.

[Rachael]

I wouldn't implement this in the engine using what Nash's implementation is doing. Mostly because I'd want to draw all the shadow sprites first, then the sprites. I would implement that simply by iterating through the sprite list twice - and then have some flag indicating if a sprite should be excluded from drawing a shadow.

Bingo. And this was exactly what I had in mind when I mentioned the C++ stuff in my other post - I just wasn't technically proficient enough to convey my thoughts into technically correct words.

And It's also for this reason which was why I was sorta "meh" on the idea of it being an engine feature, even waaaaay back when I first released this. This was more of a quick proof of concept (if anyone remembers that thread - this was made as a direct response to another user asking how to do it in DECORATE, to which I replied it's hard, but ZScript will make it easier... yadda yadda yadda)

And dpJudas - completely understood on what your reply really meant. It's still a cool idea though (shadowmapped sprites), even if it's just mere conversation at this point!

True - I guess I hadn't thought of that. I was thinking of a different goal rather than what was right in front of me.

Still - as much as possible being controlled by (and possibly even done by) ZScript would not be a bad thing. But when it comes to direct rendering stuff that is simply out of the question.

[ZZYZX]

Again, in my post, layers should only work for things that stand at the exactly same Z screen coordinate (i.e. things that are approximately at the same distance from the screen plane, with like 0.01 epsilon or so, since coordinates are float now).
Exactly so that it's easier to implement and that you don't need to deal with things like something farther away that should be drawn first because of layers.
So things can be ordered as usual, but their sorting if they are at matching distance from the screen would be decided by layers.