[Experimental] GZDoom-ZDRay - Compiled works and more

[Redneckerz]

Disclaimer: Read the credits.

This is going to be a lengthy and thorough thread. In the middle of this i have added a line for those who just want a less detailed explanation. For those who want more, well there is more.

Introduction:
I am making this thread because the contents of this originally resided at Discord. I felt it made sense to have it archived here for historic purposes, similar to the ZDuke port, and to provide a complete package for users to toy with. A lot of info was harvested from picking apart the ZDoom Discord and compiling it together for this thread.

This is provided as-is. It represents a compiled snapshot of experimental work that has since become abandoned by its creator (dpJudas). I am an archivist/editor, not a developer. It may or may not work on your system.

A special thanks to Rachael for generously hosting this and Nash for the majority grunt of getting this together and providing the latest working build.

I have named this compilation GZDoom-ZDRay, because well, that's basically what it is. I strongly suggest only using the provided custom GZDoom build in the package to toy around with this.

Do not pester Judas, Rachael, Graf or Nash with feature requests or requesting it to be ported over to the latest GZDoom.

What this is:
It is a node generator and light baking utility for ZDoom. In the words of dpJudas: ''ZDRay is a node and lightmap generator. It is based on zdbsp for the node generation and dlight for the lightmap generation. Special thanks to Randi Heit, Samuel Villarreal, Christoph Oelckers and anyone else involved in creating or maintaining those tools.''

Basically, with this you are able to get Quake-style/Unreal style lightmaps in your Doom. It supports surface-based area lights (linedef or sector) and it has extended baking features like light bouncing, 3d floors, and so forth. The results can be quite incredible, thanks to Nash:

See this post here for more.




In the Downloads directory, you will find multiple files:

gzdoom-zdray.7z:
An all-in-one package. This includes a custom GZDoom build with ZDRay support, the latest version of ZDRay as executable and the latest lightmaps test WAD with pre-built lights. Huge thanks to Nash for providing this build.
Contents:

• fm_banks (directory)
• soundfonts (directory)
• brightmaps.pk3
• gzdoom.exe: GZDoom g3.8pre-49-gd95967f2c, compiled 2018-10-30 in the console. It is a devbuild towards version 3.8.0 and should be compatible with version 3.7.0, whose thread can be found here.
• gzdoom.pk3
• lightmaptest_latest.pk3: Includes lit test WAD by Nash
• lights.pk3
• zd_extra.pk3
• zdray.exe: ZDRay as a standalone.exe.
• zdrayparameters.PNG: A screenshot of ZDRay's options prompt.
• Various dll's

zdray.7z:
This is the latest version of ZDRay as a standalone executable only.
Contents:

• zdray.exe
• map01unlit.wad: An unlit version of Nash's WAD fully baked in the gzdoom-zdray package for testing purposes.
• zdrayparameters.PNG: A screenshot of ZDRay's options prompt.

What this isn't:

• Something which will be in the latest GZDoom. Unless a variety of variables are met, this code as-is will never be in GZ.

To use ZDRay, the following parameters are available:



Downloads:
GZDoom-ZDRay
ZDRay
Readme

Stuff you may encounter:

• It *may* crash. It is experimental, after all!
• Won't work on all ZDoom slopes
• Sprites may not get correctly lit
• Setting up lights needs to be done manually in-editor
• You need to manually key in unsupported args into the things on the map
• You must create a lightconfig.txt file, it can be empty but you must at least insert 1 line
• ZDRay ideally works in tandem with a level editor. At the time, GZDoomBuilder-BugFix was the preferred solution, but you are better off using Ultimate Doom Builder
• And various other wonky stuff that may occur. This is experimental quality, so expect experimental fuckery along the way. See also the ''So how does this work, anyway?'' section for a more detailed usage on how to set this up

A more detailed background will now follow. Those who just want to download ''the goods''can exit this thread. For those who want to read further, stick by!

So how does this work, anyway?

I let Nash fill you in on this. What follows is a compiled explanation of his Discord commentary from 2019 and 2018. Be advised that Nash refers to GZDoomBuilder-Bugfix, so usage in Ultimate Doom Builder might be different, and things may contradict.

''You make your map in GZDB, then run your map through zdray.exe. The custom build of gzdoom would then be able to read that compiled data to draw complex shadows. So, basically the same workflow as building quake and unreal maps. In order to get baked lights, you have to specifically place them in the map, then run zdray on your wad. 2 ways:

• either set your lights to "static"
  or
• tag surfaces (sectors or lines; udmf only) to be light emitters. zdray will take care of the rest.

Currently (and understandably, as everything is so early in production) i use dynamic lights first for the blocking phase (because GZDB doesn't show static lighting) then when i'm finally satisfied, i convert all the dynamic lights to the UDMF static lights, by hand. then run zdray.''

dpJudas has a nice but sparse summary of the tool: ''You run a tool called zdray, like you run zdbsp today. It picks up certain UDMF properties from the map itself and sunlight properties from MAPINFO, then it bakes the lightmaps and generates the BSP nodes.''

UDMF Properties of ZDRay:

Code: Select all

linedef
{
	lightcolor = <int> (color, default: white)
	lightintensity = <float> (default: 1)
	lightdistance = <float> (default: 0, no light)
}

thing
{
	lightcolor = <int> (color)
	lightintensity = <float> (default: 1)
	lightdistance = <float> (default: 0, no light)
	lightinnerangle = <float> (default: 180)
	lightouterangle = <float> (default: 180)
}

sector
{
	lightcolorfloor = <int> (color, default: white)
	lightintensityfloor = <float> (default: 1)
	lightdistancefloor = <float> (default: 0, no light)

	lightcolorceiling = <int> (color, default: white)
	lightintensityceiling = <float> (default: 1)
	lightdistanceceiling = <float> (default: 0, no light)
}

Package details:
Originally i planned on using the package provided by a user called Kizoky on Discord, but as it turns out through some more digging, as Kizoky's compile had some issues since a user back then couldn't get things to work. dpJudas provided a proper gzdoom.exe and the overall package then seemed to work, along with test maps. Nash then was kindly enough to compile a proper build of the latest state of the lightmaps branch compiled before dpJudas abandoned it, paired with the latest version of ZDRay, with the latest lightmaptest iteration with lights pre-built.

All ive done is some clean up, making a seperate standalone archive for ZDRay and including the unlit WAD.

Credits:

• The Strife VE team for their lightmapper, of which ZDRay is derived of
• Nash Muhandes for providing the latest compiled build of the ZDRay code, lit and unlit versions of his test map, derived fancy screenshots and ZDRay descriptions
• Rachael for hosting on the ZDoom servers so that it has a permanent home
• Magnus Norddahl (dpJudas) for the code work and test maps aswell as prior test builds
• Graf Zahl for the pull requests on the Github branch
• Kizoky for compiling zdray.zip, initial build package and initial uploading over at Discord

Links:

• Github (Latest source-only): ZDRay

[olzhas1one]

What's next? Cubemaps? HDR? Realistic Physics? Ragdoll support?

In all seriousness this is a cool feature, I'd definately want to see it become an actual part of GZDoom. Imagine what kind of stuff mappers could do with this.

[Graf Zahl]

It is a cool feature, too bad that dpJudas wasn't satisfied with it to bring it to completion. I haven't found any time yet to fix the problems it still has but it is something I really would like to see in the engine.

[Redneckerz]

What's next? Cubemaps? HDR? Realistic Physics? Ragdoll support?

In all seriousness this is a cool feature, I'd definately want to see it become an actual part of GZDoom. Imagine what kind of stuff mappers could do with this.

You could get some proper baked lights going on along with shadows that significantly change the look a Doom level.

That's basically modern game making honestly. Games still do look better due to introducing a PBR pipeline to their games, but you would be amazed how important just having better lighting works for assets. That's what the next-gen is going to be: Refined lighting. Hell, RTX and Quake 2 RTX are already making strides for that (as does Reshade's Raytrace injector).

In the here and now though, This compiled stuff is basically the best one can get to have baked lights and shadows in GZDoom and its honestly more a legacy build if anything else. This solely exists for people to toy with.

But who knows. I can imagine one developer looks at this code and uses it for his project. Previously he had to compile that stuff himself (and he will have to if he wants to fork it and add changes to it) but he can atleast test/play with this with these builds now, lowering the barrier.

The visual difference is surely worth looking into, either way.

It is a cool feature, too bad that dpJudas wasn't satisfied with it to bring it to completion. I haven't found any time yet to fix the problems it still has but it is something I really would like to see in the engine.

Thanks for your comment Graf

Indeed, there are a host of limitations surrounding this - But that aside, what's there is already quite useful. Enough to have a compiled build for it. Its basically a time capsule of a feature that sadly is not quite there for GZDoom and it has outgrown its QZDoom usefulness.

With that said, now that this is out there in a playable release, ill do some work on the Wiki to describe this.

[Darkcrafter]

This is freaking awesome to have such a tool now. A got a few questions, let it not sound like requirements though

- I don't know what I'm doing wrong but seems like dyn lights don't work for me, just sky works like it would have sun.
- will it make to the official GZDoom;
- is it possible to get away from zdbsp nodebuilder as GZDoom built-in is way superior than that in terms of handling super large maps (zbsp and other don't even start with a map like this);

Doom Builder view: https://imgur.com/OuShYNz
In game view (large 7mb jpg): https://imgur.com/ZE8Yu63

- will it support gpu rendering via open cl or cuda;
- will it take 3d models into account and shade them and map geometry too;
- this question doesn't tocuh ZDRay and team but no existing doom builder version (even ultimate doom builder) opens a resulting wad and it has to be opened with slade with lightmap lump removal in order to do this.

There was a bit of slowdown if compared to the lighting without lightmaps but anyway it's exciting to see Doom is slowly getting into this territory and doesn't slow down a much because of that (hello vavoom )

[Redneckerz]

This is freaking awesome to have such a tool now. A got a few questions, let it not sound like requirements though

- I don't know what I'm doing wrong but seems like dyn lights don't work for me, just sky works like it would have sun.
- will it make to the official GZDoom;
- is it possible to get away from zdbsp nodebuilder as GZDoom built-in is way superior than that in terms of handling super large maps (zbsp and other don't even start with a map like this);

• You need to have a txt file ready, as mentioned in the thread.
• Read the thread ><. Its literally in the top part, and bolded.
• No. It is made with ZDBSP in mind. You could (i assume) have it work with the built-in nodesbuilder, but i have not read anyone trying this yet.

- will it support gpu rendering via open cl or cuda;
- will it take 3d models into account and shade them and map geometry too;
- this question doesn't tocuh ZDRay and team but no existing doom builder version (even ultimate doom builder) opens a resulting wad and it has to be opened with slade with lightmap lump removal in order to do this.

• No. Its an abandoned branch as is stated, and it works on CPU baking only.
• As far as i can tell, no. I assume with map geometry you mean if it takes into account all slopes, by which case it does not.
• The resultant WAD is supposed to run with the GZDoom build provided. To open the WAD you must not open it with the lights already baked.

Please re-read the thread because your questions have been answered already and i very much can tell that not everything was read.

[Graf Zahl]

- will it make to the official GZDoom;

If someone picks up the work and finishes it - yes. Unfortunately the entire thing is a bit outside my own comfort zone - and I really do not have the spare time to acquaint myself with the subject matter.

- is it possible to get away from zdbsp nodebuilder as GZDoom built-in is way superior than that in terms of handling super large maps (zbsp and other don't even start with a map like this);

How can exactly the same be far superior? GZDoom's internal node builder is just ZSBSP working on internal data representation, producing the exact same output.
However, with the kind of data being created here, we need an external tool due to the processing times.

- will it support gpu rendering via open cl or cuda;

Probably not. The entire thing is a lightmap generation tool, not a real time lighting algorithm. Think Quake 2, not modern games!

- will it take 3d models into account and shade them and map geometry too;

No, that's totally and entirely out of scope of how it works.

- this question doesn't tocuh ZDRay and team but no existing doom builder version (even ultimate doom builder) opens a resulting wad and it has to be opened with slade with lightmap lump removal in order to do this.

The map should be in a compatible format. What makes it throw up?

There was a bit of slowdown if compared to the lighting without lightmaps but anyway it's exciting to see Doom is slowly getting into this territory and doesn't slow down a much because of that (hello vavoom )

Static lightmaps should not be a resource consuming feature, all it needs is an additional texture layer and a bit more shader code. Unless you are running old hardware the impact should be minimal.

[Jaska]

Accidentially stumbled to this while searching other stuff. Is this still worked on?

[Graf Zahl]

Unfortunately not. It's something I wish that could be finished but it definitely requires somebody willing to merge the code into GZDoom's master. Even then, it isn't complete and will only work with levels not using advanced constructs like 3D floors or portals.

[Rachael]

but it definitely requires somebody willing to merge the code into GZDoom's master.

And how long would that be good for?

GZDoom has been going through so many sweeping refactors that it feels like a wasted effort. It'll just be out of date within the next 2 months, anyhow.

I know - in the long term, GZDoom will be better for it with all these refactors - but ... in the short term, they really do make branched work such as this project quite a nightmare to work with. It's like a bunch of conflicting major projects working in parallel to one another that somehow have to find some way to work together, and with the changes in each project being quite deep, it can be time consuming sometimes to find what needs to go where with the new master branch. These actions have also caused several QZDoom master resets because of how deep and extensive the changes are, which also goes to show that contributing via pull requests to GZDoom is also quite a nightmare, for contributors, especially for pull requests that do not get approval or rejection right away.

[Redneckerz]

Accidentially stumbled to this while searching other stuff. Is this still worked on?

This is not mean't as a blunt jab - But i am really curious as to how you could miss the 3rd alinea, the second after the Introduction where it said it is abandoned.

But no, this is not worked on. It exists in a vacuum because the code has to be merged back to master, in which case the code has to be maintained - refactors and all. It exists today for what it is - An experimental build for pretty epic lightmap support. I imagine it provides a good base how to work the ZDRay lightmapper - And essentially get a Doom-esque take on baking lights. I reckon with some modification, the standalone executable could be expanded or enhanced in various ways.

To be fair i am surprised not more people took the plunge to atleast craft a map for it. Being that its in a locked in state, you may aswell create a game with it and enjoy lightmap support that way (With various levels of success).

Ah well. Its a neat tool either way

[Nash]

To be fair i am surprised not more people took the plunge to atleast craft a map for it.

That's because there's no editor support for it, and the way the feature was hacked into the branch was very artist-unfriendly.

(and that's fair, after all, this was a proof of concept branch, the thinking was, "let's hack in some lightmaps and see what the engine can look like"... no promises were ever made by dpJudas that it was going to be production ready)

[Redneckerz]

To be fair i am surprised not more people took the plunge to atleast craft a map for it.

That's because there's no editor support for it, and the way the feature was hacked into the branch was very artist-unfriendly.

(and that's fair, after all, this was a proof of concept branch, the thinking was, "let's hack in some lightmaps and see what the engine can look like"... no promises were ever made by dpJudas that it was going to be production ready)

Would your description (Converting dynamic lights to UDMF static ones and then run through ZDRay) still apply with up to date editors like UDB?

[Steve5563]

sorry if this is the wrong place to ask these questions,
just out curiosity we have been making test maps in gzdoom builder and then exporting them as an .obj file into a 3d program called 3ds max,
in this program we have been adding ray trace lighting to the maps from the gzdoom builder file and we are able to bake the lighting to the textures and export them as a simple .png file format,
from this point we can add those textures back into gzdoom builder and play the level with those textures,
but gpu vram has limited us with only playing around with small test maps and nothing really big enough for a proper size level as texture size and resolution becomes a problem in relation to the scale of the map.

my friend also tells me that 3ds max can also bake lightmaps as well so now baking lightmaps at a low - medium resolution seems to be more practical as overall gpu vram usage can be lowered,

the question i have to ask is,
how is zdray building the lightmaps and what type of file is it outputting for the modified version of gzdoom to use them eg. is it as a texture sheet or individual .png images?
i am wondering if we can use 3ds max to build the same file as zdray creates, could this modified version of gzdoom use them?
or do we keep things basic and just make light maps as .png images and apply them manually in a modified version of the gzdoom builder level editor?

on a side note since lightmaps use another uv channel could that be an add-on to the gzdoom builder?
for example lets say gzdoom builder could have a uv channel 1 mode, this will allow users to view / modify the diffuse doom textures that are applied to the level as it currently does.
and the new uv channel 2 mode can allow the users to view / add and scale the low resolution baked lightmaps to the given sector?

would this be a better way for the dev's side of things on making the implementation of lightmaps possible or make it some what, a more simple way for gzdoom to work with?
we are not worried about having to spend the extra time manually baking and placing the light maps textures to a level if that is what we have to do to keep things simple and more realistic for the gzdoom engine development.

im just trying to brain storm ideas here and hope we can find a way to make this possible as zdray seems like it visually does the same job as 3ds max and the blender programs do.
the work flow of building a map this way is we can use the gzdoom builder to build the level and then export the map as a .obj file,
we add this file it into 3ds max or blender to dress it up, and then export the baked lightmaps / data back into a pk3 file, which is what we can currently do.
from here we add the lightmaps back into the level with modified version of gzdoom builder using the uv channel 2 mode, to visually place and finish off light map position and adjusting the scaling based on light maps resolution.
the images below are quick renders of what we have been playing around with over the last year in 3ds max and the last few pics are some baked textures in a small proof of concept test level running in gzdoom.

---

[Valken]

Looks amazing... Now I wonder with RTRT GPUs with "acceptable" performance, maybe we will see GZ-RT doom in the future.