Performance and Graphics loading

Discuss anything ZDoom-related that doesn't fall into one of the other categories.

Performance and Graphics loading

Postby Collegia Titanica » Mon Mar 05, 2018 6:24 pm

I'm curious to know how Doom/Gzdoom loads graphics, for you see, when you pair Hoover's Ultra HD textures with PB/BD and load nearly any map, it kind of stutters/freezes for a moment,this happens when you go into another sector then the game "accepts" the nature of the graphics and then it's ok, no performance issues at all. But even with this acceptance, if a so called ... wide view/place is in place you can feel that it impacts performance.

I don't know how graphics really work, all I've heard of is that cone view appear-on-demand rendering thingy. Is this lacking in gzdoom ? Could this be causing the issue ? Is this at all related to openGL ?


I'm asking this because I'm implementing good looking textures for my map. I have a great 900$ setup, which has no problem with graphic intense games like 2016, but loading very high quality textures still makes me encounter these hiccups when starting a map and when I'm moving into new areas. If this is happening on MY setup, lower fairing ones might have a problem loading my map. My current sets aren't nearly as detailed , like 1024x1024 and 1600x800 while the ones mentioned above are the 4k versions of his textures.

Why can I play anything I throw at this PC ,but a big texture pack cripples it at times ?

This is just a curiosity thing, I'm also wondering if there will be any upcoming changes which impact performance/rendering for situations like this.
Collegia Titanica
 
Joined: 25 Jan 2018
Discord: 2834

Re: Performance and Graphics loading

Postby dpJudas » Mon Mar 05, 2018 9:33 pm

GZDoom uses a classic way of handling the map resources where the first time something is used it is loaded. So for example the first time a texture needs to be displayed it first checks if it is loaded, and if it isn't then it loads the texture. Same thing for sounds and everything else.

There are two built-in improvements to this in GZDoom. One is the pre-caching, where it loads a lot of textures and models at map load time. Exactly what the rules are for picking the textures I don't know. The second improvement is a system where it caches the uncompressed textures on disk (or I at least think there is such a system). This saves it from spending time uncompressing the PNG and JPEG image files. There are others here on the forum that can hopefully fill in on the details of each of those systems.

Newer engines uses a system where all the textures exists in both a low resolution version and a high version. They load all textures at startup in the low resolution version and then uses a background thread to load the high resolution version. This allows the game to remain smooth, but if you look closely then for a split second you can see the low resolution version of the texture (*). GZDoom doesn't implement such a system. Partly because it requires help from the pk3 as it needs a low resolution version of all textures, preferably stored in a format that can be loaded very fast (i.e. uncompressed).

*) Sometimes the worker thread falls behind in such a manner that there can be several seconds of delay where you see more and more textures move to high resolution. Player Unknown's Battleground (PUBG) at ultra detail is a good example illustrating this problem.
dpJudas
 
 
 
Joined: 28 May 2016

Re: Performance and Graphics loading

Postby Chris » Tue Mar 06, 2018 12:30 am

For more modern engines, textures are usually stored in DDS files (maybe KTX) as either DXT/S3TC or RGB(A)8 format. Those formats are loaded as-is directly onto the card, and DDS/KTX allows storing premade mipmaps, which is a series of lower resolution versions of the texture. For such a file, it could quickly preload a lower mipmap level as a low-res version, and delay loading the full size version until actually needed in a background thread.

Whether or not this is a good thing to do, though, really depends. I personally don't like it when I can see a texture "load in" after the wall or surface that needs it has come into view. It just looks very distracting and unpolished. And if you have a slower mechanical drive, compared to a solid state or RAM disk or something, it will take a notable amount of time to read in. But if you do have a fast drive and an efficient rendering pipeline, along with a good prediction scheme, you may not be able to notice. And if you're using a lot of really large textures, you may not have a choice since there's only a finite amount of VRAM; you'll have to regularly purge old textures that haven't been used in a while as new ones are loaded in to keep it under control. It also allows for quicker initial loads since the low-res versions are available from the get-go, and there's no hitching whenever a new texture is seen.
User avatar
Chris
 
Joined: 17 Jul 2003

Re: Performance and Graphics loading

Postby dpJudas » Tue Mar 06, 2018 4:55 am

Chris wrote:For more modern engines, textures are usually stored in DDS files (maybe KTX) as either DXT/S3TC or RGB(A)8 format. Those formats are loaded as-is directly onto the card, and DDS/KTX allows storing premade mipmaps, which is a series of lower resolution versions of the texture

All the games I've studied over the years stored textures in their own custom format, usually generated by a baking step. Even when stored as DDS with DXT/S3TC it is usually the baking tool that generated it from a higher quality of source.

Take a normal map, for example - the content author might supply a normal map as PNG, but in order to properly mipmap a normal map it cannot use the standard scaling algorithms as used for images. If they used DDS directly then not only would it present a problem for the content author (poor general support for DDS), but they would also need a complex description of how to create that DDS file as most tools would assume its an image and not a normal map. An engine can also cut away the blue color channel for a normal map, thus saving memory. Once again, presenting this optimization to the content author is problematic, but if its done during baking both engine and author can use their preferred formats. Another advantage is the low resolution versions could be stored together to speed up load times. Since most engines already have baking for other reasons, you might as well bake the textures as well.

Chris wrote:Whether or not this is a good thing to do, though, really depends. I personally don't like it when I can see a texture "load in" after the wall or surface that needs it has come into view. It just looks very distracting and unpolished.

Totally agree. If the prediction is shit so that it loads it too late and/or the placeholder is too low resolution, then I almost prefer the hickup. PUBG is a good example of terrible prediction. But there are other games that are very good at it - GTA San Andreas is a good example of a game where I virtually never see that its streaming in the textures.
dpJudas
 
 
 
Joined: 28 May 2016

Re: Performance and Graphics loading

Postby Kinsie » Tue Mar 06, 2018 6:47 am

dpJudas wrote:PUBG is a good example of terrible prediction. But there are other games that are very good at it - GTA San Andreas is a good example of a game where I virtually never see that its streaming in the textures.
PUBG demands a minimum of 6gb of RAM and 2gb of VRAM, while GTA:SA was developed for a console with 32mb of RAM and 4mb of VRAM. Slightly different economies of scale here... :P
User avatar
Kinsie
A Concept Utterly Obsolete
 
Joined: 22 Oct 2004
Location: MAP33
Discord: Find Me...
Twitch ID: thekinsie

Re: Performance and Graphics loading

Postby dpJudas » Tue Mar 06, 2018 7:59 am

Kinsie wrote:PUBG demands a minimum of 6gb of RAM and 2gb of VRAM, while GTA:SA was developed for a console with 32mb of RAM and 4mb of VRAM. Slightly different economies of scale here... :P

That console also used a DVD for loading its assets (slooooow) vs typical a SSD for PUBG. But that doesn't really change anything - in both cases not all the textures could be in memory and had to be loaded on demand. Keep in mind that the 6gb of RAM version also has a computer of significant higher performance, so all in all it more or less amounts to the same.

The key difference between them is that GTA SA used a more intelligent strategy for loading (mainly based on the area the player is in and the LOD of the city models), while PUBG unintelligently seems to only use the "omg, I need the texture NOW" strategy and has no clear policy on cache eviction beyond last rendered. It is particular retarded in PUBG how pressing TAB makes it reload your own character's skin every single time you switch to the inventory view.
dpJudas
 
 
 
Joined: 28 May 2016

Re: Performance and Graphics loading

Postby Chris » Tue Mar 06, 2018 12:27 pm

dpJudas wrote:All the games I've studied over the years stored textures in their own custom format, usually generated by a baking step. Even when stored as DDS with DXT/S3TC it is usually the baking tool that generated it from a higher quality of source.

Take a normal map, for example - the content author might supply a normal map as PNG, but in order to properly mipmap a normal map it cannot use the standard scaling algorithms as used for images. If they used DDS directly then not only would it present a problem for the content author (poor general support for DDS), but they would also need a complex description of how to create that DDS file as most tools would assume its an image and not a normal map.

I don't doubt that there are extra (automated) steps to "bake" textures, to prepare it for use by the engine. Most paint programs I'm aware of have support for DDS, but their DXT support tends to be lacking. In regards to normal maps, that's an issue regardless. It's just due to the fact that it's abusing color to represent directional vectors, which is true no matter how you do it. This is actually where DDS/KTX can help since, unlike PNG, your baking tool can generate and store premade mipmaps, using custom algorithms that understand a given texture to be a normal map. The GPU couldn't auto-gen correct mipmaps itself, and doing that at load time would just add to loading times.

DXT is certainly not a good compression format for normal maps, though there are others that work better (like RGTC; red-green texture compression that uses only two color channels for the x/y vectors, and blue/z can be reconstructed in the shader). I've head of some interesting things going on with texture compression lately.

In any case, if you want to efficiently load textures, using a container like DDS or KTX is a must, as well as using a pixel format that can be loaded as-is onto the GPU, like raw RGB(A)8 or DXT or something. Having premade mipmaps is also good for quickly loading low-res versions of textures (and just being a good idea in general for normal maps).

Kinsie wrote:
dpJudas wrote:PUBG is a good example of terrible prediction. But there are other games that are very good at it - GTA San Andreas is a good example of a game where I virtually never see that its streaming in the textures.
PUBG demands a minimum of 6gb of RAM and 2gb of VRAM, while GTA:SA was developed for a console with 32mb of RAM and 4mb of VRAM. Slightly different economies of scale here... :P

That just proves the point, though. Despite having more resources available, PUBG still has trouble making sure textures are loaded in time, whereas GTA:SA had both less memory and slower disk access, but still made sure textures were loaded before becoming visible. But we should all know that PUBG is a good example of horrible engine optimization.
User avatar
Chris
 
Joined: 17 Jul 2003

Re: Performance and Graphics loading

Postby Collegia Titanica » Tue Mar 06, 2018 1:51 pm

Ty for the insight, purple guys know their stuff :D . Guess I'll keep em 1024x1024 and hope for the best. Anyway is there any trick/technique I can use to make the 4k textures .. I don't know .. more loadable ?

Forgot to include this gfy

https://gfycat.com/ZanyDigitalBarnowl

Also the freezes in my screen were somehow longer than what this Geforce Experience recording shows
Collegia Titanica
 
Joined: 25 Jan 2018
Discord: 2834


Return to General

Who is online

Users browsing this forum: Google [Bot], JJP, TheWalkingMicrowave and 1 guest