The Still New What Did You Last Do Thread

If it's not ZDoom, it goes here.
dpJudas
 
 
Posts: 3134
Joined: Sat May 28, 2016 1:01 pm

Re: The Still New What Did You Last Do Thread

Post by dpJudas »

Yes dynamic libraries have situations where they are advantages and desirable - in particular for plugins and system interaction. I just think anyone talking about forcing everything to shared libraries are forgetting that if an application only uses 10% of libc, then you actually need 10 applications installed using libc to even break even. That is going to happen for libc for sure, but actually only if all the applications use the same version. The moment you start to use multiple versions of libc simultaneously you're already beginning to lose out against static linking.

If you factor in something rarely used, like let's say fluidsynth, then you might only have GZDoom installed using it. If GZDoom only uses 25% of the library then you already lost a lot of disk space by insisting on it being dynamically linked. You also lost memory, since it has to load the entire DLL into memory.

As a simple illustration of this, if the only .net application you use is Ultimate Doom Builder, then the dynamic linking nature of .net means you spent over 200 megabytes of disk space just for that one program.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
Posts: 49183
Joined: Sat Jul 19, 2003 10:19 am
Location: Germany

Re: The Still New What Did You Last Do Thread

Post by Graf Zahl »

dpJudas wrote: Thu Dec 01, 2022 8:11 am If you factor in something rarely used, like let's say fluidsynth, then you might only have GZDoom installed using it. If GZDoom only uses 25% of the library then you already lost a lot of disk space by insisting on it being dynamically linked. You also lost memory, since it has to load the entire DLL into memory.

It's 40% actually. but the library is so poorly designed that it still links everything. I had to physically throw out the unneeded code to get only the 40% I actually needed. :twisted:
yum13241
Posts: 853
Joined: Mon May 10, 2021 8:08 pm
Preferred Pronouns: He/Him
Operating System Version (Optional): EndeavorOS (basically Arch)
Graphics Processor: Intel with Vulkan/Metal Support

Re: The Still New What Did You Last Do Thread

Post by yum13241 »

and the IWAD picker can be a separate Terminal or GTK2/3 or KDialog component as it was before - but then we won't be able to remove any of that code.
YAD exists. https://github.com/v1cont/yad.
The disadvantage is that it is GTK only (with some setup GTK and Qt apps can look nearly identical)
yum13241
Posts: 853
Joined: Mon May 10, 2021 8:08 pm
Preferred Pronouns: He/Him
Operating System Version (Optional): EndeavorOS (basically Arch)
Graphics Processor: Intel with Vulkan/Metal Support

Re: The Still New What Did You Last Do Thread

Post by yum13241 »

dpJudas wrote: Thu Dec 01, 2022 8:11 am Yes dynamic libraries have situations where they are advantages and desirable - in particular for plugins and system interaction. I just think anyone talking about forcing everything to shared libraries are forgetting that if an application only uses 10% of libc, then you actually need 10 applications installed using libc to even break even. That is going to happen for libc for sure, but actually only if all the applications use the same version. The moment you start to use multiple versions of libc simultaneously you're already beginning to lose out against static linking.

If you factor in something rarely used, like let's say fluidsynth, then you might only have GZDoom installed using it. If GZDoom only uses 25% of the library then you already lost a lot of disk space by insisting on it being dynamically linked. You also lost memory, since it has to load the entire DLL into memory.

As a simple illustration of this, if the only .net application you use is Ultimate Doom Builder, then the dynamic linking nature of .net means you spent over 200 megabytes of disk space just for that one program.

But as Graf put it, "you need to make sure everyone is following the rules by the letter" "I wouldn't trust any software developer to follow the rules" "there will a dev that will abuse the system and bring havoc"


You can't count on every developer to prune the stuff they don't want.

In my opinion, static linking is fine but there should be a way to force any app to use the dynamic versions if I want it to, the readme should tell the user if the statically linked version has any custom changes. (i.e, new functions in glibc, I don't expect a software dev to provide support if the user ignores the readme and removes the statically linked library, and it should rightfully crash and yell at the user) Power to the user! (almost always a good thing)
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
Posts: 49183
Joined: Sat Jul 19, 2003 10:19 am
Location: Germany

Re: The Still New What Did You Last Do Thread

Post by Graf Zahl »

yum13241 wrote: Thu Dec 01, 2022 10:18 am In my opinion, static linking is fine but there should be a way to force any app to use the dynamic versions if I want it to, the readme should tell the user if the statically linked version has any custom changes. (i.e, new functions in glibc, I don't expect a software dev to provide support if the user ignores the readme and removes the statically linked library, and it should rightfully crash and yell at the user) Power to the user! (almost always a good thing)
Sigh...

Why are you so insistent on voiding the warranty on your software?
Because that's essentially what you are asking for.

As a developer of software that can run on a multitude of systems it is of utmost importance to me to ensure that everything is checked to work together. That is a total impossibility right now due to the dynamic linking. Aside from potential bugs in my own code I always have to consider some user updating (or not updating) some library and the wrong version causing a conflict. How am I supposed to investigate this. To be blunt: I don't want to - I want to remove that possibility from the playing field if possible - by NOT giving you the option to shoot yourself in the foot.

I know of one Linux distro which edits GZDoom to dynamically link to stuff we use statically, most importanbtly the GLSL compiler. Rest assured, when I see a report popping up for that version I'll close it on sight for being an unsupported private build.

Around 2 years ago we had a situation where some issue in FluidSynth caused us such problems. It was not the laggard archive maintainers who got the report but US!
As soon as I had the ability to link statically to FluidSynth I did so - FluidSynth is now fully integrated in ZMusic in source form - and should you decide to remove that and link again to the system variant, good luck getting support for that unofficial modified version! :P
yum13241
Posts: 853
Joined: Mon May 10, 2021 8:08 pm
Preferred Pronouns: He/Him
Operating System Version (Optional): EndeavorOS (basically Arch)
Graphics Processor: Intel with Vulkan/Metal Support

Re: The Still New What Did You Last Do Thread

Post by yum13241 »

Why are you so insistent on voiding the warranty on your software?
Because that's essentially what you are asking for.
I don't inherently want to void the warranty, I'm saying that if the user wants to they should be able to, but then the warranty is void.

If there's absolutely NO reason a library is linked statically I might as well force it to use the system version. Fluidsynth (for GZDoom) has a reason. glibc typically doesn't.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
Posts: 49183
Joined: Sat Jul 19, 2003 10:19 am
Location: Germany

Re: The Still New What Did You Last Do Thread

Post by Graf Zahl »

I know what you want but it's still a bad value proposition for me as developer because a) it requires more configuration work and b) it's a given that such a feature will also be used by people that don't know what they are doing and still would call my helpdesk if things break.

I guess this is something that's hard to make the Linux community understand: It's not about YOUR convenience but about enabling developers to target your platform.
Developers do not like tech suppoprt, they especially do not like the kind of tech support that's avoidable by disabling simple error conditions. And this is such a case. Yo you will have a very hard time convinving a developer that's aware of the pitfalls to give in.
yum13241
Posts: 853
Joined: Mon May 10, 2021 8:08 pm
Preferred Pronouns: He/Him
Operating System Version (Optional): EndeavorOS (basically Arch)
Graphics Processor: Intel with Vulkan/Metal Support

Re: The Still New What Did You Last Do Thread

Post by yum13241 »

I guess this is something that's hard to make the Linux community understand: It's not about YOUR convenience but about enabling developers to target your platform.
At that point you are violating the the part of the UNIX Philosophy which says "Don't prevent the user from doing smth stupid" (which hardly anyone takes seriously) but I get your point.

Now normally I'd say "Just mark it as tainted somehow (the linux kernel does this for unsupported options, and no one's bothered to do anything about it)" or a README would convey it but I know from personal experience this is not the case. If I need to remind my Dad of something, here's what I do:

1. Get on to his computer.
2. Google "how to make an error box appear in batch" (cuz i always forget)
3. Make it autostart.
4. He'll remember when he comes back.


OR I write it in Notepad and keep it open. He can't be bothered to read a README (rightfully so, since he has a living to make)


btw, the chaotic aur (a custom arch repo, https://aur.chaotic.cx) builds GZDoom, but its an -m version, you might wanna check what they're doing.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
Posts: 49183
Joined: Sat Jul 19, 2003 10:19 am
Location: Germany

Re: The Still New What Did You Last Do Thread

Post by Graf Zahl »

yum13241 wrote: Thu Dec 01, 2022 12:23 pm At that point you are violating the the part of the UNIX Philosophy which says "Don't prevent the user from doing smth stupid" (which hardly anyone takes seriously) but I get your point.
At this point the Unix philosophy violates common business sense, i.e. in order to service you, anyone so inclined will have to put up with an unreasonable servicing effort.
I think we are back at the start now, if this is the essence of the whole matter, Linux will never ever have even the slightest inkling of becoming mainstream.

I understand that for a true nerd this is something sacred, but for any business it is a logistical nightmare. Don't get me wrong - on a system level this is a really valuable thing, but when it comes to third party software, the reasonable approach should be to make it a pleasant experience for that third party, but all these things only serve the end user - albeit merely on the surface, because the negative implications are not to be discounted.
yum13241
Posts: 853
Joined: Mon May 10, 2021 8:08 pm
Preferred Pronouns: He/Him
Operating System Version (Optional): EndeavorOS (basically Arch)
Graphics Processor: Intel with Vulkan/Metal Support

Re: The Still New What Did You Last Do Thread

Post by yum13241 »

Unix != Linux.


Some distros are more KISS (Keep It Simple, Stupid!) than others. For example, Ubuntu is HARDLY KISS, while Arch follows KISS like the Bible. I don't expect Arch to be mainstream though, and Ubuntu won't either cuz Snaps. The average user is gonna be like: 'This Leenocks thing takes forever to launch Firefox, that means Leenocks sucks! HURR DURR!"
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
Posts: 49183
Joined: Sat Jul 19, 2003 10:19 am
Location: Germany

Re: The Still New What Did You Last Do Thread

Post by Graf Zahl »

And my response to such utterings would be a loud "Fix it, goddamnit!" to the developers! :mrgreen:
User avatar
Rachael
Posts: 13793
Joined: Tue Jan 13, 2004 1:31 pm
Preferred Pronouns: She/Her

Re: The Still New What Did You Last Do Thread

Post by Rachael »

Unix philosophy made sense in the 1960's when RAM was quite a commodity and most permanent storage was tape, punchcards, or monstrous disk spindles which barely held a few kilobytes.

These days with the advent of pulseaudio and systemd, even a lot of Linux distro maintainers are beginning to see the better of it. Though there is a lot of very vocal resistance against those very projects for the very reason that they violate "Unix Philosophies" (doing more than one thing in a single program) - and for no real reason other than that, much less a practical one. It's just people being excessively conservative about change. These projects genuinely make life so much easier in Linux for so many reasons. (Incidentally they both have the same author)

No operating system is going to prevent the user from doing something stupid. It might be possible to prevent the accidental overwrite of c:\windows\system32 files by a regular user without admin permissions, but it's not possible to prevent them from booting from a CD and wiping the partition or GPT table with 0x00's to prevent the system from booting at all. You can only go so far with hand-holding users, but there is nothing wrong with at least making it more difficult to royally screw things up as they tend to like to do sometimes.
yum13241
Posts: 853
Joined: Mon May 10, 2021 8:08 pm
Preferred Pronouns: He/Him
Operating System Version (Optional): EndeavorOS (basically Arch)
Graphics Processor: Intel with Vulkan/Metal Support

Re: The Still New What Did You Last Do Thread

Post by yum13241 »

Hell, GZDoom is against the unix philosophy, I use pipewire and systemd on Linux and it works great, I'm happy they exist.
No operating system is going to prevent the user from doing something stupid.
There's nothing the OS can do about it. What they meant here is "don't prevent the user from doing what they want them to do.". Fine. let them sudo rm -rf / --no-preserve-root. Maybe just ask for a confirmation.

but there is nothing wrong with at least making it more difficult to royally screw things up as they tend to like to do sometimes.
I agree. Just don't make it impossible to do it unless it impossible.



Hell, basically everything violates the UNIX philosophy these days, if you stretch it like Rachael did. Firefox violates the UNIX philosophy! (it can read XML docs & PDFs afaik)
User avatar
Chris
Posts: 2954
Joined: Thu Jul 17, 2003 12:07 am
Graphics Processor: ATI/AMD with Vulkan/Metal Support

Re: The Still New What Did You Last Do Thread

Post by Chris »

yum13241 wrote: Thu Dec 01, 2022 10:18 am You can't count on every developer to prune the stuff they don't want.
For the most part, it's the linker that does that automatically. For a final executable, the linker looks at the defined symbols and checks if they're referenced anywhere. If not, it drops it so the function or variable is gone from the output (and it basically does this recursively). So it ultimately only includes the code and variables that are accessible from the functions the app calls (unlike when building a dynamic library, where it effectively assumes any externally visible symbol is accessible).
Graf Zahl wrote: Thu Dec 01, 2022 10:31 am Why are you so insistent on voiding the warranty on your software?
Because that's essentially what you are asking for.
Because sometimes that may be the only way to get it to work properly. Or to restore missing functionality. GZDoom has the benefit of being open source and still under active development, but there's plenty of programs that aren't. If such programs start misbehaving or loses functionality (due to a system update) or a critical security flaw is found in one of the libraries it used, replacing that library to make it work again or to fix the security hole is orders of magnitude more difficult if it's baked into the executable.

Projects like DSOAL show this off well. When Windows Vista came out and dropped accelerated DSound support, all DSound apps dropped to software mode and lost all hardware features even if you had the hardware. DSOAL aims to fix that by overriding dsound.dll and simulating hardware support, providing proper surround sound mixing with HRTF and EAX like it had pre-Vista (with the added benefit that it doesn't depend on audio hardware since it utilizes OpenAL Soft). If dsound.dll was static linked, or couldn't be replaced, because it would be "voiding the warranty on your software" to change it, this wouldn't be possible. All these programs are long out of support, so there's no warranty to void anymore, but people still want to play them and utilize the features they were capable of.

Not to say you shouldn't ever static link, but there's a balance to strike between a smaller distribution with less waste, vs easily updating dependencies, or easier modification when official support is ended. There are multiple factors to consider, in my mind, about whether any given dependency should be static linked. A significant factor is whether or not the project is FLOSS, which I'd give more lenience to static-linking (since the source is available and open to modification, there is at least always the option of rebuilding it with newer libs as needed, even if such builds aren't supported; that's the point of copyleft licenses, after all, ensuring the user can make changes to suit their needs).
Graf Zahl wrote: Thu Dec 01, 2022 10:31 am I know of one Linux distro which edits GZDoom to dynamically link to stuff we use statically, most importanbtly the GLSL compiler. Rest assured, when I see a report popping up for that version I'll close it on sight for being an unsupported private build.
And that's completely fair. Just as if someone takes GZDoom source and makes significant changes, any bug reports from that person should be closed as an unsupported private builds too. Any bugs found with unsupported modifications should be reported to whoever made and provided that modified version, and it would be their responsibility to find out if the problem occurs in a supported build. If the people who made the modifications aren't providing support for their modifications, and aren't upfront about that modified version not having any support, they're not a good source of software (most reputable distros have their own bug tracker to report bugs in various packages).

This is the same everywhere. If you modify a program (by adding mods to a game, using ENB, or something else that changes how the program is intended to behave), it's expected you'd be told the modifications aren't supported by the software vendor. You'd go to the person who made the modification if you experience problems while using it.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
Posts: 49183
Joined: Sat Jul 19, 2003 10:19 am
Location: Germany

Re: The Still New What Did You Last Do Thread

Post by Graf Zahl »

Chris wrote: Thu Dec 01, 2022 3:06 pm Because sometimes that may be the only way to get it to work properly. Or to restore missing functionality. GZDoom has the benefit of being open source and still under active development, but there's plenty of programs that aren't. If such programs start misbehaving or loses functionality (due to a system update) or a critical security flaw is found in one of the libraries it used, replacing that library to make it work again or to fix the security hole is orders of magnitude more difficult if it's baked into the executable.
I'd say the inverse is far more likely, i.e. that updating a library behind the application's back breaking some of its assumptions.
Projects like DSOAL show this off well. When Windows Vista came out and dropped accelerated DSound support, all DSound apps dropped to software mode and lost all hardware features even if you had the hardware. DSOAL aims to fix that by overriding dsound.dll and simulating hardware support, providing proper surround sound mixing with HRTF and EAX like it had pre-Vista (with the added benefit that it doesn't depend on audio hardware since it utilizes OpenAL Soft). If dsound.dll was static linked, or couldn't be replaced, because it would be "voiding the warranty on your software" to change it, this wouldn't be possible. All these programs are long out of support, so there's no warranty to void anymore, but people still want to play them and utilize the features they were capable of.
That's a VERY bad example on virtually all accounts. If DSound was statically linked, all the old hardware interface would still be there - you lost that because the systwem updated the internal library and removed functionality.
Anyway, the internal complexity here is an entirely different level.
Not to say you shouldn't ever static link, but there's a balance to strike between a smaller distribution with less waste, vs easily updating dependencies, or easier modification when official support is ended. There are multiple factors to consider, in my mind, about whether any given dependency should be static linked. A significant factor is whether or not the project is FLOSS, which I'd give more lenience to static-linking (since the source is available and open to modification, there is at least always the option of rebuilding it with newer libs as needed, even if such builds aren't supported; that's the point of copyleft licenses, after all, ensuring the user can make changes to suit their needs).
Yeah, and then you end up with distros that still ship a 4 or 5 year old OpenAL. Didn't you complain about this just a few days ago elsewhere?
Here's where the problem lies - if I cannot ensure that the proper up-to-date version of a library gets used I as the developer have a major problem on my hands.

This is the same everywhere. If you modify a program (by adding mods to a game, using ENB, or something else that changes how the program is intended to behave), it's expected you'd be told the modifications aren't supported by the software vendor. You'd go to the person who made the modification if you experience problems while using it.
Correct, but then those who make the modifications need to be honest about it that what they distribute is an altered version of the software.

Return to “Off-Topic”