The Still New What Did You Last Do Thread

[dpJudas]

I completely disagree with this stance that because there's a been a few examples of abandoned software that could be hacked a bit at the DLL transition point that this means all software should default to DLLs for all their third party dependencies. You are basically ruining the experience and performance for all supported software in order to help a few stubborn people that refuse to accept software dies without maintenance.

Also love the fact that the bible, uh I mean the unix philosophy, is brought into this. We might as well go full monty here and say anyone not understanding unix is forced to reinvent it, poorly. Notice how you apparently can't reinvent it better or even equally, because unix is god and everything they did in 1970 was perfect. The unix philosophy of every tool only doing one thing is a children's story that doesn't scale and never will. Not that it has anything to do with dynamic vs static linking, yet here we are.

[Chris]

Because sometimes that may be the only way to get it to work properly. Or to restore missing functionality. GZDoom has the benefit of being open source and still under active development, but there's plenty of programs that aren't. If such programs start misbehaving or loses functionality (due to a system update) or a critical security flaw is found in one of the libraries it used, replacing that library to make it work again or to fix the security hole is orders of magnitude more difficult if it's baked into the executable.

I'd say the inverse is far more likely, i.e. that updating a library behind the application's back breaking some of its assumptions.

And such issues can be noted and addressed. Since I'm talking about software that's no longer under support, if something's already broken or misbehaving with no prospect of it being fixed officially, a change to one of the shared libraries that may cause some other glitch but otherwise makes it overall work better, is still a net positive. And if a change to one of the shared libraries makes it worse, there'd be no reason to do it. This does work into one of the factors when deciding what should be static-linked or not, though; a library that is known to break ABI compatibility such that replacing the shared lib in-place with an updated version will more than likely than break something, dynamic linking is less useful.

That's a VERY bad example on virtually all accounts. If DSound was statically linked, all the old hardware interface would still be there - you lost that because the systwem updated the internal library and removed functionality.

And then you'd still be stuck with all that functionality locked to specific hardware that has bad drivers (presuming those drivers even continue to work). There's still the improvements in providing the functionality to everyone with any sound device, with extra features even that hardware didn't have (personalized HRTFs, UHJ, with-height surround sound, near-field emulation/compensation), and backed by a project that's still in development that can fix bugs and add even more features. It even works in Wine, where accelerated DSound wouldn't work even if you had the hardware.

Yeah, and then you end up with distros that still ship a 4 or 5 year old OpenAL. Didn't you complain about this just a few days ago elsewhere?

I'm not sure I get what you're getting at. I'm talking about the binary package you provide of your app, and the assumption you'd provide shared libraries for whatever you end up dynamic linking. Static linking a library results in fewer files and a smaller footprint (and possibly a bit more optimization), while dynamic linking makes it easier to update existing packages without having to rebuild everything. Different libraries will benefit more from one or the other, and the fact that an app is open source with a copyleft license makes more aggressive static linking of its own binary releases more tolerable to me, which is all I meant to say.

I don't know why Debian is so behind on updating OpenAL, so I can't guess what would or wouldn't help encourage them to update it. Until I can talk to someone about what the holdup is, or they decide to just update it, I don't know what will get them to move on it.

Correct, but then those who make the modifications need to be honest about it that what they distribute is an altered version of the software.

Right, and if that Linux distro isn't saying they've modified it in a way that you/upstream doesn't support, I would be annoyed too. The (L)GPL does say you're supposed to indicate if you're distributing a modified version (and provide what you've changed).

[yum13241]

That's a VERY bad example on virtually all accounts. If DSound was statically linked, all the old hardware interface would still be there - you lost that because the systwem updated the internal library and removed functionality.
Anyway, the internal complexity here is an entirely different level.

The dev could have linked a version of DSound that doesn't work with Windows Vista for example. Maybe they abandoned it before Vista anyway.


Static and dynamic distribution both have advantages but locking someone into either one of them is evil.

Yeah, and then you end up with distros that still ship a 4 or 5 year old OpenAL. Didn't you complain about this just a few days ago elsewhere?
Here's where the problem lies - if I cannot ensure that the proper up-to-date version of a library gets used I as the developer have a major problem on my hands.

*cough LTS distros *cough

I'd say the inverse is far more likely, i.e. that updating a library behind the application's back breaking some of its assumptions.

If you're relying on a hack in an old version of a library that's the dev's fault. We all know Graf hates hacks.

[Graf Zahl]

And such issues can be noted and addressed. Since I'm talking about software that's no longer under support, if something's already broken or misbehaving with no prospect of it being fixed officially, a change to one of the shared libraries that may cause some other glitch but otherwise makes it overall work better, is still a net positive. And if a change to one of the shared libraries makes it worse, there'd be no reason to do it. This does work into one of the factors when deciding what should be static-linked or not, though; a library that is known to break ABI compatibility such that replacing the shared lib in-place with an updated version will more than likely than break something, dynamic linking is less useful.

I understand what you are getting at here but it's still a very bad example because DSound is a hardware abstraction layer system library.
Things get a lot more tricky when it is about utility libraries. I would have tested my code with version X of it and cannot foretell what the next version Y might change, but I end up depending on some changes the previous version W does not have yet. Y may be an improvement but it may equally be an app breaking regression. So, in order to guarantee that the app works as intended I ship version X of the library with it to be on the safe side. Works fine on Windows, works fine on Mac, it also works fine with an AppImage - but if I make myself depend on the package management I suddenly end up in the wild west. Linux distro A ships version W and hasn't updated to X yet, distro B ships version Y and has discontinued support for version X and some users out there have recompiled version X with different settings which end up making the resulting library incompatible with my app. So in the end that channel of distribution is unfeasible because I cannot give my guarantees about stability. Thus, my app won't see a release on these channels because I cannot handle the support costs.
Your concerns of conveniently swapping out a library globally are peanuts by comparison if it means you cannot get my software. And this is where lots of Linux software stands these days. You need an insane support infrastructure by volunteers to keep it working at its current state, but that also means that it's these volunteers who decide what gets supported and what not- and like my example says, you cannot rely on these to to the right decisions - ideology often gets in the way of doing things right. And it certainly does not work for commercial software that cannot be recompiled. For that you need a stable format that keeps the entire package intact. Open Source is a requirement for this, or a provider with very deep pockets and an interest in supporting linux despite these obstacles.
Now tell me, what is more important for the platform's success on a wider field?

That's a VERY bad example on virtually all accounts. If DSound was statically linked, all the old hardware interface would still be there - you lost that because the systwem updated the internal library and removed functionality.

And then you'd still be stuck with all that functionality locked to specific hardware that has bad drivers (presuming those drivers even continue to work). There's still the improvements in providing the functionality to everyone with any sound device, with extra features even that hardware didn't have (personalized HRTFs, UHJ, with-height surround sound, near-field emulation/compensation), and backed by a project that's still in development that can fix bugs and add even more features. It even works in Wine, where accelerated DSound wouldn't work even if you had the hardware.

Like I said, it's a poor example. DSound is a system-side interface. These tend to change. Windows went from MMAPI over DSound to WASAPI - but provided compatibility fallbacks to not completely break old software.
And what did Linux do? Pretty much the same, actually - there has been ALSA, OSS, then PulseAudio and now PipeWire.
Since these are essentially low level system components you do not link them statically because they are the bridge between you and the driver. Linking these statically would be insane, as would be distributing the DLLs/SOs, because these always will need adjustment as hardware advances.
But this isn't the kind of library I am referring to here. In the end there's two classes of libraries.
1. Wrappers of system services like OpenAL - for these the sanest option is to ship a tested version along with the app and also keep it in the application directory - but not link statically to it but ship a .dll or .so, so that it can be replaced if needed (Not on macOS, actually because apps need to be signed - that's a very different problem, though...)
2. But when it comes to pure software utilities with a bit higher complexity, like for example the GLSL shader compiler I consider it insane to depend on external code. I would have tested my shaders with a very specific compiler version and therefore integrate that particular version of the compiler with my application. I can then upgrade this library on my own terms by running my test suite and checking that everything is fine - what I do not want if someone updates the library behind my back and runs into a breaking change that gets reported to me then - even though the guilty party is somewhere else entirely. (Yes, we had this happen once, which prompted us to set this library to be forcibly linked statically!) I'd still have to shoulder the support costs for the problem, which I simply do not want. But since the vast majority of depenencies falls into this category the entire argument you are bringing up falls flat. As long as the library is in shared space I cannot give any guarantee that my software tests are still valid.

Yeah, and then you end up with distros that still ship a 4 or 5 year old OpenAL. Didn't you complain about this just a few days ago elsewhere?

I'm not sure I get what you're getting at. I'm talking about the binary package you provide of your app, and the assumption you'd provide shared libraries for whatever you end up dynamic linking.
Static linking a library results in fewer files and a smaller footprint (and possibly a bit more optimization), while dynamic linking makes it easier to update existing packages without having to rebuild everything. Different libraries will benefit more from one or the other, and the fact that an app is open source with a copyleft license makes more aggressive static linking of its own binary releases more tolerable to me, which is all I meant to say.

With that I agree mostly. It's often a tricky thing to decide. As an example, for the components of ZMusic I really prefer static linking because it gives me a guarantee that the various players work as I intended - we had our problems with FluidSynth here. Of course for something like libsndfile it is beneficial to have it easily swappable, in case it adds a new format like MP3 recently - same for ZMusic as a whole. But in the end this decision is not the actual problem with Linux - as long as you ship your own SOs and can place them in the same directory as the main application and its core data, without sharing that folder with other things, all is fine. The decision whether to link something statically or dynamically would normally a case by case decision depending on what benefits my application more.
It is a problem if the app goes into a shared app binary folder, the library goes into a shared library folder and the core data gets stashed away in yet another place. That's just a recipe for trouble as software gets more complex. I want everything in one place, as it is the norm on both Windows and macOS.

I don't know why Debian is so behind on updating OpenAL, so I can't guess what would or wouldn't help encourage them to update it. Until I can talk to someone about what the holdup is, or they decide to just update it, I don't know what will get them to move on it.

See my comment above from becoming dependent on other people. This is precisely what I'd want to avoid when writing my software - and I'd need some guarantess from the OS that enables me to avoid it. The current package management system does not.

[Graf Zahl]

If you're relying on a hack in an old version of a library that's the dev's fault. We all know Graf hates hacks.

Hacks are always bad. No need to discuss. But it doesn't have to be a hack. Sometimes just making a seemingly benign improvement may affect how older code written with different assumptions may react.

Let's take a very simple example:: I have a sound loader that first tries an external library like libsndfile to load a sound, and if that fails try a few fallbacks. Now that library extends its support to a new format but for some reason only implements a subset of that format, but in the unsupported case still loads the sound but in broken form. Now my fallback won't be called anymore and the sounds I was previously able to use are now missing some infornmation, like for example some embedded loop markers.

Suddenly my program won't work anymore, even though all components are still nominally valid.
The obvious fix to this problem is to do my custom loaders first, and I surely would do that after upgrading the library - but the old app code would never work with the new library, so with libraries like this I'd really like to ensure that such bad combinations cannot be created outside of deliberate user intervention with my app installation.

[Chris]

Yeah, and then you end up with distros that still ship a 4 or 5 year old OpenAL. Didn't you complain about this just a few days ago elsewhere?
Here's where the problem lies - if I cannot ensure that the proper up-to-date version of a library gets used I as the developer have a major problem on my hands.

*cough LTS distros *cough

It's not the LTS distros that are the problem here. For whatever reason, Debian unstable (which is essentially their rolling release channel) still has an OpenAL from over 4 years ago. That was the last release before it switched from C to C++, which I wouldn't think is the problem since Mesa uses C++ too and other distros have updated, and I'm not aware of any blocker issue or compatibility problem.

[yum13241]

*Debian Sid is their rolling channel.



I'm fine with an app bringing a .dll with it, but as long as it's replaceable that's fine.

[Gez]

Dynamic linking reminds me of that day Azer Koçulu crippled the Internet by deleting one tiny teeny little bit of JavaScript code. Just a total of eleven, count them, 11, lines of code to do a rather trivial left-padding operation...

[Chris]

*Debian Sid is their rolling channel.

Which is just another name for their "unstable" channel (there are also the stable, backports, and experimental channels). Some people find the "Debian Sid" designation confusing since the way it's set up makes it look like that's going to be the codename of the next release.

Dynamic linking reminds me of that day Azer Koçulu crippled the Internet by deleting one tiny teeny little bit of JavaScript code. Just a total of eleven, count them, 11, lines of code to do a rather trivial left-padding operation...

The JavaScript ecosystem is entire mess unto itself. Whoever thought it was a good idea to automatically pull deps from their source without any oversight needs a good smack upside the head.

[yum13241]

Debian Sid != Debian Unstable


I agree with the second point. JavaScript is a mistake in the same way anime is a mistake.

[Gez]

But what would prevent the same kind of problem popping up in the glorious land All Dynamically Linked, All Constantly Updated ? What prevents a library update from including an apparently-insignificant change that ends up breaking another library, this subsequently ballooning into hundreds of other libraries breaking and thousands of applications no longer being usable?

[yum13241]

THEN you statically link the library. Don't statically link unless you have to.

[Marisa the Magician]

A chunk of the Linux userbase, as well as various developers, tend to be very stubborn and unreasonable when it comes to things that actually would be beneficial.

I personally don't see a problem with static linking if necessary, or integrating some library code into a project to make it work more seamlessly (putting Fluidsynth directly into ZMusic is a VERY welcome change).

Where I kinda draw the line, though, is in statically linking the C library. Well, unless you're using musl or the like. Statically linking glibc is a waste of 2 to 4 MB depending on how many parts of it you need.

[yum13241]

The problem with ALWAYS statically linking is that you waste too much space.


The problem with ALWAYS DYNAMICALLY linking is that things can break.


A balance of both is needed, forcing either one of them on the user is !


Also, Graf's points in what makes sense to statically and dynamically link make sense. There's some stuff that's insane to statically link however, like glibc.

[Graf Zahl]

Strangely enough, on Windows many applications do that - but that's mainly because Microsoft screwed it up again.
They made the UCRT a system library so it should be nominally safe to use - but then make it impossible to use that without the compiler runtime as well - and THIS is the part I don't want to depend on because it is far more likely to not be present on the end user's system, which causes unneeded support work.

If they shipped the compiler runtime as a system library it would be easier, but no - this is what doesn't happen.

Also, 'insanity' depends on the use case. For some stuff you cannot even risk the remote chance that there's some breakage in glibc and rather link statically instead.