[DIY] AppImage or Flatpak

Moderator: GZDoom Developers

Re: AppImage or Flatpak

Postby Graf Zahl » Thu Sep 19, 2019 3:36 am

dpJudas wrote: Another example is the freetype2 library that lacked compile time flags for certain types of usages.



Freetype is a good example of a library that takes an ultra-stupid approach for something that's supposed to be shareable. In such a case there should never be an option to configure via compiler switches. In the end there is absolutely no reliable way to use such a library from a shared location if something else had disagreements over the flags to be used but insists on using the same global variant.

Whether Flatpak or Snap are the proper solution is doubtful, though. The entire infrastructure supporting them is, like much of the rest of Linux, far too dependent on external maintainers.
In this case I think everybody could learn a lesson from macOS. Unlike much of the rest in Apple-land, the app bundles are by far the best way to package an application I have seen so far.
If we could have that everywhere it'd be a big win for everybody.
Last edited by Graf Zahl on Thu Sep 19, 2019 3:40 am, edited 1 time in total.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
 
Joined: 19 Jul 2003
Location: Germany

Re: AppImage or Flatpak

Postby dpJudas » Thu Sep 19, 2019 3:40 am

Graf Zahl wrote:Freetype is a good example of a library that takes an ultra-stupid approach for something that's supposed to be shareable. In such a case there should never be an option to configure via compiler switches. In the end there is absolutely no reliable way to use such a library from a shared location if something else had disagreements over the flags to be used but insists on using the same global variant.

Which is my point of why you can't keep all libraries like they are the same. Either a library is made to act like a system service or it is not. Clearly, libjpeg, freetype2 and llvm are not. Yet they are treated on Linux like they are.
dpJudas
 
 
 
Joined: 28 May 2016

Re: Flatpak

Postby Blzut3 » Thu Sep 19, 2019 9:58 am

Graf Zahl wrote:I'm not that worried about intentional breakage but about the inevitable bugs that will always happen. With all these libraries we are pretty much in the same situation as with operating system updates, just that you don't just have one point of failure, but many. In certain ennvironments you cannot afford that some application breaks unexpectedly because another one needs to have a certain library updated. The point of self contained packages is to minimize this inteference.

One of the points of my post is "this doesn't happen nearly as often as you think it does." Sure, if your requirements are that this can not happen at all then there's not much else you can do but hold everything constant. However I think most people don't really understand the implications of holding everything constant, since they'll happily pull the latest every time they build their container or whatever without actually vetting the changes. It is amusing how something can help 100 times without being noticed, but one failure and people shy away.

That is of course ignoring that once you take rolling distributions like Arch out of the picture, the odds of a random update breaking anything accidentally is minuscule since distros like Debian and Ubuntu don't change versions after release. Which is why I related the situation to XP/Vista, if it's going to break it's going to break during the distro version upgrades. (Side-note: After having used CentOS at work for the last 4 years I've come to learn that Redhat doesn't take version freezing quite as seriously as Debian. I believe they have a document listing packages that will never be rebased, but I can't find it at the moment.)

Another way of looking at this is why don't we recommend that every mod be distributed with the GZDoom build that it works with? And we're a community that actively tries to find edge cases to exploit. :lol:
Graf Zahl wrote:I think there's a reason why macOS app packages are self contained and normally do not depend on random libraries being installed. The Linux way is the polar opposite of sandboxing, which seems to become more and more popular as a security feature.

I'm not sure I understand which part of the app bundle you think makes it better than the way Windows or Linux apps are distributed. Keep in mind that withing the Linux filesystem hierarchy it's perfectly valid to throw your specific versions of libraries into /usr/lib/gzdoom/ and then set that as the rpath in your binary. Then your custom versions will take precedence over the system ones. Of course package static analyzers will complain since it's not following the distro guidelines of "just rebuild the world" but it is technically valid and correct.

In any case though flatpak is the closest to a macOS bundle. I have to make a correction to my last post, I did find out that flatpak does have a single file export and import, so it is possible to distribute them outside of a repo. From a distance they have similar design in that you build against a common set of libraries (the SDK) and every other resource gets bundled up.
dpJudas wrote:My issue with "the Linux way" is that there's no distinction between system critical, infrastructure libraries and helper libraries. Whether SDL is infrastructure or a helper can be debated, but essentially the QA on Debian often boils down to the attitude or approach of a single developer or packager. Even high profile products like llvm can have disastrous decisions where official packages are rather broken at times. Another example is the freetype2 library that lacked compile time flags for certain types of usages. I have encountered so many such issues over the years that I can't really agree that developers or distros learned their lesson.

Pretty much agreed here. I think with the popularity of containers, flatpak, appimage, snap, what have you, the concept of the distro as we know it is effectively dead, but they don't realize it yet. Well I think Redhat does since Fedora Silverblue is looking to change the model (not sure I agree with their specific ideas, but it's at least trying something). Perhaps it's time to let go of the central mega software repo and instead focus on curating a set of libraries that developers can rely upon.
dpJudas wrote:largely only affected applications that did stupid things

Which was the whole premise.
Blzut3
Pronounced: B-l-zut
 
 
 
Joined: 24 Nov 2004
Github ID: Blzut3
Operating System: Debian-like Linux (Debian, Ubuntu, Kali, Mint, etc) 64-bit
Graphics Processor: ATI/AMD with Vulkan Support

Re: Flatpak

Postby Graf Zahl » Thu Sep 19, 2019 10:37 am

Blzut3 wrote:Of course package static analyzers will complain since it's not following the distro guidelines of "just rebuild the world" but it is technically valid and correct.


I think the problem here is obvious. It's literally a fight against the system and in the end the system always wins.

Blzut3 wrote:I'm not sure I understand which part of the app bundle you think makes it better than the way Windows or Linux apps are distributed.


What I like about it is that it's a contained directory that in most cases can just be deleted without leaving any traces behind. The advantage over Windows installers is quite simply that it's cleaner from an organizational point of view and that it doesn't come with the risk of an installer mucking around with stuff it shouldn't touch.

Blzut3 wrote: Perhaps it's time to let go of the central mega software repo and instead focus on curating a set of libraries that developers can rely upon.


That I fully agree with. What Linux really needs is some coherence beyond the single distros. I think there's lots of users who would gladly leave Windows behind if Linux could give them something comparable with a better infrastructure than there is now. One thing I heard from the web developers I work with is that most of them use macOS because it's the only POSIX compatible system with a good GUI, but many of them do not really like Apple's products.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
 
Joined: 19 Jul 2003
Location: Germany

Re: AppImage or Flatpak

Postby Cacodemon345 » Fri Sep 20, 2019 2:20 am

For me the problem isn't even about the Linux binaries itself (Original Linux Doom still runs on modern-day systems) but the need to compile newer FOSS Linux software when your distro provides the older version. If you want to compile an old version of a Linux software that uses an older form of C/C++ and relies upon some no-longer-supported tricks in the written code, you will have to hunt down for an older compiler version and libraries' headers in some cases. Whilst Linux Doom can be compiled with some simple modifications to make it compile with the "-std=gnu9x" flag, the same can't be said for the first SDL Doom port LxDoom, which requires rewrites in some places to even compile with that said flag.

Another issue is that not all distros provide a repo to download from, in which case you will have to hunt down for the necessary libraries and binaries to get it to run some applications requiring them, which can turn into a PITA if you don't have another working Linux system to compile those (provided it is open-source). That problem is easily solved in another way by getting the binaries from a Debian package, but there's no guarantee of any compat. with the userland by then.

And then there's also the whole standardization issue with Linux. Had that happened, Linux would be better and devs wouldn't have to give Linux software in source code form by themselves.

And then we also got the Linux distros that has a rolling release model, where you are obliged to frequently update your system. Those are also the places where software can break at any moment because there's no guarantee that the libraries won't break programs. The solution is to compile it statically or compile it locally-linked to the ones in the program directory. Windows has been better at that, because Microsoft does not make frequent changes to Windows APIs making backwards-compat much better. Windows also allows a program to load libraries from its own directories, something without we wouldn't be able to run old DDraw games using Wine DLLs.

I have encountered a similar situation when I was compiling GZScoreDoom. VS2005 with nasm wouldn't compile it under Windows 10, so I had to resort to a WinXP VM to get it compiling. Which is where I realized how important binaries are for a Windows user.

Anyways, that's my whole opinion about the whole matter. I could be right or misinformed, but I can't tell...
Cacodemon345
 
Joined: 22 Dec 2017
Discord: Cacodemon345#9151
Github ID: Cacodemon345
Operating System: Windows 10/8.1/8/201x 64-bit
Graphics Processor: ATI/AMD (Modern GZDoom)

Re: AppImage or Flatpak

Postby Graf Zahl » Fri Sep 20, 2019 2:53 am

Cacodemon345 wrote:Windows also allows a program to load libraries from its own directories, something without we wouldn't be able to run old DDraw games using Wine DLLs.


The lack of this is the big issue on Linux. You can do it but it's not the default. Even starting a shell script locally requires prefixing its name with "./".
I think if this hadn't been the case there would be far more software on Linux that'd just ship its binary dependencies along with the main executable, but the setup as-is actively discourages people from doing so.

Regarding the compiler problems, you will find that on all platforms. Languages evolve and sometimes newer compilers will break badly written old code. No magic in the world can protect againd poorly written stuff from two decades ago. Sometimes you are lucky that modern compilers still swallow it (or that the code is still being maintained) but often this isn't the case and it'd just break on more modern platforms, sometimes the source no longer compiles, but often even having a binary won't work because it depends on bad hacks.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
 
Joined: 19 Jul 2003
Location: Germany

Re: AppImage or Flatpak

Postby Eonfge » Sun Sep 22, 2019 8:18 am

Graf Zahl wrote:
Cacodemon345 wrote:Windows also allows a program to load libraries from its own directories, something without we wouldn't be able to run old DDraw games using Wine DLLs.


The lack of this is the big issue on Linux. You can do it but it's not the default. Even starting a shell script locally requires prefixing its name with "./".
I think if this hadn't been the case there would be far more software on Linux that'd just ship its binary dependencies along with the main executable, but the setup as-is actively discourages people from doing so.


Getting a bit back on topic, that is what AppImage and Flatpak both try to tackle. You can package your own with it's own set of libraries. So if you must use SDL 2.0.1, then you can add that version without coming into conflict with other packages.

In other news:
https://flathub.org/apps/details/org.zdoom.GZDoom
Eonfge
 
Joined: 10 Sep 2019
Github ID: https://github.com/eonfge
Operating System: RedHat-like Linux (RHEL, Fedora, CentOS, etc) 64-bit
Graphics Processor: nVidia (Modern GZDoom)

Re: AppImage or Flatpak

Postby Rachael » Mon Sep 23, 2019 6:07 pm


Well - honestly - I am glad someone went and did it. :)
User avatar
Rachael
Webmaster
 
Joined: 13 Jan 2004
Discord: Rachael#3767
Twitch ID: madamerachelle
Github ID: madame-rachelle
Operating System: Windows 10/8.1/8/201x 64-bit
OS Test Version: No (Using Stable Public Version)
Graphics Processor: nVidia with Vulkan support

Re: AppImage or Flatpak

Postby Eonfge » Tue Sep 24, 2019 5:01 am

Rachael wrote:Well - honestly - I am glad someone went and did it. :)

That's how it often works.

In all fairness, this was not completely altruistic... I run Fedora, and there were no easy packages available. That triggered my to come into action, and to create a Flatpak that could technically run on every Linux distribution. I've also added Zandronum and I've created a short list of source prots I would like to see included on Flathub:

https://github.com/flathub/flathub/issues/1160
This will keep me occupied for a few months :D

Edit
@Rachael, as webmaster you might opt to add a Flathub link. I understand it if you decline, because it does make it a lot more official, but know that there are badges and such available for upstream projects to include:
https://flathub.org/badges
Eonfge
 
Joined: 10 Sep 2019
Github ID: https://github.com/eonfge
Operating System: RedHat-like Linux (RHEL, Fedora, CentOS, etc) 64-bit
Graphics Processor: nVidia (Modern GZDoom)

Re: AppImage or Flatpak

Postby Rachael » Tue Sep 24, 2019 9:42 am

Eonfge wrote:Edit
@Rachael, as webmaster you might opt to add a Flathub link. I understand it if you decline, because it does make it a lot more official, but know that there are badges and such available for upstream projects to include:
https://flathub.org/badges

If I do something like that, then there needs to be some way to track its usage. If Blzut3 and Graf and mental are all okay with it, the package can be uploaded directly to Github and then the link can appear on the front page.

However, Flatpak users must understand that updates to the flatpak are purely contingent on an external contributor. That means it's going to be a lot slower than actual releases.
User avatar
Rachael
Webmaster
 
Joined: 13 Jan 2004
Discord: Rachael#3767
Twitch ID: madamerachelle
Github ID: madame-rachelle
Operating System: Windows 10/8.1/8/201x 64-bit
OS Test Version: No (Using Stable Public Version)
Graphics Processor: nVidia with Vulkan support

Re: AppImage or Flatpak

Postby Eonfge » Wed Sep 25, 2019 3:55 am

Rachael wrote:If I do something like that, then there needs to be some way to track its usage.


Here you're touching on some sensitive topics. Right now, there is no easy way for anybody (even downstream maintainers like me) to see the amount of downloads on Flathub:
https://github.com/flathub/flathub/issues/177

But I can provide you with a total amount of users and installs though:
https://github.com/paulcarroty/flathub- ... s/releases
https://ahayzen.com/direct/flathub.html

Since it's release 5 days ago, GZDoom had 33 unique downloads (2 would be mine). Give it some time and lets see how the numbers evolve.

Rachael wrote:If Blzut3 and Graf and mental are all okay with it, the package can be uploaded directly to Github and then the link can appear on the front page.


That's against Flathub policy. Flathub is a Flatpak repository system, which not only hosts but also compiles and updates deltas. They compile everything based on a manifest and they can provide updates to all users worldwide. One thing they do not provide, is single downloadable files. That would introduce many headaches, because how can users get reliably run the latest version? Flatpak combines many systems: It's a repository, it's a sandboxing system, it's an updating system.

Your manifest and build info is here:
https://github.com/flathub/org.zdoom.gzdoom
So if you want to compile it yourself, you're free to do so, but that will be without Flathub integration.

Rachael wrote:However, Flatpak users must understand that updates to the flatpak are purely contingent on an external contributor.

If you want, I have no problem adding an extra disclaimer stating that this is an unofficial downstream package.

Rachael wrote:That means it's going to be a lot slower than actual releases.

I will try to always provide the latest updates, but I cannot guarantee that. One benefit will be, is that my updates can be pushed automatically to users. If I hit publish, it takes 15 minutes before it is live. Compare that with these sources:

https://pkgs.org/download/gzdoom
https://repology.org/project/gzdoom/versions

The fast majority of Linux packages it hopelessly out-of-date because they are not easy to update. Flatpak tries to fix that. Any maintainer can publish any update to all Flatpak users. Flatpak is new though, so it still has some ground to cover before it will be the default for most users.

---

Anyway, that was more text then originally planned. Hope this all makes it more clear to you. Have you already seen GZDoom in the software center?
Image of GZDoom in GNOME Software: https://imgur.com/a/nV1m1QJ
Eonfge
 
Joined: 10 Sep 2019
Github ID: https://github.com/eonfge
Operating System: RedHat-like Linux (RHEL, Fedora, CentOS, etc) 64-bit
Graphics Processor: nVidia (Modern GZDoom)

Re: AppImage or Flatpak

Postby Rachael » Wed Sep 25, 2019 5:06 am

I see. I guess I was confused. I'll discuss this with the other developers then and gather some opinions before making the final decision.
User avatar
Rachael
Webmaster
 
Joined: 13 Jan 2004
Discord: Rachael#3767
Twitch ID: madamerachelle
Github ID: madame-rachelle
Operating System: Windows 10/8.1/8/201x 64-bit
OS Test Version: No (Using Stable Public Version)
Graphics Processor: nVidia with Vulkan support

Re: AppImage or Flatpak

Postby Eonfge » Wed Sep 25, 2019 11:37 am

No problem. I'll see to it that the package gets updated and unless you together decide differently, we'll keep things how they are right now.
Eonfge
 
Joined: 10 Sep 2019
Github ID: https://github.com/eonfge
Operating System: RedHat-like Linux (RHEL, Fedora, CentOS, etc) 64-bit
Graphics Processor: nVidia (Modern GZDoom)

Re: AppImage or Flatpak

Postby Blzut3 » Wed Sep 25, 2019 4:59 pm

Eonfge wrote:That's against Flathub policy. Flathub is a Flatpak repository system, which not only hosts but also compiles and updates deltas. They compile everything based on a manifest and they can provide updates to all users worldwide. One thing they do not provide, is single downloadable files. That would introduce many headaches, because how can users get reliably run the latest version? Flatpak combines many systems: It's a repository, it's a sandboxing system, it's an updating system.

As I posted earlier there's a way to export the flatpak into a single file which can be imported offline. In my opinion offline install and the ability to archive old versions is a very important thing for an official binary.

Auto updates are the right thing to do most of the time, but occasionally there are regressions that break mods and there should always be the ability to roll back to the previous version of GZDoom if needed.
Rachael wrote:If Blzut3 and Graf and mental are all okay with it ...
However, Flatpak users must understand that updates to the flatpak are purely contingent on an external contributor.

This is exactly what I suggested earlier, so you can put me down as OK with it.
Blzut3
Pronounced: B-l-zut
 
 
 
Joined: 24 Nov 2004
Github ID: Blzut3
Operating System: Debian-like Linux (Debian, Ubuntu, Kali, Mint, etc) 64-bit
Graphics Processor: ATI/AMD with Vulkan Support

Re: AppImage or Flatpak

Postby _mental_ » Wed Sep 25, 2019 11:13 pm

Rachael wrote:If I do something like that, then there needs to be some way to track its usage. If Blzut3 and Graf and mental are all okay with it, the package can be uploaded directly to Github and then the link can appear on the front page.

Why not? If it will stop receiving updates at some point, we can always remove its mentions.
_mental_
 
 
 
Joined: 07 Aug 2011

PreviousNext

Return to Closed Feature Suggestions

Who is online

Users browsing this forum: Semrush [Bot] and 4 guests