[4.1.1] libSPIRV.so: cannot open shared object file: No such

Is there something that doesn't work right in the latest GZDoom? Post about it here.

Moderator: GZDoom Developers

Forum rules
Please construct and post a simple demo whenever possible for all bug reports. Please provide links to everything.

If you can include a wad demonstrating the problem, please do so. Bug reports that include fully-constructed demos have a much better chance of being investigated in a timely manner than those that don't.

Please make a new topic for every bug. Don't combine multiple bugs into a single topic. Thanks!

[4.1.1] libSPIRV.so: cannot open shared object file: No such

Postby vilhelmgray » Thu May 09, 2019 9:37 am

I am attempting to run GZDoom version 4.1.1 on a Linux x86_64 target. The following error is reported when the gzdoom executable is run:

Code: Select allExpand view
$ gzdoom
gzdoom: error while loading shared libraries: libSPIRV.so: cannot open shared object file: No such file or directory


It does not surprise me that libSPIRV.so is not found, because it is not installed on my system. I noticed the current gzdoom src/CMakeLists.txt file is unconditionally adding the "glslang", "SPIRV", and "OGLCompiler" libraries without first checking if they are installed on the system:

Code: Select allExpand view
if (HAVE_VULKAN)
   set( ZDOOM_LIBS ${ZDOOM_LIBS} "glslang" "SPIRV" "OGLCompiler")
endif()


These libraries should be detected first before they are added to the link list.
vilhelmgray
 
Joined: 09 May 2019
Github ID: vilhelmgray
Operating System: Other Linux 64-bit

Re: [4.1.1] libSPIRV.so: cannot open shared object file: No

Postby Rachael » Thu May 09, 2019 9:45 am

The more important thing is - will it even compile without these libraries? As far as I know, while Vulkan support was made optional, it unconditionally compiles which requires the presence of these libraries. Making the libraries optional would require a bit of work invested and I don't know if anyone's willing to do that.
User avatar
Rachael
Webmaster
 
Joined: 13 Jan 2004
Discord: Rachael#3767
Twitch ID: madamerachelle
Github ID: madame-rachelle

Re: [4.1.1] libSPIRV.so: cannot open shared object file: No

Postby _mental_ » Thu May 09, 2019 9:52 am

vilhelmgray wrote:I noticed the current gzdoom src/CMakeLists.txt file is unconditionally adding the "glslang", "SPIRV", and "OGLCompiler" libraries without first checking if they are installed on the system

vilhelmgray wrote:These libraries should be detected first before they are added to the link list.

Source code of these libraries is stored in the repository. We can create static libraries and add them unconditionally.

vilhelmgray wrote:It does not surprise me that libSPIRV.so is not found, because it is not installed on my system.

Most likely, it's quite the opposite and the library was picked from your system. We don't use dynamic libraries as targets for source code in the repository.

Rachael wrote:The more important thing is - will it even compile without these libraries? As far as I know, while Vulkan support was made optional, it unconditionally compiles which requires the presence of these libraries. Making the libraries optional would require a bit of work invested and I don't know if anyone's willing to do that.

Yes, it works with no dependencies installed. To be sure, you can check Travis configuration and build logs.
_mental_
 
 
 
Joined: 07 Aug 2011

Re: [4.1.1] libSPIRV.so: cannot open shared object file: No

Postby vilhelmgray » Thu May 09, 2019 10:48 am

_mental_ wrote:
vilhelmgray wrote:I noticed the current gzdoom src/CMakeLists.txt file is unconditionally adding the "glslang", "SPIRV", and "OGLCompiler" libraries without first checking if they are installed on the system

vilhelmgray wrote:These libraries should be detected first before they are added to the link list.

Source code of these libraries is stored in the repository. We can create static libraries and add them unconditionally.

vilhelmgray wrote:It does not surprise me that libSPIRV.so is not found, because it is not installed on my system.

Most likely, it's quite the opposite and the library was picked from your system. We don't use dynamic libraries as targets for source code in the repository.


If the intention of the makefile is to use the version of libraries provided in the source code repository, then something is going wrong since the gzdoom executable built is searching for a dynamic library.

Which part of the makefile sets up the static linking of the version of the libraries in the source code repository? Maybe we can track down where it's failing to statically link the objects for these libraries.
vilhelmgray
 
Joined: 09 May 2019
Github ID: vilhelmgray
Operating System: Other Linux 64-bit

Re: [4.1.1] libSPIRV.so: cannot open shared object file: No

Postby Rachael » Thu May 09, 2019 12:46 pm

That is most likely something going wrong with your system. What GZDoom does with most libraries (as long as they haven't been customized with bug fixes and such) is that it will use CMake to search your distro's build environment for the libraries - and if it finds them, it will use them, instead of an internal copy.

If it doesn't find them, then it will use its own internal copy, and most likely use static linking for that. If it finds an external copy it will probably dynamic link to it.

So what's likely the culprit here is that CMake is finding traces of these non-existent libraries on your system somehow. You will have to review the CMake settings by using the cmake-gui utility on your build folder and make sure that the libraries are pointed internally.

Here is an example of how it searches for the libraries - this is where the Vulkan additions are actually done - this line is the one you're most likely going to want to change to disable Vulkan completely - and these files (1, 2, 3) are the ones that CMake pulls in in order to add the Vulkan libraries.
User avatar
Rachael
Webmaster
 
Joined: 13 Jan 2004
Discord: Rachael#3767
Twitch ID: madamerachelle
Github ID: madame-rachelle

Re: [4.1.1] libSPIRV.so: cannot open shared object file: No

Postby _mental_ » Thu May 09, 2019 1:55 pm

I guess -lSPIRV linker command line switch is added because of this line.
For some reason, however, the linker found libSPIRV.so in its search paths, and linked GZDoom executable with it.
Also, the path to this library is not in dynamic loader paths. That’s why the executable fails to start.
_mental_
 
 
 
Joined: 07 Aug 2011

Re: [4.1.1] libSPIRV.so: cannot open shared object file: No

Postby Rachael » Fri May 10, 2019 1:34 am

That still falls into the HAVE_VULKAN conditional - which if disabled, should prevent that line from being added.

The trick here is to have that variable false, possibly by more thorough checks, if the system cannot fully support it.
User avatar
Rachael
Webmaster
 
Joined: 13 Jan 2004
Discord: Rachael#3767
Twitch ID: madamerachelle
Github ID: madame-rachelle

Re: [4.1.1] libSPIRV.so: cannot open shared object file: No

Postby vilhelmgray » Fri May 10, 2019 5:10 am

I tracked down the cause of the issue.

I'm running Gentoo Linux, and building GZDoom via a Gentoo ebuild. Gentoo passes several default configurations to CMake, and within these defaults I found the following:

Code: Select allExpand view
SET (BUILD_SHARED_LIBS ON CACHE BOOLEAN "")


This was causing CMake to default to dynamic linking of libraries if static linking is not explicitly requested. Setting BUILD_SHARED_LIBS to OFF allows for the static linking behavior.

However, it would still be beneficial for the GZDoom makefile to provide FORCE_INTERNAL_SPIRV, FORCE_INTERNAL_GLSLANG, and FORCE_INTERNAL_OGLCOMPILER options so that users may explicitly choose to use the internal versions of these libraries rather than leaving it up to CMake's ambiguous default behavior.

Regardless, since I found a solution for this issue, you may mark this thread as resolved.
vilhelmgray
 
Joined: 09 May 2019
Github ID: vilhelmgray
Operating System: Other Linux 64-bit

Re: [4.1.1] libSPIRV.so: cannot open shared object file: No

Postby dpJudas » Fri May 10, 2019 6:03 am

Honestly I would prefer it to always force the internal versions. That way GLSL compiler behavior is consistent and well known.

Two months ago I had to explicitly upgrade the GLSL compiler because the version Graf originally had downloaded had a bug in it - I'd really not want bug reports of that sort based on the random nature of Linux distros. If it is up the distro to pick the GLSL compiler an error like that could return, or a new one could appear. One of the advantages Vulkan has is that GLSL compiler behavior is consistent and a dynamic linking throws that right out of the window.
dpJudas
 
 
 
Joined: 28 May 2016

Re: [4.1.1] libSPIRV.so: cannot open shared object file: No

Postby Graf Zahl » Fri May 10, 2019 6:42 am

Heh, heh, one of my favorite Linux topics. Let's just wait for the first hardcore Linuxer to arrive here and explain why this is such a great feature.
In my book, the unpredictability of third party libraries makes the dynamic linking fishy at best and downright dangerous at worst.
For the GLSL compiler this is critical, though, this is a spot where consistency and reliability are far more important than saving a bit of code size. But you can almost bet that someone will submit a PR to reintroduce dynamic linking if static was enforced, because for some people static linking seems to be sacrilege.

In hindsight I wonder if it would have been possible to statically link to LLVM and work around those initialization issues that way.
User avatar
Graf Zahl
Lead GZDoom Developer
Lead GZDoom Developer
 
Joined: 19 Jul 2003
Location: Germany

Re: [4.1.1] libSPIRV.so: cannot open shared object file: No

Postby _mental_ » Fri May 10, 2019 7:38 am

It's possible to mark libraries as always static like this. I did that for DUMB only as it won't link as dynamic library anyway.
We can do the same for critical (or even all) dependencies stored in the repository.
_mental_
 
 
 
Joined: 07 Aug 2011

Re: [4.1.1] libSPIRV.so: cannot open shared object file: No

Postby dpJudas » Fri May 10, 2019 7:59 am

Thinking a bit further about this - we have to force the linking. The API of the GLSL compiler is not stable - I had to make a few minor code adjustments when I upgraded the compiler.

@Graf: totally agree with you on the sacrilege thing. It seems some people still think it is the 80's or early 90's where memory and disk was an extremely limited resource. Nowadays dynamic linking has more problems than advantages in my opinion.
dpJudas
 
 
 
Joined: 28 May 2016

Re: [4.1.1] libSPIRV.so: cannot open shared object file: No

Postby vilhelmgray » Fri May 10, 2019 9:44 pm

dpJudas wrote:Thinking a bit further about this - we have to force the linking. The API of the GLSL compiler is not stable - I had to make a few minor code adjustments when I upgraded the compiler.

@Graf: totally agree with you on the sacrilege thing. It seems some people still think it is the 80's or early 90's where memory and disk was an extremely limited resource. Nowadays dynamic linking has more problems than advantages in my opinion.


If it's critical for these libraries to be statically linked, then it's important to force it to do so in the makefile. In Gentoo, as well as many other distros, dynamic linking is treated as the preference as you've pointedly out -- so without the the makefile forcing static libraries, GZDoom will be configured for dynamic linkage and possibly result in subtle errors for users that will be difficult to track down for developers.
vilhelmgray
 
Joined: 09 May 2019
Github ID: vilhelmgray
Operating System: Other Linux 64-bit

Re: [4.1.1] libSPIRV.so: cannot open shared object file: No

Postby Chris » Fri May 10, 2019 11:00 pm

dpJudas wrote:It seems some people still think it is the 80's or early 90's where memory and disk was an extremely limited resource. Nowadays dynamic linking has more problems than advantages in my opinion.

Funny you say that, since dynamic linking was much less prevalent in the 80s and early 90s (let alone 70s). Consoles, which are more memory- and storage-limited than PCs, also stuck with static linking longer than PCs did.

Hard drives and CDs started becoming a thing for PCs in the 80s and 90s, vastly increasing storage capacity beyond floppies, and the expected amount of memory users have is always growing. Far from being a means to save memory and disk space, dynamic linking largely comes into play when you have systems designed for multi-tasking and resource sharing, where it's more about consistency and interoperability. If each app needed to maintain code for handling desktop windows or an audio device or something, you'd have conflicts out the wazoo. Even on modern multi-tasking systems, many operations are service/server-based (e.g. video and audio are handled by a separate process, and apps link to client libraries that talk with those external processes). Dynamic linking is necessary for client libraries to update along with the service/server programs, if you want to maintain backwards compatibility as the communication protocol changes. Dynamic linking is also the reason we can still run closed-source games released 10 to 20+ years ago on modern systems (Win95's gdi32.dll or kernel32.dll certainly wouldn't work on Win10, but apps that linked against them can still run), and its the reason we can make custom replacement components for those games that provide a better experience. For example, there are dsound.dll replacements that emulate the system having a hardware DS3D audio, which not only restores 3D audio for many games, but can also restore environmental audio effects (things that haven't been available for DSound apps since Vista, if you even had a hardware audio device prior to then).

Dynamic linking isn't about working around memory and storage limits, but is for completely different problems that are still (if not more) relevant on modern systems.
User avatar
Chris
 
Joined: 17 Jul 2003

Re: [4.1.1] libSPIRV.so: cannot open shared object file: No

Postby Graf Zahl » Sat May 11, 2019 12:41 am

And still, all this pales in comparison to dependency hell. The best way to avoid dependency hell is to avoid unnecessary dependencies, except to the underlying OS itself, of course.
There's a reason why these days very little software installs their DLLs directly into the Windows folder which was common practice in the early days.

But on Linux this still is a major issue because binaries not only link dynamically to the system but they link dynamically to everything by default. All fine and well if it can be guaranteed that all versions of some .so are compatible. But since this is not the case the entire setup isn't stable.
I even had it happen on macOS once with GZDoom - for some reason CMake picked up an external libJPEG which got installed by some other app and all hell broke loose because it was incompatible with GZDoom's code - the only solution was to configure the project to statically link to its own copy provided in the source package.
So why are these even configured to dynamically link? Simply because some Linux users were complaining and bitching that GZDoom was 'too bloated' and asked to change the setup. And Randi complied. Been there, done that, nothing but problems ensued.
User avatar
Graf Zahl
Lead GZDoom Developer
Lead GZDoom Developer
 
Joined: 19 Jul 2003
Location: Germany

Next

Return to Bugs

Who is online

Users browsing this forum: No registered users and 2 guests