fixing MinGW compilation

[Graf Zahl]

Well, in any sane environment each linkable module should already contain such information.

[dpJudas]

The compiler could place the information in the object files. With MSVC in Windows I haven't told the linker via command line what to link with for 20 years - #pragma comment(lib, "foobar.lib") in any random compilation unit is all I need.

Edit: seems Graf beat me to it, never mind.

[drfrag]

I could try to compile with that option but i'm without internet connection for a few days. I cannot use my mobile as access point to upload code since my SIM card is locked to prevent that (typical spanish i guess).
I updated the instructions to install the environment. For MinGW-w64 CodeBlocks is not required of course but debugging from the command line would be a nightmare then.

I've tried without SSE on a pentium 3 and the GL renderer works but software crashes (illegal instruction). I thought non SSE drawers were reintroduced for ARM.
Edit: the NO_SSE version works on my Core2 Duo.

[Chris]

Well, in any sane environment each linkable module should already contain such information.

Then you get into questions of what to do when different modules say to do different (or conflicting) things. Are there situations where you may want to specify PIE for a couple specific object files but not globally and at link time?

With MSVC in Windows I haven't told the linker via command line what to link with for 20 years - #pragma comment(lib, "foobar.lib") in any random compilation unit is all I need.

So instead of a single place to look at what's being linked (the command line), you have to look through each and every source and header file, keeping track of any of the conditions that may influence if a #pragma or the file it's in is actually utilized. Then there's the difficulty of libraries that aren't in the standard lib paths or may have different names; stuff the build system figured out and can apply to the link command, but complicates getting that info into the desired source.

[Graf Zahl]

Well, in any sane environment each linkable module should already contain such information.

Then you get into questions of what to do when different modules say to do different (or conflicting) things. Are there situations where you may want to specify PIE for a couple specific object files but not globally and at link time?

Simple:

- If the modules have conflicting settings that cannot work together, abort with an error.
- if the modules can be linked but with restrictions, print a warning.
- otherwise just do the job.

It's called "input data validation" and should be done by any piece of self-respecting software to the maximum degree that is reasonable.

[dpJudas]

Then you get into questions of what to do when different modules say to do different (or conflicting) things. Are there situations where you may want to specify PIE for a couple specific object files but not globally and at link time?

Issue a warning and/or error depending on the severity of the problem. A user of the toolchain would rather have a message about inconsistencies than a system where there's no safeguards and all you have is an executable that crashes for unknown reasons.

On a related note, at work one of my coworkers has now been searching for days why the C++/CLI integration broke between VS2015 and VS2017 (preventing us to upgrade). It crashes with a lovely 0x7fffffffffffffff access violation deep inside some .NET initialization code. I'm sure he would have appreciated if the compiler and linker would have refused to generate executables that aren't valid. In that case it's a VS toolchain problem, but essentially same thing - lousy tooling that can't detect when its own code generation is about to fail.

So instead of a single place to look at what's being linked (the command line), you have to look through each and every source and header file, keeping track of any of the conditions that may influence if a #pragma or the file it's in is actually utilized. Then there's the difficulty of libraries that aren't in the standard lib paths or may have different names; stuff the build system figured out and can apply to the link command, but complicates getting that info into the desired source.

Specifying it in the source code does have its pros and cons, but because of the awful state of build tools I find that to be the easiest way to control it. Note that I never specify the search paths this way - just the library to link with. It is particular useful for system libraries where both include and library search paths are already set up.

[Chris]

Simple:

- If the modules have conflicting settings that cannot work together, abort with an error.
- if the modules can be linked but with restrictions, print a warning.
- otherwise just do the job.

Hardly "simple". You'd first have to get that data embedded into the object file and then recognized/usable by the linker, then you have to go through each and every option to figure out which can work together, which are worthy of a warning, and which require an error. And what to do if certain platforms are more lenient with incompatible build flags (e.g. a target platform that always uses PIE/PIC, but has some sources explicitly compiled with -fPIE and some not? the build logic is trying to say something that couldn't work, but there's technically no issue on the target platform)? And even then, it won't handle all cases. If you have reason to compile a given source with certain compile switches that you don't want the linker to enable for the link, those couldn't be auto-handled (figuring out which switches that could apply to would be even more work).

Then you have the maintenance, having to go through all that again whenever you add a new compile+link option.

Specifying it in the source code does have its pros and cons, but because of the awful state of build tools I find that to be the easiest way to control it.

I can certainly see the benefit for small projects that only need standard libs. I just don't see it scaling well as you add more sources and use more libs with third-party stuff, making it a bad habit to get into if you're ever going to do anything not small.

[dpJudas]

I can certainly see the benefit for small projects that only need standard libs. I just don't see it scaling well as you add more sources and use more libs with third-party stuff, making it a bad habit to get into if you're ever going to do anything not small.

Solutions that work well for large projects tend to work equally poor for small ones. CMake, for instance, is an absolute bitch to setup unless you already have existing CMakeList.txt files to loot from (the "documentation" for CMake is a true joke). The tool defaults to something I find completely unacceptable with Visual Studio as my IDE. If it wasn't for cross platform concerns I'd never use it.

That more or less leaves me with the option (on Windows) to use the built-in solution/project system in Visual Studio. Typing in the libraries to link with there is hardly better than having a #pragma line in a source file. With the source file the linker rule follows the external references in the object file - in the project settings its drowned in noise. I doubt we will reach an agreement here, but I disagree with your implication that typing the #pragma is a bad practice. The full linker command line is no guarantee to see all the library dependencies anyway - on Linux, for example, each .so file can itself link to multiple other libraries. If that is okay for a dynamic library file, why isn't it okay for an object file?

[Chris]

on Linux, for example, each .so file can itself link to multiple other libraries. If that is okay for a dynamic library file, why isn't it okay for an object file?

Because that's an implicit dependency of the dynamic library, not your app. They're separate modules. The executable doesn't expect, need, or directly depend on what the library depends on.

[dpJudas]

That is only true for simple cases. If the executable and the dynamic library doesn't depend on the same libstdc++ library the system breaks down. Imagine the situation where both include <string>, which uses templates. The executable creates a std::string and shares it with the library, malloc'ed by one runtime, free'd by the other. Maybe even the layout of std::basic_string mismatches between them. This is the extreme case with libstdc++, but similar situations could also happen if the dynamic library doesn't wrap every library it uses internally.

[Graf Zahl]

Because that's an implicit dependency of the dynamic library, not your app. They're separate modules. The executable doesn't expect, need, or directly depend on what the library depends on.

And yet, I have seen countless cases where this caused problems on Linux because of the weak relationship between symbols and modules where calling dlsym on a given name returned something entirely inappropriate which then caused crashes, just because it had already been loaded somewhere deeper down the chain of dependencies. ZDoom itself suffered from this at some point in time because FMod exposed some stuff from libvorbis that just happened to be incompatible with what users of libvorbis expected.

That is only true for simple cases. If the executable and the dynamic library doesn't depend on the same libstdc++ library the system breaks down. Imagine the situation where both include <string>, which uses templates. The executable creates a std::string and shares it with the library, malloc'ed by one runtime, free'd by the other. Maybe even the layout of std::basic_string mismatches between them. This is the extreme case with libstdc++, but similar situations could also happen if the dynamic library doesn't wrap every library it uses internally.

Sadly, nobody seems to have any motivation to fix this mess. Which is why I never, never, EVER write dynamic libraries depending on a C++ interface. Such libraries are ultimately not shareable. Writing their code in C++ is ok - but the public interface should be plain C, both in how functions are built and which parameter types are passed.

[Graf Zahl]

Solutions that work well for large projects tend to work equally poor for small ones. CMake, for instance, is an absolute bitch to setup unless you already have existing CMakeList.txt files to loot from (the "documentation" for CMake is a true joke). The tool defaults to something I find completely unacceptable with Visual Studio as my IDE. If it wasn't for cross platform concerns I'd never use it.

I find it ok for smaller projects which can be set up with reasonable amounts of scripting. But I still wonder what kind of idiot thought that using such an inane scripting language would be a good idea. Were the syntax a bit more intuitive most of the issues would be automatically resolved.

It's still better than all the other 'tools' from the POSIX world. Autotools makes me cringe even more, for example. Its scripts tend to become utterly unreadable when becoming more complex and trying to make sense out of them is even harder than CMake.

[Chris]

That is only true for simple cases. If the executable and the dynamic library doesn't depend on the same libstdc++ library the system breaks down.

Not really. On Windows, each module in a process is directly tied to its dependencies. So if one module uses vcrt2015.dll and another module uses vcrt2017.dll or something, each module will see the symbols from their respective dependencies and not be bothered by each other. They work separately just fine. On Linux, libraries that care about compatibility while making interface changes will use versioned symbols (which gets bumped whenever an incompatible change occurs). Apps get tied to specific versions of the symbols they use depending on what they link with at build-time. So regardless of what gets loaded, as long as the appropriate symbol version exists the modules are guaranteed compatible behavior for what they were linked against (if the necessary symbol version doesn't exist, it's a load error, just as if the necessary DLL version doesn't exist).

Imagine the situation where both include <string>, which uses templates. The executable creates a std::string and shares it with the library, malloc'ed by one runtime, free'd by the other. Maybe even the layout of std::basic_string mismatches between them. This is the extreme case with libstdc++, but similar situations could also happen if the dynamic library doesn't wrap every library it uses internally.

Passing "complex" resources, such as std::string, between DLL/module boundaries has always been problematic. You can't safely pass such resources between them as anything but raw externally-owned memory if you're not sure you're using the same library since they can expect different sizes and layouts. But I don't see what that has to do with the given discussion of a library's own separate dependencies being a problem for the main app.

And yet, I have seen countless cases where this caused problems on Linux because of the weak relationship between symbols and modules where calling dlsym on a given name returned something entirely inappropriate which then caused crashes, just because it had already been loaded somewhere deeper down the chain of dependencies. ZDoom itself suffered from this at some point in time because FMod exposed some stuff from libvorbis that just happened to be incompatible with what users of libvorbis expected.

That was FMOD's fault since it exported everything it included. It could've either used the system's libvorbis.so, or not export modified versions of libvorbis functions, or renamed/versioned them. If it was just symbol overriding, it wouldn't have been a problem -- so ZDoom calls the vorbis_* methods in FMOD's shared lib instead of libvorbis.so, libvorbis doesn't break ABI so the behavior would be the same either way. It was because FMOD did break libvorbis's ABI and still exported the functions the same as libvorbis.

One can debate the merits of global symbol tables vs per-module, they each have their pros and cons. But if libraries are properly versioned where incompatible versions can exist side-by-side, it's not a problem for the app what a given library depends on.

[Graf Zahl]

That was FMOD's fault since it exported everything it included. It could've either used the system's libvorbis.so, or not export modified versions of libvorbis functions, or renamed/versioned them. If it was just symbol overriding, it wouldn't have been a problem -- so ZDoom calls the vorbis_* methods in FMOD's shared lib instead of libvorbis.so, libvorbis doesn't break ABI so the behavior would be the same either way. It was because FMOD did break libvorbis's ABI and still exported the functions the same as libvorbis.

And who guarantees that two independent libraries do not accidentally use the same symbol names? The entire setup is just a major problem waiting to blow up - utterly dependent on every single player being aware of every single other player.

[Chris]

And who guarantees that two independent libraries do not accidentally use the same symbol names?

That's what namespacing/prefixing is for. If that's not enough, if two independent libraries with independent functionality happen to use the same namespace and export clashing symbol names anyway, what would happen if those two libraries were used by a single module? You'd have problems regardless of OS. C++ name mangling could help since the function parameters also influence the mangled symbol name, making it harder to clash, though C++ has its own ABI issues (it's a right pain to properly separate a class's interface from implementation in an efficient manner, for instance).