Transition to ZMusic as DLL

[Rachael]

I just did the 'make install' thing, wiped my build folder and recreated it and I was fine.

[Matt]

Code: Select all • Expand view

make[2]: *** No rule to make target '/home/mchan223/doom/ZMusic/lib/libzmusic.so', needed by 'gzdoom'.  Stop.
make[2]: *** Waiting for unfinished jobs....
[ 99%] Building CXX object src/CMakeFiles/zdoom.dir/zzautozend.cpp.o
make[1]: *** [CMakeFiles/Makefile2:984: src/CMakeFiles/zdoom.dir/all] Error 2
make: *** [Makefile:130: all] Error 2


There's no ZMusic/lib/ folder.

Is this the cache thing people are talking about?

[_mental_]

Please read this post. Raze and GZDoom have identical configuration steps. Sorry, I cannot explain it simpler than that.

[Matt]

Compiles now, thanks!

So basically the first option in your first reply, ignoring the "Setting ZMUSIC_INCLUDE_DIR and ZMUSIC_LIBRARIES directly should not require this." stuff...

[MartinHowe]

In case anyone else has a similar problem: Mental's "-DZMUSIC_INCLUDE_DIR=..." stuff is supposed to be added to the cmake command in Graf's OP not entered as a separate command.

is there some way to tell the gzdoom stuff where ZMusic is?

Read my post above, it's exactly about how to do this.

Thanks you guys; without the context of which directory to build ZMusic from and taking into account the 'shouldn't be necessary' I wasn't sure which went where. However, it makes sense now.

[Chris]

This looks like it'll severely complicate working on and keeping keeping these projects up to date, since if there's any change to gzdoom or raze that relies on changes to zmusic (or vice-versa), you'll have to manually go and update the other without necessarily knowing ahead of time. Worse as someone who wants to work on zmusic, I might make changes that alters the API or its capabilities, how is this supposed to work? If I make the changes to zmusic and push them first, GZDoom and Raze will be in a broken/incomplete state until they're updated (and I wouldn't know how to update Raze, since I'm not at all familiar with its codebase). If I push changes to GZDoom first, it will no longer build until ZMusic is updated (at which point Raze will break until someone updates that). How are these projects supposed to stay in sync?

[Rachael]

Git supports linked subrepositories. I think that's probably the best way.

[Chris]

Git supports subrepositories. I think that's probably the best way.

I thought about that too, but anyone's who's ever used them only seems to talk of pain and torment.

[Rachael]

I am sure those stories have some merit.

Another possibility is to merge all 4 repositories (Graf's prboom, gzdoom, raze, and zmusic) and have all contributors work on the same repository. I think even having different licenses for each isn't a problem, as long as they're all separated into their own subfolders. Then - all projects using ZMusic can simply use the same ZMusic on the same commit.

[phantombeta]

I am sure those stories have some merit.

Another possibility is to merge all 4 repositories (Graf's prboom, gzdoom, raze, and zmusic) and have all contributors work on the same repository. I think even having different licenses for each isn't a problem, as long as they're all separated into their own subfolders. Then - all projects using ZMusic can simply use the same ZMusic on the same commit.

That sounds like it'd be a massive mess, and honestly far, far, far worse than submodules. You can't really separate PRs and commits by folder, so all the commits and PRs would be mixed together - and the mixed commits part would be specially hellish when doing anything that requires using the commit history, like tracking down bugs or bisecting.

[Rachael]

Bisecting will be hell too, especially to an outsider, if one of the submodules breaks.

/shrug

I am just trying to come up with ideas. They probably aren't great, but - who knows - better to try than not to, and with a few shots in the dark you might hit something viable - or god forbid, it might inspire some brainstorming in another direction.

[Chris]

You can't really separate PRs and commits by folder, so all the commits and PRs would be mixed together - and the mixed commits part would be specially hellish when doing anything that requires using the commit history, like tracking down bugs or bisecting.

The problems is precisely with keeping PRs and commits separated between projects. If a change is made to ZMusic that depends on changing GZDoom and/or Raze, or vice-versa, a commit or PR would need to change them all at the same time to ensure it all stays working. Keeping them separate would ensure it breaks when one gets changed and not the others.

The alternative, submodules, would allow GZDoom or Raze to essentially hook to a specific commit of ZMusic, so ZMusic can be changed independently, and then GZDoom or Raze can update the commit it uses at the same time it requires the newer version. This should work in theory, but as I mentioned before, anyone who's ever used submodules seems to really not like them (I've never used them personally, so I couldn't say what the problem actually is).

[Graf Zahl]

The idea here is to have a STABLE API, i.e. if new music formats get added they should become automatically usable without recompilation. I already made sure that the entire API is pure C and not a C++ class interface. If we start to treat this as an extension to GZDoom that can be changed at will, the entire idea will fall apart - this will especially mean that nobody else would ever be willing to use the library to play music.

Right now I think the only area that may need some work is the decoder part - as you recently laid out. However, since I already use the DLL for a few private modifications of other games to hook in a music player the existing API should not be changed, only added to.
This entire idea of submodules and allowing modifications in ZMusic that require changes in GZDoom and vice versa is something that should be avoided, it's not how I want to maintain this.

[Chris]

The idea here is to have a STABLE API, i.e. if new music formats get added they should become automatically usable without recompilation. I already made sure that the entire API is pure C and not a C++ class interface. If we start to treat this as an extension to GZDoom that can be changed at will, the entire idea will fall apart - this will especially mean that nobody else would ever be willing to use the library to play music.

If the idea is to have a stable API that anyone else might want to use, there's a problem that a number of things are missing the ZMusic prefix. The enum and struct types are "bare" which have a non-negligible chance to clash with other users' code, some things also have generic prefixes (like SampleType_, SoundDecoder_, or CD_), or are lacking them (ChangeMusicSetting*, FindLoopTags), also creating chances to clash. A consistent and unique prefix to all public types, enums, and functions are necessary for a library expected to be reusable by others.

Given all this about changes, I guess I would also bring up that the ZMusicCustomReader interface looks weird to me. Like an unnecessary hybridization of C-style virtual interface (with an opaque void handle for user data) and a C++-style virtual base class (a 'this' handle to base struct with pointers to functions, and user data as extra). Normally with a C-style virtual interface, the function pointers take the opaque void handle directly; there isn't a need to also give it function pointers for the other methods, which creates another level of indirection here. Alternatively, there is a way to 'extend' structs in C so the C++-style could work in C too, making the opaque void handle unnecessary.

[Graf Zahl]

If the idea is to have a stable API that anyone else might want to use, there's a problem that a number of things are missing the ZMusic prefix. The enum and struct types are "bare" which have a non-negligible chance to clash with other users' code, some things also have generic prefixes (like SampleType_, SoundDecoder_, or CD_), or are lacking them (ChangeMusicSetting*, FindLoopTags), also creating chances to clash. A consistent and unique prefix to all public types, enums, and functions are necessary for a library expected to be reusable by others.

Them it needs to be fixed. The first step was to get the code separated and working, the second will be do do the interface correctly, and since you already said that you wanted some rework of the sound decoder I didn't put too much effort into that until the specs are clear.

Given all this about changes, I guess I would also bring up that the ZMusicCustomReader interface looks weird to me. Like an unnecessary hybridization of C-style virtual interface (with an opaque void handle for user data) and a C++-style virtual base class (a 'this' handle to base struct with pointers to functions, and user data as extra). Normally with a C-style virtual interface, the function pointers take the opaque void handle directly; there isn't a need to also give it function pointers for the other methods, which creates another level of indirection here. Alternatively, there is a way to 'extend' structs in C so the C++-style could work in C too, making the opaque void handle unnecessary.

Yes, that part suffered from scaling down a truly virtual C++ class and should be cleaned up.
But again - I wanted the code to be separated first before cleaning up the interface - it was already problematic to have two projects depend on this and not being in sync because they had their own copy of it.
As long as it remains undocumented and only used in two public projects this all is easy enough to manage, so any input, both in form of comments and PRs to improve things is welcome. I already created a new branch for that.