Important - GZDoom repository now uses submodules

Here, developers communicate stuff that does not go onto the main News section or the front page of the site.
[Dev Blog] [Development Builds] [Git Change Log] [GZDoom Github Repo]

Moderator: GZDoom Developers

Re: Important - GZDoom repository now uses submodules

Postby Nash » Tue Oct 27, 2020 2:30 pm

Graf Zahl wrote:TortoiseGit has a "Submodule update..." entry in its context menu, that always does the trick. So far every time I had problems with submodules, "git submodule update" solved them, especially on macOS where all Git clients are shit, especially the expensive ones :twisted:



Ohhh, even better. Must've missed that.

User avatar
Nash
 
 
 
Joined: 27 Oct 2003
Location: Kuala Lumpur, Malaysia
Github ID: nashmuhandes

Re: Important - GZDoom repository now uses submodules

Postby Chris » Tue Oct 27, 2020 2:41 pm

Graf Zahl wrote:So far every time I had problems with submodules, "git submodule update" solved them

That seems to have worked. That gave me
Submodule path 'wadsrc_widescreen/static': checked out '9a7f5b066115d2e2d98b21e3a31021f5ca24eb47'

and wadsrc_widescreen/static now has stuff in it. Oddly, the first time I ran make afterward, I got a build error:
Could not read central directory from /home/kitty/projects/gzdoom/build/game_widescreen_gfx.pk3. (Is it a zipfile?)
Will proceed as if -u had not been specified.
Updating filter/chex.chex3/graphics/titlepic.png 1.7% [ 39527/ 40197] Deflate
Updating filter/chex/graphics/credit.png 1.9% [ 28514/ 29057] Deflate
...
Updating license.md 33.4% [ 227/ 341] Deflate
Updating mapinfo.lmp 0.0% [ 39/ 39] Stored
/home/kitty/projects/gzdoom/build/game_widescreen_gfx.pk3 contains 60 files (updated 60)
Could not rename /home/kitty/projects/gzdoom/build/game_widescreen_gfx.pk3.temp to /home/kitty/projects/gzdoom/build/game_widescreen_gfx.pk3: No such file or directory
Error copying file (if different) from "/home/kitty/projects/gzdoom/build/game_widescreen_gfx.pk3" to "/home/kitty/projects/gzdoom/build/game_widescreen_gfx.pk3".

However, running make again worked? No error this time and there's a 1.7MB game_widescreen_gfx.pk3 now.

But now we need to be told whenever there's a submodule update so we know to run "Submodule Update..." or "git submodule update", don't we? Since I don't think Git will do it automatically or tell you it needs to be done when a commit changes a submodule target commit (at least I don't remember it doing so).
User avatar
Chris
 
Joined: 17 Jul 2003

Re: Important - GZDoom repository now uses submodules

Postby Graf Zahl » Tue Oct 27, 2020 2:58 pm

That's why submodules are only good for linking to side projects that are expected to infrequently update and not updating the submodule won't break things. IMO it's fine for the widescreen pack and also for ZMusic, but not for splitting off the common engine parts of GZDoom and Raze into their own subproject.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
 
Joined: 19 Jul 2003
Location: Germany

Re: Important - GZDoom repository now uses submodules

Postby drfrag » Tue Oct 27, 2020 4:14 pm

After pulling i got a conflict and an strange error message about wadsrc_widescreen\static and asked me to choose between restoring the "file" from index or delete it from index. I decided to restore it and the merge looks okay but then i had to commit another change to that "file" and i don't know what it's about.
User avatar
drfrag
Os voy a romper a pedazos!
Vintage GZDoom Developer
 
Joined: 23 Apr 2004
Location: Spain
Discord: drfrag#3555
Github ID: drfrag666

Re: Important - GZDoom repository now uses submodules

Postby Nash » Tue Oct 27, 2020 4:16 pm

Which file was it complaining about? I made some changes:

- mapinfo.lmp was deleted
- non-Doom assets were removed out of 'master' and placed into the 'wip' branch instead
User avatar
Nash
 
 
 
Joined: 27 Oct 2003
Location: Kuala Lumpur, Malaysia
Github ID: nashmuhandes

Re: Important - GZDoom repository now uses submodules

Postby drfrag » Tue Oct 27, 2020 4:34 pm

No, the problem was merging the GZDoom repository itself. The other change appeared after updating the submodule.
Do you mind if i remove the module and add you as remote and merge your stuff directly?
User avatar
drfrag
Os voy a romper a pedazos!
Vintage GZDoom Developer
 
Joined: 23 Apr 2004
Location: Spain
Discord: drfrag#3555
Github ID: drfrag666

Re: Important - GZDoom repository now uses submodules

Postby Nash » Tue Oct 27, 2020 5:01 pm

You mean you want to embed the graphics directly into lzdoom.pk3? Sure, go ahead, but please be sure I am given full credit, thanks. :)
User avatar
Nash
 
 
 
Joined: 27 Oct 2003
Location: Kuala Lumpur, Malaysia
Github ID: nashmuhandes

Re: Important - GZDoom repository now uses submodules

Postby Blzut3 » Tue Oct 27, 2020 5:02 pm

Chris wrote:But now we need to be told whenever there's a submodule update so we know to run "Submodule Update..." or "git submodule update", don't we? Since I don't think Git will do it automatically or tell you it needs to be done when a commit changes a submodule target commit (at least I don't remember it doing so).

There are config options you can change if you want git to do the submodule stuff automatically. I forget what these are off hand, but I haven't tried them personally so I can't tell you what side effects they may have to make your life more painful than the default behavior.

Anyway, if a commit changed submodules you can tell since git status will show that the submodules are "changed." That's your cue to do a "git submodule update".

I definitely think submodules are less painful than people make them out to be (or rather I'm used to them), but hopefully the next dominant version control system (because nothing lasts forever) thinks long and hard about how to make them more intuitive.
Blzut3
Pronounced: B-l-zut
 
 
 
Joined: 24 Nov 2004
Github ID: Blzut3
Operating System: Debian-like Linux (Debian, Ubuntu, Kali, Mint, etc) 64-bit
Graphics Processor: ATI/AMD with Vulkan Support

Re: Important - GZDoom repository now uses submodules

Postby Rachael » Tue Oct 27, 2020 9:22 pm

drfrag wrote:After pulling i got a conflict and an strange error message about wadsrc_widescreen\static and asked me to choose between restoring the "file" from index or delete it from index. I decided to restore it and the merge looks okay but then i had to commit another change to that "file" and i don't know what it's about.

If you're getting that, what most likely happened is you didn't remove the folder and its contents first.

Check this commit: https://github.com/coelckers/gzdoom/com ... 15eafe0b0f

The first file (.gitmodules) enables the actual submodule in your working tree.

Then is the actual submodule (which is just a special folder in Git's perspective)

Then the last thing you notice the folder itself is actually removed. What's intended here, is to remove the entire folder, which makes way for the submodule folder.
User avatar
Rachael
Webmaster
 
Joined: 13 Jan 2004
Discord: Rachael#3767
Twitch ID: madamerachelle
Github ID: madame-rachelle
Operating System: Windows 10/8.1/8/201x 64-bit
OS Test Version: No (Using Stable Public Version)
Graphics Processor: nVidia with Vulkan support

Re: Important - GZDoom repository now uses submodules

Postby Graf Zahl » Tue Oct 27, 2020 11:33 pm

Blzut3 wrote:I definitely think submodules are less painful than people make them out to be (or rather I'm used to them), but hopefully the next dominant version control system (because nothing lasts forever) thinks long and hard about how to make them more intuitive.



A lot would be gained if git wasn't so stubborn to not automate their handling transparently - and if Github offered better support for them, like giving an option to automatically integrate their content in source packages. They are still the best option for referencing external code that is responsibly managed. The alternative would be making a copy and constant manual updates which ultimately cost a lot more time and are more prone to user errors.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
 
Joined: 19 Jul 2003
Location: Germany

Re: Important - GZDoom repository now uses submodules

Postby drfrag » Wed Oct 28, 2020 7:06 am

I did a hard reset and deleted the folder before merging and it was the same again. But that was harmless, the later commit was a submodule update i did later after all.
The right option was restore from index BTW, the other made the folder disappear and i got no submodule. The conflict was unrelated and in other file.
About your newer update for the submodule does that commit actually do anything in my repository? I updated earlier but in case i hadn't done it i mean. I guess it doesn't and i need to keep updating manually anyway, becouse the pull command in tortoisegit doesn't include an option for submodules so must be the same as when you added it for the first time. And when i update for real there's another commit then.
Nash wrote:You mean you want to embed the graphics directly into lzdoom.pk3?

No, it would be the same but keeping the graphics (and credits) in my repository too. I still don't know what i'll do, thanks.
User avatar
drfrag
Os voy a romper a pedazos!
Vintage GZDoom Developer
 
Joined: 23 Apr 2004
Location: Spain
Discord: drfrag#3555
Github ID: drfrag666

Re: Important - GZDoom repository now uses submodules

Postby drfrag » Fri Oct 30, 2020 6:51 am

In the end i deleted the submodule, why you will ask. The reason is that when i pulled the TortoiseGit dialogue said it was updated but it wasn't and i'd need to do it manually (same as adding it for the first time) and then a new commit appears everytime. So expect that commit and a merge in PRs (unless i'm wrong). They are not well integrated it seems and don't update automatically.
@Nash: https://github.com/drfrag666/gzdoom/com ... 51efd37d8e
User avatar
drfrag
Os voy a romper a pedazos!
Vintage GZDoom Developer
 
Joined: 23 Apr 2004
Location: Spain
Discord: drfrag#3555
Github ID: drfrag666

Re: Important - GZDoom repository now uses submodules

Postby Graf Zahl » Fri Oct 30, 2020 7:38 am

I may copy these assets to the repo later - but only when all work is done. Since Git and frequently changing binary data don't mix well I do not want to pollute the repo with all these different revisions of image data. - that can easily bloat the .git folder by several 100 megabytes, depending on how often the files are being edited.

Of course with a repo like yours and its countless branches for different products, things will inevitably become harder.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
 
Joined: 19 Jul 2003
Location: Germany

Re: Important - GZDoom repository now uses submodules

Postby drfrag » Fri Oct 30, 2020 7:50 am

Those assets are only in my master branch. I'll just copy and overwrite the files from the local WidePix repo the tradicional way and set the author to Nash.
I have three local repos (actually several more): Nash's, mine and yours. The issue is also present in the local copy of yours, i must update the submodule manually.
User avatar
drfrag
Os voy a romper a pedazos!
Vintage GZDoom Developer
 
Joined: 23 Apr 2004
Location: Spain
Discord: drfrag#3555
Github ID: drfrag666

Re: Important - GZDoom repository now uses submodules

Postby Graf Zahl » Fri Oct 30, 2020 8:39 am

No, those assets are not only in the master branch. They are in the reoo's store and will accumulate there, increasing the repo's size. Each new revision of such a binary file will get in there without being diff'ed against its predecessor. The only way to get rid of them is to rebase your repo and omit the commits where they were changed.

That's my concern here. As long as Nash is still editing these files, I won't copy them into the GZDoom repo.
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
 
Joined: 19 Jul 2003
Location: Germany

PreviousNext

Return to Developer Blog

Who is online

Users browsing this forum: No registered users and 1 guest