Concerning pull requests with new strings in language lump

Sat Mar 16, 2019 10:51 am

GZDoom is in the transition of its localized string handling right now.
The goal is to have the language resources in a place where it can be edited by external contributors that do not use the GZDoom Github project directly - they are hosted on Google Docs so that anyone with interest to contribute can request access to edit.

In the future the language.enu lump will no longer exist, instead, all texts will be read from a spreadsheet file within gzdoom.pk3 which is periodically updated from the public versions mentioned above.

This has a few implications on how text content needs to be handled - editing the language files within the GZDoom project is no longer possible because they no longer represent the master version of the text resources. For pull requests this means that they have to do a minor bit of added work to get the newly added strings to where they belong: Instead of editing language.enu, the new texts will have to be added to the PR as a comment, outside the actual changeset so that if the PR gets merged they can be added to the spreadsheet easily.

Please note that since the text data in the Github repo is not the master resource, any alteration of it by a PR needs to be rejected without exception.

Re: Concerning pull requests with new strings in language lu

Sat Mar 16, 2019 11:09 am

In the future the language.enu lump will no longer exist

This means "in the future you cant add new custom translations to gzdoom"? Or language lump is just for "default" translations?

Re: Concerning pull requests with new strings in language lu

Sat Mar 16, 2019 11:16 am

Apeirogon wrote:
In the future the language.enu lump will no longer exist

This means "in the future you cant add new custom translations to gzdoom"? Or language lump is just for "default" translations?
Presumably it's just for the default translations.

Re: Concerning pull requests with new strings in language lu

Sat Mar 16, 2019 11:16 am

The language lump as-is will be removed. The only piece that will remain in the old format is the untranslatable content that Dehacked needs to access.

Since the text needs to be hosted outside of Github to allow translators access to it, pull requests cannot just add text to the repo - I'd have to manually take it out again and copy to the translation spreadsheet. This is why text needs to be provided separately from the code changes.

Re: Concerning pull requests with new strings in language lu

Sat Mar 16, 2019 12:20 pm

This is very likely not going to appear in a pull request, so I may as well ask for it, here:

The Vulkan branch is very likely going to need a backend switcher in the menu, under the video menu.

I will suggest the names be as follows (and can be altered as you please):

Code:
VIDMNU_BACKEND = "Rendering Backend";
VIDMNU_BACKEND_VULKAN = "Vulkan (if available)";
VIDMNU_BACKEND_OPENGL = "OpenGL only";


For reference, the CVAR that does this is vid_backend. Right now it only works on Windows, but I do plan to change that; I just haven't gotten around to it.

Re: Concerning pull requests with new strings in language lu

Sat Mar 16, 2019 12:41 pm

This also needs to be disabled if initializing Vulkan fails, so the '(if available)' and 'only' parts are not needed.

Re: Concerning pull requests with new strings in language lu

Sat Mar 16, 2019 1:42 pm

Rachael wrote:Right now it only works on Windows

I added this CVAR to other backends.

Re: Concerning pull requests with new strings in language lu

Sat Mar 16, 2019 3:11 pm

_mental_ wrote:I added this CVAR to other backends.

Oh, I didn't realize you did that. I suppose I should look at the commits log occasionally.

Thanks.

Re: Concerning pull requests with new strings in language lu

Mon Mar 18, 2019 10:28 am

How will this affect mods that use language.en(u) to replace some strings, e.g. default death, etc.?

Re: Concerning pull requests with new strings in language lu

Mon Mar 18, 2019 10:38 am

Not at all. The old format can still be read as before. The main issue here is not the format but that with the master string tables being outside the repo, any change to the text files in there is simply not in the right place.

Re: Concerning pull requests with new strings in language lu

Tue Mar 19, 2019 6:22 am

For fork developers, what's the recommended workflow?

I have some local engine changes with some new strings that used to be in the language.enu file.

1) Does that mean I have to maintain my own copy of the Google spreadsheets, and periodically merge/sync the master sheet with mine? How do I avoid cell collisions?
2) What's the procedure to turn the Google spreadsheet into the language.csv that the engine can readily use? Is it as simple as File -> Download as -> CSV or is some other post processing tool needed?

Re: Concerning pull requests with new strings in language lu

Tue Mar 19, 2019 6:29 am

What you need to do entirely depends on your needs. If you have your own english.enu file it can remain as such, assuming you do not want to localize your text. If you have text mixed into the main one, I recommend to split them up to avoid clashes.

For updates to the main language resources, normally it should be enough to just pull in the CSVs from the main repo when they get updated. I won't update the repo for each single change, but only if either a new release is due or the changes are significant (like, right now, so I'll update this evening when I am back home.) If you absolutely need to do an update immediately, you can download the Google hosted ones as CSV and use it directly. The ones on Github are unedited Google exports.

Re: Concerning pull requests with new strings in language lu

Tue Mar 19, 2019 7:40 am

For now, I went with language.<my fork name>.csv (yay cumulative lumps!). The list of engine strings I have added is currently very small (only 3 strings so far), I can afford to maintain this file by hand.

Question: if I'm understanding the pattern correctly; when more languages are added to the engine, I have to increase the amount of commas in my language.<my fork name>.csv file, correct?

Re: Concerning pull requests with new strings in language lu

Tue Mar 19, 2019 8:04 am

You should never - ever - hand-maintain a CSV file, this is far, far too volatile and full of pitfalls.
If you hand-maintain language content, use the old format, which is still the official way to add strings to the game

CSV support is only there for directly reading in spreadsheet exports, I will give no support if you try to do it by hand.

Re: Concerning pull requests with new strings in language lu

Tue Mar 19, 2019 8:09 am

Okay, noted. Yeah, that's what I thought. It's just too clunky to do by hand anyway.