GZDoom localization development thread [Split]

[Graf Zahl]

If this used UTF-8 instead of a separate code page, it could be made part of the engine itself. As it stands it'd have to be an add-on.
Also, concerning Strife dialogues, there really should be a more versatile way to translate them, aside from recreating their entire content.

The only reason why nothing had been done on that front is that nobody was ever committed enough to do a full translation for another language, especially one with a different alphabet. I see that you seem to be really interested in making this work, so how about joining forces to do it RIGHT? I guess some issues with the font files will have to be resolved but these things should be doable.

I don't know if the FON1 and FON2 formats support more than 256 characters but again, it should be solvable. I actually would deprecate them anyway and instead add the font part of imagetool to the font loader so it can work off the raw images instead of the binary output. And once there, the rules could be simplified to allow skipping unimplemented characters.

[Kostov]

The only reason why nothing had been done on that front is that nobody was ever committed enough to do a full translation for another language, especially one with a different alphabet. I see that you seem to be really interested in making this work, so how about joining forces to do it RIGHT? I guess some issues with the font files will have to be resolved but these things should be doable.

I’m all for making this official, as long as it’s compatible with everything else in the engine. Does GZDoom already support UTF-8?

By the way, the translation contains a language menu that was sent to me in a pull request by another user a while back. If you grab these two files, you can put them in the engine.

I don't know if the FON1 and FON2 formats support more than 256 characters but again, it should be solvable. I actually would deprecate them anyway and instead add the font part of imagetool to the font loader so it can work off the raw images instead of the binary output. And once there, the rules could be simplified to allow skipping unimplemented characters.

Well, that’ll rid me and anyone else of the nightmare that is ImageTool. Thanks.

[Graf Zahl]

The engine already supports UTF-8, albeit in a somewhat ad-hoc manner by analyzing string content at run time. This will probably require some fixing to convert all strings at load time when non-UTF-8 content is parsed. Fortunately, since aside from the few binary lumps (ACS, Strife dialogues) the only one having a separate parser is Dehacked, so the majority can be done in one single place.

[Kostov]

So how can I help with getting it to work with UTF? And is there any way to contact you privately in the future about this?

[Graf Zahl]

I just had a look at the imagetool source. Holy crap, that code surely looks its age, it's a classic example of how badly software was designed 20 years ago. Let's hope I can make some sense out of that font generator. This is full of really misguided "optimizations" that pretty much mangle the image data for some supposed (probably inconsequential) space savings and this makes up the majority of code, obscuring the real algorithm.

So how can I help with getting it to work with UTF? And is there any way to contact you privately in the future about this?

The first thing I'll need is the single character images for the font named according to Unicode code points and of course the raw text data and Strife dialogue lumps (if you still got raw data for these, all the better) and the raw images for the BigFonts. With that I should have a starting point to commit to a work branch and once that is there I could give you access to that.

For private contacts, please use the forum's PM for now. I already get enough emails so stuff is far more likely to get lost in there than as a PM.

[Kinsie]

The only reason why nothing had been done on that front is that nobody was ever committed enough to do a full translation for another language, especially one with a different alphabet.

There was an effort to create a Japanese translation called Kanalang, but the way it works is fairly hacky due to the engine not supporting that kind of alphabet, and the language barrier meant it didn't really reach these forums.

Marrub and the Kanalang dev have been doing some Japanese-language support stuff using a custom ZScript-based text renderer for Lithium. I believe they looked into doing something for the engine itself, but quickly ran into some of the issues you're seeing now?

Improvements to the font code - both for creation and support for non-roman alphabets - would be greatly appreciated by GZDoom modders and players from around the world.

[Kostov]

The first thing I'll need is the single character images for the font named according to Unicode code points and of course the raw text data and Strife dialogue lumps (if you still got raw data for these, all the better) and the raw images for the BigFonts. With that I should have a starting point to commit to a work branch and once that is there I could give you access to that.

Right. Let’s continue this in PMs.

[Graf Zahl]

Marrub and the Kanalang dev have been doing some Japanese-language support stuff using a custom ZScript-based text renderer for Lithium. I believe they looked into doing something for the engine itself, but quickly ran into some of the issues you're seeing now?

What can I say? If these people have to do it on their own without any attempt to give feedback and align their work with the engine itself, it is doomed to fail. The main reason this is problematic is simply because nobody ever did a concerted attempt to get the engine ready. Cyrillic support would at least be a first step.

Of course this will inevitably look funny with all those mods replacing the fonts...

[Kinsie]

What can I say? If these people have to do it on their own without any attempt to give feedback and align their work with the engine itself, it is doomed to fail. The main reason this is problematic is simply because nobody ever did a concerted attempt to get the engine ready. Cyrillic support would at least be a first step.

In the case of Kanalang, it was developed by a non-programmer who doesn't have a fantastic grip on English. Marrub, as I understand, was basically scared off by the font code that you're currently repulsed by and couldn't really see a route to contributing in that way, so they turned their attention to other ways of helping (ZScript documentation).

In the case of this Japanese support, font glyphs can be made or sourced, and translations for the official game strings are already extant in one form or another (Doom for PC-88, Hexen for N64, this page), it's mostly the technical details of UTF-8 font support that sit in the way. Hopefully this collaboration with Undead results in a solution.

[theleo_ua]

Hi Graf Zahl

As Undead has told me in Discord, we have a phrase which cannot be translated without patching the GZDoom source. Could you please move it to language file in one of the next versions of GZDoom?

src\g_level.cpp:

Code: Select all

players[i].SetLogText ("Find help");

Thank you in advance

[Graf Zahl]

I'm a bit busy elsewhere, but I think these are all in ZScript code so you should be able to export them to LANGUAGE yourself and make a PR.

[Kostov]

I'm a bit busy elsewhere, but I think these are all in ZScript code so you should be able to export them to LANGUAGE yourself and make a PR.

It’s not, actually. I was able to deal with most of them in a single ACS script, but the first one is located in this file. Will .cpp files accept LANGUAGE code?

EDIT: Made a pull request

[Nash]

player_t::SetLogText needs to be made LANGUAGE-aware. This is easy to change.

[Graf Zahl]

No. The output for this needs to be made language aware, not the input.

[Graf Zahl]

This may be interesting for your efforts:

I just made a change that allows putting Hexen's ACS and Strife's dialogue based texts into the language files. However, because these texts are part of the game's data they are not in gzdoom.pk3 but in zd_extra.pk3.