Ability to customize the display of notifications (& more)
Moderator: GZDoom Developers
- Graf Zahl
- Lead GZDoom+Raze Developer
- Posts: 49056
- Joined: Sat Jul 19, 2003 10:19 am
- Location: Germany
Re: Ability to customize the display of notifications (& mor
A fully internationalized font would include CJK, and that's not practical.
For supplying a font with support for other languages, here's the checklist spreadsheet: https://docs.google.com/spreadsheets/d/ ... edit#gid=0
Regarding the format, just have a look at the internal fonts, they use a directory under "fonts/" containing all supported glyphs with a 4 digit hex number of their respective code point.
Greek can be ignored for now, the second attempt at translating it fell just as flat as the first one and no real work was done.
For supplying a font with support for other languages, here's the checklist spreadsheet: https://docs.google.com/spreadsheets/d/ ... edit#gid=0
Regarding the format, just have a look at the internal fonts, they use a directory under "fonts/" containing all supported glyphs with a 4 digit hex number of their respective code point.
Greek can be ignored for now, the second attempt at translating it fell just as flat as the first one and no real work was done.
- Kinsie
- Posts: 7399
- Joined: Fri Oct 22, 2004 9:22 am
- Graphics Processor: nVidia with Vulkan support
- Location: MAP33
- Contact:
Re: Ability to customize the display of notifications (& mor
The easiest way would probably be to default to the optionsmenu font if a given character is missing from the specified font. This way, the player still gets the appropriate info while OCD-heavy modders get a wake-up call to make their beautiful typeface a little fuller.Graf Zahl wrote:Unfortunately I didn't have any idea how to handle the internationalization issue. These functions may not print anything themselves, so there's simply no way to do an engine check, I guess the only way is to publicly lambast the makers for being so careless...
- Graf Zahl
- Lead GZDoom+Raze Developer
- Posts: 49056
- Joined: Sat Jul 19, 2003 10:19 am
- Location: Germany
Re: Ability to customize the display of notifications (& mor
Yes, but that needs to be coded into the new output function. Wanna bet that this is the part which most people won't do?
I already added a CanPrint function to the font class.
I already added a CanPrint function to the font class.
- Marisa the Magician
- Posts: 3886
- Joined: Fri Feb 08, 2008 9:15 am
- Preferred Pronouns: She/Her
- Operating System Version (Optional): (btw I use) Arch
- Graphics Processor: nVidia with Vulkan support
- Location: Vigo, Galicia
- Contact:
Re: Ability to customize the display of notifications (& mor
CanPrint doesn't seem to work over here, it apparently always returns true for any string.
- Graf Zahl
- Lead GZDoom+Raze Developer
- Posts: 49056
- Joined: Sat Jul 19, 2003 10:19 am
- Location: Germany
Re: Ability to customize the display of notifications (& mor
What font, what text?
In the tests I made it correctly recognized if it was printable.
In the tests I made it correctly recognized if it was printable.
- Marisa the Magician
- Posts: 3886
- Joined: Fri Feb 08, 2008 9:15 am
- Preferred Pronouns: She/Her
- Operating System Version (Optional): (btw I use) Arch
- Graphics Processor: nVidia with Vulkan support
- Location: Vigo, Galicia
- Contact:
Re: Ability to customize the display of notifications (& mor
Lemme just isolate this.
Edit: here. It should print "new game" in english, russian, japanese and korean, with the included font, and fall back to NewSmallFont if it can't. But all it does is always use the font (which results in nothing drawn for the other 3 lines).
Edit: here. It should print "new game" in english, russian, japanese and korean, with the included font, and fall back to NewSmallFont if it can't. But all it does is always use the font (which results in nothing drawn for the other 3 lines).
Re: Ability to customize the display of notifications (& mor
It works as expected on Windows though. The problem is caused by differences in locale handling. Selecting UTF-8 locale explicitly fixes this issue.Marisa Kirisame wrote:CanPrint doesn't seem to work over here, it apparently always returns true for any string.
- Graf Zahl
- Lead GZDoom+Raze Developer
- Posts: 49056
- Joined: Sat Jul 19, 2003 10:19 am
- Location: Germany
Re: Ability to customize the display of notifications (& mor
Are you certain that this doesn't break anything else? What precisely does get messed up here? I'd rather fix the actual cause than to muck around with C's broken-by-design locale feature.
How does "C.UTF-8" work?
How does "C.UTF-8" work?
Re: Ability to customize the display of notifications (& mor
Of course I’m not. But I’m pretty sure that standard conforming iswalpha() cannot be used with C locale either.Graf Zahl wrote:Are you certain that this doesn't break anything else?
Like I wrote in the commit message, iswalpha() returns zero for any alphabetic character that isn’t Latin.Graf Zahl wrote:What precisely does get messed up here?
That’s why CanPrint returns true for pretty much any argument.
The single option I see is to hardcode table for alpha category in order to avoid using iswalpha().Graf Zahl wrote:I'd rather fix the actual cause than to muck around with C's broken-by-design locale feature.
It works on Ubuntu 16.04, although this locale seems to specific for some Linux distros according to Google.Graf Zahl wrote:How does "C.UTF-8" work?
- Graf Zahl
- Lead GZDoom+Raze Developer
- Posts: 49056
- Joined: Sat Jul 19, 2003 10:19 am
- Location: Germany
Re: Ability to customize the display of notifications (& mor
_mental_ wrote: The single option I see is to hardcode table for alpha category in order to avoid using iswalpha().
... which makes me wonder, why the hell this isn't even properly defined for a Unicode function. Somehow the C and C++ committees always manage to mess up supposedly simple things. Unicode is quite anal about what category a code point belongs to there, is basically no chance for a letter not being a letter in another language and vice versa.
I guess with such fucked up situations, rolling out one's own table is the only viable solution - like I already did for upper- and lowercase because the language provides no usable facilities.
- Marisa the Magician
- Posts: 3886
- Joined: Fri Feb 08, 2008 9:15 am
- Preferred Pronouns: She/Her
- Operating System Version (Optional): (btw I use) Arch
- Graphics Processor: nVidia with Vulkan support
- Location: Vigo, Galicia
- Contact:
Re: Ability to customize the display of notifications (& mor
My locale is en_GB.UTF-8, though.
Also standards are crap IMHO. With the addition of C library and compiler developers (mis)interpreting them in their own special ways too shit's even worse.
Also standards are crap IMHO. With the addition of C library and compiler developers (mis)interpreting them in their own special ways too shit's even worse.
- Graf Zahl
- Lead GZDoom+Raze Developer
- Posts: 49056
- Joined: Sat Jul 19, 2003 10:19 am
- Location: Germany
Re: Ability to customize the display of notifications (& mor
Your locale is not important here, but the fact that the entire GZDoom code was written against the "C" locale. Change that and lots of stuff might break. Software that mucks around with the locale setting has to be extremely careful about the libraries it uses - most simply cannot deal with different locales at all.Marisa Kirisame wrote:My locale is en_GB.UTF-8, though.
Re: Ability to customize the display of notifications (& mor
Different library implementations have different set of characters in 'alpha' category. So yes, setting locale isn’t enough as it may fail to detect a letter from some specific character.
The thing is Unicode standard has a predefined list of alphabetic characters. Have no idea why it’s not the same in all implementations.
In such situation, own function is not so bad idea indeed. UnicodeData.txt from the standard contains category information important to us.
Unfortunately, this file has some quirks with a few ranges omitted from it. Tomorrow I plan to make a better fix for this problem.
The thing is Unicode standard has a predefined list of alphabetic characters. Have no idea why it’s not the same in all implementations.
In such situation, own function is not so bad idea indeed. UnicodeData.txt from the standard contains category information important to us.
Unfortunately, this file has some quirks with a few ranges omitted from it. Tomorrow I plan to make a better fix for this problem.
- Graf Zahl
- Lead GZDoom+Raze Developer
- Posts: 49056
- Joined: Sat Jul 19, 2003 10:19 am
- Location: Germany
Re: Ability to customize the display of notifications (& mor
My guess is that the C standard goons were overthinking it and the GCC and Clang people dilligently following a broken spec where Microsoft had been doing this 'properly' since 1992, long before there was a coding standard and now can't backpedal because it'd break lots of software._mental_ wrote: The thing is Unicode standard has a predefined list of alphabetic characters. Have no idea why it’s not the same in all implementations.
- Chris
- Posts: 2940
- Joined: Thu Jul 17, 2003 12:07 am
- Graphics Processor: ATI/AMD with Vulkan/Metal Support
Re: Ability to customize the display of notifications (& mor
Because C++ has a standard locale that's supposed to be set by default on launch (and that locale is defined to clearly specify what is valid for things like is[w]alpha). A better question would be why C or C++ hasn't defined a Unicode locale that any program can set, or update the classic/"C" locale with if it can maintain enough "C" locale compatibility that nothing serious would break._mental_ wrote:The thing is Unicode standard has a predefined list of alphabetic characters. Have no idea why it’s not the same in all implementations.
Though as it is, ctype functions like is[w]alpha are already fundamentally broken for multi-character encoding schemes. isalpha will fail for anything that can't be represented by a single char (extended UTF-8 characters don't work), and iswalpha for anything that can't be represented by a single wchar_t (on Windows, extended UTF-16 characters above 0xffff shouldn't work, and that does include some CJK characters). So unless wchar_t is 32-bit, if a desired codepoint to test is outside of wchar_t's range (e.g. greater than 0xffff on WIndows) it will fail regardless of locale. C++11 added a char32_t type to represent UTF-32 characters without multi-character encoding, but there's no ctype functions that can accept it that don't also require an explicit locale parameter (making it very clunky, though does avoid problems with global state negatively affecting external code).
TL;DR locales in C/C++ suck. Probably better to find a string or character handling library that better understands and works with Unicode/UTF-8/16/32.