[4.1.3] Controller options appear in chinese

Forum rules
Please don't bump threads here if you have a problem - it will often be forgotten about if you do. Instead, make a new thread here.

Post a reply

Smilies
:D :) :( :o :shock: :? 8-) :lol: :x :P :oops: :cry: :evil: :twisted: :roll: :wink: :geek: :ugeek: :!: :?: :idea: :arrow: :| :mrgreen: :3: :wub: >:( :blergh:
View more smilies

BBCode is OFF
Smilies are ON

Topic review
   

Expand view Topic review: [4.1.3] Controller options appear in chinese

Re: [4.1.3] Controller options appear in chinese

by Graf Zahl » Mon Jul 01, 2019 4:32 am

dpJudas wrote: Nowadays I always use UTF-8 for everything as I believe that was the encoding that ultimately won out as the superior way. The null bytes and endianess issues of UTF-16 makes it unattractive.
Me, too. I made one exception in GZDoom where I use u32string for the console's command line - in this case it was more convenient to have equal code point size than saving space.

dpJudas wrote: I think if Win32 had been designed today they would have gone with UTF-8 as well, but at the time it seemed like a good idea as UCS-2 was the original size of the unicode character set.
Most likely, yes. This was a classic case of an early adopter being screwed by how things developed.
What Microsoft can be faulted for is not adjusting for far too long. Now it's going to be a long time until the last ANSI traces get eliminated from existing software.

Re: [4.1.3] Controller options appear in chinese

by dpJudas » Mon Jul 01, 2019 3:55 am

Good point about the wchar_t for the portability - didn't know that. I only used std::wstring in one project, which was strictly Windows where it worked fine.

Nowadays I always use UTF-8 for everything as I believe that was the encoding that ultimately won out as the superior way. The null bytes and endianess issues of UTF-16 makes it unattractive. I think if Win32 had been designed today they would have gone with UTF-8 as well, but at the time it seemed like a good idea as UCS-2 was the original size of the unicode character set.

Re: [4.1.3] Controller options appear in chinese

by Graf Zahl » Mon Jul 01, 2019 3:37 am

I wouldn't use std::wstring because it's not portable - on most POSIX systems wchar_t is a 32 bit type.
That said, basically the only software still using ANSI is either crappy enterprise software that has lived long enough or backwards looking Open Source, like old games where the developers often suffer from some irrational desire to continue supporting Windows 9x at the expense of modernization.

GZDoom only had it this long because there was no pressing need to change.

Re: [4.1.3] Controller options appear in chinese

by dpJudas » Mon Jul 01, 2019 2:46 am

In my own programs I don't support ANSI (targeting both only made sense around Windows 95 where the unicode support was a redistributable and memory extremely scarce). In this case I only added the #ifdef because the code was written in such an old form to begin with.

As for tutorials, I think it is ultimately because most people don't understand character sets very well. Or why the dual form existed in the first place (for porting from DOS). If they did, they would use the modern approach which is to decide either to use internal strings as UTF-8 (std::string) or UTF-16 (std::wstring) and then add two functions for converting to and from that into the other form. ANSI is never the correct solution.

Re: [4.1.3] Controller options appear in chinese

by Graf Zahl » Mon Jul 01, 2019 12:28 am

dpJudas wrote:OK, I plugged in my Logitech joystick, set up some breakpoints and pushed a fix.

The issue was that UNICODE is defined, which in turn defines IDirectInput8 to IDirectInput8W instead of IDirectInput8A. The original code clearly assumed this wasn't the case. It used an explicit query for the ANSI interface, which was no longer correct.

Up until the localization effort, all low level ZDoom code was strictly ANSI. Thanks to lots of type casting and generally bad style of the Windows API it's not really surprising that something was overlooked.
The #ifdef UNICODE is quite redundant, though, it cannot be undefined anymore. I don't really understand why nearly all the Windows tutorials stlll try to convey the dual-format approach and just don't simply say "Unicode is the only way"... :?

Re: [4.1.3] Controller options appear in chinese

by drfrag » Sun Jun 30, 2019 1:32 pm

Well done. Good to see i was not very far off the mark this time. :)

Re: [4.1.3] Controller options appear in chinese

by dpJudas » Sun Jun 30, 2019 1:15 pm

OK, I plugged in my Logitech joystick, set up some breakpoints and pushed a fix.

The issue was that UNICODE is defined, which in turn defines IDirectInput8 to IDirectInput8W instead of IDirectInput8A. The original code clearly assumed this wasn't the case. It used an explicit query for the ANSI interface, which was no longer correct.

Re: [4.1.3] Controller options appear in chinese

by drfrag » Sun Jun 30, 2019 12:52 pm

After looking a bit into it i noticed it could not be the language autodetection thing. Besides i don't know chinese or japanese? but i don't think those people write using matrices. :) So it didn't look like real chinese.
I know we need another version of the interface or doing a conversion or two, is there a function to do the conversion in the engine? As expected i looked at the code but i didn't get very far.
Most likely i've got no idea of what i'm talking about but what about using IID_IDirectInput8W instead?

Re: [4.1.3] Controller options appear in chinese

by dpJudas » Sun Jun 30, 2019 4:43 am

As far as I know, Graf changed the internal representation in GZDoom strings to be UTF-8. IID_IDirectInput8A does not return UTF-8 strings, it returns ANSI strings. The code in i_dijoy.cpp simply assigns the values as-is to its FString.

Based on the stuff phantombeta wrote, it looks like there are two missing character conversions? First from ANSI to UTF-8, then from UTF-8 to UTF-16?

The code in i_dijoy.cpp seems a bit crazy in general - mixing dinput 3, dinput 8, xinput, WMI and rawinput. What could possibly go wrong? ;)

Re: [4.1.3] Controller options appear in chinese

by phantombeta » Sun Jun 30, 2019 3:59 am

It's actually some mix up between non-UTF-16 and UTF-16. Windows uses UTF-16 for a bunch of stuff, so it's the most likely culprit.
Taking the correct text, saving it into a text file as UTF-8, then opening the file as UTF-16 LE in Sublime Text gives some very similar results to what we see in GZDoom:

Image
Not quite the exact same, but it's invalid input, anyway, so differences are to be expected.

Re: [4.1.3] Controller options appear in chinese

by Rachael » Sun Jun 30, 2019 2:40 am

drfrag wrote: I guess may be it's due to the removal of the automatic language detection name not being in unicode format?
Yeah, I was gonna say... what makes you think the automatic language detection has anything to do with it? But then I saw you strike that out.

I had a peek at the i_dijoy.cpp file, and needless to say it's a very ancient work that has not been updated in a very long time.

If it is what you stated you suspect it is, however, it would not be consistently kanji script. Kanji takes up a very small portion of the Unicode table, and if there was an issue converting from CP 1272 to Unicode then the letters would be seemingly random from all over the place.

The automatic language detection removal seems to be a more likely culprit, but I have no way to confirm this right now. The joystick code is calling WMI to get the list of joysticks. It still has ancient code in there to support Windows 2000 that should probably be removed at this point.

I suspect this is one of those things that's not going to be fixed unless it's by a Windows developer who has a controller.

Re: [4.1.3] Controller options appear in chinese

by drfrag » Sat Jun 29, 2019 12:12 pm

chinese.png
Language selection doesn't matter.

[4.1.3] Controller options appear in chinese

by drfrag » Sat Jun 29, 2019 5:13 am

In old versions the name of the controller (logitech) and the axes were in spanish, language selection (auto or enu) doesn't matter.
Now they are in chinese? with the exception of the X, Y and Z letters. There are some square characters with numbers inside like matrices too.
In the vintage build there are no specific names i guess becouse i didn't pick those characters.
I guess may be it's due to the removal of the automatic language detection name not being in unicode format?

Top