[4.1.3] Controller options appear in chinese

[drfrag]

In old versions the name of the controller (logitech) and the axes were in spanish, language selection (auto or enu) doesn't matter.
Now they are in chinese? with the exception of the X, Y and Z letters. There are some square characters with numbers inside like matrices too.
In the vintage build there are no specific names i guess becouse i didn't pick those characters.
I guess may be it's due to the removal of the automatic language detection name not being in unicode format?

[drfrag]

chinese.png

Language selection doesn't matter.

[Rachael]

I guess may be it's due to the removal of the automatic language detection name not being in unicode format?

Yeah, I was gonna say... what makes you think the automatic language detection has anything to do with it? But then I saw you strike that out.

I had a peek at the i_dijoy.cpp file, and needless to say it's a very ancient work that has not been updated in a very long time.

If it is what you stated you suspect it is, however, it would not be consistently kanji script. Kanji takes up a very small portion of the Unicode table, and if there was an issue converting from CP 1272 to Unicode then the letters would be seemingly random from all over the place.

The automatic language detection removal seems to be a more likely culprit, but I have no way to confirm this right now. The joystick code is calling WMI to get the list of joysticks. It still has ancient code in there to support Windows 2000 that should probably be removed at this point.

I suspect this is one of those things that's not going to be fixed unless it's by a Windows developer who has a controller.

[phantombeta]

It's actually some mix up between non-UTF-16 and UTF-16. Windows uses UTF-16 for a bunch of stuff, so it's the most likely culprit.
Taking the correct text, saving it into a text file as UTF-8, then opening the file as UTF-16 LE in Sublime Text gives some very similar results to what we see in GZDoom:


Not quite the exact same, but it's invalid input, anyway, so differences are to be expected.

[dpJudas]

As far as I know, Graf changed the internal representation in GZDoom strings to be UTF-8. IID_IDirectInput8A does not return UTF-8 strings, it returns ANSI strings. The code in i_dijoy.cpp simply assigns the values as-is to its FString.

Based on the stuff phantombeta wrote, it looks like there are two missing character conversions? First from ANSI to UTF-8, then from UTF-8 to UTF-16?

The code in i_dijoy.cpp seems a bit crazy in general - mixing dinput 3, dinput 8, xinput, WMI and rawinput. What could possibly go wrong?

[drfrag]

After looking a bit into it i noticed it could not be the language autodetection thing. Besides i don't know chinese or japanese? but i don't think those people write using matrices. So it didn't look like real chinese.
I know we need another version of the interface or doing a conversion or two, is there a function to do the conversion in the engine? As expected i looked at the code but i didn't get very far.
Most likely i've got no idea of what i'm talking about but what about using IID_IDirectInput8W instead?

[dpJudas]

OK, I plugged in my Logitech joystick, set up some breakpoints and pushed a fix.

The issue was that UNICODE is defined, which in turn defines IDirectInput8 to IDirectInput8W instead of IDirectInput8A. The original code clearly assumed this wasn't the case. It used an explicit query for the ANSI interface, which was no longer correct.

[drfrag]

Well done. Good to see i was not very far off the mark this time.

[Graf Zahl]

OK, I plugged in my Logitech joystick, set up some breakpoints and pushed a fix.

The issue was that UNICODE is defined, which in turn defines IDirectInput8 to IDirectInput8W instead of IDirectInput8A. The original code clearly assumed this wasn't the case. It used an explicit query for the ANSI interface, which was no longer correct.

Up until the localization effort, all low level ZDoom code was strictly ANSI. Thanks to lots of type casting and generally bad style of the Windows API it's not really surprising that something was overlooked.
The #ifdef UNICODE is quite redundant, though, it cannot be undefined anymore. I don't really understand why nearly all the Windows tutorials stlll try to convey the dual-format approach and just don't simply say "Unicode is the only way"...

[dpJudas]

In my own programs I don't support ANSI (targeting both only made sense around Windows 95 where the unicode support was a redistributable and memory extremely scarce). In this case I only added the #ifdef because the code was written in such an old form to begin with.

As for tutorials, I think it is ultimately because most people don't understand character sets very well. Or why the dual form existed in the first place (for porting from DOS). If they did, they would use the modern approach which is to decide either to use internal strings as UTF-8 (std::string) or UTF-16 (std::wstring) and then add two functions for converting to and from that into the other form. ANSI is never the correct solution.

[Graf Zahl]

I wouldn't use std::wstring because it's not portable - on most POSIX systems wchar_t is a 32 bit type.
That said, basically the only software still using ANSI is either crappy enterprise software that has lived long enough or backwards looking Open Source, like old games where the developers often suffer from some irrational desire to continue supporting Windows 9x at the expense of modernization.

GZDoom only had it this long because there was no pressing need to change.

[dpJudas]

Good point about the wchar_t for the portability - didn't know that. I only used std::wstring in one project, which was strictly Windows where it worked fine.

Nowadays I always use UTF-8 for everything as I believe that was the encoding that ultimately won out as the superior way. The null bytes and endianess issues of UTF-16 makes it unattractive. I think if Win32 had been designed today they would have gone with UTF-8 as well, but at the time it seemed like a good idea as UCS-2 was the original size of the unicode character set.

[Graf Zahl]

Nowadays I always use UTF-8 for everything as I believe that was the encoding that ultimately won out as the superior way. The null bytes and endianess issues of UTF-16 makes it unattractive.

Me, too. I made one exception in GZDoom where I use u32string for the console's command line - in this case it was more convenient to have equal code point size than saving space.

I think if Win32 had been designed today they would have gone with UTF-8 as well, but at the time it seemed like a good idea as UCS-2 was the original size of the unicode character set.

Most likely, yes. This was a classic case of an early adopter being screwed by how things developed.
What Microsoft can be faulted for is not adjusting for far too long. Now it's going to be a long time until the last ANSI traces get eliminated from existing software.