On-Screen Keyboard Improvements (AKA InputGrid)

[Nash]

Recently got myself a Steam Deck and have been taking small steps to improve the engine's default state to be controller-friendly.

While progress is good so far, one of the remaining pain points is the Input Grid (for the on screen keyboard).

There are two things I think that need to be done to fix it:

1) An option to allow it to appear anywhere that text entry is allowed. This includes the console. Obviously if a person is playing with a mouse and keyboard, this is largely unnecessary, but if someone enters the console from a mouse or gamepad (this is possible when using the menu), it should automatically show the OSK. Possibly with some CVars to block it completely, or to unconditionally show it no matter how the console is entered, as per GZDoom norm.

2) Proper keyboard layout with a graphical IME switcher, with the IME defaulting to the user's current language. Basically this would mean the numbers 1234567890 on the top row, qwertyuiop on the next, etc. With shift and altgr support, with punctuation as dictated by the IME. Custom IME's could easily be implemented if a user supplies a text file with the requisite layout, and should be defined internally in GZDoom under the same format. And for people with dvorak or whatever keyboards this would be possible too.

I'm mostly putting this here because I think it's going to require more than just 2 or so devs working on it in order to make this work. Most of it can be done in ZScript, there wouldn't be a lot of C++ changes required to make this work.

[Rachael]

Point 1 is probably going to be the hardest part, getting it to work inside the console, I know it can be done but I haven't yet figured out how much work this is going to take, and that's definitely going to be the bulk of the C++ work done for this.

[Nash]

I should also mention:

Another thing I noticed, and this is weirdly specific and I honestly have no idea why it's happening:

On the Steam Deck, when you summon the built-in OSK (for example, say you want to type some commands in the GZDoom console) -- typing simply does not work. You can tap the letters all you want, but nothing gets typed into GZD's console.

HOWEVER! The tilde key on the OSK works (closes the console, as expected). Also the Enter key works (it just submits a new line).

The OSK also works in-game - tapping space on the OSK makes DoomGuy jump.

This is very strange. I'm convinced this is a bug. From whose side though, I can't say.

Point 1 is probably going to be the hardest part, getting it to work inside the console, I know it can be done but I haven't yet figured out how much work this is going to take.

Yeah. I'll have to take a closer look at it (cannot at the time of this post) but I believe the InputGrid is currently heavily married to the menu mode right now. The 'gamestate' for the various console modes cannot mix with GS_MENUSCREEN.

[Graf Zahl]

The input grid isn't even a fully featured keyboard, if you need something better for the Steam Deck I think it may be a better idea to look for a third party solution that can be adapted. That thing was written at a time when ZDoom was ASCII and English only and essentially meant to allow entering savegame names with the controller, nothing more.

Regarding controllers in general, there's also this open suggestion: viewtopic.php?t=78734
I think if we really want to improve controller support it is a losing proposition to keep using our ancient controller code from early ZDoom years. I have taken a look at what other engines do with SDL and controllers and the code looks a lot better than the crusty stuff we have currently (in 3 platform dependent variants no less, so have fun keeping that operable.) The only downside is that we'll need someone with the time to get the SDL backend running on Windows so that we can start a serious evaluation of our options.