Accessibility on Linux is Broken in version GZDoom g4.13pre

[stormdragon2976]

I have the latest version installed from git as of this morning. Previously, in versions up to and including "GZDoom g4.11.3-m - 2023-10-26 15:18:50 -0400 - SDL version" Orca is able to read the window, making it possible to pick the game you want from the list.

With the latest version from git, the window is completely silent with Orca. The full string containing the version information is "GZDoom g4.13pre-87-gd697eafbe-m - 2024-07-29 00:19:56 -0400 - SDL version".

Thanks for all the hard work that goes into this. It's been countless hours of fun, and will continue to be for years to come.

[dpJudas]

I'm afraid this requires someone with Linux knowledge about how to declare the window contents to accessibility tools. I do wonder how you're going to survive in a game of Doom if you need accessibility help just to get through the wad selection dialog though.

[axredneck]

I believe it's because GZDoom no longer uses GTK.

[dpJudas]

Correct, however going back to GTK is not an option for other reasons.

[stormdragon2976]

Ahh, that would explain the sudden loss of accessibility. What is the new GUI toolkit? If it were QT5 or later, it should also work.

There is an accessibility mod for actually playing the game. It works quite well. It is how I am able to play and actually beat the game.

[Rachael]

It's a custom in-house one that for too many reasons got picked and favored over the standard ones (GTK3, Qt, etc). Those came with too much baggage and to be quite honest they aren't as easy to use cross-platform as they should be. GTK3 for instance has some glibc dependencies that makes it much harder to use on Windows.

Having some experience with this, I understand and agree with dpJudas's decision to create a widget library from scratch. I have created an application that uses the raw widget code for Cocoa and Win32, and at first it even was programmed to use raw X11 code, but then I moved to GTK3 afterward only because it interfaces with Linux better and it also has some features that Win32 and Cocoa had that raw X11 were missing, it also interfaces directly with Wayland too. To put it bluntly: GUI programming is a nightmare to break the ice with. You definitely can't do it without creating libraries of your own at some point if you get into it too deeply, even if those libraries are nothing more than "put widget on screen and make widget do something when clicked".

There is absolutely no cross-platform standard that works well and works natively for all 3 major systems without a bunch of fuckery tied into them.

This is the new library that GZDoom is currently using: https://github.com/dpjudas/ZWidget/ - I am sure that accessibility could be built into it, but they'd need to be built by users experienced in accessibility and the ecosystems within which they are used.

[stormdragon2976]

That makes sense. The small bit of GUI development I have done wasn't much fun and had challenges for cross platform too, and it wasn't anything major at all. I did find that if you launch gzdoom without an X session or wayland, it brings up a small command line interface, 1 freedoom, 2 freedoom2, 0 to exit. I went searching for a command line flag to access that even in the GUI session, but couldn't find anything. Could a flag be added to bring up this interface? It would solve the inaccessibility problem, because it provides all the same functionality, well the important stuff anyway. It doesn't have the radio buttons for selecting video stuff, but if those are needed, most likely someone who can see the GUI would be around to help.

[dpJudas]

As Rachael mentions, there's nothing technically preventing zwidget from supporting accessibility tools.

The issue is more that I'm not aware of any actual standard for accessibility on X11 or Wayland. My understanding so far is that the accessibility tools there tap directly into the two mainstream UI toolkits for the platform (Gtk+ and Qt). If there's a relatively simple interface other toolkits can implement on Linux to add support I am not against adding it in principle.

Regarding alternatives, if you can find a GZDoom launcher on Linux that has proper accessibility tool support you could use that. I think some of them are based on Qt.

[stormdragon2976]

QT and GTK send information and events through at-spi, which is how they are accessible to Orca. I only know the bare minimum about how it works, so I will contact the developers and ask for clarification and documentation. I will post what I find out here, but as far as I know, any application can be made accessible if it sends these events to at-spi. Hopefully more information coming soon.

[dpJudas]

Thanks for that. Just need to implement the at-spi2 dbus interface then. Of course the big question is how much work there is in doing so - most of the libraries they wrote for it seems to bring in the whole gnome package. In the meantime I think your best option is to use a third party launcher written in a UI toolkit with at-spi support.

[stormdragon2976]

I looked into the launchers available in the AUR, and there was one called supergzdoom which may work. The labeling for the button I found was a little off, but nothing that can't be fixed. For now, I'm personally just using 4.11, it still had GTK. For me personally, I know where each of the entries are in the list, so I can do it without speech. The problem is for newcomers. I think I can work around this in my launcher until accessibility is added. There's also the -iwad flag, so we are not out of options.

I received this response from the Orca email list, so it seems that at-spi is not the only method.

Message starts here:

I believe AccessKit (https://github.com/AccessKit/accesskit) is ready, or
close to being ready, for use by cross-platform C and C++ toolkits like the
ZWidget toolkit that GZDoom is using now. The C bindings, which can also be
used from C++, are in this repository:

https://github.com/AccessKit/accesskit-c

Included in that repository, and in the pre-built binary packages, is an
example of using AccessKit across all three major desktop platforms. This
example happens to use SDL, but it shows how to use the three AccessKit
platform adapters directly.

My one reservation, and the reason I say it might only be "close to being
ready", is that we likely need to do more work to make the library easy to
integrate into various build systems, especially since the library itself is
written in Rust.

Hope this helps,
Matt

[Rachael]

One thing that would be really helpful and this is truly a question that can only be answered properly by someone who has the needs this would provide for: What is an overview for how such a feature would work? What are all the things we'd need to plan for, what would you need, in order for this to be fully functional? Being specific about everything, including the workflows would help a lot, too. I hope I am not asking too much here, but I also just can't ask this of any random person, either.

I hope you understand, that asking this isn't an implicit promise to finish this feature, that depends a lot on how hard it will be, but I can tell you for certain it's not as simple as "drop library in and hit compile" and that is why I felt the need to make this inquiry. My goal in asking this question is to have a final picture to shoot for, with what the feature will look like, and maybe that will be something we can at least start working towards. And who knows, if we start the process along, maybe someone who has a better understanding of what is necessary can help us to finish it. That kind of input would be immensely valuable.

[dpJudas]

Generally speaking the way this works is that the UI framework and the application exports a different interface to accessibility applications. The accessibility app then uses that interface to do what it needs to do. For example in order to navigate a listview, the app tells via the interface that there's a listview named "IWADS" and then the accessibility app can navigate it by sending messages back that are kind of like keyboard input, but designed to be more specific in purpose.

On Windows the accessibility interface is the IAccessible COM abstraction that a HWND can return. On Linux its the at-spi2 interface. Who knows what Mac got. On websites its certain attributes on DOM elements. For any of them the app exports the interface and the accessibility app consumes it.

In principle its fairly simple what needs to be done: export an accessible interface for PushButton, ListView, TabControl and CheckBox. The hard part is the fact that it needs to be done 3 times: one time for each platform. Oh and that accessibility documentation unfortunately in general is really poor on all three platforms.

[stormdragon2976]

@Rachael to answer your work flow question, in my case, 99.9% of my usage is GZDoom launches, it starts with the list of iwads focused. I use up and down arrow to find the one I want and press enter. That's the whole thing. I know there are other settings available because I did explore the GUI, but I don't think I have ever personally used any of them.

After talking to some other blind people who use it, most of them never even tabbed through the other stuff and just use the list to find the desired iwad.

[Rachael]

So basically, enabling tab and arrow key navigation, and having it speak the currently selected item, should hopefully cover a good percentage of what you need, for now?