TTS/Screenreader support for blind players

Moderator: GZDoom Developers

SanyaWaffles
Posts: 800
Joined: Thu Apr 25, 2013 12:21 pm
Preferred Pronouns: They/Them
Operating System Version (Optional): Windows 11 for the Motorola Powerstack II
Graphics Processor: nVidia with Vulkan support
Location: The Corn Fields
Contact:

Re: TTS/Screenreader support for blind players

Post by SanyaWaffles »

MartinHowe wrote:As a disabled person myself, I would have had a much better life if I'd been born in 1995 instead of 1965, but there's nothing I can do about it; only hope (and, in my job, work) for a better life for people like me in 50 year's time.
Yeah, I can understand. I have a form of autism myself and have a hard time working in America, but I do my best. I do my best to work daily. It's tough, can't deny it.

You made some good points there MartinHowe.
User avatar
Enjay
 
 
Posts: 26517
Joined: Tue Jul 15, 2003 4:58 pm
Location: Scotland
Contact:

Re: TTS/Screenreader support for blind players

Post by Enjay »

objectinspace wrote:what is the point of this project?
Primarily to get the Doom engine running on modern hardware with a modern renderer and to (massively) expand the modding capabilities while remaining compatible with as many legacy mods as possible. Anything else, including accessibility for disabled users, is a bonus - but not a main goal. GZDoom is already more accessible than the original Doom games.
objectinspace wrote:I'm actually extremely offended by this answer, on a deep personal level. Your attitude is ignorant, hateful, bigoted, ableist--and most importantly, wrong.
You are offended - well, that's disappointing, but that's all it is. (I tend to agree with the Stephen Fry quote about people being offended. (Though perhaps not so hard-over as that quote implies, and I certainly have sympathy in this particular case. There are times, IMO, when being offended is an appropriate response.)) However, I feel that your response could be described using a very similar set of words as the ones you use.

I get that being ignored, marginalised and dismissed for a disability (or for any reason for that matter) is hugely frustrating, hurtful, sometimes even dangerous and a daily occurrence for many people. It sucks, I really do get that (which is why I feel that the Stephen Fry position is too harsh in this case). But GZDoom is a hobby project; anything and everything that has been done with it is as a free gift to the community. Coming out swinging at the people who invest thousands of hours to make it really isn't a good way forward. Demanding action from someone who has no obligation to you and who does not have the time/skills/whatever to do what you want and attacking them by calling them names for it isn't going to help either. Surely the irony in you using language likely to be interpreted as offensive to describe someone who offended you will not be lost on you.

Now that is out of the way...

I'm just wondering about this from the opposite perspective. Does some sort of overlay software exist that can do OCR and then do a TTS conversion on the fly? Would it take too much processing power? Would it be too confused by "fancy" fonts?

I agree with the views expressed by most of the contributors to this thread. I think that they are realistic and correct in the context of GZDoom. I'll not rehash them because they have been well covered many times already. But it strikes me that if some sort of software could sit on a person's computer and convert text on images to speech in real time it would be the answer to this.

I'm not suggesting that this would be something for the GZDoom devs to do (far from it) but it strikes me as the kind of thing that probably should already be in development (or already exist) somewhere and, if it does exist, then the people responsible would be the ones best placed to make it more universal to allow it to be used in games - any games - and a whole bunch of other programs too. It wouldn't just enable blind and visually impaired people to play a nearly 30 year old niche retro game, but any game! Does such a thing exist?
User avatar
Graf Zahl
Lead GZDoom+Raze Developer
Lead GZDoom+Raze Developer
Posts: 49056
Joined: Sat Jul 19, 2003 10:19 am
Location: Germany

Re: TTS/Screenreader support for blind players

Post by Graf Zahl »

Enjay wrote: I'm just wondering about this from the opposite perspective. Does some sort of overlay software exist that can do OCR and then do a TTS conversion on the fly? Would it take too much processing power? Would it be too confused by "fancy" fonts?
You may do something like that if all you used is simple to read stock fonts - and even then OCR is still hit and miss. Doing it with graphical fonts - especially extremely low res ones blown up 5-6 times - is not going to work. Having done some processing of books into a digital format I can assure you that it just won't work. Most OCR software will already bail out if you feed it some blackletter printing from early 20th century Germany, and these are known fonts, not some random collections of pixels.
User avatar
Enjay
 
 
Posts: 26517
Joined: Tue Jul 15, 2003 4:58 pm
Location: Scotland
Contact:

Re: TTS/Screenreader support for blind players

Post by Enjay »

Fair enough. My experience with OCR is limited.

I tried it many years ago to convert scanned paper documents to text and it had so many errors that it was actually easier to re-type the documents in most cases. More recently, however, at work we needed to do the same thing but the OCR worked well enough that the errors were minimal and all that was needed was a quick proof read. So because things had improved for that job, I wondered if it might be possible elsewhere. However, I can certainly understand how the low-res pixelated graphical fonts from a 320x200 game would be difficult.

Perhaps one day AI will be intelligent enough to look at shapes and make sense of them in a similar way to how humans can - but not yet I guess.
User avatar
Alando1
Posts: 224
Joined: Mon Apr 15, 2013 6:46 pm
Location: New York, USA

Re: TTS/Screenreader support for blind players

Post by Alando1 »

A screen reader would be helpful, indeed. However, my team and I who are working on the Toby Accessibility Mod have been trying to think about if it is possible to have specific audio queues based on what the Doom skull cursor is highlighting. It would be nice if we could define unique sounds for the Doom menu - Perhaps something with ZScript?

If anyone wants to check out the Toby Accessibility Mod, you can find it here: viewtopic.php?f=4&t=71349

-Alando1
SanyaWaffles
Posts: 800
Joined: Thu Apr 25, 2013 12:21 pm
Preferred Pronouns: They/Them
Operating System Version (Optional): Windows 11 for the Motorola Powerstack II
Graphics Processor: nVidia with Vulkan support
Location: The Corn Fields
Contact:

Re: TTS/Screenreader support for blind players

Post by SanyaWaffles »

@Alando1 - I appreciate the work you're doing. I don't think I ever have said that.

There are ways of doing menus with custom sounds in the menus in theory to add such a feature. Check and see how ZScript implements it.
Proydoha
Posts: 53
Joined: Thu Jan 21, 2016 2:25 am

Re: TTS/Screenreader support for blind players

Post by Proydoha »

Menu problem can be attacked from a different angle. You can load into any map bypassing menus by providing suitable launch parameters. Usage of various launchers is not unknown concept for doom players.

Maybe somebody can make a launcher that would compose all launch parameters needed as well as editing an .ini file if settings need to be changed while providing all sounds needed in the launcher and not in the game.
User avatar
wildweasel
Posts: 21706
Joined: Tue Jul 15, 2003 7:33 pm
Preferred Pronouns: He/Him
Operating System Version (Optional): A lot of them
Graphics Processor: Not Listed
Contact:

Re: TTS/Screenreader support for blind players

Post by wildweasel »

Proydoha wrote:Menu problem can be attacked from a different angle. You can load into any map bypassing menus by providing suitable launch parameters. Usage of various launchers is not unknown concept for doom players.

Maybe somebody can make a launcher that would compose all launch parameters needed as well as editing an .ini file if settings need to be changed while providing all sounds needed in the launcher and not in the game.
This doesn't necessarily mean the menu problem shouldn't be tackled at some point; what if the player needs to change game options, or save the game?
User avatar
Alando1
Posts: 224
Joined: Mon Apr 15, 2013 6:46 pm
Location: New York, USA

Re: TTS/Screenreader support for blind players

Post by Alando1 »

Graf Zahl wrote:
objectinspace wrote: ???

If empowering the 35 million plus people with sight loss around the world to play Doom has "very little payoff in the end," what is the point of this project? I don't imagine , for example, adding OpenGL/DX11 support to a game that's some 30 years old was trivial either, but it was done, because people wanted it.
How many of these play Doom? You seem to be the first one to come around.
I'm really sorry but this is a hobby project - I simply do not have the time to invest into a feature that may ultimately be used by a handful of users. I surely won't say "no" if someone stepped up to implement it, but as things stand, I see zero chance for it to happen.
@Graf Zahl - Back in 2017, a gentleman named Toby (who the wad is named after) reached out to me. There's a documentary called "Gaming Through New Eyes" that he showed me. Here's the link to the video: https://youtu.be/P7n9s7yBlGw

In fact, there are many blind/visually impaired gamers out in the world. I've teamed up with not just Toby, but SightlessKombat and IllegallySighted. These gentlemen have been helping me and my programmer, Jarewill, to further improve the accessibility features. A handful of blind/visually impaired people comment on some of my YouTube videos about the mod. If they have any questions or concerns, I do my best to help them. All in all, there are many people who are blind and want to play games such as Doom. I'm willing to do what I can to give them the chance to have fun with an amazing game.

I am very grateful there are many people who support my project and I'm very grateful I get to work with a group of amazing individuals who helped shape this project into what it is today. As I have pointed out on the main project thread regarding the accessibility mod, if anyone would like to contribute something to the project, they are more than welcome.

-Alando1
Guest

Re: TTS/Screenreader support for blind players

Post by Guest »

Howdy folks! I think the menus can be read with Capture 2 Text and Minispeaker which are both free software. The issue is the fancy font should probably be replaced in the main menu with font similar to that of the settings menu and brightened up while reducing the background sto something like a solid black color. I've tried to read the menus with Capture 2 text but it wants to read the background as well and won't recognize dark text that is against the busy background. It did work... kinda... but I imagine it MIGHT be improved with these changes. Also, OCR with the NVDA screen reader MAY work as well... but I have not yet tested it. Lion OCR is an addon to NVDA and works in a different way but still using Windows 10s OCR engine but is yet another option.

Using Capture to text reads whatever you highlight with a hotkey and the mouse then puts it in the clipboard as well as a popup window I'd recomend turning off. Minispeaker simply reads everything copied to the clipboard and so using them both together can create a sort of rough screen reader.
It is not perfect but does work in other games.

Capture 2 Text
https://sourceforge.net/projects/capture2text/

Minispeaker
http://speakcomputer.com/text-to-speech ... eaker.aspx
Post Reply

Return to “Closed Feature Suggestions [GZDoom]”