[games_access] Toolkits for making GL rendered content screen reader accessible?

Fri Jan 13 13:12:59 EST 2017

You mean developing natively rather than using an engine? Apart from all the nice toys that you get with engines a big issue is that developing natively means you're only developing for a single platform, so it's a totally unrealistic ask for a developer to give up cross platform publishing and have to develop natively for each platform they want to release on.

It causes all manner of problems between gamers and developers, developers just telling gamers that they've investigated it and it's not technically/financially possible (for above reasons, but not explaining those reasons), gamers having seen games with similar mechanics that are accessible, so reacting angrily to what they see as devs lying.. really unfortunate. Especially as blind gamers are such a great demographic to work with.

There would be another hacky way of doing it, which would be for the engine to abuse the notification system, and send a notification out whenever an element receives focus. Working on push rather than pull, with focus management handled by the engine rather than the OS.

That way the OS never needs to see anything inside the game. But from what I've been told from one of the platforms there wouldn't be much point in doing that for their platform at least, as apparently it would be a similar level of work to do that as change how the rendering works to output native accessible UI (properly accessible without needing a layer).

________________________________
From: games_access <games_access-bounces at igda.org> on behalf of Brandon Keith Biggs <brandonkeithbiggs at gmail.com>
Sent: 13 January 2017 17:24
To: IGDA Games Accessibility SIG Mailing List
Subject: Re: [games_access] Toolkits for making GL rendered content screen reader accessible?

Hello,
One major problem that I have heard from game developers is that using stock widgets is too difficult. It is hard to customize them and they don't work well with animation.
Has anyone ever tried making an animated game with native widgets?
I think Unity would need to implement some kind of fake native cover to the different elements on the screen. If there are stock widgets or shapes, they could be assigned a button type or whatnot. What would be the feasibility of using IAccessible2 in a Unity project?
Thanks,

Brandon Keith Biggs<http://brandonkeithbiggs.com/>

On Fri, Jan 13, 2017 at 8:09 AM, Ian Hamilton <i_h at hotmail.com<mailto:i_h at hotmail.com>> wrote:

Not at all unfortunately, games have more diverse ranges of typefaces and typography than most other media, which is an accessibility issue in itself, for people who have difficulty reading.. e.g. a tiny bespokely made ornate fantasy font in full caps.

It's something that could/should be done just a single time by the engine developer. That way it should just work like native iOS app development.. the UI accessibility mostly just works (e.g. the accidental blind accessibility here, just from having logically named UI elements), all that's left for game developers is just a bit of tweaking, which is a much more realistic ask than the current option of dumping your engine and starting again from scratch.

No articles about the BBC tests beyond a little bit that I wrote here -

http://ian-hamilton.com/screenreaders-and-game-engines/

It's just that simple of thing of one layer being CPU and one being GPU, which kills the idea due to the tiny bus width on mobiles. It's fine as a hacky workaround for games that have static visuals so can happily run at 2fps, but many games don't work with that.

Again things like that and TOLK are just hacky workarounds to try and substitute for work that game engines should have taken care of themselves, game engines are where the issue lies.

It isn't an issue that's unique to game development, you see the same thing anywhere where frameworks are used. But because there are a small range of popular ones in gamedev it would be very easy to make a difference, particularly as the vast majority of games that have a mechanic suited to screenreader accessibility are made in Unity anyway. Just Unity fixing it would make a staggering difference.

Ian

________________________________
From: games_access <games_access-bounces at igda.org<mailto:games_access-bounces at igda.org>> on behalf of Brandon Keith Biggs <brandonkeithbiggs at gmail.com<mailto:brandonkeithbiggs at gmail.com>>
Sent: 13 January 2017 15:19
To: IGDA Games Accessibility SIG Mailing List
Subject: Re: [games_access] Toolkits for making GL rendered content screen reader accessible?

Hello,
Are most games using a limited number of typeface and fonts?
If so, one could just tell an OCR engine that this is A, this is B and whatnot. It would probably be more accurate than current OCR. Also, most players have more free time to create an accessibility layer than the game devs do. anything that would take the work and give it to the non technical community would be more than worth it.

Do you have any articles or talks about the BBC Unity accessibility experience? I really want to learn as much as possible about what they did, what worked, what didn't work and whatnot. I would also love to reference their case in a paper I am writing.
Thanks,

Brandon Keith Biggs<http://brandonkeithbiggs.com/>

On Fri, Jan 13, 2017 at 6:47 AM, Ian Hamilton <i_h at hotmail.com<mailto:i_h at hotmail.com>> wrote:

Yes, BBC tried it, didn't work sadly!

Reason being that the layer is rendered by the CPU and the visual by the GPU, and mobiles only have a tiny bus width, so constantly switching between the two layers do to the updates caused framerates to slow to an unplayable crawl. So BBC just put an end to developing with Unity for cross platform web games and use HTML5 instead.

For PC you can use Tolk, Skullgirls managed to get it working nicely for their UI. It still needs to be actively integrated by a developer, it only outputs what is passed out to it so is still far more work than it should be, involving focus management and passing the info out to all be handled at a game developer level, and doesn't work with all screenreaders or cross-platform either. But it's something at least. That's about as good as it gets for now, until if/when engine devs integrate something like it internally.

https://davykager.com/projects/tolk/

I don't think OCR would have much chance of working with most games due to the kind of typefaces and typography used, but most of the text, text strings for copy and text labels for UI elements, is already there inside the game anyway. Just (just..) need the engine devs to implement a way of exposing it.

That's easily my top thing I'd like to see happen in accessibility, such a frustrating barrier.

Ian

________________________________
From: games_access <games_access-bounces at igda.org<mailto:games_access-bounces at igda.org>> on behalf of Brandon Keith Biggs <brandonkeithbiggs at gmail.com<mailto:brandonkeithbiggs at gmail.com>>
Sent: 13 January 2017 14:20
To: IGDA Games Accessibility SIG Mailing List
Subject: [games_access] Toolkits for making GL rendered content screen reader accessible?

Hello,
I am wondering what research or work has been done on making GL rendered content accessible to screen reader users?
Ian was mentioning on the Unity Screen reader accessibility thread:
https://feedback.unity3d.com/suggestions/screen-reader-accessibility
That Apple had an idea to replicate the UI above the app, but I can't find any information about this at all. Are there any articles or papers on Apple's trial? Was this invisible UI ever tried on the PC?

I think that a layer like this would be very useful, especially if the application had some sort of image or character recognition to present new content to the screen reader.
Are there any other applications or companies who have made (or tried to make) any programs to add accessibility to prebuilt apps and particularly GL rendered apps and games?
Thanks,

Brandon Keith Biggs<http://brandonkeithbiggs.com/>

_______________________________________________
games_access mailing list
games_access at igda.org<mailto:games_access at igda.org>
https://pairlist7.pair.net/mailman/listinfo/games_access
The main SIG website page is http://igda-gasig.org

_______________________________________________
games_access mailing list
games_access at igda.org<mailto:games_access at igda.org>
https://pairlist7.pair.net/mailman/listinfo/games_access
The main SIG website page is http://igda-gasig.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist7.pair.net/pipermail/games_access/attachments/20170113/773369b8/attachment.htm>