Unity Plugin for UI Accessiblity

Updates on the game have been scarce of late, I know. I’ve been looking into making my mobile games accessible and as a result felt forced to take on a new side project.

Accessible Mobile Games

Not everybody realizes that blind people are avid smartphone and tablet users too, since both Android and iOS have integrated screen readers. These help navigating around the screen, reading out buttons and text and preventing accidental clicks. There are also a few special gestures that can be drawn on screen that will trigger the Back or the Home button.

Blind Man Walking with a stick

The problem is that I am using Unity for my games, which doesn’t produce apps that work with these screen-readers. This is perfectly alright, it’s just in the nature of the design on which Unity is built.

No Unity Solution – yet

I searched up and down the Internet for a solution, but all I could find was other people facing the same problem. They either gave up on making their apps accessible, or had to re-implement their menus, and record additional audio or use Text-To-Speech engines. If you want to read a great story from one such developer, let me point you to this Gamasutra Article.

Woman standing excitedly in front of the Unity Asset Store.

My usual asset store experience.

My hope was to find a plugin to do some of the heavy lifting for me, but for the first time, the mighty Asset Store, my trusted friend, who usually has a solution for every problem known to man, came up empty. There are a number of Text-To-Speech engines, but nothing that automatically handles the UI navigation, reads out the elements, makes sure buttons can be triggered, sliders can be moved and toggles can be flipped – let alone listen to any magic gestures.

So I decided I will write such a plugin myself then.
I have no clue whether the world at large has any need for this (I guess not, or someone would have done it already), but I do, so let’s get started.

Feature Wishlist

In short, what I would like is a way to mark up my UI (uGUI) with accessibility tags. The plugin has to then automatically allow the users to navigate the menus using gestures, interact with the UI and have a text-to-speech system read out what’s on screen.

There will obviously be some manual setup involved, but a good plugin will make things work with minimal effort. Everything should work on Android and iOS, including Text-to-Speech.

First Steps – UI Navigation

Unfortunately, there seems to be no gold standard of UI navigation for the blind. Both Android and iOS have their own way of navigating through the screen. Sure, both use swipes, but that is almost where the similarities end. On Android, you can swipe down to jump to the next element on the screen. On iOS devices you swipe right. Swiping down will jump to the next UI container instead. On Android you can jump to the top of the screen by swiping up and then immediately down. On iOS you can read from the top of the screen by doing a two-finger upward swipe. The list goes on forever.

I can only assume that the respective users would like to keep using the gestures that they are familiar with – so a good plugin will have to support the appropriate navigation depending on the platform.  Heck, while I am at it, I might as well throw in listening to the keyboard and make everything work on PC as well.

Continue Reading with Part 2 here.

10 thoughts on “Unity Plugin for UI Accessiblity

  1. James Gale says:

    Hi there,

    It’s pretty late here so I haven’t read through all of your posts on your Unity Accessibility Plugin, but I did read that you were having some issues with Google Talk Back and single swipe gestures.

    I’ve been working in mobile games since 2005 (including native Android) and quite keen on making games accessible.

    If you would like a hand at tackling problems like that I would be happy to have a tinker and share my findings, let me know if I might be of some use 🙂

    Like

    • Thank you for the offer, James! I’d love to write you an email in a few weeks when I plan to tackle the Google TalkBack issues again. First I want to finish the current feature set, hence the delay. Thanks again!

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s