Unity Accessibility Plugin – Update 8 – Supported UI Elements

Contract Work has been keeping me busy the past two months, so I haven’t found the time to post any updates on this blog until today.

Still Alive, just not Blogging!

The internet is short lived – and people ask if you gave up on a project the moment you aren’t blogging about it daily (and spend your time working on it instead for example).
So here is a quick check-in-slash-update-slash-still-alive ping!

The post will be short and possibly full of typos – I’m still knee deep in contracting work and hard pressed to divide my time equally between work, play, work on side projects and sleep. The latter is drawing the short straw almost every night.

Full Windows Screen Reader Support

I recently added proper Screen Reader support for Windows to my feature wishlist after reading this excellent article by Ian Hamilton: Screen Readers and Game Engines

Ian and I had a great chat on Twitter and thanks to his article I learned about Tolk. Tolk is a screen reader abstraction library.

See, on Windows there is more than just one screen reader – users get to choose which one they prefer. Or which one they can afford. JAWS for example has a price tag around $1000.

Instead of writing custom code to support every potential screen reader out there, software could simply include Tolk. This greatly reduces the amount of code one has to write. Tolk then does the actual interfacing with the various screen readers. It currently supports six different screen readers.

This is on my future wishlist for now and I will have to investigate it further. Most likely I will release the plugin only for mobile first and then add Windows support at a later time. And then I will have to look into screen reader support for MAC. (No Linux planned at this point.)

Most UI elements now supported

As for the core of the plugin, the basic functionality of the plugin is complete for mobile. That doesn’t mean the plugin is ready to be released, just that it is working and usable. While that sounds like almost the same thing, it really is not. There is lots of code cleanup and refactoring to be done.

 

A menu screen with a label, buttons, a dropdown list, a toggle, a slider, a text edit box and a scroll list.

A screenshot of my test user interface.

Here is a complete list of UI elements that are now supported:

  • Labels
  • Drop-down Lists
  • Buttons
  • Toggles
  • Sliders
  • Scroll Lists
  • Text Edit Boxes (Windows and iOS only at the moment)

The scroll lists actually turned out to be quite tricky and need a little more coding love to handle special use cases. But that might actually deserve a blog post all on its own.

That’s it for today.
Still alive, still eating cake and still working on this plugin!

Surprise: Blind users have established interface conventions, too!

Imagine the following scenario:
You want to delete a file from your computer. A dialog pops up on your screen and a text message asks: “Are you sure?”. Below the text are two rectangular shaped gray areas. One reads “OK” and the other “Cancel”.

An alert asking the user whether he really wants to delete the file. The options are "OK" and "Cancel".

Raise your hand if you know how to click the OK button.

Did you know that you need to move your mouse cursor over the rectangle that reads “OK”, – and then, while hovering over it, click the left button on your mouse exactly one time?

Yes? You did?

Well of course you did!

We all know the basics of navigating user interfaces. We know to left-click on buttons, and to right-click for context menus. We use the mouse wheel and expect a page to scroll. We instinctively search for the close button in the upper corner of a window.

Those are established user interface conventions – and they are a good thing.

As a developer, it means not having to explain the interface basics to your users. You can safely assume that players know how to activate the button labeled “Play” in your main menu. You can just focus on teaching your players the rules of your actual gameplay.

Surprise: Blind users have interface conventions, too!

It shouldn’t come as a big surprise that such interface navigation conventions exist for blind users as well.
It shouldn’t – but it seems that it does.

Often enough, when developers do go the extra mile to add blind accessibility to their games, they create their own interface navigation rules. One of their sighted programmers will spend an afternoon thinking about how it could work and then implement it. If he’s pressed for time he might not even look at how other games did it before. As a result, these games start out by reading a page of instructions to their users at the beginning. And that is about as exciting as reading a phone book.

Why break established conventions?

Not only is it boring to listen to instructions. Breaking conventions can also lead to losing users and turning them away from your app. Consider this: Every app that does this has their own navigation scheme. A blind user has to learn the quirks of each one and then try not to forget as they switch apps. Maybe they will remember the important basics, but if a game implements special gestures for things not used very often, let’s say for Pause for example, it will quickly be forgotten.

You don’t ask your sighted users to press Ctrl-K to click buttons, or right-click to scroll text, do you?
The same should be true for making your menus accessible. Don’t reinvent the wheel, don’t invent your own navigation scheme. Go with what’s already established.

Remember – in order to even start your app, the blind user will have to navigate to it and activate somehow. They know how to do that. They are already using an interface. If your app could be navigated the same way that the user navigated their operating system, you wouldn’t need instructions at all.

Not building on what users already know is creating an unnecessary entry barrier. And for blind users especially, more barriers is the last thing they need.

Made by Apple.

For mobile accessibility, the conventions have pretty much been established by Apple, because their VoiceOver screen reader is the gold standard. The majority of blind smart phone and tablet users are using Apple devices, and these devices are running VoiceOver.

The folks at Apple have done a great job with VoiceOver. Apple received an Access Award from the American Foundation for the Blind in 2009 and a Helen Keller Achievement Award in 2015, and the Bray Award for Accessibility Innovation in 2016 from the American Council of the Blind.
Whether you are an Apple fan or hater, following the really simple interface conventions from VoiceOver is a good place to start.

The freedom to be impatient

Whether you implement your own system or go with established conventions is ultimately up to you, of course. If there is one thing to be said about blind mobile users, it’s that they are incredibly patient and resilient. If your game is good, heck, even if it is just OK, they will probably be willing to jump through the hoops of learning how to navigate your menus either way.
For now.

A sighted user will uninstall a game within seconds if he finds the controls frustrating. There are enough other games out there – the Google Play store counted 2 million apps in February 2016 (Source: Statista.com). Blind users don’t have that option. It won’t surprise you to hear that the number of accessible games is extremely limited.

But that number is growing.
And here’s why:

There around 21 million blind or visually impaired people over the age of 18 in the US alone (Source: AFB). In my opinion it’s one of the few customer groups still left in the mobile market that aren’t yet over-saturated with games. In other words, there is money to be made. The challenge to create games specifically for the blind is also attractive to game designers. If it’s financially viable and artistically interesting… all that’s left is the technical barrier.

And the easier it gets to create accessible apps and games, the more developers will happily take up the task. I hope that my plugin will help with that as well. This is a good thing, because everybody wins. Blind people get more software and more developers can make a living doing what they love.

There will come a time when blind users have a choice. They will have the freedom that we sighted users already have: The freedom to be impatient.

And unless a game has outstanding gameplay – those apps with unusual and tricky interfaces will be the first to be ignored.

I’m guilty

Remember that time pressed programmer I mentioned, who implemented her own accessible menus without regards to existing conventions? Yeah, that was me. I created an elaborate interface system, based on long taps and circular menus. But I’ve learned my lesson earlier this year and deleted all that code. I found a forum with lots of blind gamers and asked them for their help and feedback, and got plenty. It’s when my idea to write a plugin for Unity was born. If you would like to see just how helpful this huge community is, here is the link to the original forum thread.

All apps made accessible with my plugin will automatically be working similar to VoiceOver. Like I said, I am not a fan of reinventing the wheel. You can read more about the plugin and how to make games for sighted and blind users for Windows, iOS and Android in Unity here: Accessibility Plugin for Unity

Further Reading

Unity Accessibility Plugin – Update 7 – Explore By Touch

As of today, the Unity Accessibility Plugin has a new feature: Explore By Touch!

Explore By What Now?

Explore By Touch is an accessibility feature on smartphones that let’s blind users run their finger over the screen – and the screen reader will read out what’s under their fingertip. The screen reader also makes sure that no buttons are accidentally pressed or an app is started unintentionally. This works on both Google TalkBack and iOS VoiceOver.

A blindfolded man is swinging a stick at a pinata. He misses and hits his girlfriend instead.

This was my original impression of Explore By Touch. Typical rookie mistake of a sighted person.

I originally thought that this is one feature that I wouldn’t need to re-create for the accessibility plugin, because I figured that no one would be using it much. I assumed no one wanted to blindly poke around on the screen to discover buttons. Why would anyone do that when swiping left and right navigates safely through the user interface?

Think again, you sighted fool!

As we all know, making blind assumptions is a bad thing. Pun intended.

Why would anyone use Explore By Touch?
The answer is simple. Because it is faster!

Navigating menus is a necessity, not fun. Doing it slowly by stepping through all elements on the screen one by one doesn’t make it any better.
If you already roughly know where a button is located, then using Explore By Touch is the next best thing to actually seeing where it is and clicking it directly.

Sure, the first time you get hit with a new menu, you might just trigger the Read From Top function and have the entire screen read out to you. But after that, you can poke around and find out where things are. And being blind doesn’t mean you cannot remember roughly where a button is located on the screen.

Thanks for the Feedback

I wouldn’t have come to this conclusion if I hadn’t received some very clear feedback from an actually blind person, who straight up told me the plugin would be pretty much useless without Explore By Touch. His opinion matched what I read in an article by Matt Gemmel in which he covers Myths about visually impaired users. After that I put my phone back in accessibility mode for a while and ultimately had to agree. It made me want that feature, too.

I am happy to announce that my plugin’s Explore By Touch is now fully functional.
Also: Thank you for your honest feedback, Scott!