Unity Accessibility Plugin – Update 17 – Hide and Seek with TalkBack

One thing that continues to keep me on my toes is screen reader detection. And when I say screen reader, of course what I really mean is Android TalkBack.

Screen Reader Detection – and why it’s Important for your Sighted Users

My plugin needs to know whether a screen reader is running, so it can spring into action and turn itself on.
And this needs to work flawlessly. 

We recently released a Crafting Kingdom update that turned on accessibility mode by accident on some devices – and we got swamped with feedback like this:
My phone started muttering to me and none of the buttons work!“.
(Note: I edited out the many colorful expletives that embellished these complaints.)

Man looking in horror at his cellphone, which is telling him to double tap a button to activate it.

“Help! What’s going on? It speaks!”

See, if you’re sighted, using a screen reader is not something that comes naturally. In fact, it is almost counter-intuitive. I’ve watched several friends and relatives stab at their screen helplessly, and after about 10 seconds they had enough. Our actual users will probably have less patience.

In a best case scenario they’ll will simply uninstall the game. They might also leave a bad review. If they spent money they’ll likely want a refund.
In a nutshell, better to err on the side of caution and only activate the plugin when a screen reader has been detected for sure.

I worked through the weekend to find a fix, because the only other option was to disable accessibility completely.

Detecting TalkBack on Android – the Pain Continues

Every major cellphone brand has their own version of Android. Samsung even has their own version of TalkBack, Samsung Voice Assistant. Code that works on one device is not guaranteed to work on another. And every update to Android can potentially cause more trouble. This happened with the Android Oreo release.

As it turns out, that update is incompatible with my TalkBack detection code. I had three different mechanisms implemented, just to be on the safe side. After the update, one of them would crash, and the other two would always report it was turned off.

Woman asks TalkBack whether it is running in the background. TalkBack hesitates and then replies 'Maybe'

Is it me, or is TalkBack dodging me…?

I rewrote my code and tried what I thought was a clever trick: I checked the device’s settings to see if accessibility was enabled. It worked great on my pure clean Android Nexus test devices. But when I tested it on an LG phone, the settings would always claim TalkBack was running, even when it was off. I was pulling my hair out at this point.

Not all devices were affected, but some of the popular ones were – most Samsung Galaxy S devices for example. Samsung has a market share of around 35% in the US. Not a user base you want to ignore. In fact, my company just bought yet another Samsung phone as a test device because of this.

Here’s the Solution – and the Source Code

In the end, I solved the problem by asking the phone for all accessibility services that are both currently enabled and provide spoken feedback. If there were any at all, it’s fairly safe to say that a screen reader is on – or that the user is familiar enough with them to not freak out if the phone started talking.

That worked on all test devices that I could find lying around in the office. It also worked on ten different devices in the Firebase test lab. That’s a Google service that runs your app on a plethora of different devices for you and provides error reports and video recordings. (Hint: If you haven’t yet taken a look at the Pre-Launch report in your Google Play developer console, don’t ignore it any longer.)

I want to spare other developers the pain, because Googleing the issue will lead to a lot of out-of-date code that won’t produce the correct result.
So here’s a code snippet for ya (works as of March 4th, 2017):

import android.accessibilityservice.AccessibilityServiceInfo;
import android.view.accessibility.AccessibilityManager;

private static boolean isScreenReaderActive(Context context) 
{
  AccessibilityManager am = (AccessibilityManager) context.getSystemService(ACCESSIBILITY_SERVICE);
  List<AccessibilityServiceInfo> enabledScreenReaderServices = am.getEnabledAccessibilityServiceList(AccessibilityServiceInfo.FEEDBACK_SPOKEN);

  return !enabledScreenReaderServices.isEmpty();
}

Disclaimer

Whenever I complain about Android TalkBack I feel the urge to put a disclaimer at the end of my blog, to make sure I’m not mistaken for an Apple fan girl, who’s bashing on Android out of principle alone.
I’m not.

If you don’t believe me just come by the office on one of those days that I have to create new certificates and mobile provisioning profiles for our games on the Apple Store. See how much love I have for Apple then.

3 thoughts on “Unity Accessibility Plugin – Update 17 – Hide and Seek with TalkBack

  1. Not sure how to reach you, but I’ve been enjoying Blindie Match. I was simply wondering how difficult it would be to add braille support to that game? With a grid such as found in this game, it would easily be playable with a braille display. Especially if you use one letter code for each gem. Anyway, just thought I’d try to reach out and suggest it. Martin

    Like

    • Hi Martin,
      that sounds like a pretty great idea. It makes sense, with the gameplay being grid based anyhow.
      I don’t know exactly how difficult it would be to add. From what I have already research, it seems that it would be easier to add on iOS than on Android. I don’t own any Braille display that I could hook up to my computer or iOS device though, so for the time being, I am unable to implement this. But I like the idea a lot, so I put it on my TODO list, and I’ll keep occasionally checking on eBay for a compatible device.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s