Unity Accessibility Plugin – Update 6 – Text Editing

This week I extended the list of UI element types that plugin supports. The basic functionality of the Unity Accessibility Plugin is almost complete. I will give a complete overview of what is supported in my next post. Today, I want to write about text edit boxes.

Android is refusing to cooperate.

Android Logo standing with crossed arms, looking at me with crossed arms, telling me I can't do what I am trying.Android TalkBack or BackTalk?

Text Editing is complex

Text editing is a huge and complicated topic, with lots of functionality. Just think about everything you can do in a text editor, from moving the caret (insertion point) forward and backward, selecting, copying, deleting and pasting text, changing font size, color and so on.

There is no way I could re-create all of that functionality – which of course also works differently with iOS VoiceOver and Android TalkBack. But luckily I don’t have to.

The Keyboard is Native

On Windows, you can use an actual keyboard to type in the text. The plugin reads out what you typed to make it a little safer. On mobile devices, blind users will have to use the virtual onscreen keyboard.

Luckily, the onscreen keyboard on iOS and Android is a native overlay. When the edit field receives focus, the onscreen keyboard is brought up automatically. And it’s input is automatically directed into the edit field in Unity. On iOS, this was a breeze. If VoiceOver is running, it will spring into action and read out what the user types. On Android? Not so much.

Google TalkBack – Unsupported

Unlike VoiceOver, which is happy to stay in the background until needed, TalkBack will have to be disabled or suspended for my plugin to work. Which means it won’t spring into action automatically once the keyboard comes up. The user won’t even know that the keyboard is active.

This had me running around in circles for the past few days. I cannot access what key is currently under the finger on the onscreen keyboard, so I cannot even have the plugin read it out. There’s no way to access TalkBack directly to resume or suspend it. The plugin doesn’t receive any input from the touchscreen, because it is filtered out. Heck, the system doesn’t even let me detect whether TalkBack is suspended. I can tell when it is off completely, but not when it is on, but currently suspended.

Of course Android cannot allow apps to suspend TalkBack. Apple has a clear advantage since they control the hardward and the software. Android devices don’t have that physical Home button that all iOS devices have. There is no way a user could quit an app if it suspended TalkBack and then didn’t offer enough decent accessibility by itself. The device would be stuck.

The only solution I could come up with would be if TalkBack had a Limited mode. This mode would allow direct input through, but still listened for the magic gestures for Home and the global or local context menu. It would ensure that users could always quit an app but allow most apps to function fairly normally.

Compromise

Unfortunately there is no TalkBack Limited mode. There is a compromise however, though I don’t like it.

TalkBack will allow multi-finger gestures through to the apps, and pretend they’re one-finger gestures. So, technically, TalkBack wouldn’t have to be disabled for Unity. The user just would have to awkwardly use two fingers for all gestures (and three fingers for a double tap for some reason). I tried it, and it isn’t comfortable. But at least it would work.

A street sign pointing into the direction of "Good Enough"Do I really want to go down that road?

Currently I don’t really see a way to make edit boxes work on Android with TalkBack suspended. It’s either forcing the user to use two fingers for everything, or asking him to resume TalkBack for the edit field, then disable it again when done. That is a far cry from the user friendly experience I am aiming for.

But for the time being, I will postpone this for another day, so I don’t start pulling my hair out.

On a personal note

I know this isn’t the first post in which I focus on the issues with Google’s TalkBack. This wasn’t an intentional choice, it just so happened that I came across these issues and wrote about them. It doesn’t say anything about my general attitude towards Android. This is not supposed to be a rant, and I am certainly not an Apple fan girl just jumping at every opportunity to bash on their competition. I exchanged my personal iPhone for an Android phone a few years ago, and found that both operating systems have their advantages and disadvantages. All I care about in the scope of these posts is how easy or difficult it is for me to develop the features I want for them.

3 thoughts on “Unity Accessibility Plugin – Update 6 – Text Editing

  1. i was also making a plugin for voice over. Just a small doubt, are you passing the text to ios plugin so that ios handles the speech part? i was using a text to speech plugin to speak out the text?

    Is there a way we can pass the info to ios so that it handles all speaking as in native voice over?

    Like

    • Yes, iOS can handle the TTS output natively, so you don’t need a full blown TTS plugin with its own speech synthesis.
      The amount of code required is surprisingly small.

      I would recommend you take a look at this plugin on the asset store. It is only $5, and it includes full source code which will show you how to do that:
      Easy TTS (I am neither the creator of that plugin, nor affiliated with it in any way – it’s just a really good and really small plugin.)

      Like

      • Thanks for the reply

        I know about TTS system, but i was asking about voice over system. I was able to make a working plugin but the problem came when it was connected to Braille display (Focus blue). None of the tts text was channeled to the display. Native voice over apps are working perfectly.

        My plugin structure is like this, if button selected, the label text is passed to tts plugin to speak out.

        I was asking is there a way we can trigger the underlying voice over system itself.

        Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s