This week I extended the list of UI element types that plugin supports. The basic functionality of the Unity Accessibility Plugin is almost complete. I will give a complete overview of what is supported in my next post. Today, I want to write about text edit boxes.
Android is refusing to cooperate.
Text Editing is complex
Text editing is a huge and complicated topic, with lots of functionality. Just think about everything you can do in a text editor, from moving the caret (insertion point) forward and backward, selecting, copying, deleting and pasting text, changing font size, color and so on.
There is no way I could re-create all of that functionality – which of course also works differently with iOS VoiceOver and Android TalkBack. But luckily I don’t have to.
The Keyboard is Native
On Windows, you can use an actual keyboard to type in the text. The plugin reads out what you typed to make it a little safer. On mobile devices, blind users will have to use the virtual onscreen keyboard.
Luckily, the onscreen keyboard on iOS and Android is a native overlay. When the edit field receives focus, the onscreen keyboard is brought up automatically. And it’s input is automatically directed into the edit field in Unity. On iOS, this was a breeze. If VoiceOver is running, it will spring into action and read out what the user types. On Android? Not so much.
Google TalkBack – Unsupported
Unlike VoiceOver, which is happy to stay in the background until needed, TalkBack will have to be disabled or suspended for my plugin to work. Which means it won’t spring into action automatically once the keyboard comes up. The user won’t even know that the keyboard is active.
This had me running around in circles for the past few days. I cannot access what key is currently under the finger on the onscreen keyboard, so I cannot even have the plugin read it out. There’s no way to access TalkBack directly to resume or suspend it. The plugin doesn’t receive any input from the touchscreen, because it is filtered out. Heck, the system doesn’t even let me detect whether TalkBack is suspended. I can tell when it is off completely, but not when it is on, but currently suspended.
Of course Android cannot allow apps to suspend TalkBack. Apple has a clear advantage since they control the hardward and the software. Android devices don’t have that physical Home button that all iOS devices have. There is no way a user could quit an app if it suspended TalkBack and then didn’t offer enough decent accessibility by itself. The device would be stuck.
The only solution I could come up with would be if TalkBack had a Limited mode. This mode would allow direct input through, but still listened for the magic gestures for Home and the global or local context menu. It would ensure that users could always quit an app but allow most apps to function fairly normally.
Unfortunately there is no TalkBack Limited mode. There is a compromise however, though I don’t like it.
TalkBack will allow multi-finger gestures through to the apps, and pretend they’re one-finger gestures. So, technically, TalkBack wouldn’t have to be disabled for Unity. The user just would have to awkwardly use two fingers for all gestures (and three fingers for a double tap for some reason). I tried it, and it isn’t comfortable. But at least it would work.
Currently I don’t really see a way to make edit boxes work on Android with TalkBack suspended. It’s either forcing the user to use two fingers for everything, or asking him to resume TalkBack for the edit field, then disable it again when done. That is a far cry from the user friendly experience I am aiming for.
But for the time being, I will postpone this for another day, so I don’t start pulling my hair out.
On a personal note
I know this isn’t the first post in which I focus on the issues with Google’s TalkBack. This wasn’t an intentional choice, it just so happened that I came across these issues and wrote about them. It doesn’t say anything about my general attitude towards Android. This is not supposed to be a rant, and I am certainly not an Apple fan girl just jumping at every opportunity to bash on their competition. I exchanged my personal iPhone for an Android phone a few years ago, and found that both operating systems have their advantages and disadvantages. All I care about in the scope of these posts is how easy or difficult it is for me to develop the features I want for them.