Unity Accessibility Plugin – Update 8 – Supported UI Elements

Contract Work has been keeping me busy the past two months, so I haven’t found the time to post any updates on this blog until today.

Still Alive, just not Blogging!

The internet is short lived – and people ask if you gave up on a project the moment you aren’t blogging about it daily (and spend your time working on it instead for example).
So here is a quick check-in-slash-update-slash-still-alive ping!

The post will be short and possibly full of typos – I’m still knee deep in contracting work and hard pressed to divide my time equally between work, play, work on side projects and sleep. The latter is drawing the short straw almost every night.

Full Windows Screen Reader Support

I recently added proper Screen Reader support for Windows to my feature wishlist after reading this excellent article by Ian Hamilton: Screen Readers and Game Engines

Ian and I had a great chat on Twitter and thanks to his article I learned about Tolk. Tolk is a screen reader abstraction library.

See, on Windows there is more than just one screen reader – users get to choose which one they prefer. Or which one they can afford. JAWS for example has a price tag around $1000.

Instead of writing custom code to support every potential screen reader out there, software could simply include Tolk. This greatly reduces the amount of code one has to write. Tolk then does the actual interfacing with the various screen readers. It currently supports six different screen readers.

This is on my future wishlist for now and I will have to investigate it further. Most likely I will release the plugin only for mobile first and then add Windows support at a later time. And then I will have to look into screen reader support for MAC. (No Linux planned at this point.)

Most UI elements now supported

As for the core of the plugin, the basic functionality of the plugin is complete for mobile. That doesn’t mean the plugin is ready to be released, just that it is working and usable. While that sounds like almost the same thing, it really is not. There is lots of code cleanup and refactoring to be done.

 

A menu screen with a label, buttons, a dropdown list, a toggle, a slider, a text edit box and a scroll list.

A screenshot of my test user interface.

Here is a complete list of UI elements that are now supported:

  • Labels
  • Drop-down Lists
  • Buttons
  • Toggles
  • Sliders
  • Scroll Lists
  • Text Edit Boxes (Windows and iOS only at the moment)

The scroll lists actually turned out to be quite tricky and need a little more coding love to handle special use cases. But that might actually deserve a blog post all on its own.

That’s it for today.
Still alive, still eating cake and still working on this plugin!

Unity Accessibility Plugin – Update 7 – Explore By Touch

As of today, the Unity Accessibility Plugin has a new feature: Explore By Touch!

Explore By What Now?

Explore By Touch is an accessibility feature on smartphones that let’s blind users run their finger over the screen – and the screen reader will read out what’s under their fingertip. The screen reader also makes sure that no buttons are accidentally pressed or an app is started unintentionally. This works on both Google TalkBack and iOS VoiceOver.

A blindfolded man is swinging a stick at a pinata. He misses and hits his girlfriend instead.

This was my original impression of Explore By Touch. Typical rookie mistake of a sighted person.

I originally thought that this is one feature that I wouldn’t need to re-create for the accessibility plugin, because I figured that no one would be using it much. I assumed no one wanted to blindly poke around on the screen to discover buttons. Why would anyone do that when swiping left and right navigates safely through the user interface?

Think again, you sighted fool!

As we all know, making blind assumptions is a bad thing. Pun intended.

Why would anyone use Explore By Touch?
The answer is simple. Because it is faster!

Navigating menus is a necessity, not fun. Doing it slowly by stepping through all elements on the screen one by one doesn’t make it any better.
If you already roughly know where a button is located, then using Explore By Touch is the next best thing to actually seeing where it is and clicking it directly.

Sure, the first time you get hit with a new menu, you might just trigger the Read From Top function and have the entire screen read out to you. But after that, you can poke around and find out where things are. And being blind doesn’t mean you cannot remember roughly where a button is located on the screen.

Thanks for the Feedback

I wouldn’t have come to this conclusion if I hadn’t received some very clear feedback from an actually blind person, who straight up told me the plugin would be pretty much useless without Explore By Touch. His opinion matched what I read in an article by Matt Gemmel in which he covers Myths about visually impaired users. After that I put my phone back in accessibility mode for a while and ultimately had to agree. It made me want that feature, too.

I am happy to announce that my plugin’s Explore By Touch is now fully functional.
Also: Thank you for your honest feedback, Scott!

3D Audio with Stereo Headphones

Today I want to write about an audio experiment I did for a game that I’m developing using my Unity Accessibility Plugin. It’s an accessible cooking game in which you prepare food for customers.

Problem 

The game pans the voices of customers to the left or right, depending on where they are standing. But when playing the game blind, I find it hard to quickly tell where a customer is. It always takes me a second and it requires a little concentration. This might not be a show-stopper, but it keeps bugging me.

Spatial Audio could be the solution.

Woman with headphones on has her eyes closed and focuses on the whether the sounds are left, right or center.

One burger to the left, two hot dogs to the right…

3D Vision and 3D Hearing

Never heard of spatial audio? Don’t worry, I won’t delve into the details. In a nutshell, it is a clever way of playing back audio so that the listener has a 3D audio effect using only stereo headphones. It’s not even that complicated. Humans only have two ears, so stereo is technically all we need.

Just like your brain uses the differences in the images from your two eyes and gives you 3D vision, it uses the differences in sound between your two ears and gives you 3D hearing.

The picture shows sound sources that are placed at different locations around a head.

Even with only two ears, you can always tell where a sound is coming from. Image Source

Here Is The General Idea

Because sound waves travel through the air (at the speed of sound), a sound coming from your left would reach your left ear first, and then your right ear. Furthermore, your head is in the way of the sound and will block part of it. This is called the head shadow, which dampens some frequencies from the sound before it reaches your other ear. While the differences are very small, your brain picks up on them and uses this very subtle information to place the sound in the world around you. There’s more playing into it, like sound reflected off walls close to you etc, but I said I wouldn’t delve into the science too far.

If you haven’t yet, listen to the famous barber shop example (skip to 2:54 to get the idea):

 

Famous Games using Spatial Audio

A game can reproduce this 3D effect artificially, by delaying the playback on the far ear and filtering frequencies. The only requirement for it to work is that the player wears headphones.

Games for visually impaired or blind players sometimes use spatial audio to allow players to find their way in the dark. Famous examples are the Papa Sangre games and The Nightjar. If you want to check them out, get The Nightjar if you like Benedict Cumberbatch, or Papa Sangre II if you are a fan of Sean Bean (no need to play Papa Sange I first).

Actors Benedict Cumberbatch and Sean Bean inside a voice recording studio. They look cute.

The Nightjar and Papa Sangre II have pretty darn good voice acting in my totally biased opinion.

Unity and Spatial Audio

Somethin’ Else, the developers of Papa Sange and Nightjar, have written their own audio engine and their results are impressive. I wanted to give spatial audio a try in my Sandbox dev environment, before bringing it into my game. But I am fan of not reinventing the wheel. I didn’t want to implement my own spatial audio system when there are some out there already.

There are plugins for spatial audio available for Unity. Here are the three that I found in the Asset Store: Spatial Audio, AstoundSound and dearVR. The last one looked the best, but the first one was the cheapest. For just a quick test that would do. But dearVR is now on my WishList and I will be hoping for a sale in the future.

Direct Comparison

Which one works better?
The majority of people should be able to tell the direction of the voice much faster when listening to the spatial audio example.
But have a listen for yourself.

Put on your headphones and try to determine whether the voice is coming from the left, the right or from the front. How fast are you able to tell the direction? Do you need to focus on it?

Tip: After you’ve listened to the spatial audio example, listen to the stereo one for a second time. It seems a lot worse now, doesn’t it?

I know the spatial audio sounds like it is panned more to the side than the stereo one, but that is not the case – it’s just the effect that the brain creates from the sounds.

EDIT: I just learned that the SoundCloud player above doesn’t work with all screen readers. Here are alternative links to the audio files:
Stereo Results
Spatial Results

Conclusion

While the stereo panning works “good enough” for the simplistic setting of my cooking game, it seems to me that the spatial audio requires less effort to determine where the sound is coming from. For that reason, I will put in the work and switch over to the spatial audio library. The difference might not be huge, but I am hoping it will make the gameplay feel more intuitive, because blind players need to pay less attention to locating the customer. After all, the gameplay is supposed to be about cooking, not about telling direction.

Notes for those interested in the topic:

  • If you want to play around with spatial audio in Unity but are unable to spend money on plugins, good news: Unity has just released a demo Spatialization SDK. You can find it here. I gave it a quick test and it seems to be working fine. Hint: If you check out the demos, you need to select the HRTF demo.
  • If you want to read more about the topic of spatial audio, but are bummed out by all the “binaural beats for better sleep” nonsense search results, try searching for ‘HRTF’ instead.
  • The developer Somethin’ Else used to license their audio engine (Papa Engine) to other developers. While this is no longer the case, their website hints that there might be a Unity compatible version in the future (see here). There is also a forum thread that claims the engine might become open source entirely. You can read about it here.

 

Dev Update #14 – Concept Graphics and Aspect Ratios

Have your base (classes) covered!

In spite of what I wrote in my previous blog post, I realized I need to create one more department before I can finally tackle the Build Menu: The Management department. Like the Design department, this one works differently from the production floors. This makes a difference from the coding point of view – as I want a base class to take care of all floors, and then handle only specialized stuff in derived classes. And like I said, I want all my bases covered.

Here is the mockup graphic for the Management floor:
Management

The idea was that a Management floor can employ up to four development directors. Each director would allow for one game to be in production. This way the player can work on multiple games at the same time. But I made a pretty noobish mistake when creating the above mockup – I hadn’t given any thought to the aspect ratio of my floors.

Aspect Ratio, anyone?

When creating the actual game graphics for this floor based on this mockup, I ran into some trouble. I hadn’t really thought about aspect ratios and screen width when I created those images. That was a big mistake, as it turns out. The mockup screens were too wide. There was simply no room left on the right side of the screen for the two big purple buttons, not unless I scaled the height down.  But then everything else in the floor would become tiny. To counteract, I would have to scale up the characters. But then I didn’t have enough room above the head of the managers to put the actual game icons. My characters were simply too big.

So I redesigned that department completely. Here’s the result (middle floor):

BlogPost15

Instead of placing the icons above the characters’ heads, they were placed next to them, sorta like a PowerPoint screen in the background. It actually gives it a nice manager look and feel. And as a bonus, this will free up some screen space at the top of each floor where I have been wanting to put a floor number. In another change, the Release and Trash buttons on the right are done away with. Instead, scripts can be released or trashed by clicking on them and opening up their details dialog.

I definitely learned my lesson and will create future mockups in a sensible aspect ratio.

Now, this time for real – next up is going to be the build menu.

Dev Update #13 – Code and Design Departments

It should have come as no surprise to me that spending my free time working on a game directly clashes with writing blog posts in that same free time. And one of these things simply comes more natural to me than the other. So it’s been five months… Time for an update.

I really want to start working on the build menu, but before I can do that, I need a few floors that I can actually build. So I created basic prefabs for the Code and Design departments.

Here are the results.
BlogPost12The Code department layout (top) represents the basic layout for all the basic production floors. 3D Art, Animation, Sound etc… they all will share the same basic setup with three queue slots. The Design department (bottom) works differently and needed a special setup. So it made sense to create these first and cover all my bases.

In my original game design document, the workers would all sit with their backs towards the player. This was mainly due to my poor drawing skills. But for the actual game I found this to be too boring. So instead, I turned them all around, so that they would be facing the camera.

The floors all use the same background graphic for the floor, which is actually grayscale. The graphic is tinted in realtime with the color of the floor.

Please keep in mind that the floors are not really finished yet. The coder for example doesn’t even yet have a computer in front of her. But it’s good enough to work with. Now I can start creating the menu that let’s me build floors into my game studio tower.

Dev Update #12 – Game Name Generator

I like to get all unpleasant work out of the way first – but I allowed myself a little fun task this week: A Game Name Generator!

blog_gamenamesFor my game Game Tower I need a bit of code that randomly generates game names for the virtual games the player creates.

This was probably the most fun task of all the work so far!

Pattern Matching

I went for a simple pattern matching system. The generator picks a pattern at random from a long list and then replaces the placeholder wildcards with random words from others lists.
Here are a few examples of such patterns:

<name> and the <noun>
<noun> Tycoon

The generator would take such a pattern and replace the wildcards with words from a list of pre-approved words. The wildcard determines which words match – there a nouns, verbs, names and plural forms of nouns. When all wildcards are replaced it spits out the finished game name, for example:

Jesse and the Unicorn
Waldo and the Zombie
My Mom and the Warlock
Zombie Tycoon

It’s soo much fun!

I added the name generator to my debug menu (see Dev Update #9) so I can test it out and generate new game names when I feel like it. It works great and I find myself adding new words or patterns to my database every few days, because it is simply that much fun.

GameNameGenerator

Good ideas come … while in bed.

All patterns and words are kept inside my trusted Google Spreadsheet. I have a macro that can convert them into a string list quickly, so I can get it into my code fast. I can access the spreadsheet from anywhere – and this has already paid off.

Like all coders (I assume) I have a lot of good ideas in bed. When I have a fun idea for a new pattern or a noun to add to the list. I can just reach for my phone and punch it in.

More updates and screenshots from the game in progress coming up soon!

Dev Update #11 – Tip of the Day instead of a Tutorial

I would like to see if it’s possible to make Game Tower a game that can be played without a needing tutorial.

It’s not that I would shy away from implementing one. But when I pick up a game, I would like to start actually playing it right away. So this is an experiment I will try. There’s two plans of attack I have for this: Giving the player more features only over time and a Tip Of The Day.

Tip of the Day

I like loading screen-tips in games, because I can easily ignore them when I don’t want them. A tutorial forces me to click at a sequence of buttons in a specific order and is much more restrictive. That is fine for other games, but feels like overkill for such a simple game like Game Tower.

BlogPost11More coder art for your enjoyment! Ok, I know that this is bad, even by my standards. But my plan is to get the entire game done and working. Then I can see to it that the graphics are replaced by someone with actual talent.

So I created this quick’n’dirty Splash Screen and a Tip of the Day database. A random tip is displayed every time the game is started. The game will load the player’s tower in the background while the screen is displayed. There is a minimum display time of 4 seconds, to give the user a chance to read it even if the loading is done faster.

Easy to Learn, Difficult to Master

Probably everybody has heard of Bushnell’s law, that good games are easy to learn and difficult to master. The actual quote is a bit longer and also mentions rewarding the player (here is an interesting blog post about it). But I’ll stick with the short-hand version for now.

Applying that to my no-tutorial experiment, it should be possible to make the game easy enough in the beginning that there is no need for a tutorial before the player can start actually playing. I will have to do some more trial-and-error testing, but my current plan is to provide the player with three basic floors instead of an empty tower at game start. That way he can immediately jump in and start producing games, rather than worrying about building the correct floors, hiring people etc. Essentially I want the player to learn the core gameplay first – and then get into the finer details as he levels up.

The level-up design should come in really supportive here as well. Since the player unlocks more departments and larger game designs by leveling up, the complexity of the game already starts out small and grows larger over time.

If at first you don’t succeed…

…make sure you don’t blindly ignore if something is just not working.

Making Game Tower playable without a tutorial is something I want to try. But I won’t stick to it like it’s set in stone. If in the end I discover that the absence of a tutorial makes the game to confusing, I will mark the experiment as failed and provide a tutorial. After all, the game is meant to be played.

P.S.: I am also playing around with different names. That’s why that screenshot reads “Game Studio Manager” instead of “Game Tower”.