I've always looked to provide good accessibility support in my apps, but in my latest work, I've needed to build parts of the UI within a native OpenGL view, which means that there is no Android widgets or components on which one can hang a contentDescription or similar.
In theory, it shouldn't be too hard to implement "talkback" functionality regardless; just check whether talkback is switched on when the user taps a button and if it's the first tap, play the talkback text, and if a second consecutive, do what the button would normally do. But - somewhat to my surprise - there does not seem to be any APIs for this, or at least no documentation for them that I could find.
Anyone who has tried to work with the Android talkback functionality directly (or tried alternate solutions)? I'd hate to abandon those users who benefit from Talkback if there is a way to work around its seemingly limited implementation.
Related
There are apps like Texpand which are able to replace text in any EditText view - even of views which are part of other apps. Looking at the app-info this is happening without any requested permissions. I'm scratching my head how this is done - my (rookie) understanding is that each app resides in its own separated sandbox, so it should not have direct access to other apps views?
I looked for possible global events which could be provided by any central manager, but found nothing. More likely I would expect the replacement to be done passively (that means without the app being aware of the actual EditText), but checking for possible bindings or user dictionaries I found nothing promising either.
Looking at my Android system it seems the app is neither using permissions nor installing a keyboard. Additionally I don't see any entries in my user dictionary. Does anybody have an idea how the described functionality could actually be achieved?
Texpand's Google Play posting indicates that it uses Accessibility Services. Accessibility services are a set of APIs Android offers to help build tools to allow non-standard interactions with apps (such as audio descriptions/voice commands) to expand access to the platform to people with an impairment that might otherwise prevent them from using a touch-screen/smart-phone.
These include the ability to take action on the behalf of a user, such as filling in text fields.
I am trying to make my app more accessible. I am having a hard time finding helpful things because there isn't a lot of documentation (at least I could not find it).
In my app, Talkback does not announce the element type for ImageViews. What I basically want is for Talkback to announce my contentDescription for the ImageView and follow it up with ", Image".
This link states that " Many accessibility services, such as TalkBack and BrailleBack, automatically announce an element's type after announcing its label, so you shouldn't include element types in your labels. For example, "submit" is a good label for a Button object, but "submitButton" isn't a good label." but it does not specify which element types it announces and which it does not.
https://developer.android.com/guide/topics/ui/accessibility/apps.html
Does anyone have any idea if Talkback announces "Image" after the contentDescription for ImageViews?
When does Talkback announce a link as a "Link"? Or is it the developer's responsibility to add it at the end of the contentDescription? Can I make talkback announce clickable text as a "Link"?
Any help/information/pointers is greatly appreciated. Thanks in advance.
A: don't add stuff to the end of the content description. It is an accessibility violation and in almost ALL circumstances just makes things less acessible (will explain more later).
B: A lot of contextual things are communicated to TalkBack users via earcons (bips, beeps, etc), you may just not be noticing.
C: Yes, this is confusing and difficult to determine, no images are not announced, though I think this is for good reason. For example, an image with a click listener may just be a fancy styled button. In iOS there are traits for you to change for this, Android has omitted this highly useful feature, so we're stuck with odd workarounds. The ideal solution would be for the Accessibility APIs to allow the developer to communicate this information.
As for links, typically inline links in text views are announced (basically anything android detects and underlines automatically), but otherwise are not. So, in practice A LOT of links are missed.
Now, as for when you should/should not supply this information yourself. If unsure, just don't and you'll obtain a reasonably high level of accessibility by following the above guidelines. In fact, any of the considerations below are really just fighting the Android OS limitations, and are their problem! However, the Android Accessibility Ecosystem is very weak, and if you want to provide a higher level of accessibility, it is understandable, however, difficult! In your attempts you can actually end up working against yourself. Let me explain:
In Accessibility there is a line between providing information and consistency. By adding contextual information to a content description you are walking along this line. What if Google said "We're not going to share contextual information, add it yourself!".
You would have buttons in music players across different music playing apps that announce in TalkBack like this:
App1: "Play, Button"
App2: "Play, Actionable"
App3: "Play, Clickable"
Do we have consistency here? Now a final example!
App4: "Play, Double Tap to click if you're on TalkBack, Hit enter if you're a keyboard user, use your select key for SwitchAccess users....."
Notice how complicated App4's Play Button is, this is illustrating that the information that TalkBack is using is NOT JUST FOR TALKBACK. The accessibility information from you app is consumed by a multitude of Accessibility services. When you "Hack" contextual information onto a content description, sure it might sound better for a TalkBack user, but what have you done to Braille Back users? To SwitchAccess users? In general, the content description of an element should describe the element, and leave contextual information for TalkBack to calculate/users to figure out given proximity to other controls.
TO ANSWER YOUR TWO PARTICULAR ISSUES (Images and Links):
What I recommend for images is in the content description make it obvious that the thing your describing is an image.
Let's say you have a picture of a kitten.
BAD: A kitten, Image
Good: A picture of a kitten.
See here, if TalkBack fails to announce this as an image (which it will) the user still gets the idea that it is a picture. You have added contextual information to the content description in a way that has ACTUALLY BETTER DESCRIBED THE CONTROL. This is actually the most accessible solution! Go figure.
Links:
For links this gets a bit tricky. There is a great debate in Accessibility about what constitutes a link vs a button. In the web browser world, this is a good debate. However, in native mobile I think we have a very clear definition:
Any button that when activated takes you away from your current Application Context, is a link.
The fact that the Context (Capital C Context!!!) is going to change significantly when a user interacts with a control is EXCEPTIONALLY important information.
TalkBack fails to recognize links A LOT, and as such, for this very important piece of information, if you find that TalkBack is not sharing this information, go ahead and append ", link" to your content description for this element. THIS IS THE ONLY EXCEPTION TO THIS RULE I HAVE FOUND, but believe it is a good exception. Reason being, YES it does add a violation or two for other Assistive Technologies, but it conveys important enough information to justify doing so. YOU CANNOT create a WCAG 2.0 compliant application of reasonable complex User Interface using the Android Accessibility APIs. They have too many limitations, you simply can't do everything you need to do to accomplish this without "hacks". So, we have to make judgement calls sometimes.
I am implementing a viewpager which has multiple hints to a particular functionality. Now when I activate talkback feature it reads the content in the current view. I also want the talkback to say as "swipe left for more hints" how can I do that.
Easy answer: Don't. This may be a sensible thing to say if the user is using TalkBack Gestures. But what if they're using an attached USB keyboard in conjunction with TalkBack? Is this really the ONLY way to reach those items. Probably not. And if it is, you're app may work great for TalkBack gestures, but is broken for switch access and keyboard access. TalkBack users know how to use TalkBack, switch users know how to use switches, etc. Use good, common design patterns in your layouts, and let users figure things out for themselves.
Let's explain this in a different way. Let's say you have a button. You have marked this thing up so it looks like a button. Do you have a big sign next to your button that says "HEY, CLICK YOUR MOUSE HERE TO ACTIVATE THIS BUTTON"... no of course not. Users know how to use buttons. Use reasonable design idioms, and users should understand how your UI works, independent of whether they are using TalkBack, Switches, Keyboards, etc. It is misleading, inappropriate, and actually LESS accessible to include TalkBack gesture specific instructions on how to perform actions.
See WCag 2.0 Guideline 4.1
I'm developing an Android webapp (an app built with Android Studio mainly based on a single webview which loads remote content) that is meant to be used by visually impaired people.
Standard accessibility features, such as those provided by Android itself and Talkback, are completely useless for my use case, because this app is meant to be used in a completely non standard way, by using the headset button as the only input, and spoken text as the output.
The webapp is NOT meant to be used by people who are not visually impaired alike. We already have a completely different version of the webapp for that. However, I do want the spoken texts to be displayed on screen, mainly for testing/debugging purposes.
So, I don't need nor want to take advantage of Google Talkback, and my webapp, built from scratch using native TTS called from JavaScript (and TTS events, such as ending of utterances, triggering JavaScript code), already works fine when TalkBack is disabled.
However, my app is going to be used mostly on phones whose users DO have accessibility and TalkBack enabled (or perhaps other accessibility extensions - hopefully not, given the nearly nonexistent standardization of the framework for accessibility features).
So I need to prevent it from interfering with my own calls to the TTS engine.
Which is complicated as hell.
Also, I need it to be compatible with Android 2.x through 4.x.
On older Android versions, the problem is relatively small, as TalkBack doesn't inject scripts into the webview, and the webview is almost opaque, its contents being "invisible" to Talkback.
On newer versions, it's a nightmare.
Android doesn't provide a way to simply disable accessibility features from the app, which is ridiculous.
The AccessibilityManager.interrupt() method doesn't seem to do the trick, and it's practically undocumented, so I can hardly figure out in which ways I could try to use it.
There doesn't seem to be any documentation about how the injected scripts work and how I could try to interact with them.
And to screw things further, I'm reading that since 4.4.something, the scripts formerly injected by TalkBack are now replaced by something else, integrated into the very web engine, or something like that. So maybe even any hack I can figure out for TalkBack injected scripts may not work with the most recent versions.
Is there some workaround/hack to completely disable TalkBack from my app, for my app, when my app is running?
Is there a way to prevent the injection of javascript code from TalkBack (when the script injection is enabled systemwise)? (obviously without disabling javascript altogether in my webview, since I do use my own javascript)
Is there at least some documentation of those scripts available? Or their code? So that I could at least try to "fool" them?
Is there a way I can make some parts of the webview's content "invisible" to TalkBack?
Or any other suggestion in order to be able to have my app do its own call to TTS while avoiding Talkback to interfere with them, and render text on screen while preventing TalkBack from interacting with it?
I've been working with TalkBack and Android for about a month now and find the same frustrations you have - little to no documentation other than reading thousands of lines of source code. In 4.4 the WebView component changed to use the Chromium engine, which is from what I can tell the reason the scripts injected aren't exactly the same. Both still seem to be using ChromeVox. There are no public APIs for disabling TalkBack, it can be done with reflection if you're included to deal with the headaches that come with that. You're best bet is to disable ChromeVox (it's injected into your WebView automatically on 4.4+ and on lower 4.x assuming the web accessibility setting is enabled) from the JS side.
I can only test on 4.4 now, but this does the trick for me:
cvox.ChromeVox.host.activateOrDeactivateChromeVox(false);
https://code.google.com/p/google-axs-chrome/source/browse/trunk/chromevox/chromevox/injected/init_document.js?r=193
I went through some posts, but most of them discuss placing a image or a string with information on it.
I however need to place button that manipulates some call features.
Is there a method to do that universally works throughout all android distributions?
(I thought to pop up custom screen with my button above usual screen. Is that good idea, or is there more straightforward way to achieve this?)
Thanks a lot
On stock devices there isn't going to be a legit way to put your own button "on" any portion of the Dialer application Activities (including incoming call screen). This kind of functionality would require the Dialer to explicitly provide an API for it. The stock system prevents applications in the background from placing their own clickable regions "on top of" whatever is currently in the foreground . (It's a good thing too, the bad reasons to do this far outweigh the legit ones.)
If you are wanting to do this you're going to have to look into building your own version of the OS that allows for it. OR potentially if you unlock your device enough to be able to install your own application that handles all of the functionality of the Dialer. Then you could provide an API for a third party application to do it(or just "bake in" your button to the Dialer) But I imagine that being able to get it all hooked up correctly to actually make your calls would be mighty difficult.