Using Android 4.2.2
I'm trying to write an AccessibilityService, and have most of the required features. I'm drawing on an overlay and allowing the user to select/cligk highlighted items via a bluetooth switch (the purpose is a disabled client wanting to interact with an android device using only one switch).
Whilst parsing a screen, I can get the root accessibiltyNodeInfo object, and all its children. I can highlight on the screen all such elements, and click a desired one by the .performAction() method.
On the home screen, there are 3 "panes" available, with the middle one being shown. Swipe left or right to see the others (standard launcher behaviour). There is a node that reports isScrollable = true, but the Action Flags do not report ACTION_SCROLL_FORWARD or ACTION_SCROLL_BACKWARDS. How do I scroll such a node, if I cannot call .performAction() on it as it does not support scrolling? Why does it report isScrollable = true if its not somehow scrollable?
Any help appreciated - thanks.
You cannot scroll a node that cannot perform ACTION_SCROLL_FORWARD or ACTION_SCROLL_BACKWARDS.
The Accessibility framework on Android is experimental at best and is plagued with inconsistencies such as the one you mentioned. In general, it is best not to rely on any of the is[Property]() methods. Instead, you should test for the property you are interested in yourself, after calling getActions() or getActionsList() on the node.
Related
short version: navigating into a RecyclerView with TalkBack gestures forces you to swipe through every item before you can move on, is that normal? Is there a standard way to navigate out and onto the next element?
I'm trying to get accessibility features working smoothly in an app I'm working on, but I've run into a problem with part of the UI. I'm using RecyclerViews sized so you can only see one item at a time, snapping to the centre, so you can swipe to change the current value from a range. It's basically the NumberPicker approach but directly in the UI, instead of a separate dialog.
The issue is by doing swipe gestures (Default linear navigation) through the UI, as soon as it lands on the RecyclerView, it starts on the current item and then navigates through every single item in the list. It won't exit the RecyclerView until it's hit the last item and has nowhere else to go.
My question is, is this normal? I'm new to TalkBack and I can't find much info on what's expected in every situation. I know my situation is a little unusual (since navigating the RecyclerView implicitly means changing the current item) but just being trapped inside the list until you go through every item seems a really strange way to navigate in general.
I know there are solutions like adding navigation headings etc (which is how the Play Store seems to handle this with their "infinite" horizontally scrolling app selections) but I just wanted to check I wasn't missing something. TalkBack announces "in list" and "out of list" when you tap on/outside of the RecyclerView, so it seems like there should be a way to explicitly navigate in and out with a gesture too...? Would users be used to switching to Controls or Headings navigations to do this?
Thanks! I really want to make this work for everyone but it's hard to know if I've done something wrong, or if I'm just bumping into limitations with the standard components
In my Android app I have a custom layout that is being used as a button - it consists of some TextViews and an ImageView, additionally it has some gradient background.
I'm aligning my app now to conform to the Accessibility rules. In order to do so, I would need to convert this layout into a button, so that TalkBack can correctly indicate the action, that this whole layout is clickable and serves like a button.
I know that on iOS there is a possibility to set the UIAccessibilityTraits to treat such view as a button - this kind of solution would save me a huge amount of work in terms of migration.
Is there any similar solution on Android for that? What approach should I follow in order to make this layout recognized correctly by TalkBack?
No, there's no concept of accessibility traits on Android - but you can still get a good accessibility experience without needing to specifically convert your layout into a Button.
Generally, it's most important that TalkBack (or whatever accessibility service is being used - remember, it's not just TalkBack) is able to detect that the widget is clickable and to be able to read a coherent description of what it does. The additional information that it's a button, specifically, isn't super useful, especially because there are so many different kinds of UI elements that it's often a very ambiguous question whether something even is a button.
You can test this by selecting it in TalkBack and confirming that it reads the content description properly, says something along the lines of "Double tap to activate," and performs the correct action when you double tap.
If it's not correct, make sure the content description, clickable flag, and click action are set correctly on the widget's AccessibilityNodeInfo.
I am creating a script that explore different android applications. If the script finds a scrollable component during the exploration this one must be scrolled. The problem is that sometimes the scroll direction can be horizontal or vertical and I would like to know it before perform the scrolling action.
Given an AccessibilityNodeInfo of a scrollable component (e. g. RecyclerView), is it possible to know in which directions the RecyclerView is scrollable (vertical or horizontal)?
The presence of ACTION_SCROLL_UP/DOWN/LEFT/RIGHT is the API that gives you this information. Unfortunately, the original API of ACTION_SCROLL_FORWARD/BACKWARD is very common and doesn't tell you which direction. So, like pretty much everything in the accessibility API, it depends on the app and widget sharing the information with you.
I currently retrieve the root node of the active window with getRootInActiveWindow(). Afterwards, I perform a breadth first search to get a list of all nodes.
My questions:
How can I traverse this list of nodes to get the focus order? Is this list ordered according to the nodes' focus order already? Is there a different approach of retrieving the focus order?
Thanks in advance.
Ultimately, no. You cannot, and the accessibility api documentation on this is very misleading.
One might think that the "focusSearch(int direction)" function of AccessibilityNodeInfo would be the way to go. But, ultimately it is broken, and if you go back into the inards of the Android Accessibility APIs you end up finding a function that's documented as such:
Searches for the nearest view in the specified direction that can take the input focus.
COOL, just what we want right? HOLD ON A SECOND. This function calls a function
AccessibilityInteractionClient.getInstance().focusSearch(mConnectionId, mWindowId, mSourceNodeId, direction);
And this function is documented as such:
Finds the accessibility focused {#link android.view.accessibility.AccessibilityNodeInfo}. The search is performed in the window whose id is specified and starts from the node whose accessibility id is specified.
Dissappointingly, you will also find that this function does neither of those jobs (directional focus search, or directional accessibility focus search) well.
Ultimately, when an element in TalkBack gains accessibility focus due to the focus of an element, it's actually responding to the actual focus event coming from the OS and NOT any calculation that TalkBack is doing on input focus ordering.
In order to do this, you would basically have to implement the entire logic from the OS for Tab Ordering yourself, with less information the system itself has to do it, because the System has real android.widget.View objects to deal with, not fake AccessibilityNodeInfo objects with limited information. Any way you can come up to do this is going to be exceptionally fragile, and not guaranteed to line up with what the system actually does.
Your inclination to try a breadth first traversal of "input focusable" (the focusable property of AccessibilityNodeInfo) is a pretty reasonable solution. The problem is going to be, that the real system Tab Ordering is going to be a subset of that, which ignores things like elements existing on different windows and such.
Seriously, open up the TalkBack settings screen, turn on TalkBack, connect a hardware keyboard, go crazy on the tab and shift tab keys, and marvel at how you can't put focus on the "Settings" button. Your traversal function would get this wrong, because the "Settings" button is focusable but somehow, you can't put focus there. The reason why digs into some serious AOSP nonsense that I will omit, and no, I'm not sure if the information that would keep this corner case from coming up in your theoretical algorithm is available to AccessibilityServices.
To summarize, the breadth first traversal can theoretically work, but there are going to be a ton of corner cases, and no I'm not sure if the information to deal with each corner case correctly is available to AccessibilityServices. Also, notably, because of this particular issue, if you'd like the solution to be correct across 4.4, 5.0 and 7.0+ devices, you will have to have different variations of this calculation for different OS versions. YEP, those corner cases are going to be different! (Queue Evil Laughter).
My honest recommendation... give up.
I have an Android Recyclerview which has some more rows of item.
In the sense
Recyclerview comprises of
Row 1 ->> TextView , below that one more textview
Row 2 ->> TextView , below that one more textview
Issue is that, whenever I turn on the Talkback, it reads out the entire Recyclerview in one go, which is not expected, it should read one item at a time depending on the focussed item.
Expected behavior is - Read component on Focus when d-pad is moved onto it.
Any help??
Contrary to ChrisCM's answer, you should NOT set...
android:importantForAccessibility="no"
... on your RecyclerView. This will disable all native accessibility functionality related to your list, including the ACTION_SCROLL_FORWARD / ACTION_SCROLL_BACKWARD actions (Accessibility Source), and the ability to append "In List" / "Out of List" context to accessibility announcements.
Instead, you should set...
android:focusable="true"
android:focusableInTouchMode="true"
...on the root layout of your List items, which will enable them to gain focus individually when the accessibility user navigates to them.
It would appear that at some point in the construction process your Recycler View is marked as importantForAccessibility. When a container view is marked as important for accessibility it gathers all of the information within that container and announces it as the content description.
This should remedy the situation.
android:importantForAccessibility="no" //Or "auto"
If at no point in code did you set this otherwise, this would appear to be a bug with your flavor of Android. This is certainly not desirable default behavior.
Edit:
Changed "no" to "auto". Really we just want it to not be "yes", which is the value that creates the poor behavior. Auto behaves better with Switch Control on modern devices.
Been investigating this on and off for a bit, I don't think there's a Android OS Version agnostic solution here. Android APIs have changed their definition of Focusability vs Accessibility Focusability too many times across too many versions.