Accessibility: Talkback, WebView and user's locale - android

I have developed an app that includes a Webview. I would like to make my app fully accessible, so for the webview element I would like for TalkBack to read html elements such as "Heading", "Banner", "EditText" in a fully accessible way.
I have seen that the TTS process in WebView is done through JS injection via Chromium AccessibilityInjector.java class. This injects this script into the page, which only has the messages in English. The result is that when a device is in another language, the TTS reads these html element in English regardless.
Now I cannot debug or extend the chromium webclient, so how can I make TTS to read my page according to the user's locale?
EDIT: I am using jQuery Mobile by the way.

Just in case someone stumbles into this problem: I had to apply an ugly workaround to get over this. Whenever I load a page and TalkBack is enabled, I reinject the javascript variables containing the text to be read, with their localized counterparts. For instance, for Spanish text:
view.loadUrl("javascript:window.setTimeout(function(){" +
"window.console.log(\"Injecting messages.\");" +
"cvox.TestMessages[\"chromevox_input_type_text\"] = {message: \"cuadro de edición\"};" +
"cvox.TestMessages[\"chromevox_input_type_radio\"] = {message: \"botón de opción\"};" +
"cvox.TestMessages[\"chromevox_selected\"] = {message: \"seleccionado\"};" +
"cvox.TestMessages[\"chromevox_unselected\"] = {message: \"no seleccionado\"};" +
"cvox.TestMessages[\"chromevox_radio_selected_state\"] = {message: \"seleccionado\"};" +
"cvox.TestMessages[\"chromevox_radio_unselected_state\"] = {message: \"no seleccionado\"};" +
"cvox.TestMessages[\"chromevox_input_type_submit\"] = {message: \"botón\"};" +
"cvox.TestMessages[\"chromevox_input_type_button\"] = {message: \"botón\"};" +
"cvox.TestMessages[\"chromevox_tag_button\"] = {message: \"botón\"};" +
"}, 2000)");
Note that I insert a timeout before injecting the variables -- this is to prevent chromevox from being injected after my injection thus making the solution useless.
I know this is an ugly patch, but I could not find any better solutions without access to the chromium webview classes.

Related

How to Ebay oauth process for Android?

I have managed to achieve HTTP requests with Volley to an API without auth, but with Ebay API things are becoming harder.
https://github.com/eBay/ebay-oauth-android-client
The above link has the an example, yet it is not filled with a real case scenario.
ApiSessionConfiguration.initialize
(
apiEnvironment = ApiEnvironment.PRODUCTION,
apiConfiguration = ApiConfiguration(
"<Client ID>",
"<Redirect Uri>",
"<space separated scopes>"
)
)
Client ID appears on Ebay Developer Account as is and i guess "XXXX-XXXX-PRD-XXXX-XXXX" something on that lines is correct.
Redirect Uri on the other hand seems more confusing. Which string needs to be placed there? The "RuName" that is similar to the Client ID XXXX-XXXX-XXXX-XXXX? Or something like "http://..."?
Space separated scopes hopefully are; "https://api.ebay.com/oauth/api_scope/sell.marketing.readonly" + " " + "https://api.ebay.com/oauth/api_scope/sell.marketing" ... and so on.
I have further doubts about the part that requires having a second activity to be overriden on the manifest but i guess that will be better asked separatedly.

WebView app (PhoneGap Build), not saving localStorage after restart

For some reason, when I restart my PhoneGap app - it looses the localStorage vales that were stored before! I'm saving them in the normal way:
localStorage.setItem("foo","value");
This stores it just fine. However, when you restart the app (or leave the device off for a random amount of time), it seems to randomly loose the data. I've found a heck of a lot of posts about this - but no definative answer on how to get it to be persistent in a PhoneGap Build WebView app,
Any suggestions are much welcomed!
This seems to be quite a common problem with WebView apps:
Android 2.3.6 + Phonegap + localStorage
Android - Making Webview DomStorage persistant after app closed
I can't find a solution that works with PhoneGap Build apps though
An actual example I'm using, is:
var current_id = parseInt(currentId) + 1;
localStorage.setItem("entry_"+current_id,save_string);
localStorage.setItem("entryId",current_id);
..and then to extract it (not that this is important, as the problem is with the data going missing, and not with accessing it)
for (var i = 0; i < localStorage.length; i++){
if (localStorage.key(i).match("entry_")) {
outputString += "\n" + localStorage.getItem(localStorage.key(i));
}
}
I'm wondering if maybe upgrading from PhoneGap Build cli-5.2.0 to cli-6.0.0 may help. I will do this, and give it a whirl.
I guess another option, would be to use a SQL database to locally store the device (its just a bit trickier to setup, and means re-writing my code)
UPDATE: Not the ideal solution - but I have now moved the app over to use WebSQL for the app. It was a bit tricky to get the hang of (never used it before) - but seems to do the job, and shouldn't loose the data :)
EDIT
i tried it like this and it worked:
var current_id = parseInt(currentId) + 1;
localStorage.setItem("entry_"+current_id,save_string);
localStorage.setItem("entryId",current_id);
/*
//this is for checking, what is stored in localStorage
console.log("length: " + localStorage.length);
for(var i = 0; i < localStorage.length; i++) {
console.log(localStorage.key(i));
}
*/
var myEntryIdFromStorage = localStorage.getItem("entryId");
var myItem = localStorage.getItem("entry_" + myEntryIdFromStorage);
Old answer for clarification
How do you get your localstorage?
normally you should store items like you did:
var permanentStorage = window.localstorage;
permanentStorage.setItem("foo", "bar");
and get them back by initializing the permanentStorage the same way and:
//assuming you have permanentStorage in the same script file
//or else you have to initialize it again:
//var permanentStorage = window.localstorage;
var myItem = permanentStorage.getItem("foo");
console.log("myItem: " + myItem);
The method store item uses two parameters: the identifier and the data itself. Please check, that the identifier with which you store your data is the same as the one, with which you get it back.
Do you get any errors? Is the return (stored in my example in myItem) null or undefined or just an empty string? Does this fail in the browser or on the device?
You could clarify your question by providing more code or error messages!

Titanium: Show and Create Contact Intent

I was trying to export contacts from my application to the native android contacts.
I found the following solution here on this site: Titanium: How to add Contact in Phone book in Android?.
And it kinda works. The intent gets started. Only problem is, that android does not recognize most of the Extras i put in. So almost every field is blank. It does not matter if I replace contactModel with a simple String, the result is the same.
So i was wondering if the keys are simply wrong, but there seems no proper documentation on appcelerator. Probably something has changed over the past years or I am just missing something. Does anybody know how to do it the right way.
Code Snippet:
if (OS_ANDROID) {
var intent = Ti.Android.createIntent({
action : 'com.android.contacts.action.SHOW_OR_CREATE_CONTACT',
data : 'mailto:' + contactModel.get('contact_first_name') + ' ' + contactModel.get('contact_last_name')
});
intent.putExtra('email', contactModel.get('contact_email'));
intent.putExtra('email_type', 'Work');
intent.putExtra('phone', contactModel.get('contact_mobile_number'));
intent.putExtra('phone_type', 'mobile');
intent.putExtra('name', contactModel.get('contact_first_name') + ' ' + contactModel.get('contact_last_name') );
intent.putExtra('address', addressModel.get('address_street') + ", " + addressModel.get('address_city'));
intent.putExtra('address_type', 'Work');
Ti.Android.currentActivity.startActivity(intent);
}
Thx in advance. :)
The first parameter of putExtra() also accepts one of the constants Ti.Android.... So instead of email you would probably have to use Ti.Android.EXTRA_EMAIL.
I think you can find quite a lot in the docs, e.g. here:
EXTRA constant properties in Ti docs
EXTRA constants in Android docs
Android Intents in Ti docs

EXTRA_AVAILABLE_VOICES always returns eng-GBR only. Why?

I am using the following snippet to log all available (and unavailable) voices currently on phone:
ArrayList<String> availableVoices = intent.getStringArrayListExtra(TextToSpeech.Engine.EXTRA_AVAILABLE_VOICES);
String availStr = "";
for (String lang : availableVoices)
availStr += (lang + ", ");
Log.i(String.valueOf(availableVoices.size()) + " available langs: ", availStr);
ArrayList<String> unavailableVoices = intent.getStringArrayListExtra(TextToSpeech.Engine.EXTRA_UNAVAILABLE_VOICES);
String unavailStr = "";
for (String lang : unavailableVoices)
unavailStr += (lang + ", ");
Log.w(String.valueOf(unavailableVoices.size()) + " unavailable langs: ", unavailStr);
The logged result is somehwat bewildering, since I know beyond certainty that I have multiple languages installed and I can even hear the TTS speaking in eng-USA, yet the log shows:
1 available langs: eng-GBR,
30 unavailable langs: ara-XXX, ces-CZE, dan-DNK, deu-DEU, ell-GRC,
eng-AUS, eng-GBR, eng-USA, spa-ESP, spa-MEX, fin-FIN, fra-CAN,
fra-FRA, hun-HUN, ita-ITA, jpn-JPN, kor-KOR, nld-NLD, nor-NOR,
pol-POL, por-BRA, por-PRT, rus-RUS, slk-SVK, swe-SWE, tur-TUR,
zho-HKG, zho-CHN, zho-TWN, tha-THA,
Why is this inconsistent behavior? (note that eng-GBR appears in both the available and unavailable lists...)
It turns out that as far as text-to-speech in Android 2.x goes, it's the wild west out there: Every and any installed 3rd-party TTS engine can modify the output of this EXTRA_AVAILABLE_VOICES function however they desire, regardless whether checked/unchecked or selected/unselected as default.
I just tried uninstalling all TTS engines from my phone, leaving only the hard-coded Pico, and the result match exactly what I expected:
6 available voices: deu-DEU, eng-GBR, eng-USA, spa-ESP, fra-FRA,
ita-ITA,
0 unavailable voices:
I don't mind the output of this function dynamically refer to the currently selected (i.e. default) TTS engine, but the fact is that once a 3rd party TTS engine is installed, this function's output doesn't make any sense, because it ignores any settings.
Also note that the name misleading: It's available languages, not voices!
I am posting this answer with the hope that it will help someone save the time & agony of discovering this the hard way.

best practice for specifying pronunciation for Android TTS engine?

In general, I'm very impressed with Android's default text to speech engine (i.e., com.svox.pico). As expected, it mispronounces some words (as do I) and it therefore occasionally needs some pronunciation guidance. So I'm wondering about best practices for phonetically spelling out those words that the pico TTS engine mispronounces.
For example, the correct pronunciation of the bird Chachalaca is CHAH-chah-LAH-kah. Here is what the TTS engine produces:
mTts.speak("Chachalaca", TextToSpeech.QUEUE_ADD, null); // output: chuh-KAL-uh-KUH
mTts.speak("CHAH-chah-LAH-kah", TextToSpeech.QUEUE_ADD, null); // output: CHAH-chah-EL-AY-AYCH-dash-kuh
mTts.speak("CHAHchahLAHkah", TextToSpeech.QUEUE_ADD, null); // output: CHA-chah-LAH-ka
mTts.speak("CHAH chah LOCKah", TextToSpeech.QUEUE_ADD, null); // output: CHAH-chah-LAH-kah
Here are my questions.
Is there a standard phonetic spelling recognized by the Android TTS engine?
If not, are there some general rules for making custom pronunciation spellings that will make the spellings more likely to be correct in future TTS engines/versions?
It appears that the Android TTS engine ignores text case. What is the best way to specify emphasis?
By the way, this is what the TTS engine writes to logcat:
V/TtsService( 294): TTS processing: CHAH chah LOCKah
V/TtsService( 294): TtsService.setLanguage(eng, USA, )
I/SVOX Pico Engine( 294): Language already loaded (en-US == en-US)
I/SynthProxy( 294): setting speech rate to 100
I/SynthProxy( 294): setting pitch to 100
[UPDATE]
I tried passing an XML document to TextToSpeech.speak() as follows:
String text = "<?xml version=\"1.0\"?>" +
"<speak version=\"1.0\" xmlns=\"http://www.w3.org/2001/10/synthesis\" " +
"xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" " +
"xsi:schemaLocation=\"http://www.w3.org/2001/10/synthesis " +
"http://www.w3.org/TR/speech-synthesis/synthesis.xsd\" " +
"xml:lang=\"en-US\">" +
"That is a big car! " +
"That <emphasis>is</emphasis> a big car! " +
"That is a <emphasis>big</emphasis> car! " +
"That is a huge bank account! " +
"That <emphasis level=\"strong\">is</emphasis> a huge bank account! " +
"That is a <emphasis level=\"strong\">huge</emphasis> bank account!" +
"</speak>";
mTts.speak(text, TextToSpeech.QUEUE_ADD, null);
As Android Eve suggested, the TTS engine read only the XML body (i.e., the comments about the big car and the huge bank account). I didn't realize the TTS engine was capable of parsing XML documents. However, I did not hear any emphasis in the TTS output.
[UPDATE 2]
I simplified the question to whether or not Android TTS supports Speech Synthesis Markup Language here.
JW answered my question at the tts-for-android group:
Hi Greg,
The Pico engine recognizes the tag with the XSAMPA alphabet.
There are no easy rules to derive a certain pronunciation from the orthograpy, but you can use intuitive spellings and trial and error. Capitalizing and hyphens will introduce more problems than solving them. Using different spellings and introducing extra word boundaries (spaces) can work.
The emphasis tag and the exclamation mark will not change the synthesis result. Use , , and commands instead.
Some examples of the proper syntax for specifying the pronunciation using the SSML phoneme tag are in these tests of TextToSpeech.
Even with these simple test SSML documents, there are warning messages posted to logcat about the SSML document not being well-formed. So I opened an issue about these seemingly incorrect logcat messages to the Android issue tracker.
The syntax for specifying an x-SAMPA sequence to SVOX pico is
String text = "<speak xml:lang=\"en-US\"> <phoneme alphabet=\"xsampa\" ph=\"d_ZIn\"/>.</speak>";
mTts.speak(text, TextToSpeech.QUEUE_ADD, null);
Although more examples would be helpful, a good reference for x-SAMPA is at http://en.wikipedia.org/wiki/Xsampa If I compile a couple dozen examples, I'll post them to that Wikipedia page.
One answer for all 3 questions: Look at the SSML specifications: http://www.w3.org/TR/speech-synthesis/
For example, to specify emphasis, you use the emphasis element, e.g.
<?xml version="1.0"?>
<speak version="1.0" xmlns="http://www.w3.org/2001/10/synthesis"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.w3.org/2001/10/synthesis
http://www.w3.org/TR/speech-synthesis/synthesis.xsd"
xml:lang="en-US">
That is a <emphasis> big </emphasis> car!
That is a <emphasis level="strong"> huge </emphasis>
bank account!
</speak>

Categories

Resources