are there any characters that, if included in an SMS message (body), might cause problems with being sent?
edit: Actually I only care really about any characters that can be input by user with keyboard (but in any language).. don't know if that makes my question easier to answer or not...
There's no straightforward answer to this as it depends on a variety of factors, but this standard lays out the basics and I expect you can use this to deduce which characters to focus on:
http://en.wikipedia.org/wiki/GSM_03.38
I would advise doing extensive testing of the characters you use, on different handsets, and in your target locales.
Related
I would like to make the talkback say a simple mathematical expression like: "2 - (3/5)"
However, when it reads, talkback skips the parentheses, thus affecting the expression to take on a different meaning.
Thanks to #QuentinC I was able to read more questions on the subject. Would this fall under "it's best to leave it at that and not edit" cases, or should I instead be looking for ways to force talkback to speak every character?
Background
TextView always had issues with RTL (Right-To-Left) languages. Since I know only how to read Hebrew (in addition to English), I will talk about its issues:
Text alignment (and I'm not talking about gravity) . As an RTL language, Hebrew puts words from right to left (compared to English which is the opposite).
For demonstrating how annoying it is, imagine that instead of showing "Hello world." you usually get ".Hello world" . This could be easily fixed if you had it in a single sentence, but it's harder when there are multiple punctuations characters.
Vowels positions. Hebrew doesn't require vowels in order to read text, but sometimes it's very hard to read without them (especially the bible). For vowels, Hebrew has what is called "NIKUD", which are actually like dots inside the letters. The problem in Android was that they were usually positioned in the wrong location .
For demonstrating how annoying it is, imagine that instead of showing "Hello world." you usually get ".eHlol owrld" . Even if you try to fix it (put the vowels always one character after the current one), the position in the letter wasn't correct (imagine that the "e" in "Hello" would be like above the "H", for example) .
Only on version 4.2 (read here, under "Native RTL support") , Google has fixed all of the Hebrew related issues (or at least it seems so).
The problem
the problems with Hebrew has caused each Israeli carrier and each custom ROM maker have its own solution of how to fix the different issues, which makes it practically impossible to handle RTL text on pre 4.2 devices.
Things can get even more frustrating in case the text include both Hebrew and English letters.
What I've tried
I've read many websites talking about those problems, and I've tried many variants of the solutions, none has solved the problem on all devices:
Some suggest to put the character '\u200F' (or '\u202D') at the end/start/both of the text.
Some suggest using Html.fromHtml() method and put something special there.
Some even suggest to use the WebView instead (and maybe use WebSettings.setDefaultTextEncodingName() ).
The question
Is there a definite solution for this problem?
I would assume the best thing is that because Android 4.2 solves this, and Android is open source, we should have its TextView imported into a library that we can use, but Google hasn't provided such a library yet.
Sadly, I don't think there's a good solution ("good" meaning "does the job and is readily available"). For our own Android apps that support Hebrew, we use a custom rendering mechanism that we developed over many years. The rendering mechanism does everything: proprietary fonts; bidirectional (bidi) analysis; glyph placement; line break analysis; text flow; etc. Some of the problems trying to use native Android text handling capabilities (especially pre-4.2) are:
Really crappy fonts. However, you can package third-party fonts like DejaVu that are pretty good. The right font can do wonders with positioning of nekudot—and te'amim1, if you need that. (I agree with you about how important correct pointing placement is; reading Hebrew text with misplaced nekudot is like reading a screen-full of CAPTCHAs.)
Buggy bidi analysis. What makes it worse is that the bugs seem to be different for different versions of Android. Modifying the text to include strategically placed bidi formatting codes (RTL mark; LTR mark; etc.) can overcome many of these bugs (see the discussion here, which isn't specific to Android). However, it's a nuisance to do this and, because of the inconsistencies among Android versions, it is difficult to predict in advance what help the framework is going to need.
No (or poorly thought out) framework-level awareness of right-to-left issues. For instance, good luck getting the scroll bar to display on the left side of a Hebrew TextView. For our apps, we had to build an entire scroll-bar system just to get this to work how we wanted. (Good think Android is open source!)
Poor line and word break analysis. At least one early version of Android on which we tested thought that each nikud mark was a word boundary. When it comes to line breaks, the system often doesn't know how to handle Hebrew punctuation like maqaf, gershayim, or sof pasuk.
Some of the newer Unicode characters (like HOLAM HASER FOR VAV—U+05BA—new to Unicode 5.0) are not recognized as Hebrew script by the system.
My recommendation is that, unless you are prepared to build a top-to-bottom text handling system yourself, you give up on high-quality text display on pre-4.2 versions of Android, particularly if you need to support nekudot and te'amim. Also, plan to use the techniques I mentioned in the first two points above.
1 biblical cantillation marks
As of August 2013, Android has posted API documentation for a Bi Directional Formatter which might suit your needs. This is contained in the Android Support v4 library which I believe should run in versions prior to Android 4.2.
Refer to:
http://developer.android.com/reference/android/support/v4/text/BidiFormatter.html
I have been tasked to create a new android 3rd party keyboard that supports customized emojis (My own Icons) from assets.
I want to implement a softkeyboard with my own emoji icons without using UniCode or my custom UniCode.
Questions:
If I create a custom emoji, with some string of characters which does not map to the standard set of emojis, and text this message to a friend with the customized app/keyboard, what shows up on their device? The regular ASCII characters string? or the image.
I have read two ways to add image to textView.
Html.ImageGetter
Spannable Image (String consisting of image)
Which way should i prefer?
Is there anyway to display(send) the customized emoji on the recipients device without downloading the app/keyboard?
Is it possible to send text with Image(Emoji) to other apps like facebook,skype and for messaging.
Need suggestions.
Simple Words
I simply want to send my custom(Emoji icon) to other apps as this app does with out using unicode or with my custom UniCode.
Thanks.
To answer the first part of your question, by definition Emoji are encoded characters - they are a part of unicode. See here:
http://emojipedia.org/unicode-8/
There are many references to this if you look. You will also discover that for a long time Apple and Google used two different sets. They are now merged, but then Android manufacturers and carriers have added their own emoji "versions."
Changing the keyboard to have custom images will not change the data that is transmitted to the other device. So, to answer the next part of your question: what shows up on their device is whatever the ASCII or Unicode character that was transmitted, not what the sender "typed."
In other words, to answer the next part of you question, generally speaking there is not a way to send custom characters to another device without them having your app. A keyboard would not suffice because apps do the job of displaying text/images. So unless an app knows that you are the content provider or source or whatever of the image, it will display whatever it knows to do. So, a custom keyboard won't even display custom emoji on your own device, unless you are also using your own app.
I said "generally not possible" because here are your options:
You can become a part of the Unicode Consortium (http://unicode.org/) and submit your emoji images for approval to go into a future version of Unicode. There are future emoji already in the works, FYI. That will likely take several years, by the way, and it's unlikely they will approve commercially biased images. However, unicode has the capacity to handle billions of characters and is hardly even close to being full (Unicode 16, not Unicode 8 - Unicode 8 is full). Even then, the Android team would need to adopt it and include it in a future release like smileys and the current set of emoji are.
You build your own app with your own emoji and get people on both sides of the communication to download it, like everyone else does. IMO, this not ideal for anyone but the developer of the app. Still, the ones that people enjoy I applaud for their work and success. That industry is fickle and difficult to really gain a presence in.
I'm a part of sdmmllc.com - and we're trying to develop a messaging "platform" exactly for situations like yours. We want to allow messaging apps to "discover" other messaging apps, incorporate features like custom emoji, without the user getting confused or having to download tons of apps. This is similar to plug-ins in web browsers. Our developers love us, our users love us, but it's a slow process.
Develop a competing platform. (And good luck with that - no one really seems to be getting the concept, except the few developers we have, and the hundreds of users that download our app every day and love our idea and platform... but there's no money it so far...)
you can only use those uniCode which are supported. you cannot add your own for generic use. But you can use it with in your app and between your app. It is not possible.
In short it is not possible to create your own Unicode. But you can do it with app to app. and on both ends you have to store those character in database. and match them when they get..
Background
TextView always had issues with RTL (Right-To-Left) languages. Since I know only how to read Hebrew (in addition to English), I will talk about its issues:
Text alignment (and I'm not talking about gravity) . As an RTL language, Hebrew puts words from right to left (compared to English which is the opposite).
For demonstrating how annoying it is, imagine that instead of showing "Hello world." you usually get ".Hello world" . This could be easily fixed if you had it in a single sentence, but it's harder when there are multiple punctuations characters.
Vowels positions. Hebrew doesn't require vowels in order to read text, but sometimes it's very hard to read without them (especially the bible). For vowels, Hebrew has what is called "NIKUD", which are actually like dots inside the letters. The problem in Android was that they were usually positioned in the wrong location .
For demonstrating how annoying it is, imagine that instead of showing "Hello world." you usually get ".eHlol owrld" . Even if you try to fix it (put the vowels always one character after the current one), the position in the letter wasn't correct (imagine that the "e" in "Hello" would be like above the "H", for example) .
Only on version 4.2 (read here, under "Native RTL support") , Google has fixed all of the Hebrew related issues (or at least it seems so).
The problem
the problems with Hebrew has caused each Israeli carrier and each custom ROM maker have its own solution of how to fix the different issues, which makes it practically impossible to handle RTL text on pre 4.2 devices.
Things can get even more frustrating in case the text include both Hebrew and English letters.
What I've tried
I've read many websites talking about those problems, and I've tried many variants of the solutions, none has solved the problem on all devices:
Some suggest to put the character '\u200F' (or '\u202D') at the end/start/both of the text.
Some suggest using Html.fromHtml() method and put something special there.
Some even suggest to use the WebView instead (and maybe use WebSettings.setDefaultTextEncodingName() ).
The question
Is there a definite solution for this problem?
I would assume the best thing is that because Android 4.2 solves this, and Android is open source, we should have its TextView imported into a library that we can use, but Google hasn't provided such a library yet.
Sadly, I don't think there's a good solution ("good" meaning "does the job and is readily available"). For our own Android apps that support Hebrew, we use a custom rendering mechanism that we developed over many years. The rendering mechanism does everything: proprietary fonts; bidirectional (bidi) analysis; glyph placement; line break analysis; text flow; etc. Some of the problems trying to use native Android text handling capabilities (especially pre-4.2) are:
Really crappy fonts. However, you can package third-party fonts like DejaVu that are pretty good. The right font can do wonders with positioning of nekudot—and te'amim1, if you need that. (I agree with you about how important correct pointing placement is; reading Hebrew text with misplaced nekudot is like reading a screen-full of CAPTCHAs.)
Buggy bidi analysis. What makes it worse is that the bugs seem to be different for different versions of Android. Modifying the text to include strategically placed bidi formatting codes (RTL mark; LTR mark; etc.) can overcome many of these bugs (see the discussion here, which isn't specific to Android). However, it's a nuisance to do this and, because of the inconsistencies among Android versions, it is difficult to predict in advance what help the framework is going to need.
No (or poorly thought out) framework-level awareness of right-to-left issues. For instance, good luck getting the scroll bar to display on the left side of a Hebrew TextView. For our apps, we had to build an entire scroll-bar system just to get this to work how we wanted. (Good think Android is open source!)
Poor line and word break analysis. At least one early version of Android on which we tested thought that each nikud mark was a word boundary. When it comes to line breaks, the system often doesn't know how to handle Hebrew punctuation like maqaf, gershayim, or sof pasuk.
Some of the newer Unicode characters (like HOLAM HASER FOR VAV—U+05BA—new to Unicode 5.0) are not recognized as Hebrew script by the system.
My recommendation is that, unless you are prepared to build a top-to-bottom text handling system yourself, you give up on high-quality text display on pre-4.2 versions of Android, particularly if you need to support nekudot and te'amim. Also, plan to use the techniques I mentioned in the first two points above.
1 biblical cantillation marks
As of August 2013, Android has posted API documentation for a Bi Directional Formatter which might suit your needs. This is contained in the Android Support v4 library which I believe should run in versions prior to Android 4.2.
Refer to:
http://developer.android.com/reference/android/support/v4/text/BidiFormatter.html
Is it possible to restrict voice search search widget to look for a match near to a given set of words. For example if I am using it to search over a list of names, its not meaningful as names are often corrected to some words.
If you use 3rd party Android recognition from Nuance (The people behind DragonDictate), it supports a "grammar mode" where you can somewhat restrict the phrases that will be recognised during recognition.
Importantly, if you add unusual names into a Custom Vocabulary, they SHOULD become recognizable (Complex pronunciation issues aside)
You can find information if you dig through:
http://dragonmobile.nuancemobiledeveloper.com ,
looking for 'Custom Vocabularies'. Grammar mode is essentially a special mode of custom vocabularies.
At the time of writing, there was a document here that makes some mention of grammar mode:
http://dragonmobile.nuancemobiledeveloper.com/downloads/custom_vocabulary/Guide_to_Custom_Vocabularies_v1.5.pdf - It only really becomes clear when you try to progress in their provisioning web GUI.
You have to set up an account, and jump through other hoops, but there is a free tier. This is the only potential way I have found to constrain a recognition vocabulary.
Well, short of running up PocketSphinx, but that is still described as a 'Research' 'PreAlpha'.
No, I don't work for Nuance. Not sure anyone does. They may have all been eaten by zombies. You would guess as much reading their support forums. They never reply.
No, unfortunately this is not possible.
You could also look at recognition from AT&T.
They have a very feature-rich web API, including full grammar support. I only found out about it recently!
1,000,000 transactions per month for free. Generous!
Look for 'AT&T API Program'. Weird name.
Link at time of writing:
http://developer.att.com/apis/speech
Unfortunately for me, no Australian accent language models at time of writing. Only US and UK English. Boooo.
EDIT: Some months after I submitted the above, AT&T retired the service mentioned. It seems everyone just wants a 'dumbed down' API where you just call a recognizer, and it returns words. Sure, that is of course the holy grail, but a properly designed, constrained grammar will generally work better. As someone with speech skills, the minimalism of the common Speech APIs today is really frustrating...