Background
TextView always had issues with RTL (Right-To-Left) languages. Since I know only how to read Hebrew (in addition to English), I will talk about its issues:
Text alignment (and I'm not talking about gravity) . As an RTL language, Hebrew puts words from right to left (compared to English which is the opposite).
For demonstrating how annoying it is, imagine that instead of showing "Hello world." you usually get ".Hello world" . This could be easily fixed if you had it in a single sentence, but it's harder when there are multiple punctuations characters.
Vowels positions. Hebrew doesn't require vowels in order to read text, but sometimes it's very hard to read without them (especially the bible). For vowels, Hebrew has what is called "NIKUD", which are actually like dots inside the letters. The problem in Android was that they were usually positioned in the wrong location .
For demonstrating how annoying it is, imagine that instead of showing "Hello world." you usually get ".eHlol owrld" . Even if you try to fix it (put the vowels always one character after the current one), the position in the letter wasn't correct (imagine that the "e" in "Hello" would be like above the "H", for example) .
Only on version 4.2 (read here, under "Native RTL support") , Google has fixed all of the Hebrew related issues (or at least it seems so).
The problem
the problems with Hebrew has caused each Israeli carrier and each custom ROM maker have its own solution of how to fix the different issues, which makes it practically impossible to handle RTL text on pre 4.2 devices.
Things can get even more frustrating in case the text include both Hebrew and English letters.
What I've tried
I've read many websites talking about those problems, and I've tried many variants of the solutions, none has solved the problem on all devices:
Some suggest to put the character '\u200F' (or '\u202D') at the end/start/both of the text.
Some suggest using Html.fromHtml() method and put something special there.
Some even suggest to use the WebView instead (and maybe use WebSettings.setDefaultTextEncodingName() ).
The question
Is there a definite solution for this problem?
I would assume the best thing is that because Android 4.2 solves this, and Android is open source, we should have its TextView imported into a library that we can use, but Google hasn't provided such a library yet.
Sadly, I don't think there's a good solution ("good" meaning "does the job and is readily available"). For our own Android apps that support Hebrew, we use a custom rendering mechanism that we developed over many years. The rendering mechanism does everything: proprietary fonts; bidirectional (bidi) analysis; glyph placement; line break analysis; text flow; etc. Some of the problems trying to use native Android text handling capabilities (especially pre-4.2) are:
Really crappy fonts. However, you can package third-party fonts like DejaVu that are pretty good. The right font can do wonders with positioning of nekudot—and te'amim1, if you need that. (I agree with you about how important correct pointing placement is; reading Hebrew text with misplaced nekudot is like reading a screen-full of CAPTCHAs.)
Buggy bidi analysis. What makes it worse is that the bugs seem to be different for different versions of Android. Modifying the text to include strategically placed bidi formatting codes (RTL mark; LTR mark; etc.) can overcome many of these bugs (see the discussion here, which isn't specific to Android). However, it's a nuisance to do this and, because of the inconsistencies among Android versions, it is difficult to predict in advance what help the framework is going to need.
No (or poorly thought out) framework-level awareness of right-to-left issues. For instance, good luck getting the scroll bar to display on the left side of a Hebrew TextView. For our apps, we had to build an entire scroll-bar system just to get this to work how we wanted. (Good think Android is open source!)
Poor line and word break analysis. At least one early version of Android on which we tested thought that each nikud mark was a word boundary. When it comes to line breaks, the system often doesn't know how to handle Hebrew punctuation like maqaf, gershayim, or sof pasuk.
Some of the newer Unicode characters (like HOLAM HASER FOR VAV—U+05BA—new to Unicode 5.0) are not recognized as Hebrew script by the system.
My recommendation is that, unless you are prepared to build a top-to-bottom text handling system yourself, you give up on high-quality text display on pre-4.2 versions of Android, particularly if you need to support nekudot and te'amim. Also, plan to use the techniques I mentioned in the first two points above.
1 biblical cantillation marks
As of August 2013, Android has posted API documentation for a Bi Directional Formatter which might suit your needs. This is contained in the Android Support v4 library which I believe should run in versions prior to Android 4.2.
Refer to:
http://developer.android.com/reference/android/support/v4/text/BidiFormatter.html
Related
Why do some Android projects list a "CHAR_LIMIT" in comments above each string in strings.xml? For example, from https://android.googlesource.com/platform/frameworks/base/+/master/core/res/res/values/strings.xml:
<!-- [CHAR LIMIT=25] String for confirmation button to enable a feature gated by the battery saver warning-->
<string name="confirm_battery_saver">OK</string>
<!-- [CHAR_LIMIT=NONE] Battery saver: Feature description, with a "learn more" link. -->
<string name="battery_saver_description_with_learn_more">To extend battery life, Battery Saver:\n·Turns on Dark theme\n·Turns off or restricts background activity, some visual effects, and other features like \u201cHey Google\u201d\n\n<annotation id="url">Learn more</annotation></string>
Googling turns up a few other examples, but no explanation. I don't believe Android documentation covers this. And unlike tags like <xliff:g> it doesn't seem to be an official, functional component of Android. My best guess is that it's a convention some projects use to indicate "when you translate this, make sure the translated version doesn't have more than X characters or else it will break the UI!" Or perhaps, vice versa, "if you're using this String, make sure the UI still looks good with a String X characters long"
My thought is that the UI should always be built to be as flexible as is reasonable, given the possibility for translations of different lengths, and the possibility of different text sizes. Also that translators should strive for the translation to be more or less the same size as the original text, instead of going from, say, 10 characters to 50 characters when translating. So I would think this "CHAR_LIMIT" would be completely unnecessary and could be dropped. (I ask, because I'm touching up an old open source project that uses this.)
Is my understanding of "CHAR_LIMIT" correct, or what is it for, and should it be used?
when you translate this, make sure the translated version doesn't have more than X characters or else it will break the UI!" Or perhaps, vice versa, "if you're using this String, make sure the UI still looks good with a String X characters long
That's pretty much my understanding of it.
I searched around a bit for where this clue is used, and I found only a single place: stringslint.py. This is a lint script that seems to be used on strings.xml files. Maybe the project you're talking about used this script? The file docs say:
Enforces common Android string best-practices. It ignores lint
messages from a previous strings file, if provided.
Usage: stringslint.py strings.xml
Usage: stringslint.py strings.xml old_strings.xml
In general:
Errors signal issues that must be fixed before submitting, and are only used when there are no false-positives.
Warnings signal issues that might need to be fixed, but need manual inspection due to risk of false-positives.
Info signal issues that should be fixed to match best-practices, such as providing comments to aid translation.
So that goes along what you were saying. It's a clue for translators but can also be used for lint checks, hence the easily parseable syntax.
I also tried to search for pages explaining how one would translate the Android System, hoping CHAR LIMIT would be mentioned somewhere. But it doesn't seem like the community can contribute their own translation, so I found no such page.
Note that there's also a BACKUP_MESSAGE_ID that's declared in the same way for some string resources. It seems related to the system Settings app, but I don't know any more that than.
EDIT: I found another script using it: https://chromium.googlesource.com/chromium/src/+/master/tools/grit/grit/tool/android2grd.py, although this time it's CHAR-LIMIT.
I have been tasked to create a new android 3rd party keyboard that supports customized emojis (My own Icons) from assets.
I want to implement a softkeyboard with my own emoji icons without using UniCode or my custom UniCode.
Questions:
If I create a custom emoji, with some string of characters which does not map to the standard set of emojis, and text this message to a friend with the customized app/keyboard, what shows up on their device? The regular ASCII characters string? or the image.
I have read two ways to add image to textView.
Html.ImageGetter
Spannable Image (String consisting of image)
Which way should i prefer?
Is there anyway to display(send) the customized emoji on the recipients device without downloading the app/keyboard?
Is it possible to send text with Image(Emoji) to other apps like facebook,skype and for messaging.
Need suggestions.
Simple Words
I simply want to send my custom(Emoji icon) to other apps as this app does with out using unicode or with my custom UniCode.
Thanks.
To answer the first part of your question, by definition Emoji are encoded characters - they are a part of unicode. See here:
http://emojipedia.org/unicode-8/
There are many references to this if you look. You will also discover that for a long time Apple and Google used two different sets. They are now merged, but then Android manufacturers and carriers have added their own emoji "versions."
Changing the keyboard to have custom images will not change the data that is transmitted to the other device. So, to answer the next part of your question: what shows up on their device is whatever the ASCII or Unicode character that was transmitted, not what the sender "typed."
In other words, to answer the next part of you question, generally speaking there is not a way to send custom characters to another device without them having your app. A keyboard would not suffice because apps do the job of displaying text/images. So unless an app knows that you are the content provider or source or whatever of the image, it will display whatever it knows to do. So, a custom keyboard won't even display custom emoji on your own device, unless you are also using your own app.
I said "generally not possible" because here are your options:
You can become a part of the Unicode Consortium (http://unicode.org/) and submit your emoji images for approval to go into a future version of Unicode. There are future emoji already in the works, FYI. That will likely take several years, by the way, and it's unlikely they will approve commercially biased images. However, unicode has the capacity to handle billions of characters and is hardly even close to being full (Unicode 16, not Unicode 8 - Unicode 8 is full). Even then, the Android team would need to adopt it and include it in a future release like smileys and the current set of emoji are.
You build your own app with your own emoji and get people on both sides of the communication to download it, like everyone else does. IMO, this not ideal for anyone but the developer of the app. Still, the ones that people enjoy I applaud for their work and success. That industry is fickle and difficult to really gain a presence in.
I'm a part of sdmmllc.com - and we're trying to develop a messaging "platform" exactly for situations like yours. We want to allow messaging apps to "discover" other messaging apps, incorporate features like custom emoji, without the user getting confused or having to download tons of apps. This is similar to plug-ins in web browsers. Our developers love us, our users love us, but it's a slow process.
Develop a competing platform. (And good luck with that - no one really seems to be getting the concept, except the few developers we have, and the hundreds of users that download our app every day and love our idea and platform... but there's no money it so far...)
you can only use those uniCode which are supported. you cannot add your own for generic use. But you can use it with in your app and between your app. It is not possible.
In short it is not possible to create your own Unicode. But you can do it with app to app. and on both ends you have to store those character in database. and match them when they get..
I'd like to localize an iOS and Android app for over 25 different languages using a custom font. The problem is no new fonts cover that type of ground. What's the current best practice for this problem in app development?
I've only come up with the following 2 solutions, however I'm unsure either are possible or a good idea.
1.) Hire a font designer to create a massive custom font across at least 3 different weights (regular, bold, italic). But that could be extremely costly considering the app license for some single-weight simplified Chinese fonts are 5k alone.
2.) Use a custom font that covers about 10 languages thanks to Latin characters (e.g. Proxima Nova) and then similar-looking fonts for unsupported languages.
It seems to me the current best practice is to use a custom font that covers a bulk of Latin-based languages and all unsupported languages fallback to the device fonts. But I've experienced problems there as well particularly with localizing dynamic third-party data from Facebook connect. If I'm in America and my friend in China has Chinese characters in their username a custom font outputs little square glyphs instead of falling back on the device's Chinese character set.
In any case both solutions add quite a bit of file size to the app which itself could be a deal-breaker. For solution 2 I've also considered using static images instead of embedding additional fonts, but that also presents a problem in localizing dynamic third-party data and creates a ton of work if the app should ever need updated.
Any suggestions would be greatly appreciated. Thanks.
Background
TextView always had issues with RTL (Right-To-Left) languages. Since I know only how to read Hebrew (in addition to English), I will talk about its issues:
Text alignment (and I'm not talking about gravity) . As an RTL language, Hebrew puts words from right to left (compared to English which is the opposite).
For demonstrating how annoying it is, imagine that instead of showing "Hello world." you usually get ".Hello world" . This could be easily fixed if you had it in a single sentence, but it's harder when there are multiple punctuations characters.
Vowels positions. Hebrew doesn't require vowels in order to read text, but sometimes it's very hard to read without them (especially the bible). For vowels, Hebrew has what is called "NIKUD", which are actually like dots inside the letters. The problem in Android was that they were usually positioned in the wrong location .
For demonstrating how annoying it is, imagine that instead of showing "Hello world." you usually get ".eHlol owrld" . Even if you try to fix it (put the vowels always one character after the current one), the position in the letter wasn't correct (imagine that the "e" in "Hello" would be like above the "H", for example) .
Only on version 4.2 (read here, under "Native RTL support") , Google has fixed all of the Hebrew related issues (or at least it seems so).
The problem
the problems with Hebrew has caused each Israeli carrier and each custom ROM maker have its own solution of how to fix the different issues, which makes it practically impossible to handle RTL text on pre 4.2 devices.
Things can get even more frustrating in case the text include both Hebrew and English letters.
What I've tried
I've read many websites talking about those problems, and I've tried many variants of the solutions, none has solved the problem on all devices:
Some suggest to put the character '\u200F' (or '\u202D') at the end/start/both of the text.
Some suggest using Html.fromHtml() method and put something special there.
Some even suggest to use the WebView instead (and maybe use WebSettings.setDefaultTextEncodingName() ).
The question
Is there a definite solution for this problem?
I would assume the best thing is that because Android 4.2 solves this, and Android is open source, we should have its TextView imported into a library that we can use, but Google hasn't provided such a library yet.
Sadly, I don't think there's a good solution ("good" meaning "does the job and is readily available"). For our own Android apps that support Hebrew, we use a custom rendering mechanism that we developed over many years. The rendering mechanism does everything: proprietary fonts; bidirectional (bidi) analysis; glyph placement; line break analysis; text flow; etc. Some of the problems trying to use native Android text handling capabilities (especially pre-4.2) are:
Really crappy fonts. However, you can package third-party fonts like DejaVu that are pretty good. The right font can do wonders with positioning of nekudot—and te'amim1, if you need that. (I agree with you about how important correct pointing placement is; reading Hebrew text with misplaced nekudot is like reading a screen-full of CAPTCHAs.)
Buggy bidi analysis. What makes it worse is that the bugs seem to be different for different versions of Android. Modifying the text to include strategically placed bidi formatting codes (RTL mark; LTR mark; etc.) can overcome many of these bugs (see the discussion here, which isn't specific to Android). However, it's a nuisance to do this and, because of the inconsistencies among Android versions, it is difficult to predict in advance what help the framework is going to need.
No (or poorly thought out) framework-level awareness of right-to-left issues. For instance, good luck getting the scroll bar to display on the left side of a Hebrew TextView. For our apps, we had to build an entire scroll-bar system just to get this to work how we wanted. (Good think Android is open source!)
Poor line and word break analysis. At least one early version of Android on which we tested thought that each nikud mark was a word boundary. When it comes to line breaks, the system often doesn't know how to handle Hebrew punctuation like maqaf, gershayim, or sof pasuk.
Some of the newer Unicode characters (like HOLAM HASER FOR VAV—U+05BA—new to Unicode 5.0) are not recognized as Hebrew script by the system.
My recommendation is that, unless you are prepared to build a top-to-bottom text handling system yourself, you give up on high-quality text display on pre-4.2 versions of Android, particularly if you need to support nekudot and te'amim. Also, plan to use the techniques I mentioned in the first two points above.
1 biblical cantillation marks
As of August 2013, Android has posted API documentation for a Bi Directional Formatter which might suit your needs. This is contained in the Android Support v4 library which I believe should run in versions prior to Android 4.2.
Refer to:
http://developer.android.com/reference/android/support/v4/text/BidiFormatter.html
NB: My whole senario is for only android version ICS.
My Goal is to render text having complex script/indic script. In ICS, this feature has been added in WebView (and so Browser). If any indic text is rendered in Browser or WebView it renders correctly. But in other widgets (TextView,EditText) it renders broken.
Now my goal is to re-use the code for properly drawing indic text in webcore and use that code to draw indic text in a custom widget.
I also tested using HTML5 canvas in browser and it can render indic text fine.
I have tested android's android.graphics.Canvas and it failed to render indic text properly.
libskia seriously lacks of documentation so i cant work out how to render text using this.
I checked the objdump of libwebcore.so and saw that it depends on lib "skia" "libicu". So i'm assuming i can draw indic text using these libraries.
Can anyone suggest how can i draw indic text using skia and icu ? or Can anyone point to specific code segment in libwebcore ?
There are a couple of alternate text stacks that are in common use, but neither is directly exposed through the Android NDK. At the base level, you need to:
Break your text into runs of a single script, in a single font, in a
single direction, on a single line.
Convert the sequence of characters in each run into a sequence of
glyphs.
Lay out those glyphs in 2D space, potentially going back to step (1)
if your linebreak didn't work out.
Rasterize the glyphs in the chosen positions.
The most popular stack for this is approximately (1) fribidi, (2) harfbuzz-ng, (3) more harfbuzz-ng, (4) libfreetype. All of these projects are pretty easy to get running on Android; documentation quality varies.
There's also the Pango project, which sits on top of the stack just described, but provides a higher level interface. Unfortunately, Pango has quite a few dependencies, such that it might be more effort to incorporate Pango than to just use the stack directly. If nothing else, Pango serves as great documentation for how to use the stack.
As an alternate stack, the various parts of libicu can replace fribidi, harfbuzz-ng, and Pango (the last with the nearly-undocumented ParagraphLayout class); you'll still need libfreetype to actually deal with fonts and rasterization. Be aware that parts of libicu are being moved over to harbuzz-ng, so these stacks are not completely distinct; and while parts of libicu are both excellent and updated regularly, other parts are a but fuzzier.