Develop Reference

special characters in Json feed getting inserted as unknown characters in sqllite database

In my Android app, Json feed may have special characters of French, German and Spanish language like "Atlético Madrid vs Málaga". Feed works fine when i see them in browser but sqllite shows unknown characters when the same data is being inserted. On the recommendations by experts on StackOverFlow in other posts, I made these changes in HttpConnections as well as in InputStream to support UTF-8 but it's not working. I was told that sqllite supports UTF-8 on its own and no config changes are required. What's wrong i am doing ? Please help. Thanks.
InputStream isC = null;
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestProperty("Accept-Charset", "UTF-8");
conn.setConnectTimeout(10000);
conn.setRequestMethod("GET");
conn.setDoInput(true);
conn.setRequestProperty("Content-type", "application/json;charset=UTF-8");
conn.connect();
int response = conn.getResponseCode();
isC = conn.getInputStream();
JsonReader jsonReader = new JsonReader(new InputStreamReader(isC, "UTF-8"));
jsonReader.setLenient(true);
jsonReader.beginObject();

you're decoding as UTF-8, which supports ASCII charachers only.
If a non-ascii character (á, ó, ú, í, é) is encountered, you'll get weird behaviour.
also, make sure the column you're saving to in the database supports Unicode - in MsSql, it'd need to be a nvarchar field. Not sure what the SQLite equivalent is...

Android ByteArrayOutputStream corrupting HTTP GET JSONArray

I'm using this code to parse a JSON array I'm getting from my server.
try {
URL u = new URL("http://54.68.139.250/get_user_likes");
HttpURLConnection conn = (HttpURLConnection) u.openConnection();
conn.setRequestMethod("GET");
conn.connect();
InputStream is = conn.getInputStream();
byte[] b = new byte[1024];
ByteArrayOutputStream baos = new ByteArrayOutputStream();
while ( is.read(b) != -1)
baos.write(b);
String JSONResp = new String(baos.toByteArray());
JSONArray arr = new JSONArray(JSONResp);
for (int i=0; i < arr.length(); i++) {
result.add(convertArticle(arr.getJSONObject(i)));
}
return result;
}
catch(Throwable t) {
t.printStackTrace();
}
return null;
This code works great on my phone. Unfortunately, when I'm using a Genymotion emulator with the virtual device of Google Nexus 7, the JSON array is slightly altered. 95% of the JSON array is fine, but it is truncated near the very end and is randomly missing about 4 characters of the json array at character 1253 so I'm getting:
org.json.JSONException: Expected ':' after top_id at character 1253 of [{"top_id":6,"top_url":
I'm thinking this is some memory problem with the emulator. Its base memory is 1024. Increasing that amount though doesn't change anything.
Any tips as to the reason behind the problem would be greatly appreciated. Also, feel free to comment on my code if you see room for improvement. :)

That's weird.
I can think of two things to try:
Check the encoding of the server and the encoding of the String constructor.
It's possible that the server is decoding with, say, Latin-1 (ISO-8859-1) and the String is encoding with UTF-8. But JSON is supposed to be sent in Unicode and the default charset for Android is UTF-8. Check the HTTP Content-type response header; it should say: application/json; charset=utf-8. If the charset part isn't there, do some investigation and find out what character set your server is using for decoding to HTTP stream. And check that the default charset on the phone and the emulator is the same; it should be UTF-8.
Try calling flush() on the ByteArrayOutputStream after you read the data and before you construct the String.
It may be that there is a slight difference between the phone OS and the emulator OS on how stream data is transferred/buffered/flushed.
If flush() doesn't work, try rewriting the code without using ByteArrayOutputStream. You could, for example, wrap the input stream with an InputStreamReader, which reads characters, not bytes, then append the characters using a StringBuilder or StringBuffer.
One way you could make the code better is to use JSONReader instead of JSONArray or JSONObject. The JSONReader would wrap an InputStreamReader which in turn wraps the HTTP input stream. It can be faster and more memory efficient since you don't have to read the entire input stream before starting to parse the data. When the server is sending a LOT of JSON data, that can make a big difference.

You should check the return value of is.read(). Change
while ( is.read(b) != -1)
baos.write(b);
to
int nread;
while ( (nread=is.read(b)) != -1)
baos.write(b, 0, nread);

Convert a unicode string for a webservice - equivalent to php utf8_encode in Android

My Android app needs to connect to a webservice that will decode a unicode string using utf8_decode. How can I encode my string in my application in a similar way as php utf8_encode?
I have found CharsetEncoder but am not sure how to use it.
Thanks for your advice!

You can do this using just the String class, but just a note about converting text to/from UTF and ISO. When decoding from UTF-8 to ISO-8859-1 will cause "replacement characters" (�) to appear in your text when unsupported characters are found.
To encode texts:
byte[] utf8 = new String(latin1, "ISO-8859-1").getBytes("UTF-8");
or
byte[] latin1 = new String(utf8, "UTF-8").getBytes("ISO-8859-1");

How to import text from a webpage into a textview?

How do I get text from a basic HTML page and show it in a TextView.
I want to do it this way because it will look better than having a webview showing the text.
There is only one line on the html page. I can change it to a txt file if needed.
Could you also provide a quick example?

You would need to download the HTML first using something like HttpClient to retrieve the data from the Internet (assuming you need to get it from the Internet and not a local file). Once you've done that, you can either display the HTML in a WebView, like you said, or, if the HTML is not complex and contains nothing other than some basic tags (<a>, <img>, <strong>, <em>, <br>, <p>, etc), you can pass it straight to the TextView since it supports some basic HTML display.
To do this, you simply call Html.fromHtml, and pass it your downloaded HTML string. For example:
TextView tv = (TextView) findViewById(R.id.MyTextview);
tv.setText(Html.fromHtml(myHtmlString));
The fromHtml method will parse the HTML and apply some basic formatting, returning a Spannable object which can then be passed straight to TextView's setText method. It even supports links and image tags (for images, though, you'll need to implement an ImageGetter to actually provide the respective Drawables). But I don't believe it supports CSS or inline styles.
How to download the HTML:
myHtmlString in the snippet above needs to contain the actual HTML markup, which of course you must obtain from somewhere. You can do this using HttpClient.
private String getHtml(String url)
{
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet(url);
try
{
HttpResponse response = client.execute(request);
BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
String line;
StringBuilder builder = new StringBuilder();
while((line = reader.readLine()) != null) {
builder.append(line + '\n');
}
return builder.toString();
}
catch(Exception e)
{
//Handle exception (no data connectivity, 404, etc)
return "Error: " + e.toString();
}
}
It's not enough to just use that code, however, since it should really be done on a separate thread (in fact, Android might flat out refuse to make a network connection on the UI thread. Take a look at AsyncTasks for more information on that. You can find some documentation here (scroll down a bit to "Using Asynctask").

Android WebView can't show copyright or trademark symbols?

When I display bullet-points, copyright symbols, trademark signs in a web browser, they
look fine.
// bullets: http://losangeles.craigslist.org/wst/acc/2900906683.html
// bullets: http://losangeles.craigslist.org/lac/acc/2902059059.html
// bullets: http://indianapolis.craigslist.org/acc/2867115357.html
// bullets: http://indianapolis.craigslist.org/ofc/2885697780.html
// bullets: http://indianapolis.craigslist.org/ofc/2887554512.html
// copyright: http://chicago.craigslist.org/nwc/acc/2854640931.html
But I get "question marks inside triangles" when I use an Android WebView with:
web.loadDataWithBaseURL(null, myHtml, null, "UTF-8", null);
Should I be using a different encoding?
Should I be searching/replacing certain characters myself... 1-by-1?

Try using WebView settings
myWebView = (WebView)findViewById(R.id.mywebView);
WebSettings settings = myWebView.getSettings();
settings.setDefaultTextEncodingName("UTF-8");

I've run into this problem before. I would make sure that your myHtml String already has good encoding before you load it into your WebView. You can check that by logging it using Log.d(). If the encoding is wrong in that String, that it won't show properly in WebView either. You'll see those weird characters in LogCat.
If that is the case, you'll want to make sure that when you're reading the data into your myHtml String, that you use something like an InputStreamReader and pass it "UTF-8" as the character encoding.
I would change the line of code that you're using from:
BufferedReader buffer = new BufferedReader(new InputStreamReader(content), 1000);
to:
BufferedReader buffer = new BufferedReader(new InputStreamReader(content, "UTF-8"), 1000);
This version of the constructor is documented to:
Constructs a new InputStreamReader on the InputStream in. The character converter that is used to decode bytes into characters is identified by name by enc. If the encoding cannot be found, an UnsupportedEncodingException error is thrown.
at http://developer.android.com/reference/java/io/InputStreamReader.html and look at the second one.
EDIT: If that doesn't work, you could try using:
String s = EntityUtils.toString(entity, HTTP.UTF_8);
which is from Android Java UTF-8 HttpClient Problem