Develop Reference

How to download String file which contain special characters of slovenia

I am trying to download the json file which contains slovenian characters,While downloading json file as a string I am getting special character as specified below in json data
"send_mail": "Po�lji elektronsko sporocilo.",
"str_comments_likes": "Komentarji, v�ecki in mejniki",
Code which I am using
URL url = new URL(f_url[0]);
URLConnection conection = url.openConnection();
conection.connect();
try {
InputStream input1 = new BufferedInputStream(url.openStream(), 300);
String myData = "";
BufferedReader r = new BufferedReader(new InputStreamReader(input1));
StringBuilder totalValue = new StringBuilder();
String line;
while ((line = r.readLine()) != null) {
totalValue.append(line).append('\n');
}
input1.close();
String value = totalValue.toString();
Log.v("To Check Problem from http paramers", value);
} catch (Exception e) {
Log.v("Exception Character Isssue", "" + e.getMessage());
}
I want to know how to get characters downloaded properly.

You need to encode string bytes to UTF-8. Please check following code :
String slovenianJSON = new String(value.getBytes([Original Code]),"utf-8");
JSONObject newJSON = new JSONObject(reconstitutedJSONString);
String javaStringValue = newJSON.getString("content");
I hope it will help you!

Decoding line in while loop can work. Also you should add your connection in try catch block in case of IOException
URL url = new URL(f_url[0]);
try {
URLConnection conection = url.openConnection();
conection.connect();
InputStream input1 = new BufferedInputStream(url.openStream(), 300);
String myData = "";
BufferedReader r = new BufferedReader(new InputStreamReader(input1));
StringBuilder totalValue = new StringBuilder();
String line;
while ((line = r.readLine()) != null) {
line = URLEncoder.encode(line, "UTF8");
totalValue.append(line).append('\n');
}
input1.close();
String value = totalValue.toString();
Log.v("To Check Problem from http paramers", value);
} catch (Exception e) {
Log.v("Exception Character Isssue", "" + e.getMessage());
}

It's not entirely clear why you're not using Android's JSONObject class (and related classes). You can try this, however:
String str = new String(value.getBytes("ISO-8859-1"), "UTF-8");
But you really should use the JSON libraries rather than parsing yourself

When creating the InputStreamReader at this line:
BufferedReader r = new BufferedReader(new InputStreamReader(input1));
send the charset to the constructor like this:
BufferedReader r = new BufferedReader(new InputStreamReader(input1), Charset.forName("UTF_8"));

problem is in character set
as per Wikipedia Slovene alphabet supported by UTF-8,UTF-16, ISO/IEC 8859-2 (Latin-2). find which character set used in server, and use the same character set for encoding.
if it is UTF-8 encode like this
BufferedReader bufferedReader= new BufferedReader(new InputStreamReader(inputStream), Charset.forName("UTF_8"));
if you had deffrent character set use that.

I have faced same issue because of the swedish characters.
So i have used BufferedReader to resolved this issue. I have converted the Response using StandardCharsets.ISO_8859_1 and use that response. Please find my answer as below.
BufferedReader r = new BufferedReader(new InputStreamReader(response.body().byteStream(), StandardCharsets.ISO_8859_1));
StringBuilder total = new StringBuilder();
String line;
while ((line = r.readLine()) != null)
{
total.append(line).append('\n');
}
and use this total.toString() and assigned this response to my class.
I have used Retrofit for calling web service.

I finally found this way which worked for me
InputStream input1 = new BufferedInputStream(conection.getInputStream(), 300);
BufferedReader r = new BufferedReader(new InputStreamReader(input1, "Windows-1252"));
I figured out by this windows-1252, by putting json file in asset folder of the android application folder, where it showed same special characters like specified above,there it showed auto suggestion options to change encoding to UTF-8,ISO-8859-1,ASCII and Windows-1252, So I changed to windows-1252, which worked in android studio which i replicated the same in our code, which worked.

Converting String to Android JSONObject loses utf-8

I am trying to get a (JSON formatted) String from a URL and consume it as a Json object. I lose UTF-8 encoding when I convert the String to JSONObject.
This is The function I use to connect to the url and get the string:
private static String getUrlContents(String theUrl) {
StringBuilder content = new StringBuilder();
try {
URL url = new URL(theUrl);
URLConnection urlConnection = url.openConnection();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
String line;
while ((line = bufferedReader.readLine()) != null) {
content.append(line + "\n");
}
bufferedReader.close();
} catch(Exception e) {
e.printStackTrace();
}
return content.toString();
}
When I get data from server, the following code displays correct characters:
String output = getUrlContents(url);
Log.i("message1", output);
But when I convert the output string to JSONObject the Persian characters becomes question marks like this ??????. (messages is the name of array in JSON)
JSONObject reader = new JSONObject(output);
String messages = new String(reader.getString("messages").getBytes("ISO-8859-1"), "UTF-8");
Log.i("message2", messages);

You're telling Java to convert the string (with key message) to bytes using ISO-8859-1 and than to create a new String from these bytes, interpreted as UTF-8.
new String(reader.getString("messages").getBytes("ISO-8859-1"), "UTF-8");
You could simply use:
String messages = reader.getString("messages");

You can update your code as the following:
private static String getUrlContents(String theUrl) {
StringBuilder content = new StringBuilder();
try {
URL url = new URL(theUrl);
URLConnection urlConnection = url.openConnection();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(urlConnection.getInputStream(), "utf-8"));
String line;
while ((line = bufferedReader.readLine()) != null) {
content.append(line).append("\n");
}
bufferedReader.close();
} catch(Exception e) {
e.printStackTrace();
}
return content.toString().trim();
}

You've got two encoding issues:
The server sends text encoded in a character set. When you setup your InputStreamReader, you need to pass the encoding the server used so it can be decoded properly. The character encoding is usually given in the Content-type HTTP response, in the charset field. JSON is typically UTF-8 encoded, but can also be legally UTF-16 and UTF-32, so you need to check. Without a specified encoding, your system environment will be used when marshalling bytes to Strings, and vice versa . Basically, you should always specify the charset.
String messages = new String(reader.getString("messages").getBytes("ISO-8859-1"), "UTF-8"); is obviously going to cause issues (if you have non-ascii characters) - it's encoding the string to ISO-8995-1 and then trying to decode it as UTF-8.
A simple regex pattern can be used to extract the charset value from the Content-type header before reading the inputstream. I've also included a neat InputStream -> String converter.
private static String getUrlContents(String theUrl) {
try {
URL url = new URL(theUrl);
URLConnection urlConnection = url.openConnection();
InputStream is = urlConnection.getInputStream();
// Get charset field from Content-Type header
String contentType = urlConnection.getContentType();
// matches value in key / value pair
Pattern encodingPattern = Pattern.compile(".*charset\\s*=\\s*([\\w-]+).*");
Matcher encodingMatcher = encodingPattern.matcher(contentType);
// set charsetString to match value if charset is given, else default to UTF-8
String charsetString = encodingMatcher.matches() ? encodingMatcher.group(1) : "UTF-8";
// Quick way to read from InputStream.
// \A is a boundary match for beginning of the input
return new Scanner(is, charsetString).useDelimiter("\\A").next();
} catch(Exception e) {
e.printStackTrace();
}
return null;
}

Not sure if this will help, but you might be able to do something like this:
JSONObject result = null;
String str = null;
try
{
str = new String(output, "UTF-8");
result = (JSONObject) new JSONTokener(str).nextValue();
}
catch (Exception e) {}
String messages = result.getString("messages");

How to handle 500 Internal Server Error when using JSON

I am relatively new to Android and I am using JSON to get data from a server. On the first loop at line 22, the StringBuilder contains, 500 Internal Server Error and then the jArray ends up coming back null. How can I handle this error?
public static JSONObject getJSON() {
String jsonString = "";
InputStream inStream = null;
//http post
JSONObject jArray = null;
try {
HttpClient httpClient = new DefaultHttpClient(new BasicHttpParams());
HttpPost httpPost = new HttpPost(WS_URL);
httpPost.setHeader("Content-type", "application/json");
HttpResponse response = httpClient.execute(httpPost);
HttpEntity entity = response.getEntity();
inStream = entity.getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(inStream, "UTF-8"), 8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
inStream.close();
jsonString = sb.toString();
jArray = new JSONObject(jsonString);
//outputTransactions(jArray);
} catch (IOException e) {
e.printStackTrace();
} catch (JSONException e) {
e.printStackTrace();
}
return jArray;
}

Though its a late reply but it might help others. You need to check the response status from the server before parsing it as JSON.
For ex.
int status_code=response.getStatusLine().getStatusCode();
if(status_code!=200){
Log.d("MYLOG","ERROR! Response status is"+status_code);
}
else{
inStream = entity.getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(inStream, "UTF-8"), 8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
inStream.close();
// Rest of your code......
}
or Optionally you can check the status code and display the error to the user
Like:
else if(status_code==404){
Log.d("MYLOG","Sorry! Page not found! Check the URL ");
}else if(status_code==500){
Log.d("MYLOG","Server is not responding! Sorry try again later..");
}
Hope it helps for newbies like you :-)

A "500 Internal Server" error means the server had a problem responding to your request. You are not getting a JSON string response.
Then when you try to create your jArray, the string is not valid JSON and the JSONObject cannot parse it - it returns "null" as you say.
You can parse the server response to see if it contains this string, and then create whatever jArray object you want, but you can't get a JSON object from a non-JSON string.

Take a look at this answer: https://stackoverflow.com/a/8148785/1974614
You should check the statusCode against 500

You should consider use a library to handle the REST requests like: http://square.github.io/retrofit/
If you use a library like that you can get an object from json when a success response is available and other when an error occur.
MyApi mylogin = restAdapter.create(MyApi.class); //this is how retrofit create your api
mylogin.login(username,password,new Callback<String>() {
#Override
public void success(String s, Response response) {
//process your response if login successfull you can call Intent and launch your main activity
}
#Override
public void failure(RetrofitError retrofitError) {
retrofitError.printStackTrace(); //to see if you have errors
}
});
}

I got the same problem like you and I solved it because I missed a part while adding GSON .jar files to adding my serverside project. I think you should carrefully add external libraries to your project too like me.With these links I could aware of problem .
LINK 1
LINK 2

Encoding Issue with HttpUrlConnection in Android

I want to send an XML message to a server from my Android Mobile app via HTTP post.
I tried it with HttpUrlConnection, following these steps:
URL url = new URL(vURL);
HttpUrlConnection conn = (HttpURLConnection) url.openConnection();
conn.setDoInput(true);
conn.setDoOutput(true);
// Adding headers (code removed)
conn.setRequestProperty("Content-Type", "text/xml; charset=utf-16");
OutputStream out = new BufferedOutputStream(conn.getOutputStream());
// Adding XML message to the connection output stream
// I have removed exception handling to improve readability for posting it here
out.write(pReq.getBytes()); // here pReq is the XML message in String
out.close();
conn.connect();
Once I get the response, the stream reading part is in done this manner:
BufferedReader in = null;
StringBuffer sb;
String result = null;
try {
InputStreamReader isr = new InputStreamReader(is);
// Just in case, I've also tried:
// new InputStreamReader(is, "UTF-16");
// new InputStreamReader(is, "UTF-16LE");
// new InputStreamReader(is, "UTF-16BE");
// new InputStreamReader(is, "UTF-8");
in = new BufferedReader(isr);
sb = new StringBuffer("");
String line = "";
while ((line = in.readLine()) != null)
sb.append(line);
in.close();
result = sb.toString();
} catch (Exception e) {
e.printStackTrace();
}
Now the result string I get is in some unreadable format/encoding.
When I try the same thing with HttpClient it works correctly. Here is the streaming reading part once I get an HttpResponse after the HttpClient.execute call:
BufferedReader in = null;
InputStream is;
StringBuffer sb;
String decompbuff = null;
try {
is = pResponse.getEntity().getContent();
InputStreamReader isr = new InputStreamReader(is);
in = new BufferedReader(isr);
// Prepare the String buffer
sb = new StringBuffer("");
String line = "";
while ((line = in.readLine()) != null)
sb.append(line);
in.close();
// gZip decompression of response. Note: message was compressed before
// posting it via HttpClient (Posting code is not mentioned here)
decompbuff = Decompress(sb.toString());
} catch (Exception e) {
e.printStackTrace();
}
return decompbuff;
Some help is appreciated in understanding the problem.

One (severe) problem could be that you're ignoring the encoding of input and output.
Input
conn.setRequestProperty("Content-Type", "text/xml; charset=utf-16");
OutputStream out = new BufferedOutputStream(conn.getOutputStream());
// Adding XML message to the connection output stream
// I have removed exception handling to improve readability for posting it here
out.write(pReq.getBytes()); // <-- you use standard platform encoding
out.close();
better:
out.write(pReq.getBytes("UTF-16"));
Output
You probably ignored compression, which would better look like this (taken from DavidWebb):
static InputStream wrapStream(String contentEncoding, InputStream inputStream)
throws IOException {
if (contentEncoding == null || "identity".equalsIgnoreCase(contentEncoding)) {
return inputStream;
}
if ("gzip".equalsIgnoreCase(contentEncoding)) {
return new GZIPInputStream(inputStream);
}
if ("deflate".equalsIgnoreCase(contentEncoding)) {
return new InflaterInputStream(inputStream, new Inflater(false), 512);
}
throw new RuntimeException("unsupported content-encoding: " + contentEncoding);
}
// ...
InputStream is = wrapStream(conn.getContentEncoding(), is);
InputStreamReader isr = new InputStreamReader(is, "UTF-16");
in = new BufferedReader(isr);
sb = new StringBuffer("");
String line = "";
while ((line = in.readLine()) != null)
sb.append(line); // <-- you're swallowing linefeeds!
in.close();
result = sb.toString();
It would be better to let the XML-Parser consume your InputStream directly. Don't create a JAVA string, but let the parser scan the bytes. It will automatically detect the encoding of the XML.
Generally there might be still an issue, because we don't know what type of UTF-16 you use. Can be BigEndian or LittleEndian. That's why I asked, if you really need UTF-16. If you don't have to treat with some asian languages, UTF-8 should be more efficient and easier to use.
So the "solution" I gave you is not guaranteed to work - you have to fiddle with UTF-16 BE/LE a bit and I wish you good luck and patience.
Another remark: in your example above you first construct the String and then Decompress it. That is the wrong order. The stream comes compressed (gzip, deflate) and must be decompressed first. Then you get the String.

HttpUrlConnection.getInputStream returns empty stream in Android

I make a GET request to a server using HttpUrlConnection.
After connecting:
I get response code: 200
I get response message: OK
I get input stream, no exception thrown but:
in a standalone program I get the body of the response, as expected:
{"name":"my name","birthday":"01/01/1970","id":"100002215110084"}
in a android activity, the stream is empty (available() == 0), and thus I can't get
any text out.
Any hint or trail to follow? Thanks.
EDIT: here it is the code
Please note: I use import java.net.HttpURLConnection; This is the standard
http Java library. I don't want to use any other external library. In fact
I did have problems in android using the library httpclient from apache (some of their anonymous .class can't be used by the apk compiler).
Well, the code:
URLConnection theConnection;
theConnection = new URL("www.example.com?query=value").openConnection();
theConnection.setRequestProperty("Accept-Charset", "UTF-8");
HttpURLConnection httpConn = (HttpURLConnection) theConnection;
int responseCode = httpConn.getResponseCode();
String responseMessage = httpConn.getResponseMessage();
InputStream is = null;
if (responseCode >= 400) {
is = httpConn.getErrorStream();
} else {
is = httpConn.getInputStream();
}
String resp = responseCode + "\n" + responseMessage + "\n>" + Util.streamToString(is) + "<\n";
return resp;
I see:
200
OK
the body of the response
but only
200
OK
in android

Trying the code of Tomislav I've got the answer.
My function streamToString() used .available() to sense if there is any data received,
and it returns 0 in Android. Surely, I called it too soon.
If I rather use readLine():
class Util {
public static String streamToString(InputStream is) throws IOException {
StringBuilder sb = new StringBuilder();
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
String line;
while ((line = rd.readLine()) != null) {
sb.append(line);
}
return sb.toString();
}
}
then, it waits for the data to arrive.
Thanks.

You can try with this code that will return response in String:
public String ReadHttpResponse(String url){
StringBuilder sb= new StringBuilder();
HttpClient client= new DefaultHttpClient();
HttpGet httpget = new HttpGet(url);
try {
HttpResponse response = client.execute(httpget);
StatusLine sl = response.getStatusLine();
int sc = sl.getStatusCode();
if (sc==200)
{
HttpEntity ent = response.getEntity();
InputStream inpst = ent.getContent();
BufferedReader rd= new BufferedReader(new InputStreamReader(inpst));
String line;
while ((line=rd.readLine())!=null)
{
sb.append(line);
}
}
else
{
Log.e("log_tag","I didn't get the response!");
}
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return sb.toString();
}

The Stream data may not be ready, so you should check in a loop that the data in the stream is available before attempting to access it.
Once the data is ready, you should read it and store in another place like a byte array; a binary stream object is a nice choice to read data as a byte array. The reason that a byte array is a better choice is because the data may be binary data like an image file, etc.
InputStream is = httpConnection.getInputStream();
byte[] bytes = null;
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] temp = new byte[is.available()];
while (is.read(temp, 0, temp.length) != -1) {
baos.write(temp);
temp = new byte[is.available()];
}
bytes = baos.toByteArray();
In the above code, bytes is the response as byte array. You can convert it to string if it is text data, for example data as utf-8 encoded text:
String text = new String(bytes, Charset.forName("utf-8"));