I'm struggling with an issue while sending Json data to the server. I guess there is some issue with the bad characters which are not expected at start of UTF-8 format.
I used CharDecoder to replace all the malformed utf-8 characters and here is the code.
// Construct the Decoder
CharsetDecoder utf8Decoder = Charset.forName("UTF-8").newDecoder();
utf8Decoder.onMalformedInput(CodingErrorAction.REPLACE);
utf8Decoder.onUnmappableCharacter(CodingErrorAction.REPLACE);
// Configure to replace Malformed input with space
utf8Decoder.replaceWith(" ");
// Construct ByteBuffer
ByteBuffer byteBuff = ByteBuffer.wrap(text.getBytes());
try {
// Process the text.
CharBuffer parsed = utf8Decoder.decode(byteBuff);
return new String(parsed.array());
} catch (CharacterCodingException e) {
e.printStackTrace();
}
This is not helping me. When I look at the column line of Json post data where parser is complaining, it is a space character.
Json to post is
{"body":{"messageSegments":[{"type":"Text","text":"This is a link "},{"type":"Mention","id":"005GGGGGG02g6MMIAZ"},{"type":"Text","text":" ish"}]},"capabilities":{"questionAndAnswers":{"questionTitle":"https:\/\/www.google.co.nz\/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-16"}}}
Error is
[{"errorCode":"JSON_PARSER_ERROR","message":"Invalid UTF-8 start byte 0xa0 at [line:1, column:139]"}]
Any leads please.
Thanks,
Sree
0xa0 is a Unicode Character 'NO-BREAK SPACE' (U+00A0)
not the usual space 0x20
visually hard to notice the difference, but some frameworks do not like it.
Convert characters - the other characters and test it.
Helpful link http://www.ietf.org/rfc/rfc4627.txt
Related
I am working on my very first Android application, I am getting some Arabic messages in the JSON response from a web service in two different formats. when I display one of them get translated correctly but other get printed as it is in the encoded message.
Here is the first one:
\u0635\u0641\u0631 \u0627\u0644\u0645\u0638\u0641\u0631
This is converted to proper Arabic string as intended.
but
کامران
does not, I was expecting that its a UTF-8 encoded message, but I'm unable to convert it. can anyone help me to understand this encoded message?
Here is how I tried to convert but its unchanged:
public String decodeString(String encodedString) {
try {
return new String(encodedString.getBytes(), "UTF-8");
} catch(Exception e){
e.printStackTrace();
return encodedString;
}
}
any help is appreciated.
Thank you very much for your time and assistance in this matter.
When I check (کامران) message here: http://www.cafewebmaster.com/online_tools/utf8_decode I get the correct response.
Please also share some details on the encoding scheme i-e what is the difference between both encodings.
کامران Looks like the HTML encoding of the Unicode code points. You'll need to decode the HTML, using e.g Apache commons StringEscapeUtils.unescapeHtml().
Here's the gradle dependency for the library:
compile 'commons-lang:commons-lang:2.6'
Sorry for asking this question twice times, but I did research 2 hours...
I have problem with Base64 decode String in my Android.
For example:
encoded string: VgFzJ1+TrFa7WsXS5w==
The results in javascript and PHP: Vs'_¬V»ZÅÒç
The result in my Android: Vs'_��V�Z���
I found a solution from this page Android PHP Base64 decode with different results
They say
By this You convert the input String content to ISO-8859-1 encoded byte stream that will be decoded from base64.
This is my function decoder based on what they say:
byte[] b = Base64.decode(data.getBytes("ISO-8859-1"), Base64.DEFAULT);
return new String(b, "UTF-8");// event if remove UTF8
But still got the same error
Sorry, I have resolved by myself
It's very simple to get string from ISO-8859-1
return new String(b, "ISO-8859-1")
I'm using this code to parse a JSON array I'm getting from my server.
try {
URL u = new URL("http://54.68.139.250/get_user_likes");
HttpURLConnection conn = (HttpURLConnection) u.openConnection();
conn.setRequestMethod("GET");
conn.connect();
InputStream is = conn.getInputStream();
byte[] b = new byte[1024];
ByteArrayOutputStream baos = new ByteArrayOutputStream();
while ( is.read(b) != -1)
baos.write(b);
String JSONResp = new String(baos.toByteArray());
JSONArray arr = new JSONArray(JSONResp);
for (int i=0; i < arr.length(); i++) {
result.add(convertArticle(arr.getJSONObject(i)));
}
return result;
}
catch(Throwable t) {
t.printStackTrace();
}
return null;
This code works great on my phone. Unfortunately, when I'm using a Genymotion emulator with the virtual device of Google Nexus 7, the JSON array is slightly altered. 95% of the JSON array is fine, but it is truncated near the very end and is randomly missing about 4 characters of the json array at character 1253 so I'm getting:
org.json.JSONException: Expected ':' after top_id at character 1253 of [{"top_id":6,"top_url":
I'm thinking this is some memory problem with the emulator. Its base memory is 1024. Increasing that amount though doesn't change anything.
Any tips as to the reason behind the problem would be greatly appreciated. Also, feel free to comment on my code if you see room for improvement. :)
That's weird.
I can think of two things to try:
Check the encoding of the server and the encoding of the String constructor.
It's possible that the server is decoding with, say, Latin-1 (ISO-8859-1) and the String is encoding with UTF-8. But JSON is supposed to be sent in Unicode and the default charset for Android is UTF-8. Check the HTTP Content-type response header; it should say: application/json; charset=utf-8. If the charset part isn't there, do some investigation and find out what character set your server is using for decoding to HTTP stream. And check that the default charset on the phone and the emulator is the same; it should be UTF-8.
Try calling flush() on the ByteArrayOutputStream after you read the data and before you construct the String.
It may be that there is a slight difference between the phone OS and the emulator OS on how stream data is transferred/buffered/flushed.
If flush() doesn't work, try rewriting the code without using ByteArrayOutputStream. You could, for example, wrap the input stream with an InputStreamReader, which reads characters, not bytes, then append the characters using a StringBuilder or StringBuffer.
One way you could make the code better is to use JSONReader instead of JSONArray or JSONObject. The JSONReader would wrap an InputStreamReader which in turn wraps the HTTP input stream. It can be faster and more memory efficient since you don't have to read the entire input stream before starting to parse the data. When the server is sending a LOT of JSON data, that can make a big difference.
You should check the return value of is.read(). Change
while ( is.read(b) != -1)
baos.write(b);
to
int nread;
while ( (nread=is.read(b)) != -1)
baos.write(b, 0, nread);
My Android app needs to connect to a webservice that will decode a unicode string using utf8_decode. How can I encode my string in my application in a similar way as php utf8_encode?
I have found CharsetEncoder but am not sure how to use it.
Thanks for your advice!
You can do this using just the String class, but just a note about converting text to/from UTF and ISO. When decoding from UTF-8 to ISO-8859-1 will cause "replacement characters" (�) to appear in your text when unsupported characters are found.
To encode texts:
byte[] utf8 = new String(latin1, "ISO-8859-1").getBytes("UTF-8");
or
byte[] latin1 = new String(utf8, "UTF-8").getBytes("ISO-8859-1");
Hi um using both of these options one at a time to encode an image and send it over http post in android
String encodedImage = Base64.encodeToString(imageData, Base64.URL_SAFE + Base64.NO_WRAP);
String encodedImage = Base64.encodeToString(
imageData,Base64.DEFAULT);
It's convert the image in to encoded string sends it over http post.
In WCF side um trying to decode the image as follows and the encoded image string comes as in encoded format.
byte[] contents = Convert.FromBase64String(encodedImage.Trim());
After this it throws an exception saying
The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or a non-white space character among the padding characters.
what can possibly go wrong . I have a no clue of this. I will be thankful if anyone can guide me.