I have a client / server app written using Android and I'm using the standard org.json package classes bundled with android to do the parsing and creating.
I've been getting weird characters appearing on the server side right in the middle of the json strings generated for example (not the full one, because its big):
{!lo":"es_MX","id":2791884,"os":"8"}
As you can see the (!) exclamation mark appears randomly instead of a double quote. I also get other random characters appearing mid string. It's very bizarre.
Here is the code which creates the JSON object...
JSONObject jsonObject = new JSONObject();
jsonObject.put("key", someValue);
Here is the code which sends..
HttpPost type = new HttpPost(<server url here>);
List<NameValuePair> params = new ArrayList<NameValuePair>();
params.add(new BasicNameValuePair("v", jsonObject.toString()));
type.setEntity(new UrlEncodedFormEntity(params, HTTP.UTF_8));
httpClient.execute(type); // This is a DefaultHttpClient
I say random, but the exclamation mark in this exact position is consistent in many errors, but not every-time. About 5 messages that get this error, among tens of thousands per day. And usually not the contents of the values inserted into the json, but the characters (such as the quote character above) that define the structure of the message, which suggests to me that this isn't a character set issue.
Has anyone come across this?
it seems you are composing string in other format, and on receiving text decode in another format
like iso to utf.
It looks like your sender is not properly setting the character set. Spanish will have symbols not present in regular ASCII or most Windows encodings, you need to use UTF-8:
Content-Type: text/html; charset=utf-8
Without knowing which HTTP exchange you're using (read more), it is not possible to give you an exact code snippet to fix the problem - but that should be easy enough to figure out.
You give not enough information. Radical method to fix your problem is just replace all (!) characters to (").
string.replaceAll("!", "\"");
I guess it is server side issue.
I had also simmilar problem. Let me write much more to describe my environment. My server was returning data in json format. But my problem was connected with special chars like ąść. YOu should know, json_encode() will return from server in this case string text as a null.
I know, it sucks! So I added mysql_query('SET CHARACTER SET utf8'); before my selction for items from database. This allowed me to take strings from server with special diacritics letters.
Now on the app site, I was taking data from server by GET method. First I was storing result data into InputStream. Then I was packing it into InputStreamReader and byte by byte I was appending it into stringBuilder. That's ready appended text was converting by toString() ready string. Then I was putting it to new JsonArray(readyString). However I discovered some parts of text for json had weird chars.. Especially in that places where were special letters like żóć. For example "description":"aaa" was throwing "descriptionPffa":"aa"null:`.
I decided to try another way for converting result from data. In places where I was converting data from server I used method below. At the end, wgen I got byteArrayOutputStream object I changed it to new String(byteArray) and then somehow it worked with new JsonArray(new String(byteArray))!
public class Streams {
public static byte[] getBytes(InputStream is) {
int len;
int size = 1024;
byte[] buf = new byte[0];
try {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
buf = new byte[size];
while ((len = is.read(buf, 0, size)) != -1)
bos.write(buf, 0, len);
buf = bos.toByteArray();
} catch (IOException e) {
e.printStackTrace();
}
return buf;
}
}
Print the json on the client (using Log.d or similar) and see if it contains weird characters before sending it to the server.
Related
I have a problem when try to use hashtable with large data. I have a text file that contains over 111000 record, and when it reach 75000, outofmemory exception was thrown, so does anyone have any solution for this?
The problem happen at the line :
while ((text = reader.readLine())!= null);
and because of the java.lang.String <init> inside the readLine(), but I think the problem come from the hashtable too big to keep store data. I tested it on my xperia Neo and it failed although it run quite good on another samsung device.
The object SoundUnit which I keep on hashtable has a structure below:
private String filename;
private int start;
private int end;
just above 3 field inside SoundUnit object.
Here is the piece of code I use to read data from text to store on hashtable:
reader = new BufferedReader(new InputStreamReader(new FileInputStream(unitSelectionFile), "UTF-8"));
String text=null;
do {
if (text.length() != 0) {
if (mainHash.get(text)==null)
mainHash.put(key, soundUnit);
}
} while ((text = reader.readLine())!= null);
Yes, most likely the hashtable is simply to large. I did a test on a Galaxy Nexus, which is a rather new phone, and it ran out of memory after 230k entries, with unique keys and values about 8 chars long. I would suggest to put your data in a database instead.
Your reader should not be an issue, since BufferedReader only uses a small amount of memory.
You should get the InputStream from the response and pass it to any xml handler for parsing it. This should do the trick because by doing this your xml will be parsing dynamically as you get the inputstream.
First, I have researched this question a lot. I learned how to read from my text file in the assets folder. The problem is that I don't want to end up with a string because the file is actually an arraylist written using json and inside that arraylist are objects. I need to access one object only. Here's what I mean:
I have a Book class, which has an int called chapter, an int called title, and an int called pageNum. I created several Book objects and added them to an ArrayList. Then I used the following code to write my ArrayList to a text file (in a regular java project):
ArrayList<Book> aList = new ArrayList<Book>();
aList.add(book1);
//etc...add more Book objects here...
File aFile = new File("books.txt");
FileOutputStream aFileStream = new FileOutputStream(aFile);
JSONOutputStream jsonOut = new JSONOutputStream(aFileStream);
jsonOut.writeObject(aList);
jsonOut.close();
That code creates a text file which I then put into the /assets folder in my Android project because I want it included with the app. In a non-Android java project I could simply use the following code to repopulate an ArrayList so that I could parse Book obects from specific indexes:
File bFile = new File("books.txt");
FileInputStream bFileStream = new FileInputStream(bFile);
JSONInputStream jsonIn = new JSONInputStream(bFileStream);
Arraylist<Book> bList = (ArrayList<Book>) jsonIn.readObject();
Book aBook = bList.get(253); //some arbitrary index
The json code I'm using comes from quickconnectfamily.org. You have to add a file called qc_json.jar to the build path of your project. http://www.quickconnectfamily.org/qcjson/more.html
The problem in Android is when I read the file using InputStream, I can only get the entire file into a string, the code above doesn't work in Android. I can't wrap JSONInputStreams around an InputStream, only around a FileInputStream. But it seems I am unable to use
FileInputStream.
So what I need is a way to create an ArrayList rather than a string in my Android app.
Without giving away too much about my app, the app basically generates a random number and creates a Book object from that index in the ArrayList. Then the user is quizzed with info from that specific book. Sounds silly, but the real app is much cooler.
I'm open to solutions, alternative methods of storing objects in a text file, etc. Please don't simply post criticism about my grammar, syntax, or application idea. I'm pretty new to app development and I couldn't care less about personal opinions. If anyone wants to see
more code I can upload it, but it doesn't seem necessary at this point. Thanks.
I figured out the solution. I connected my Evo 4G and tested the app and it worked. Here's what I did:
I used the following method to read from the file, which is what I was doing before:
InputStream is = appContext.getAssets().open("books.txt");
int size = is.available();
buffer = new byte[size];
is.read(buffer);
is.close();
String bufferString = new String(buffer);
When you do this, you end up with a String of the entire file. This is what I was able to do before, but what I wanted was a way to convert the String to an ArrayList of Book objects. Here's how I accomplished this:
//convert string to JSONArray
jsonArray = new JSONArray(bufferString);
//parse an Object from a random index in the JSONArray
JSONObject anObject = jsonArray.getJSONObject(randomNum);
Book aBook = new Book();
aBook.setTitle((String) anObject.get("title"));
//you can continue to set the different attributes of the Book object using key/value pairs from the Book class (e.g. setPageNum, setChapter, etc).
I don't know, maybe that was obvious to some people, but I really couldn't find any examples that did this. In a different question someone mentioned using the json library native to Android org.json and so I tried that and it worked.
If you check my previous questions, you will see that they are all in some way related to "\" or "/" for Android and why my implementations of code wasn't working when other people's versions were.
I now know why mine wasn't working.
I am developing for a live client who has access to a content management system, from which I am getting the data. Other than the general checks, they can post anything they want to the site.
They are posting sizes in inches; e.g. 5-1/2
It is this, and this alone, which is screwing up my Restful json.
For example, 1 eigth has become
1\\\/8
Currently, I am doing a string rewrite at the WCF point to catch these 'fractions' and turn them into decimal just so I can continue development. But I can't code for every eventuality and Android/Eclipse fails at JSONArray json=new JSONArray(result);
Would appreciate any input on this.
Dave
On reflection, and further investigation, it isn't the escaped fractions causing the problem.
It is something more fundamental.
Will close the question.
I have searched high and wide for an answer to this, and have finally found it.
I will share for anyone else experiencing the same issue:
It is the WCF Rest service.
Learning WCF and Android at the same time led me to believe that the response from WCF should be a String serialized in the Json format.
To do this, a .Net object, array or whatever would go through DataContractJsonSerializer before being returned as a String to Android for further parsing.
Something like this:
Dim stream1 As MemoryStream = New MemoryStream
Dim ser As DataContractJsonSerializer = New DataContractJsonSerializer(GetType(myType))
ser.WriteObject(stream1, myThing)
Dim _json As String = Encoding.UTF8.GetString(stream1.ToArray())
stream1.Close()
return _json
Wrong.
Keep your object, array or whatever and return that instead; WCF will take care of the proper escaping for you.
For example (this is VB);
IService:
<OperationContract()> _
<WebGet(BodyStyle:=WebMessageBodyStyle.WrappedRequest, RequestFormat:=WebMessageFormat.Json, ResponseFormat:=WebMessageFormat.Json, UriTemplate:="/MyKit/{AccountID}")> _
Function GetKit(ByVal AccountID As String) As MyKit
Service:
Public Function GetKit(ByVal AccountID As String) As MyKit Implements IService1.GetKit
Dim allKit As New MyKit() //Your object
objDal.CommandText = 'run some sql here - or whatever
Using dr As SqlDataReader = "blah"
//populate your object
End Using
Return allKit //return the object, not the string representation of it
End Function
Using DataContractJsonSerializer for sending as Json to Android from WCF effectively 'pre-escapes' the data. When it gets to Android, the Json parser is unable to handle it, because it also escapes the data.
I have a .Net WCF Rest service returning Json to be consumed by an Android app.
In debug, the WCF service correctly has the return value (Json) as:
{"BaseLoyaltyPoints":1480,"BonusLoyaltyPoints":0,"BrandId":1414, [etc...] }
Also in debug, when it returns to Notepad, the return value has changed to:
{\"BaseLoyaltyPoints\":1480,\"BonusLoyaltyPoints\":0,\"BrandId\":1414, [etc...] }
And when it gets to my Android app, it has become:
{\\"BaseLoyaltyPoints\\":1480,\\"BonusLoyaltyPoints\\":0,\\"BrandId\\":1414, [etc...] }
This is the boilerplate code I am using to serialize the Json:
Dim stream1 As MemoryStream = New MemoryStream
Dim ser As DataContractJsonSerializer = New DataContractJsonSerializer(GetType(FullProduct))
ser.WriteObject(stream1, Me)
Dim _json As String = Encoding.UTF8.GetString(stream1.ToArray())
stream1.Close()
The Android code to get the Json is:
HttpClient httpclient = new DefaultHttpClient();
HttpGet request = new HttpGet(getString(R.string.CONST_RestService) + "/json/Product/" + productID);
ResponseHandler<String> handler = new BasicResponseHandler();
result = httpclient.execute(request, handler);
jObject = new JSONObject(result);
What's going on?
Thanks
Dave
The JSON Data needs to be an evaluable JavaScript String - the " character needs to be escaped to \" (the character " as opposed to the String limitor "), and the \ in that expression needs to be escaped as well (because \ also is a special character). So it has been that string all along, it was just printed differently everytime.
Take this JavaScript for an example:
object = JSON.parse("{\"hello\":\"World\"}");
alert(object.hello);
The following is more of a guess than actual knowledge, because I do not know how exactly the classes you used behave, but I think that it's about right.
As you can see, the " characters need to be escaped to \", so your .NET JSON serializer does just that. Everything is fine, interpreting this as JS would work as expected.
Now what is probably going on is that your JSONObject constructor does not expect Strings to be already properly escaped, so it does that itself. To clarify: When you say "with one escape character", you probably mean something like this:
String workingJSONString = "{\"Hello\":\"World\"}"
right? The problem here is that Java has the same escaping rules as JavaScript - this is not what you get from your HTTPRequest, what you get is
String youGotThis = "{\\\"Hello\\\":\\\"World\\\"}"
Because there are literal backslashes in your String, and those need to be escaped as well. I am pretty certain that that is what is going on, and you'd probably either have to tell your .NET JSON serializer not to apply escape rules or find something that constructs a JSONObject from a properly escaped JSON string - or remove the unnecessary escapes yourself.
I hope this helped - but again, I am not fully certain so you should just check out if it actually behaves as I said.
I'm using JSON Framework to update the data of client side from server. I have 30 Pojo Classes and i received the Http Response from server. I create the Object of Reader by using the method
InputStream instream = entity.getContent();
reader = new InputStreamReader(instream, "UNICODE");
And i pass it in the Json Object like this
synchronizationResponse = gson.fromJson(reader, SynchronizationResponse.class);
But this line is giving outOfMemory Exception.
I write the response in a file and found the size of file is around 12MB.
So is their any way to split the response in multiple response.So that i can read and write simultaneously to avoid OOM exception.
Looking for Help
Difficult to say, as long as we do not know, what your SynchronisationResponse Class looks like.
Whatever your content, if you can split it into sub-sections, do so and serialize those.
You can use a JsonReader to parse a large stream of JSON without running out of memory. The details are in the documentation, but basically you create a JsonReader:
JsonReader reader = new JsonReader(new InputStreamReader(instream, "UNICODE"));
and write a recursive descent parser using reader.nextName() to get keys, reader.nextString(), reader.nextInt(), etc. to get values, and reader.beginObject(), reader.endObject(), reader.beginArray(), reader.endArray() to mark object boundaries.
This allows you to read and process one piece at a time, at any granularity, without ever holding the entire structure in memory. If the data is long-lived, you can store it in a database or a file.