I am getting an OutOfMemoryException when processing a very long JSON response
returned by the code below:
HttpClient httpclient = new DefaultHttpClient();
HttpGet httpGETRequest = new HttpGet(url);
HttpResponse response = httpclient.execute(httpGETRequest);
return EntityUtils.toString(response.getEntity());
Even though I use System.gc() before I call the http request, the exception still occurs.
I tried to look for the cause of the problem by saving the response on a notepad and check its size, just to find out that it is a 5mb file.
What are the things that should be done in order to avoid the exception?
Thanks for your help in advance.
In this cause, we should avoid large data responses from server. I requested a pagination functionality from the server and problem solved.
I don't recommend that you use System.gc() on Android. What you would need to do is use streaming and avoid loading the whole content in memory if it can be avoided.
https://google-gson.googlecode.com/svn/trunk/gson/docs/javadocs/com/google/gson/JsonStreamParser.html
If you need to access parts of the JSON data more readily rather than doing it one chunk at at time, you can transform the data as it is being read and store it in a SQL database that you can query later.
http://developer.android.com/training/basics/data-storage/databases.html
However, if you have embedded binary data such as an image as part of the JSON object you can't do anything about it anymore because it will read the embedded binary data into memory as one chunk. There's no easy way around this aside from writing your own JSON streamer.
Generally
This general technique can be applied not just to Android, but even application servers dealing with large input data such as XML as well.
Calling System.GC() is not guaranteed to run GC. In fact its not a good practice to programmatically invoke GC. For parsing large json data, use streaming APIs in whichever library you use to encode/decode json. One such popular library is Jackson processor.
5mb is not that big data to cause out of memory, maybe dump the memory of the app and analyze for memory leaks. You could sump using adb dumpsys
Tool. Find more about how to use that in What's the Android ADB shell "dumpsys" tool and what are its benefits?
try {
HttpResponse response = httpclient.execute(httppost);
HttpEntity entity = response.getEntity();
inputStream = entity.getContent();
// json is UTF-8 by default
BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream, "UTF-8"), 8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
result = sb.toString();
inputStream.close();
} catch (Exception e) {
Log.d(TAG, "String was to large" + e.toString());
}
I think if you have problem to load 5Mb string into memory (totally reasonable size), streaming JSON parser will probably not help, because you need to have the result of parsing in memory, which will not necessarily be much smaller. I'd suggest you to do the memory profiling using Android Studio built-in memory monitor, to check how much free memory you have and how you're using it in general.
Related
I'm using this code to parse a JSON array I'm getting from my server.
try {
URL u = new URL("http://54.68.139.250/get_user_likes");
HttpURLConnection conn = (HttpURLConnection) u.openConnection();
conn.setRequestMethod("GET");
conn.connect();
InputStream is = conn.getInputStream();
byte[] b = new byte[1024];
ByteArrayOutputStream baos = new ByteArrayOutputStream();
while ( is.read(b) != -1)
baos.write(b);
String JSONResp = new String(baos.toByteArray());
JSONArray arr = new JSONArray(JSONResp);
for (int i=0; i < arr.length(); i++) {
result.add(convertArticle(arr.getJSONObject(i)));
}
return result;
}
catch(Throwable t) {
t.printStackTrace();
}
return null;
This code works great on my phone. Unfortunately, when I'm using a Genymotion emulator with the virtual device of Google Nexus 7, the JSON array is slightly altered. 95% of the JSON array is fine, but it is truncated near the very end and is randomly missing about 4 characters of the json array at character 1253 so I'm getting:
org.json.JSONException: Expected ':' after top_id at character 1253 of [{"top_id":6,"top_url":
I'm thinking this is some memory problem with the emulator. Its base memory is 1024. Increasing that amount though doesn't change anything.
Any tips as to the reason behind the problem would be greatly appreciated. Also, feel free to comment on my code if you see room for improvement. :)
That's weird.
I can think of two things to try:
Check the encoding of the server and the encoding of the String constructor.
It's possible that the server is decoding with, say, Latin-1 (ISO-8859-1) and the String is encoding with UTF-8. But JSON is supposed to be sent in Unicode and the default charset for Android is UTF-8. Check the HTTP Content-type response header; it should say: application/json; charset=utf-8. If the charset part isn't there, do some investigation and find out what character set your server is using for decoding to HTTP stream. And check that the default charset on the phone and the emulator is the same; it should be UTF-8.
Try calling flush() on the ByteArrayOutputStream after you read the data and before you construct the String.
It may be that there is a slight difference between the phone OS and the emulator OS on how stream data is transferred/buffered/flushed.
If flush() doesn't work, try rewriting the code without using ByteArrayOutputStream. You could, for example, wrap the input stream with an InputStreamReader, which reads characters, not bytes, then append the characters using a StringBuilder or StringBuffer.
One way you could make the code better is to use JSONReader instead of JSONArray or JSONObject. The JSONReader would wrap an InputStreamReader which in turn wraps the HTTP input stream. It can be faster and more memory efficient since you don't have to read the entire input stream before starting to parse the data. When the server is sending a LOT of JSON data, that can make a big difference.
You should check the return value of is.read(). Change
while ( is.read(b) != -1)
baos.write(b);
to
int nread;
while ( (nread=is.read(b)) != -1)
baos.write(b, 0, nread);
I am a beginner on Android app development. I want to confirm if my approach is correct and as per best practices in Android world.
I have an android application that needs textual data (no graphic, video). The data comes from REST based web service as JSON string. I consume this web service using HttpGet object, parse json string using JSONArray and display data.
All this happens in a button click.
My questions are:
Is there a better (or android-style) approach to do the same?
What is the preferred approach to retrieve and post graphic contents to REST based web service?
Any help is most appreciated.
Thanks
Please find my inline commnets,
All this happens in a button click.
My questions are:
Is there a better (or android-style) approach to do the same?
Ideal approach to trigger the Webservice calls in Async task or in a service, so your UI thread will not be blocked till HTTP fires and get the response.
What is the preferred approach to retrieve and post graphic contents to REST based web service?
The graphics content will be usually base64 when you try to retrieve it from the backend.
Refer this example : http://androidtrainningcenter.blogspot.in/2012/03/how-to-convert-string-to-bitmap-and.html
for the posting the graphics to the server
I'm going to assume that you know the path and filename of the image that you want to upload. Add this string to your NameValuePair using image as the key-name.
Sending images can be done using the HttpComponents libraries. Download the latest HttpClient (currently 4.0.1) binary with dependencies package and copy apache-mime4j-0.6.jar and httpmime-4.0.1.jar to your project and add them to your Java build path.
You will need to add the following imports to your class.
import org.apache.http.entity.mime.HttpMultipartMode;
import org.apache.http.entity.mime.MultipartEntity;
import org.apache.http.entity.mime.content.FileBody;
import org.apache.http.entity.mime.content.StringBody;
Now you can create a MultipartEntity to attach an image to your POST request. The following code shows an example of how to do this:
public void post(String url, List nameValuePairs) {
HttpClient httpClient = new DefaultHttpClient();
HttpContext localContext = new BasicHttpContext();
HttpPost httpPost = new HttpPost(url);
try {
MultipartEntity entity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
for(int index=0; index < nameValuePairs.size(); index++) {
if(nameValuePairs.get(index).getName().equalsIgnoreCase("image")) {
// If the key equals to "image", we use FileBody to transfer the data
entity.addPart(nameValuePairs.get(index).getName(), new FileBody(new File (nameValuePairs.get(index).getValue())));
} else {
// Normal string data
entity.addPart(nameValuePairs.get(index).getName(), new StringBody(nameValuePairs.get(index).getValue()));
}
}
httpPost.setEntity(entity);
HttpResponse response = httpClient.execute(httpPost, localContext);
} catch (IOException e) {
e.printStackTrace();
}
}
Hope this helps!
Hey refer this link which will train you better in parsing json data as you required in Android App. Loading the graphic required some lazy loading mechanism which you can refer it from here.
How do I get text from a basic HTML page and show it in a TextView.
I want to do it this way because it will look better than having a webview showing the text.
There is only one line on the html page. I can change it to a txt file if needed.
Could you also provide a quick example?
You would need to download the HTML first using something like HttpClient to retrieve the data from the Internet (assuming you need to get it from the Internet and not a local file). Once you've done that, you can either display the HTML in a WebView, like you said, or, if the HTML is not complex and contains nothing other than some basic tags (<a>, <img>, <strong>, <em>, <br>, <p>, etc), you can pass it straight to the TextView since it supports some basic HTML display.
To do this, you simply call Html.fromHtml, and pass it your downloaded HTML string. For example:
TextView tv = (TextView) findViewById(R.id.MyTextview);
tv.setText(Html.fromHtml(myHtmlString));
The fromHtml method will parse the HTML and apply some basic formatting, returning a Spannable object which can then be passed straight to TextView's setText method. It even supports links and image tags (for images, though, you'll need to implement an ImageGetter to actually provide the respective Drawables). But I don't believe it supports CSS or inline styles.
How to download the HTML:
myHtmlString in the snippet above needs to contain the actual HTML markup, which of course you must obtain from somewhere. You can do this using HttpClient.
private String getHtml(String url)
{
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet(url);
try
{
HttpResponse response = client.execute(request);
BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
String line;
StringBuilder builder = new StringBuilder();
while((line = reader.readLine()) != null) {
builder.append(line + '\n');
}
return builder.toString();
}
catch(Exception e)
{
//Handle exception (no data connectivity, 404, etc)
return "Error: " + e.toString();
}
}
It's not enough to just use that code, however, since it should really be done on a separate thread (in fact, Android might flat out refuse to make a network connection on the UI thread. Take a look at AsyncTasks for more information on that. You can find some documentation here (scroll down a bit to "Using Asynctask").
I am using the HttpPut to communicate with server in Android, the response code I am getting is 500.After talking with the server guy he said prepare the string like below and send.
{"key":"value","key":"value"}
now I am completely confused that where should i add this string in my request.
Please help me out .
I recently had to figure out a way to get my android app to communicate with a WCF service and update a particular record. At first this was really giving me a hard time figuring it out, mainly due to me not knowing enough about HTTP protocols, but I was able to create a PUT by using the following:
URL url = new URL("http://(...your service...).svc/(...your table name...)(...ID of record trying to update...)");
//--This code works for updating a record from the feed--
HttpPut httpPut = new HttpPut(url.toString());
JSONStringer json = new JSONStringer()
.object()
.key("your tables column name...").value("...updated value...")
.endObject();
StringEntity entity = new StringEntity(json.toString());
entity.setContentType("application/json;charset=UTF-8");//text/plain;charset=UTF-8
entity.setContentEncoding(new BasicHeader(HTTP.CONTENT_TYPE,"application/json;charset=UTF-8"));
httpPut.setEntity(entity);
// Send request to WCF service
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpResponse response = httpClient.execute(httpPut);
HttpEntity entity1 = response.getEntity();
if(entity1 != null&&(response.getStatusLine().getStatusCode()==201||response.getStatusLine().getStatusCode()==200))
{
//--just so that you can view the response, this is optional--
int sc = response.getStatusLine().getStatusCode();
String sl = response.getStatusLine().getReasonPhrase();
}
else
{
int sc = response.getStatusLine().getStatusCode();
String sl = response.getStatusLine().getReasonPhrase();
}
With this being said there is an easier option by using a library that will generate the update methods for you to allow for you to update a record without having to manually write the code like I did above. The 2 libraries that seem to be common are odata4j and restlet. Although I haven't been able to find a clear easy tutorial for odata4j there is one for restlet that is really nice: http://weblogs.asp.net/uruit/archive/2011/09/13/accessing-odata-from-android-using-restlet.aspx?CommentPosted=true#commentmessage
Error 500 is Internal Server error. Not sure if this answers your question but I personally encountered it when trying to send a data URI for an animated gif in a PUT request formatted in JSON but the data URI was too long. You may be sending too much information at once.
I have a text file on my server. I want to open the text file from my Android App and then display the text in a TextView. I cannot find any examples of how to do a basic connection to a server and feed the data into a String.
Any help you can provide would be appreciated.
Try the following:
try {
// Create a URL for the desired page
URL url = new URL("mysite.com/thefile.txt");
// Read all the text returned by the server
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
String str;
while ((str = in.readLine()) != null) {
// str is one line of text; readLine() strips the newline character(s)
}
in.close();
} catch (MalformedURLException e) {
} catch (IOException e) {
}
(taken from Exampledepot: Getting text from URL)
Should work well on Android.
While URL.openStream will work, you would be better off using the Apache HttpClient library that comes with Android for HTTP. Among other reasons, you can use content encoding (gzip) with it, and that will make text file transfers much smaller (better battery life, less net usage) and faster.
There are various ways to use HttpClient, and several helpers exist to wrap things and make it easier. See this post for more details on that: Android project using httpclient --> http.client (apache), post/get method (and note the HttpHelper I included there does use gzip, though not all do).
Also, regardless of what method you use to retrieve the data over HTTP, you'll want to use AysncTask (or Handler) to make sure not to block the UI thread while making the network call.
And note that you should pretty much NEVER just use URL.openStream (without setting some configuration, like timeouts), though many examples show that, because it will block indefinitely if you the server is unavailable (by default, it has no timeout): URL.openStream() Might Leave You Hanging.
Don't forget to add internet permissions to the manifest when taking net resources: (add in manifest).