Long Internet operations with limited bandwidth - android

yesterday I published my application which I tested on my phone and worked as intended. When my friends downloaded it using 3G and not WiFi my application failed to download all the content and as a result it crashed. I used a headless fragment which runs an AsyncTask in order to download the content (which is some photos) my guess is that it took a lot of time for some photos and skipped them, throwing some timeOut exception. My question is would this be avoided if instead of an fragment I used a service to run my AsyncTask and download the content?
private ArrayList<Monument> processJsonData(JSONObject jsonObj) throws IOException{
try{
JSONArray posts=jsonObj.getJSONArray(TAG_POSTS);
ArrayList<Monument> monuments = new ArrayList<Monument>();
for (int i=0; i<posts.length(); i++){
JSONArray attachments = c.optJSONArray(TAG_ATTACHMENTS);
if(attachments!=null){
int lengthSize;
if(attachments.length()<3)
lengthSize=attachments.length();
else
lengthSize=3;
for(int j=0;j<lengthSize;++j){
JSONObject atta = attachments.getJSONObject(j);
JSONObject images = atta.optJSONObject(TAG_IMAGES);
if(images!=null){
JSONObject medium = images.getJSONObject(TAG_MEDIUM);
String url_image = medium.getString(TAG_URL_IMAGE);
String id = atta.getString("id");
String filename =title.replace(" ","")+id+".nomedia";
File destination = new File(MyApplication.getPhotoStorage() ,filename);
URL url = new URL (url_image);
InputStream is = url.openStream();
OutputStream os = new FileOutputStream(destination);
byte[] b = new byte[2048];
int length;
while ((length = is.read(b)) != -1) {
os.write(b, 0, length);
}
is.close();
os.close();
localPhotosUrl.add(destination.getAbsolutePath());
}
}
}
}
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
Edit so I made these changes in my code now I'm dealing with the connectionTimeout exception but I can't catch it properly
#Override
protected String doInBackground(String... args) {
JSONParser jParser = new JSONParser();
JSONObject jsonObj = jParser.getJSONFromUrl(url);
Log.d("check1",url);
try {
listOfObjects.addAll(processJsonData(jsonObj));
} catch (Exception e) {
e.printStackTrace();
onDownloadFailed(this);
} finally {
jsonObj=null;
}
return "done";
}
protected void onDownloadFailed(downloadUrl task) {
System.out.println(task.tag+" failed to download");
if(dtask1.cancel(true))
Log.d("TASK1", "Canceled");
if(dtask2.cancel(true))
Log.d("TASK2", "Canceled");
if(dtask3.cancel(true))
Log.d("TASK3", "Canceled");
if(dtask4.cancel(true))
Log.d("TASK4", "Canceled");
mCallbacks.onDownloadFailed();
}
private ArrayList<Monument> processJsonData(JSONObject jsonObj) throws IOException, SocketException, JSONException{
JSONArray attachments = c.optJSONArray(TAG_ATTACHMENTS);
if(attachments!=null){
int lengthSize;
if(attachments.length()<3)
lengthSize=attachments.length();
else
lengthSize=3;
for(int j=0;j<lengthSize;++j){
JSONObject atta = attachments.getJSONObject(j);
JSONObject images = atta.optJSONObject(TAG_IMAGES);
if(images!=null){
JSONObject medium = images.getJSONObject(TAG_MEDIUM);
String url_image = medium.getString(TAG_URL_IMAGE);
String id = atta.getString("id");
String filename =title.replace(" ","")+id+".nomedia";
File destination = new File(MyApplication.getPhotoStorage() ,filename);
try{
URL url = new URL (url_image);
InputStream is = url.openStream();
OutputStream os = new FileOutputStream(destination);
byte[] b = new byte[2048];
int length;
while ((length = is.read(b)) != -1) {
os.write(b, 0, length);
}
is.close();
os.close();
localPhotosUrl.add(destination.getAbsolutePath());
}catch (SocketException e) {
throw new SocketTimeoutException();
}
}
}
}

Alright, so I don't know much about JSON, but that shouldn't affect this answer as a whole I don't think. It looks like your issue could be solved by better use of Exception handling.
At the moment you are not really handling the exception at all, merely catching it and printing the stack trace. Also, because the entire method is inside one try{ } statement the method is exiting if there is a problem dealing with any one of the attachments. Instead, you could include a try{ } block inside of your for loop. This way, if any one of the loop blocks failed (due to say an unstable connection), you can use the catch block to j--; and then Thread.sleep(4000);. That way, when an exception is thrown in the loop it will be caught, the loop will be jumped back to try the same section again, and there will be a pause to to wait for a better connection.
Example (Not tested);
for (int j = 0; j < lengthSize; ++j) {
try{
JSONObject atta = attachments.getJSONObject(j);
JSONObject images = atta.optJSONObject(TAG_IMAGES);
if (images != null) {
JSONObject medium = images.getJSONObject(TAG_MEDIUM);
String url_image = medium.getString(TAG_URL_IMAGE);
String id = atta.getString("id");
String filename = title.replace(" ", "") + id + ".nomedia";
File destination = new File(MyApplication.getPhotoStorage(), filename);
URL url = new URL(url_image);
InputStream is = url.openStream();
OutputStream os = new FileOutputStream(destination);
byte[] b = new byte[2048];
int length;
while ((length = is.read(b)) != -1) {
os.write(b, 0, length);
}
is.close();
os.close();
localPhotosUrl.add(destination.getAbsolutePath());
}
}catch (Exception e){
Thread.sleep(4000);
j--;
}
}
You may also want to create a counter to see how many attempts have been made. If the code makes too many attempts you can then assume that it will never work and return from the method.
I hope this helps. Let me know how you get on.
Also, as an aside you should never really catch Exception. Better to catch the specific exception/s you are expecting so that you can deal with them differently depending on the Exception subtype. You don't want to catch a RunTimeException and try to handle it when there may be no effective way of doing so.

No. This won't be avoided. You cannot expect a connection to last for a long time. Anything can happen on the phone (battery exhausted, network signal lost, etc.)
You must code your app so that it properly checks if it has all the resources available and to retry (or better, resume) downloads that failed.
To increase your chances, break your resources in relatively small downloads instead of a huge file. Then the user does not have to download all from scratch when using 3G for instance.

Related

Randomize array of questions while reading from text file android

This is a android quiz app code snippet which load the question from text file.
I want to shuffle the question and answer after every next click so how can i implement random function ?
https://github.com/gitssk/quizfun/blob/master/src/ssk/quizfun/QuizFunActivity.java
https://github.com/gitssk/quizfun/blob/master/res/raw/questions.txt
private void loadQuestions() throws Exception {
try {
InputStream questions = this.getBaseContext().getResources()
.openRawResource(R.raw.questions);
bReader = new BufferedReader(new InputStreamReader(questions));
StringBuilder quesString = new StringBuilder();
String aJsonLine = null;
while ((aJsonLine = bReader.readLine()) != null) {
quesString.append(aJsonLine);
}
Log.d(this.getClass().toString(), quesString.toString());
JSONObject quesObj = new JSONObject(quesString.toString());
quesList = quesObj.getJSONArray("Questions");
Log.d(this.getClass().getName(),
"Num Questions " + quesList.length());
} catch (Exception e){
} finally {
try {
bReader.close();
} catch (Exception e) {
Log.e("", e.getMessage().toString(), e.getCause());
}
}
}
https://github.com/gitssk/quizfun/blob/master/src/ssk/quizfun/QuizFunActivity.java
I will refrain from posting much code because I think you should attempt it on your own. It is seriously not that tough. I will give you an approach though.
You have quesList = quesObj.getJSONArray("Questions");. So quesList is the list of questions that is a JSONArray. You want to shuffle this. Just do this:
Get the length of the quesList array. Let's call it len.
Create a simple arrayList called quesOrder containing integers 0 to len.
List<Integer> quesOrder = new ArrayList<>();
for (int i = 0; i <= len; i++)
{
quesOrder.add(i);
}
Once you have the quesOrder array. Just do Collections.shuffle(quesOrder);. Now when you get questions from your quesList array, just get the index from quesOrder list. And you will have a randomized selection. Put it together in a function for convenience.

asynctask download file network connection drop

I have an application that downloads multiple files (and checks the download speed) within an Asynctask. However, when I am connected to 3G (and there is a handover) or the device switches form WiFi to 3G, my application freezes and I can not handle this problem
#Override
protected Integer doInBackground(String... sUrl) {
try {
speed=0; //initial value
int i=0;
while ((i<sUrl.length)) {
URL url = new URL(sUrl[i]);
URLConnection connection = url.openConnection();
connection.setReadTimeout(10000);
connection.connect();
int fileLength = connection.getContentLength();
// download the file
InputStream input = new BufferedInputStream(url.openStream());
OutputStream output = new FileOutputStream(file);
byte data[] = new byte[1024*1024]; //1MB buffer
long total = 0;
int count;
long start = System.currentTimeMillis();
while ((count = input.read(data)) != -1) {
total += count;
publishProgress((int) (total * 100 / fileLength));
output.write(data, 0, count);
}
long finish = System.currentTimeMillis();
long tempSpeed= (fileLength *8)/(finish-start);
if (tempSpeed>speed) {
speed=tempSpeed;
}
output.flush();
output.close();
input.close(); // connection is closed
i++;
}
}catch(SocketTimeoutException e) {
exceptions.add(e);
messageProgressDialog.setCancelable(true);
AlertDialog alertDialog;
alertDialog = new AlertDialog.Builder(gui.getActivity()).create();
alertDialog.setTitle("Network problem");
alertDialog.setMessage("connection dropped");
alertDialog.show();
} catch (IOException e) {
exceptions.add(e);
messageProgressDialog.setCancelable(true);
AlertDialog alertDialog;
alertDialog = new AlertDialog.Builder(gui.getActivity()).create();
alertDialog.setTitle("IOException");
alertDialog.setMessage("IOException error");
alertDialog.show();
} catch(Exception e) {
exceptions.add(e);
AlertDialog alertDialog;
alertDialog = new AlertDialog.Builder(gui.getActivity()).create();
alertDialog.setTitle("Exception");
alertDialog.setMessage("Exception error");
alertDialog.show();
}
return 1;
}
I have read many topics on stackoverflow, however none of them could help me solve the prolem I have with my application. I have the try/catch clause, but it doesn't seem to make a difference. When I am using 3G and the phone connects to annother antenna, or there is a network problem the application freezes. What can I do ?
I have found the problem. It was this line InputStream input = new BufferedInputStream(url.openStream()); I use as inputstream the url. I replaced it with this line: InputStream input = new BufferedInputStream(connection.getInputStream()); Now it seems to work better, when it times-out, the application crashes.
}catch(SocketTimeoutException e) {
String erro=Integer.toString(count);
Log.d("error socketTimeout",erro);//1st log
exceptions.add(e);
onPostExecute(1);
Log.d("error sockteTimeout", "here"); //2nd log to see if onPostExecute worked
AlertDialog alertDialog;
alertDialog = new AlertDialog.Builder(gui.getActivity()).create();
alertDialog.setTitle("Network problem");
alertDialog.setMessage("connection dropped");
alertDialog.show();
This is the catch that I use. It seems that when I try to call the within the catch clause the onPostExecute method my application crashes.
protected void onPostExecute(Integer result) {
Log.d("onpost was called", "here");
messageProgressDialog.setMessage("Your speed is: " + speed +" KBit/sec");
messageProgressDialog.setCancelable(true);
}
Looking on the Log viewer, only the 1st Log appears. When onPostExecute is called, the app crashes. onPostExecute is never actually executed.
Any ideas?
Your problem is likely related to your handling of the return value of input.read(). If a socket is closed, input.read() may return 0 (it may also return -1 in other error situations). If it does return 0, then your loop will hang. I suggest something like:
while ((count = input.read(data)) > 0) {
That way, your loop will run while you are still making progress on the download.

IndexOutOfBoundsException error happening depending on Android OS version

I am desperatly trying to fix a bug that:
always happens in my emulator for Android versions 2.2, 2.3
never happens in emulator android versions 4.*
never happens in a real device (android 4.*)
It is the following IndexOutOfBoundsException exception:
java.lang.RuntimeException: Unable to start activity
ComponentInfo{<myapppackage>}: java.lang.IndexOutOfBoundsException:
Invalid index 39, size is 0
In my app I am fecthing data from a json file that I am displaying as text. I've isoleted where the bug is coming from, it is when I call this method:
public String getItemValue(int id, String s) {
List<JsonItems> list = new ArrayList<JsonItems>();
try {
// CONVERT RESPONSE STRING TO JSON ARRAY
JSONArray ja = new JSONArray(s);
// ITERATE THROUGH AND RETRIEVE
int n = ja.length();
for (int i = 0; i < n; i++) {
// GET INDIVIDUAL JSON OBJECT FROM JSON ARRAY
JSONObject jo = ja.getJSONObject(i);
// RETRIEVE EACH JSON OBJECT'S FIELDS
JsonItems ji = new JsonItems();
ji.id = jo.getInt("id");
ji.text= jo.getString("text");
list.add(ji);
}
} catch (Exception e) {
e.printStackTrace();
}
return list.get(id).text;
}
My class JsonItems is very basic:
public class JsonItems{
int id;
String text;
}
Sample from my json file:
[
{"id":0,"text":"some text 0"},
{"id":1,"text":"some text 1"},
{"id":2,"text":"some text 2"}
]
Here is how I process content of my json file into a String
public static String fromJsonFileToString(String fileName, Context c) {
//JSONArray jArray = null;
String text = "";
try {
InputStream is = c.getAssets().open(fileName);
int size = is.available();
byte[] buffer = new byte[size];
is.read(buffer);
is.close();
text = new String(buffer);
} catch (IOException e) {
throw new RuntimeException(e);
}
return text;
}
Once again I repeat: the IndexOutOfBoundsException NEVER happens on a device with Android 4.* , it only happens when I test the app on emulators with Android 2.*
Any idea where it is coming from?
Thanks
First problem:
You are not reading your input stream correctly. Calling available literally does just that - it returns you the amount of data that is available to be read, when the call is made. This number may, or may not represent the entire content of the file.
Reading material for you:
How to Read a File in Java.
Writing and Creating Files
Note that there are helper libraries like Apache Commons IO that make it possible to read file contents in a single line of code (IOUtils.toString(inputStream)). Android doesn't support Java 7 yet but a noteworthy alternative is available in that release, with the Files.readAllLines method. In any case, you can make the below shown changes to your file reading code and it should work better:
public static String fromFileToString(String fileName, Context context) {
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(
context.getAssets().open(fileName)));
String line = null;
StringBuilder builder = new StringBuilder(1024);
while ((line = reader.readLine()) != null) {
builder.append(line);
}
return builder.toString();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
Second problem:
You do not do any bound check, to make sure that the argument you pass into your 'search' method:
public String getItemValue(int id, String s)
Does not exceed the length of the list of items you eventually calculate:
return list.get(id).text;
// ^
// -------------
// 'id' could be larger than the list size!
In any case, your current design doesn't at all match what you are really trying to do, aka, to determine the element in the JSON array that has an 'id' field matching what you supply to the method. You need to process the JSON data as a map, in order to be able to do this.
public String getItemValue(int id, String json) {
if(json == null || json.trim().equals("")) return null;
Map<Integer, JsonItems> map = new HashMap<Integer, JsonItems>(4);
try {
JSONArray ja = new JSONArray(json);
int n = ja.length();
for (int i = 0; i < n; i++) {
JSONObject jo = ja.getJSONObject(i);
JsonItems ji = new JsonItems();
ji.id = jo.getInt("id");
ji.text = jo.getString("text");
map.put(Integer.valueOf(ji.id, ji);
}
} catch(NumberFormatException nfe) {
throw new IllegalArgumentException("Invalid JSON format");
} catch (Exception e) {
throw new RuntimeException(e);
}
JsonItems item = map.get(id);
return item != null ? item.text : null;
}
Some quick notes:
JsonItems should be called JsonItem, to conform to good Java naming standards
You should really parse and store your JSON just once, to improve performance
You are really only using a minimal subset of your JSON data, you could actually determine the matching node within your for loop and directly return its value, without having to use an intermedia Java bean object

AsyncTask - slow download

I am using AsyncTask to download ~50 MB files from internet. Sometimes, when I download this file, progress bar gain is very slow (even when I am on Wi-Fi). And after minute, phone shows me, download complete, but the file itself has only ~100kB, no more. But when I restart device, and try to download file, download is executed briefly and quick. Has anyone faced same problem? Do I need to erase same download memory before downloading new file? I am downloading file to Environment.externalStoryDirectory().
Thx
Calling download from activity:
mProgressDialog = new ProgressDialog(ItemDetails.this);
mProgressDialog.setTitle("Downloading");
mProgressDialog.setMessage("Downloading sth...");
mProgressDialog.setIndeterminate(false);
mProgressDialog.setMax(100);
mProgressDialog.setProgressStyle(ProgressDialog.STYLE_HORIZONTAL);
DownloadMapTask downloadFile = new DownloadMapTask(ItemDetails.this);
downloadFile.execute(web_location_url);
mProgressDialog.show();
Download Async Task (two methods):
#Override
protected String doInBackground(String... urls) {
int count;
PATH=maps_loc+"/Android/data/test/maps/";
try {
URL url = new URL(urls[0]);
HttpURLConnection connection2 = (HttpURLConnection) url.openConnection();
connection2.setRequestMethod("GET");
connection2.setDoOutput(true);
connection2.connect();
int lenghtOfFile = connection2.getContentLength();
File apkdir = new File(PATH);
apkdir.mkdirs();
File newInstall = new File(PATH, name+".tmp");
InputStream input = new BufferedInputStream(url.openStream());
OutputStream output = new FileOutputStream(newInstall);
byte data[] = new byte[1024];
long total = 0;
while ((count = input.read(data)) != -1 && running==true) {
total += count;
publishProgress((int) (total * 100 / lenghtOfFile));
output.write(data, 0, count);
}
output.flush();
output.close();
input.close();
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
public void onProgressUpdate(Integer... args) {
ItemDetails.mProgressDialog.setProgress(args[0]);
}
Some servers will close the connection if the client has slow speed and the download takes long time, which can be the case if your program is connected to the Internet through mobile data not Wi-Fi.
You should consider supporting download resume in your program to not start from scratch every time.
I do not think there is sort of download memory that you need to clear. I have an app that can easily downloads over 50MB with no problems.
Also, you might consider obtaining a lock for both Wi-Fi and processor to keep your program running until the download finishes.
Edit
In your code, try to print the value lenghtOfFile after the line int lenghtOfFile = connection2.getContentLength(); to make sure that it is the same as the actual file size you are downloading.
Below is alternative example code which supports resume that I am using in my projects. (it is just to illustrate the idea, you will need to modify the code to your needs)
HttpClient httpClient = new DefaultHttpClient();
HttpGet request = new HttpGet(new URI(fileURL)));
HttpResponse response;
InputStream is = null;
FileOutputStream fos = null;
try {
boolean continueDownloading = false;
String tmpFileName = fileName + "_tmp";
outputFile = new File(downloadFolder, tmpFileName);
if (outputFile.exists()) {
localFileLength = outputFile.length();
if (localFileLength > 0) {
continueDownloading = true;
}
if (continueDownloading) {
request.addHeader("Range", "bytes=" + localFileLength + "-");
}
response = httpClient.execute(request);
long remoteFileLength = 0;
Header contentLengthHeader = response.getFirstHeader("Content-Length");
if (contentLengthHeader != null) {
remoteFileLength = Integer.parseInt(contentLengthHeader.getValue());
}
long downloaded = 0;
if (continueDownloading) {
downloaded = localFileLength;
}
long fullFileLength = downloaded + remoteFileLength;
fos = new FileOutputStream(outputFile, true);
is = response.getEntity().getContent();
byte[] buffer = new byte[DOWNLOAD_BUFFER_SIZE];
int len = 0;
while ((len = is.read(buffer)) != -1 && isDownloading) {
fos.write(buffer, 0, len);
downloaded += len;
}
fos.flush();
boolean success = downloaded == fullFileLength;
if (success) {
outputFile.renameTo(new File(downloadFolder, fileName));
}
} catch (Throwable ex) {
ex.printStackTrace();
} finally {
// clean up resources
}
Try using downloadManager instead of downloading manually , there are many advantages to using it.
Here is an example for it : DownloadManager Example
and take a look at the documentations : DownloadManager

Android force closes when reading from internal storage 5 times in a row

In my android app, I am reading a file from internal storage every time a new game loads.
The first 4 times I do this, it works fine, but on the fifth time it force closes.
Here is my code
private String readFromInternalStorage(String filename) {
FileInputStream fis=null;
byte[] bytes = new byte[1000000];
try {
fis=startGame.openFileInput(filename);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
try {
fis.read(bytes);
} catch (IOException e) {
e.printStackTrace();
}
return new String(bytes);
}
While messing around with the code, I noticed that if I change the length of the byte array, it changes the amount of times I can read a file without it force closing. If I change the length to 2000000, it closes after the second time and if I change it to 100000 it closes after the eighth time. I'm pretty clueless as to why this would happen because I am creating a new byte array every time the method is called so I wouldn't think that the size would change anything.
Update:
After going back and doing some more testing it seems like file input has nothing to do with why my app is force closing. When this code is commented out, the app will load five levels in a row without force closing so I thought that it was the problem, but it still force closes after eight tries so clearly there's something else that's not working. Thanks for your help anyway.
I don't see a "close()" in your code:
http://docs.oracle.com/javase/6/docs/api/java/io/FileInputStream.html#close%28%29
You shouldn't hard-code the array size. Besides you should use finally, in order to make sure the FileInputStream is closed, even when failed.
Here's a code sample that shows how it should be done:
FileInputStream fis;
String info = "";
try {
fis = mContext.openFileInput(this.fileName);
byte[] dataArray = new byte[fis.available()];
if (dataArray.length > 0) {
while (fis.read(dataArray) != -1) {
info = new String(dataArray);
}
Log.i("File Reading" , "Success!");
isOk = true;
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e1) {
e1.printStackTrace();
} finally {
fis.close();
}
a safe version of what you do is e.g.:
private String readFromInternalStorage(String filename) {
FileInputStream fis = null;
File file = new File(startGame.getFilesDir(), filename);
long size = file.length();
// impossible to have more than that (= 2GB)
if (size > Integer.MAX_VALUE) {
Log.d("XXX", "File too big");
return null;
}
int iSize = (int) size;
try {
fis = new FileInputStream(file);
// part of Android since API level 1 - buffer can scale
ByteArrayBuffer bb = new ByteArrayBuffer(iSize);
// some rather small fixed buffer for actual reading
byte[] buffer = new byte[1024];
int read;
while ((read = fis.read(buffer)) != -1) {
// just append data as long as we can read more
bb.append(buffer, 0, read);
}
// return a new string based on the large buffer
return new String(bb.buffer(), 0, bb.length());
} catch (FileNotFoundException e) {
Log.w("XXX", e);
} catch (IOException e) {
Log.w("XXX", e);
} catch (OutOfMemoryError e) {
// this could be left out. Keep if you read several MB large files.
Log.w("XXX", e);
} finally {
// finally is executed even if you return in above code
// fis will be null if new FileInputStream(file) throws
if (fis != null) {
try {
fis.close();
} catch (IOException e) {
// ignored, nothing can be done if closing fails
}
}
}
return null;
}

Categories

Resources