I need to download a few large zip files into my application (each approx 25mb) however it seems quite slow (5 minutes +) and when we test the same files being downloaded on an iPad it's downloading several times faster. I've considered using Volley, however it seems asynctask is the best for large files (from what I've read).
Does anyone have any suggestions or ideas on how I might be able to download/write these files faster?
My current implementation is show below:
My AsyncTaskExample:
#Override
protected String doInBackground(String... sUrl) {
InputStream input = null;
OutputStream output = null;
HttpURLConnection connection = null;
try {
URL url = new URL(sUrl[0]);
connection = (HttpURLConnection) url.openConnection();
connection.connect();
// expect HTTP 200 OK, so we don't mistakenly save error report
// instead of the file
if (connection.getResponseCode() != HttpURLConnection.HTTP_OK) {
return "Server returned HTTP " + connection.getResponseCode()
+ " " + connection.getResponseMessage();
}
// this will be useful to display download percentage
// might be -1: server did not report the length
int fileLength = connection.getContentLength();
// download the file
input = connection.getInputStream();
output = new FileOutputStream("/sdcard/file_name.extension");
byte data[] = new byte[1024];
long total = 0;
int count;
while ((count = input.read(data)) != -1) {
// allow canceling with back button
if (isCancelled()) {
input.close();
return null;
}
total += count;
// publishing the progress....
if (fileLength > 0) // only if total length is known
publishProgress((int) (total * 100 / fileLength));
output.write(data, 0, count);
}
} catch (Exception e) {
return e.toString();
} finally {
try {
if (output != null)
output.close();
if (input != null)
input.close();
} catch (IOException ignored) {
}
if (connection != null)
connection.disconnect();
}
return null;
}
AsyncTask should be used only for relatively short background processes (i.e. processes that last a few seconds). From the docs:
AsyncTask is designed to be a helper class around Thread and Handler
and does not constitute a generic threading framework. AsyncTasks
should ideally be used for short operations (a few seconds at the
most.) If you need to keep threads running for long periods of time,
it is highly recommended you use the various APIs provided by the
java.util.concurrent package such as Executor, ThreadPoolExecutor and
FutureTask.
For long operations you should use a Service:
A Service is an application component representing either an
application's desire to perform a longer-running operation while not
interacting with the user or to supply functionality for other
applications to use.
Related
I have a SQLite database file in my server, and from time to time my Android App checks if there is a new SQLite database file. If true the App downloads the File and replaces the old database.
The problem is, that some times the new database file gets corrupted and the App start to crashing and never recovers if I dont manualy clean the app in the Android Settings.
My question is, there is a way to check the integrity of SQLite Database after the Downloaded?
This is my code for download the new Database from the server this code is placed in an AssyncTask :
protected Boolean doInBackground(String... Url) {
try {
URL url = null;
if(Url[0].equals("")){
mSyncDate = mConnectionManager.getSyncDate();
url = new URL(Constants.HF_SERVER_DATABASE+"db_fxbus_"+convertDateToFormatYYYYMMDD(mSyncDate.getServerDate())+".sqlite");
}else{
url = new URL(Url[0]);
}
URLConnection connection = url.openConnection();
connection.connect();
// this will be useful so that you can show a typical 0-100% progress bar
int fileLength = connection.getContentLength();
mDB.getReadableDatabase();
// download the file
InputStream input = new BufferedInputStream(url.openStream());
Log.i(TAG, "Path:"+mContext.getDatabasePath("HorariosDoFunchal").getAbsolutePath());
OutputStream output = new FileOutputStream(mContext.getDatabasePath("HorariosDoFunchal").getAbsolutePath());
startWriting = true;
byte data[] = new byte[1024];
long total = 0;
int count;
while ((count = input.read(data)) != -1) {
total += count;
// publishing the progress....
publishProgress((int) (total * 100 / fileLength));
output.write(data, 0, count);
//Log.i(TAG, "Executing ...");
}
//Log.i(TAG, "Finish ...");
output.flush();
output.close();
input.close();
} catch (Exception e) {
Log.e(TAG, e.toString());
return false;
}
return true;
}
Look into:
pragma integrity_check;
it will scan the Database and check it for errors and other things too.
More info(and more commands) can be found at this link:
http://www.sqlite.org/pragma.html
also check out the documentation of isDatabaseIntegrityOk().
You could try to use PRAGMA integrity_check (or Android's equivalent isDatabaseIntegrityOk()), but this checks only the database structure for errors, and can detect only errors where it can prove that the structure is wrong.
To be able to detect all errors (especially in your own data), you need to compute a checksum for the entire database file.
I'm new to Android and developing a file downloading app with a ProgressDialog which shows the downloading percentage.
I use AsyncTask and here is the trouble part of my code.
protected String doInBackground(String... f_url){
int count;
try {
URL url = new URL(f_url[0]);
URLConnection conn = url.openConnection();
conn.connect();
// getting file length
int lenghtOfFile = conn.getContentLength();
// input stream to read file - with 8k buffer
InputStream input = new BufferedInputStream(url.openStream(), 8192);
File direct = new File(folder);
if(!direct.exists()) {
direct.mkdirs();
}
// Output stream to write file
OutputStream output = new FileOutputStream(apkPath);
byte data[] = new byte[1024];
long total = 0;
while ((count = input.read(data)) != -1) {
total += count;
// publishing the progress....
// After this onProgressUpdate will be called
publishProgress(""+(int)((total*100)/lenghtOfFile));
// writing data to file
output.write(data, 0, count);
}
// flushing output
output.flush();
// closing streams
output.close();
input.close();
} catch (Exception e) {
Log.e("Error: ", e.getMessage());
}
return null;
}
My issue is this code works really well on Android API 16 (JB) but not on API 19 (KitKat). On KitKat devices, the progress bar percentage does not update (always 0). After checking the codes, I found conn.getContentLength() returns -1 when I run it on KitKat. So it can not update the progress. But it returns correct file size when I run it on API 16 (JB).
Can somebody please help me to solve this?
Thank you in advance.
Have you read Migrating to WebView in Android 4.4: http://developer.android.com/guide/webapps/migrating.html
Blockquote
If you call methods on WebView from any thread other than your app's UI thread, it can cause unexpected results. For example, if your app uses multiple threads, you can use the runOnUiThread() method to ensure your code executes on the UI thread:
runOnUiThread(new Runnable() {
#Override
public void run() {
// Code for WebView goes here
}
});
You can try this:
conn.setRequestProperty("Accept-Encoding", "identity");
I have followed some online tutorials and created this code to download the files that i have hosted in dropbox
I am using async task to do this
// AsyncTask to download a file
private class DownloadTask extends AsyncTask<String, Integer, String> {
private Context context;
public DownloadTask(Context context) {
this.context = context;
}
#Override
protected String doInBackground(String... sUrl) {
// take CPU lock to prevent CPU from going off if the user
// presses the power button during download
PowerManager pm = (PowerManager) context
.getSystemService(Context.POWER_SERVICE);
PowerManager.WakeLock wl = pm.newWakeLock(
PowerManager.PARTIAL_WAKE_LOCK, getClass().getName());
wl.acquire();
try {
InputStream input = null;
OutputStream output = null;
HttpURLConnection connection = null;
try {
URL url = new URL(sUrl[0]);
connection = (HttpURLConnection) url.openConnection();
connection.connect();
// expect HTTP 200 OK, so we don't mistakenly save error
// report
// instead of the file
if (connection.getResponseCode() != HttpURLConnection.HTTP_OK)
return "Server returned HTTP "
+ connection.getResponseCode() + " "
+ connection.getResponseMessage();
// TODO
File file = new File(Environment
.getExternalStorageDirectory().getPath()
+ "/kathmandu.map");
if (file.exists()) {
Log.i("File Exists", "Code Gets here, file exists");
return "exists";
// if (connection.getResponseCode() ==
// HttpURLConnection.HTTP_NOT_MODIFIED) {
//
// return null;
// }
}
// this will be useful to display download percentage
// might be -1: server did not report the length
int fileLength = connection.getContentLength();
Log.i("Length", String.valueOf(fileLength));
// download the file
input = connection.getInputStream();
output = new FileOutputStream(Environment
.getExternalStorageDirectory().getPath()
+ "/kathmandu.map");
byte data[] = new byte[4096];
long total = 0;
int count;
while ((count = input.read(data)) != -1) {
// allow canceling with back button
if (isCancelled())
return null;
total += count;
// publishing the progress....
if (fileLength > 0) // only if total length is known
publishProgress((int) (total * 100 / fileLength));
output.write(data, 0, count);
}
} catch (Exception e) {
return e.toString();
} finally {
try {
if (output != null)
output.close();
if (input != null)
input.close();
} catch (IOException ignored) {
}
if (connection != null)
connection.disconnect();
}
} finally {
wl.release();
}
return null;
}
I call the download code when the download options menu is clicked.
final DownloadTask downloadTask = new DownloadTask(MapActivity.this);
downloadTask
.execute("https://dl.dropboxusercontent.com/u/95497883/kathmandu-2013-8-12.map");
mProgressDialog
.setOnCancelListener(new DialogInterface.OnCancelListener() {
#Override
public void onCancel(DialogInterface dialog) {
downloadTask.cancel(true);
}
});
The code works fine but at the times the outputstream does not write full file and exits. Everything looks okay. The file is downloaded but it is corrupted.
The getContentLength() also returns -1 so i cannot check if the whole file has been downloaded using the content length. The file is a offline vector map and i need it to display offline maps. The corrupted file causes a runtime exception while trying to access it. Is there is any way to ensure that the file has been downloaded correctly.
Also i would like to provide the data with the app itself. Can i put this in the assets folder of my app. What is the best way to access the files in the assets folder during runtime.
Your assets folder is not writable as it is a part of the apk. you can of course use your application's sandbox storage (using Environment.getDir() ) or external storage (using Environment.getExternalStorageDirectory()) like you have done in your code.
I think using the DownloadManager would be a great idea to achieve exactly what you want please refer : http://developer.android.com/reference/android/app/DownloadManager.html
a short solution
DownloadManager.Request req=new DownloadManager.Request(url);
req.setAllowedNetworkTypes(DownloadManager.Request.NETWORK_WIFI
| DownloadManager.Request.NETWORK_MOBILE)
.setTitle("Downloading")
.setDescription("Map is Being Downloaded")
.setDestinationInExternalPublicDir(Environment.getExternalStorageDirectory,
"+/maps_app/something.map");
I try to create an application that can download files from a server in order to measure the speed that I can get with the particular server. I do this by using the Asynctask class. All the files that I want to download are located on the same directory. My question is, how can I download the subsequent files by keeping the connection and not by creating every time a new one? I know that for a TCP connection, there must be a 3-way-handshake established, before downloading a file. I want to connect to the server and then keep the connection and perform the download.
My code looks like this
#Override
protected Integer doInBackground(String... sUrl) {
try {
speed=0; //initial value
int i=0;
while ((i<sUrl.length)) {
URL url = new URL(sUrl[i]); //sUrl[] contains the links that i want
// for example http://myserver.net/file1.jpg, http://myserver.net/file2.jpg ... etc
URLConnection connection = url.openConnection();
connection.connect(); //connection to be established 3WAY HANDSHAKE
int fileLength = connection.getContentLength();
// download the file
InputStream input = new BufferedInputStream(url.openStream());
OutputStream output = new FileOutputStream(file);
byte data[] = new byte[1024];
long total = 0;
int count;
long start = System.currentTimeMillis();
while ((count = input.read(data)) != -1) {
total += count;
publishProgress((int) (total * 100 / fileLength));
output.write(data, 0, count);
}
long finish = System.currentTimeMillis();
long tempSpeed= (fileLength *8)/(finish-start);
if (tempSpeed>speed) {
speed=tempSpeed;
}
output.flush();
output.close();
input.close(); // connection is closed
i++;
}
}catch(Exception e) {
exceptions.add(e);
}
return 1;
}
By creating a new connection I loose time (for the download speed), because of the 3way handsharke . Also when transfering files in TCP, there is something called a tcp window (when you dowload data, initialy you start with low speed transmission, and if the connection is good this rate increases).
How can I do the above without creating and tearing down the connection for each file?
Looking at you code, you keep receiving until the socket is close at the other side, so there is no way of using the same socket since it's closed. If you can program both, the server and the client, then, I would suggest one possible way of doing it, and that is with a protocol, instead of receiving the file directly, the first packet you get is an integer which indicates the size of the file you're going to receive. If that length is 0 (cero) it means there no more files and the connection should be closed.
At the server:
While (the_are_files_to_send)
{
Socket.write((int) FileSize);
Socket.write(file's content);
}
Socket.Write(0); // No more files;
Socket.Close();
At the Client:
While ((size = Socket.read(buffer, 0, 4)) != -1)
{
int FileLength = convert_to _int(buffer);
if (FileLength==0) break;
Socket.read(FileLength bytes);
}
Socket.Close();
I am using AsyncTask to download ~50 MB files from internet. Sometimes, when I download this file, progress bar gain is very slow (even when I am on Wi-Fi). And after minute, phone shows me, download complete, but the file itself has only ~100kB, no more. But when I restart device, and try to download file, download is executed briefly and quick. Has anyone faced same problem? Do I need to erase same download memory before downloading new file? I am downloading file to Environment.externalStoryDirectory().
Thx
Calling download from activity:
mProgressDialog = new ProgressDialog(ItemDetails.this);
mProgressDialog.setTitle("Downloading");
mProgressDialog.setMessage("Downloading sth...");
mProgressDialog.setIndeterminate(false);
mProgressDialog.setMax(100);
mProgressDialog.setProgressStyle(ProgressDialog.STYLE_HORIZONTAL);
DownloadMapTask downloadFile = new DownloadMapTask(ItemDetails.this);
downloadFile.execute(web_location_url);
mProgressDialog.show();
Download Async Task (two methods):
#Override
protected String doInBackground(String... urls) {
int count;
PATH=maps_loc+"/Android/data/test/maps/";
try {
URL url = new URL(urls[0]);
HttpURLConnection connection2 = (HttpURLConnection) url.openConnection();
connection2.setRequestMethod("GET");
connection2.setDoOutput(true);
connection2.connect();
int lenghtOfFile = connection2.getContentLength();
File apkdir = new File(PATH);
apkdir.mkdirs();
File newInstall = new File(PATH, name+".tmp");
InputStream input = new BufferedInputStream(url.openStream());
OutputStream output = new FileOutputStream(newInstall);
byte data[] = new byte[1024];
long total = 0;
while ((count = input.read(data)) != -1 && running==true) {
total += count;
publishProgress((int) (total * 100 / lenghtOfFile));
output.write(data, 0, count);
}
output.flush();
output.close();
input.close();
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
public void onProgressUpdate(Integer... args) {
ItemDetails.mProgressDialog.setProgress(args[0]);
}
Some servers will close the connection if the client has slow speed and the download takes long time, which can be the case if your program is connected to the Internet through mobile data not Wi-Fi.
You should consider supporting download resume in your program to not start from scratch every time.
I do not think there is sort of download memory that you need to clear. I have an app that can easily downloads over 50MB with no problems.
Also, you might consider obtaining a lock for both Wi-Fi and processor to keep your program running until the download finishes.
Edit
In your code, try to print the value lenghtOfFile after the line int lenghtOfFile = connection2.getContentLength(); to make sure that it is the same as the actual file size you are downloading.
Below is alternative example code which supports resume that I am using in my projects. (it is just to illustrate the idea, you will need to modify the code to your needs)
HttpClient httpClient = new DefaultHttpClient();
HttpGet request = new HttpGet(new URI(fileURL)));
HttpResponse response;
InputStream is = null;
FileOutputStream fos = null;
try {
boolean continueDownloading = false;
String tmpFileName = fileName + "_tmp";
outputFile = new File(downloadFolder, tmpFileName);
if (outputFile.exists()) {
localFileLength = outputFile.length();
if (localFileLength > 0) {
continueDownloading = true;
}
if (continueDownloading) {
request.addHeader("Range", "bytes=" + localFileLength + "-");
}
response = httpClient.execute(request);
long remoteFileLength = 0;
Header contentLengthHeader = response.getFirstHeader("Content-Length");
if (contentLengthHeader != null) {
remoteFileLength = Integer.parseInt(contentLengthHeader.getValue());
}
long downloaded = 0;
if (continueDownloading) {
downloaded = localFileLength;
}
long fullFileLength = downloaded + remoteFileLength;
fos = new FileOutputStream(outputFile, true);
is = response.getEntity().getContent();
byte[] buffer = new byte[DOWNLOAD_BUFFER_SIZE];
int len = 0;
while ((len = is.read(buffer)) != -1 && isDownloading) {
fos.write(buffer, 0, len);
downloaded += len;
}
fos.flush();
boolean success = downloaded == fullFileLength;
if (success) {
outputFile.renameTo(new File(downloadFolder, fileName));
}
} catch (Throwable ex) {
ex.printStackTrace();
} finally {
// clean up resources
}
Try using downloadManager instead of downloading manually , there are many advantages to using it.
Here is an example for it : DownloadManager Example
and take a look at the documentations : DownloadManager