Develop Reference

Download code in android downloads incomplete files

I have followed some online tutorials and created this code to download the files that i have hosted in dropbox
I am using async task to do this
// AsyncTask to download a file
private class DownloadTask extends AsyncTask<String, Integer, String> {
private Context context;
public DownloadTask(Context context) {
this.context = context;
}
#Override
protected String doInBackground(String... sUrl) {
// take CPU lock to prevent CPU from going off if the user
// presses the power button during download
PowerManager pm = (PowerManager) context
.getSystemService(Context.POWER_SERVICE);
PowerManager.WakeLock wl = pm.newWakeLock(
PowerManager.PARTIAL_WAKE_LOCK, getClass().getName());
wl.acquire();
try {
InputStream input = null;
OutputStream output = null;
HttpURLConnection connection = null;
try {
URL url = new URL(sUrl[0]);
connection = (HttpURLConnection) url.openConnection();
connection.connect();
// expect HTTP 200 OK, so we don't mistakenly save error
// report
// instead of the file
if (connection.getResponseCode() != HttpURLConnection.HTTP_OK)
return "Server returned HTTP "
+ connection.getResponseCode() + " "
+ connection.getResponseMessage();
// TODO
File file = new File(Environment
.getExternalStorageDirectory().getPath()
+ "/kathmandu.map");
if (file.exists()) {
Log.i("File Exists", "Code Gets here, file exists");
return "exists";
// if (connection.getResponseCode() ==
// HttpURLConnection.HTTP_NOT_MODIFIED) {
//
// return null;
// }
}
// this will be useful to display download percentage
// might be -1: server did not report the length
int fileLength = connection.getContentLength();
Log.i("Length", String.valueOf(fileLength));
// download the file
input = connection.getInputStream();
output = new FileOutputStream(Environment
.getExternalStorageDirectory().getPath()
+ "/kathmandu.map");
byte data[] = new byte[4096];
long total = 0;
int count;
while ((count = input.read(data)) != -1) {
// allow canceling with back button
if (isCancelled())
return null;
total += count;
// publishing the progress....
if (fileLength > 0) // only if total length is known
publishProgress((int) (total * 100 / fileLength));
output.write(data, 0, count);
}
} catch (Exception e) {
return e.toString();
} finally {
try {
if (output != null)
output.close();
if (input != null)
input.close();
} catch (IOException ignored) {
}
if (connection != null)
connection.disconnect();
}
} finally {
wl.release();
}
return null;
}
I call the download code when the download options menu is clicked.
final DownloadTask downloadTask = new DownloadTask(MapActivity.this);
downloadTask
.execute("https://dl.dropboxusercontent.com/u/95497883/kathmandu-2013-8-12.map");
mProgressDialog
.setOnCancelListener(new DialogInterface.OnCancelListener() {
#Override
public void onCancel(DialogInterface dialog) {
downloadTask.cancel(true);
}
});
The code works fine but at the times the outputstream does not write full file and exits. Everything looks okay. The file is downloaded but it is corrupted.
The getContentLength() also returns -1 so i cannot check if the whole file has been downloaded using the content length. The file is a offline vector map and i need it to display offline maps. The corrupted file causes a runtime exception while trying to access it. Is there is any way to ensure that the file has been downloaded correctly.
Also i would like to provide the data with the app itself. Can i put this in the assets folder of my app. What is the best way to access the files in the assets folder during runtime.

Your assets folder is not writable as it is a part of the apk. you can of course use your application's sandbox storage (using Environment.getDir() ) or external storage (using Environment.getExternalStorageDirectory()) like you have done in your code.
I think using the DownloadManager would be a great idea to achieve exactly what you want please refer : http://developer.android.com/reference/android/app/DownloadManager.html
a short solution
DownloadManager.Request req=new DownloadManager.Request(url);
req.setAllowedNetworkTypes(DownloadManager.Request.NETWORK_WIFI
| DownloadManager.Request.NETWORK_MOBILE)
.setTitle("Downloading")
.setDescription("Map is Being Downloaded")
.setDestinationInExternalPublicDir(Environment.getExternalStorageDirectory,
"+/maps_app/something.map");

0-byte files not detected when downloading files on Android

I have an app for Android which downloads hundreds of files from the Internet. Some files turn out to be 0-byte after download. The app attempts to detect such cases and delete such files after download but sometimes it fails. The problem is more frequent on Android 4.x devices.
Here is the method which does the downloading. I gets the number of actually read bytes from inputStream.read(buffer).
public class Utils
{
public static class DownloadFileData
{
int nTotalSize;
int nDownloadedSize;
}
public interface ProgressCallback
{
void onProgress(long nCurrent, long nMax);
}
public static boolean downloadFile(String sFileURL, File whereToSave, DownloadFileData fileData, ProgressCallback progressCallback)
{
InputStream inputStream = null;
FileOutputStream fileOutput = null;
try
{
URL url = new URL(sFileURL);
URLConnection connection = url.openConnection();
//set up some things on the connection
connection.setDoOutput(true);
connection.connect();
fileOutput = new FileOutputStream(whereToSave);
inputStream = connection.getInputStream();
fileData.nTotalSize = connection.getContentLength();
fileData.nDownloadedSize = 0;
byte[] buffer = new byte[1024];
int bufferLength = 0; //used to store a temporary size of the buffer
// now, read through the input buffer and write the contents to the file
while ((bufferLength = inputStream.read(buffer)) > 0)
{
// if interrupted, don't download the file further and return
// also restore the interrupted flag so that the caller stopped also
if (Thread.interrupted())
{
Thread.currentThread().interrupt();
return false;
}
// add the data in the buffer to the file in the file output stream
fileOutput.write(buffer, 0, bufferLength);
// add up the size so we know how much is downloaded
fileData.nDownloadedSize += bufferLength;
if (null != progressCallback && fileData.nTotalSize > 0)
{
progressCallback.onProgress(fileData.nDownloadedSize, fileData.nTotalSize);
}
}
return true;
}
catch (FileNotFoundException e)
{
return false; // swallow a 404
}
catch (IOException e)
{
return false; // swallow a 404
}
catch (Throwable e)
{
return false;
}
finally
{
// in any case close input and output streams
if (null != inputStream)
{
try
{
inputStream.close();
inputStream = null;
}
catch (Exception e)
{
}
}
if (null != fileOutput)
{
try
{
fileOutput.close();
fileOutput = null;
}
catch (Exception e)
{
}
}
}
}
Here is the piece of code which processes the downloads. Since sometimes the number of read bytes is incorrect (it is > 0 and the real file has the size 0 bytes) I check the size of the downloaded file with outputFile.length(). But this again gives a value > 0 even though the file is really 0 byte. I tried to also just create a new file and read its size with recheckSizeFile.length(). Still the size is determined as > 0 while it's really 0 byte.
Utils.DownloadFileData fileData = new Utils.DownloadFileData();
boolean bDownloadedSuccessully = Utils.downloadFile(app.sCurrenltyDownloadedFile, outputFile, fileData, new Utils.ProgressCallback()
{
... // progress bar is updated here
});
if (bDownloadedSuccessully)
{
boolean bIsGarbage = false;
File recheckSizeFile = new File(sFullPath);
long nDownloadedFileSize = Math.min(recheckSizeFile.length(), Math.min(outputFile.length(), fileData.nDownloadedSize));
// if the file is 0bytes, it's garbage
if (0 == nDownloadedFileSize)
{
bIsGarbage = true;
}
// if this is a video and if of suspiciously small size, it's
// garbage, too
else if (Utils.isStringEndingWith(app.sCurrenltyDownloadedFile, App.VIDEO_FILE_EXTENSIONS) && nDownloadedFileSize < Constants.MIN_NON_GARBAGE_VIDEO_FILE_SIZE)
{
bIsGarbage = true;
}
if (bIsGarbage)
{
++app.nFilesGarbage;
app.updateLastMessageInDownloadLog("File is fake, deleting: " + app.sCurrenltyDownloadedFile);
// delete the garbage file
if (null != outputFile)
{
if (!outputFile.delete())
{
Log.e("MyService", "Failed to delete garbage file " + app.sCurrenltyDownloadedFile);
}
}
}
else
{
... // process the normally downloaded file
}
I am not sure but I think there is a bug in Android with reading file size. Has anyone seen a similar problem? Or am I maybe doing something wrong here?
Thanks!
EDIT: how i determine that the files are 0-byte:
all the files which get downloaded go thru the described routines. When I then later view the download folder with a file browser (Ghost Commander), some of the files (like maybe 10%) are 0-byte. They can't be played by a video player (shown as "broken file" icon).

It looks to me like your problem is that you only check for "garbage" files if the Utils.downloadFile call returns true. If the download fails in the getInputStream call or the first read, you will have created a file with zero length which will never be deleted.

You should call flush() on your FileOutputStream to ensure that all data is written to the file. This should make your issue with 0-byte files occur less often.
To check for 0 byte files using File.length() should work properly. Can you open a shell (adb shell) on the device and run ls -l to see the byte count displayed by it is 0 (maybe your file manager has some weird issues). Also please debug (or put some log statements) that sFullPath contains the correct file paths. I can't see where sFullPath gets set in your code above and why you don't just use outputFile but recreate another File object.

Need help debugging my code

I need some input about my code.
Basically, I have a method to load music from Class A
public void onListItemClick(ListView parent, View v, int position, long id){
musicIndex = cursor.getColumnIndexOrThrow(MediaStore.Audio.Media.DATA);
cursor.moveToPosition(position);
filePath = cursor.getString(musicIndex);
fileName = new File(filePath).getName();
playMusic();//Play the selected music
}
public void playMusic(){
if(mPlayer.isPlaying()){
mPlayer.reset();
}
try{
mPlayer.setDataSource(filePath);
mPlayer.prepare();
mPlayer.start();
BeatDetection beatDetect = new BeatDetection();
beatDetect.init();
}catch (Exception e){
}
}
That method will call the init() method in Class B
public void init() throws Exception{
energy = 0;
variance = 0;
constant = 0;
isBeat = false;
sensitivity = 0;
dBuffer = new float[sampleRate / bufferSize];
eBuffer = new float[sampleRate / bufferSize];
timer = System.currentTimeMillis();
MusicLoad msc = new MusicLoad();
totalMs = 0;
seeking = true;
//msc.printText();
decode(msc.fileName, 25, 40);
}
In that method, it initializes everything and call the decode() method
public void decode(String path, int startMs, int maxMs)
throws IOException, javazoom.jl.decoder.DecoderException {
debug();
File in = new File(path);
InputStream inStream = new BufferedInputStream(new FileInputStream(in), 8 * 1024);
ByteArrayOutputStream outStream = new ByteArrayOutputStream(1024);
try {
Bitstream bitstream = new Bitstream(inStream);
Decoder decoder = new Decoder();
boolean done = false;
while (! done) {
Header frameHeader = bitstream.readFrame();
if (frameHeader == null) {
done = true;
} else {
totalMs += frameHeader.ms_per_frame();
if (totalMs >= startMs) {
seeking = false;
}
if (! seeking) {
SampleBuffer output = (SampleBuffer) decoder.decodeFrame(frameHeader, bitstream);
if (output.getSampleFrequency() != 44100 || output.getChannelCount() != 2) {
throw new javazoom.jl.decoder.DecoderException("mono or non-44100 MP3 not supported", null);
}
short[] pcm = output.getBuffer();
for (short s : pcm) {
outStream.write(s & 0xff);
outStream.write((s >> 8 ) & 0xff);
}
}
if (totalMs >= (startMs + maxMs)) {
done = true;
}
}
bitstream.closeFrame();
}
byte[] abAudioData = outStream.toByteArray();
calculation(abAudioData);
} catch (BitstreamException e) {
throw new IOException("Bitstream error: " + e);
} catch (DecoderException e) {
Log.w("Decoder error", e);
throw new javazoom.jl.decoder.DecoderException("Error",e);
} finally {
inStream.close();
}
}
Don't mind reading all the code lines. If you guys notice I put debug() in the beginning to see whether the method is called or not. At this point, the debug() is properly called. However, if I put the debug() after the line File in = new File(path);, the debug() will not be called anymore. It seems like the code is stop running at that point.
The ultimate result is, I can load and play the song without any problem. However, the decode() is not called and there is no error whatsoever. I'm stuck at pointing out the problem at this point. So if there's any input please help me.
EDIT: After I tried tracing the "path" variable, it returns NULL so the error is NullPointerException. Seems like the "fileName" variable from Class A is not passed to Class B. Any suggestion?

If you are using Eclipse with ADT then it's very easy to debug your Android apps, just add a breakpoint (probably in the new File(...) line) and see what happens.
My guess here is that File in = new File(path); probably is throwing a IOException in your decode method, that exception is bubbling first to init() and then to playMusic(), where it is caught by try catch block. Your catch is empty so you are not seeing anything. Try debugging as I said or add some logging info in the catch block.

This is just something to look at, but from the doc page
http://developer.android.com/reference/java/io/File.html#File%28java.lang.String%29
"The actual file referenced by a File may or may not exist. It may also, despite the name File, be a directory or other non-regular file."
If you had the path wrong, it may be trying to create the file and you may not have the correct permission to do so. Perhaps: WRITE_EXTERNAL_STORAGE.

I know this post is old, but I just wanted to show how to get the file path to read/write files for others that come across this post as I have:
String filePath = myContext.getFilesDir().getPath().toString() + "/sysout.log";
File file = new File(filePath);
These two lines will create (open if it exists, and overwrite) a file named "sysout.log" in the folder /data/data/com.app.name/files/; myContext is just the current context. Using this technique alleviates problems with defining your own path name. Hope this helps someone.

Android app freezes irregularly when downloading files

I'm making an Android app where the user can download files from a FTP-server. For the ftp parts I am using apache.org.commons-net package.
When I have connected to server I get a list of the filenames, and then I want to download each file. I start the download routine which runs in a thread of it's own.
The problem I'm experiencing is, that if I have say 6 files on the server, and I run this code on my emulator, it will download the first two files, and then just freeze (with the progressbar hanging on 34 %). When I run it on my phone it will download three files and freeze.
If I debug my way through the code on the emulator it will download all six files just fine and not freeze.
Does anyone have any idea what might be the problem?
Thanks in advance,
LordJesus
This is my code (the client is already initialized):
private void downloadFiles2() {
dialog = new ProgressDialog(this);
dialog.setCancelable(true);
dialog.setMessage("Loading...");
// set the progress to be horizontal
dialog.setProgressStyle(ProgressDialog.STYLE_HORIZONTAL);
// reset the bar to the default value of 0
dialog.setProgress(0);
dialog.setMax(DownloadCount);
// display the progressbar
dialog.show();
// create a thread for updating the progress bar
Thread background = new Thread (new Runnable() {
public void run() {
InputStream is = null;
try {
for (String filename : fileNames) {
is = client.retrieveFileStream(filename);
byte[] data = new byte[1024];
int x = 0;
x = is.read(data, 0, 1024);
boolean downloadIsNewer = true;
File fullPath = new File(path + "/" + filename);
Log.d("FTP", "Starting on " + filename);
if (fullPath.exists()) {
downloadIsNewer = checkIfNewer(data, fullPath);
}
if (downloadIsNewer) {
Log.d("FTP", "Need to download new file");
FileOutputStream fos = new FileOutputStream(fullPath);
fos.write(data,0,x);
while((x=is.read(data,0,1024))>=0){
fos.write(data,0,x);
}
fileText += filename + " - downloaded OK." + FileParser.newline;
is.close();
client.completePendingCommand();
fos.flush();
fos.close();
}
else {
Log.d("FTP", "No need to download");
is.close();
fileText += filename + " - own copy is newer." + FileParser.newline;
}
// active the update handler
progressHandler.sendMessage(progressHandler.obtainMessage());
}
client.logout();
client.disconnect();
}
catch (Exception e) {
// if something fails do something smart
}
}
});
// start the background thread
background.start();
}
Handler progressHandler = new Handler() {
public void handleMessage(Message msg) {
dialog.incrementProgressBy(1);
InfoTextView.setText(fileText);
if(dialog.getProgress()== dialog.getMax())
{
dialog.dismiss();
}
}
};

OK, found the problem:
When the boolean downloadIsNewer is false I do not call client.completePendingCommand(). When I add that before the line is.close() it works like a charm.

Unzipping files in a faster way than using the java.util.zip in Android

I need unzip a .zip file of 2.5mb(1087 files - *.html, *.css and *.db) in android, i have used java.util.zip, it works fine, but i need improve the performance, the unzip process last 1.10 minutes, i need reduce this time.
I have followed some recomendations for improve the performance, for example :
Use BufferedInputStream, FileOutputStream and BufferedOutputStream.
Read the zip in blocks :
byte data[] = new byte[2048];
while ((counter = bisMediaFile.read(data, 0, 2048)) != -1)
{
bosMediaFile.write(data, 0, counter);
}
Is there any way to improve my code?. I was searching third party zip programs to use programatically, for example i tried the 7ZipJBinding, but it looks like android doesn't support this, because i referenced the sevenzipjbinding.jar and sevenzipjbinding-AllPlatforms.jar but i get an error : "Native Libraries Detected in sevenzipjbinding-AllPlatforms". At 7zip homepage there are versions for MAC, Windows, Linux, but i didn't see anything about android.
Could you please recommend any other library to unzip files in android?
This is my all code :
public static void processZipFile(String strBinaryPath,String strExtractPath, String strDestinationDBPath) throws Exception
{
ZipFile zipInFile = null;
try
{
if (strExtractPath != null)
{
zipInFile = new ZipFile(strBinaryPath);
for (Enumeration<? extends ZipEntry> entries = zipInFile.entries(); entries.hasMoreElements();)
{
ZipEntry zipMediaEntry = entries.nextElement();
if (zipMediaEntry.isDirectory())
{
File mediaDir = new File(String.format("%s\\%s", strExtractPath, zipMediaEntry.getName()));
mediaDir.mkdirs();
}
else
{
BufferedInputStream bisMediaFile = null;
FileOutputStream fosMediaFile = null;
BufferedOutputStream bosMediaFile = null;
try
{
String strFileName = String.format("%s\\%s", strExtractPath, zipMediaEntry.getName());
File uncompressDir = new File(strFileName).getParentFile();
uncompressDir.mkdirs();
//if is a database file, extract to other path : android.movinginteractive.com/databases
if(strFileName.contains(".db"))
strFileName = String.format("%s\\%s", strDestinationDBPath, ExtractDBName(zipMediaEntry.getName()));
bisMediaFile = new BufferedInputStream(zipInFile.getInputStream(zipMediaEntry));
fosMediaFile = new FileOutputStream(strFileName);
bosMediaFile = new BufferedOutputStream(fosMediaFile);
int counter;
byte data[] = new byte[2048];
while ((counter = bisMediaFile.read(data, 0, 2048)) != -1)
{
bosMediaFile.write(data, 0, counter);
}
}
catch (Exception ex)
{
throw ex;
}
finally
{
if (bosMediaFile != null)
{
bosMediaFile.flush();
bosMediaFile.close();
}
if (bisMediaFile != null)
bisMediaFile.close();
}
}
}
}
}
catch (Exception ex)
{
throw ex;
}
finally
{
if (zipInFile != null)
zipInFile.close();
File flZipToDelete = new File(strBinaryPath);
if(flZipToDelete.exists())
flZipToDelete.delete();
}
}

I'm sure you could find a C or C++ code snippet for unzipping files and run it through the Android NDK. That said, I'm not sure what performance gains you might get.