I'm working on a project that downloads a zip file and unzips locally. The issue I'm hitting is that the unzip process works like 5% of the time.
It's a mystery to me at this point because sometimes it works, but most of the time it throws data or crc errors. It'll even switch between erros even though the zip file hasn't changed.
I've tried zip files that were created by numerous tools wondering if the format was incorrect. But to no avail. Even zips created in the terminal don't work.
Here's my unzipping code:
try {
String _location = model.getLocalPath();
FileInputStream fin = new FileInputStream(localFile);
ZipInputStream zin = new ZipInputStream(fin);
ZipEntry ze = null;
byte[] buffer = new byte[1024];
while((ze = zin.getNextEntry()) != null) {
if(_cancel) break;
System.out.println("unzipping " + ze.getName());
if(ze.isDirectory()) {
File f = new File(_location + ze.getName());
f.mkdirs();
} else {
FileOutputStream fout = new FileOutputStream(_location + ze.getName());
for(int c = zin.read(buffer); c > 0; c = zin.read(buffer)) {
fout.write(buffer,0,c);
}
zin.closeEntry();
fout.close();
}
}
zin.close();
if(_cancel) {
handler.post(dispatchCancel);
return;
}
} catch(Exception e) {
System.out.println("UNZIP ERROR!");
System.out.println(e.getMessage());
System.out.println(e.toString());
e.printStackTrace();
}
And here's how I typically create the zip file.
$>zip -r myzip.zip myzip/
Here are the two error outputs:
java.util.zip.ZipException: CRC mismatch
at java.util.zip.ZipInputStream.readAndVerifyDataDescriptor(ZipInputStream.java:209)
at java.util.zip.ZipInputStream.closeEntry(ZipInputStream.java:173)
at com.XX.XX.XXIssueDownloader$7.run(XXIssueDownloader.java:222)
at java.lang.Thread.run(Thread.java:1020)
java.util.zip.ZipException: data error
at java.util.zip.ZipInputStream.read(ZipInputStream.java:336)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at com.XX.XX.XXIssueDownloader$7.run(XXIssueDownloader.java:219)
at java.lang.Thread.run(Thread.java:1020)
Anyone have any idea why I might get these errors? I'm not getting anywhere with these.
There are two things very important when loading Zip files.
Make sure you're using a request method that doesn't contain the Accept-Encoding: header. If it's in the request then the response is not a zip file, it's a gzip compressed zip file. So if you're writing that directly to disk while it's downloading then it won't actually be a zip file. You can use something like this to load the zip file:
URL url = new URL(remoteFilePath);
URLConnection connection = url.openConnection();
InputStream in = new BufferedInputStream(connection.getInputStream());
FileOutputStream f = new FileOutputStream(localFile);
//setup buffers and loop through data
byte[] buffer = new byte[1024];
long total = 0;
long fileLength = connection.getContentLength();
int len1 = 0;
while((len1 = in.read(buffer)) != -1) {
if(_cancel) break;
total += len1;
_Progress = (int) (total * 100 / fileLength);
f.write(buffer,0,len1);
handler.post(updateProgress);
}
f.close();
in.close();
When using input and out streams, do NOT use the read(buffer) or write(buffer) method, you need to use read/write(buffer,0,len). Otherwise what you're writing or reading may end up with garbage data in it. The former (read(buffer)) will always read the entire buffer, but there may actually not be a full buffer, for example if the last iteration of the loop only read 512 bytes. So here's how you'd unzip the file:
String _location = model.getLocalPath();
FileInputStream fin = new FileInputStream(localFile);
ZipInputStream zin = new ZipInputStream(fin);
ZipEntry ze = null;
while((ze = zin.getNextEntry()) != null) {
if(_cancel) break;
System.out.println("unzipping " + ze.getName());
System.out.println("to: " + _location + ze.getName());
if(ze.isDirectory()) {
File f = new File(_location + ze.getName());
f.mkdirs();
} else {
byte[] buffer2 = new byte[1024];
FileOutputStream fout = new FileOutputStream(_location + ze.getName());
for(int c = zin.read(buffer2); c > 0; c = zin.read(buffer2)) {
fout.write(buffer2,0,c);
}
zin.closeEntry();
fout.close();
}
}
zin.close();
Related
I'm downloading a file using Download Manager and saving into Download Folder.
After download finish i'm picking up the folder path like this way:
int uriIndex = cursor.getColumnIndex(DownloadManager.COLUMN_LOCAL_URI);
String downloadedPackageUriString = cursor.getString(uriIndex);
Then i need to use this path to unzip the file downloaded. The code to unzip it is below:
SouceFile is the path from downloadmanager.
unzip(String sourceFile, String destinationFolder)
try {
zis = new ZipInputStream(new BufferedInputStream(new FileInputStream(sourceFile)));
ZipEntry ze;
int count;
byte[] buffer = new byte[BUFFER_SIZE];
while ((ze = zis.getNextEntry()) != null) {
String fileName = ze.getName();
fileName = fileName.substring(fileName.indexOf("/") + 1);
File file = new File(destinationFolder, fileName);
File dir = ze.isDirectory() ? file : file.getParentFile();
Log.i("MainService", "Unzipping fileName: " + fileName);
file_path = destinationFolder + "/" + fileName;
if (!dir.isDirectory() && !dir.mkdirs())
throw new FileNotFoundException("Invalid path: " + dir.getAbsolutePath());
if (ze.isDirectory()) continue;
FileOutputStream fout = new FileOutputStream(file);
try {
while ((count = zis.read(buffer)) != -1) {
fout.write(buffer, 0, count);
}
} finally {
fout.close();
}
list_filenames.add(file_downloaded);
}
Log.d("MainService", "TAM:" + tam);
} catch (IOException ioe) {
Log.d("MainService", "Oiiiiiiiiii " + ioe);
return list_filenames;
} finally {
if (zis != null)
try {
zis.close();
} catch (IOException e) {
}
}
The code to unzip works on Android 6 (My phone), but on Android Oreo + it doesnt.
I'm getting (No such file or directory) from zis = new ZipInputStream(new BufferedInputStream(new FileInputStream(sourceFile)));
Reinforcing: I'm not getting this error on Android 6. Just on 8+
Thanks for any help.
I tried some suggestions from other guys here with similar problem, but doesnt works for me.
I'm not getting this error on Android 6
At most, you are not getting this error on the one device that you tested on Android 6.0. There are many device models running Android 6.0, not just one.
COLUMN_LOCAL_URI is supposed to give you a string representation of a Uri. That will not work with FileInputStream, because a Uri is not a file.
Replace:
zis = new ZipInputStream(new BufferedInputStream(new FileInputStream(sourceFile)));
with:
zis = new ZipInputStream(new BufferedInputStream(cr.openInputStream(Uri.parse(sourceFile))));
...where cr is a ContentResolver that you get by calling getContentResolver() on some Context.
To unzip zip file I've used classes from package java.util.zip* by refering this and it works correctly however to unzip a file of 40MB it takes 59 seconds. When I tried same zip file on iPhone project (we are developing app for both platforms - Android & iPone & that have functionality to unzip zip file & save unzipped content to SDCARD-Android or document directory - iPhone ), it takes only 14 seconds. The iphone app uses ziparchive.
So my question are:
1.From above experiment it clears that unzipping & file write operation to SDCARD in Java consumes more time as compared to iPhone app, so I decided to use C/C++ level unzipping & file write operation using NDK.
Is this right choice ?
2.I have searched on google, stackoverflow & some suggested to use minizip but there no sufficient help on how to use minizip in android. Is anyboday have tried minizip for android ?
3.I also tried NDK development for libz to achieve my goal, as Libz is added in NDK but not getting how to use it. Is anybody tried libz in NDK?
4.Is there any other Framework in Java or C/C++ that unzip large zip file & write them to SDCARD in less time ?
Please help me.
Here is my Java Unzip Code :
public String unzip() {
String result;
try {
FileInputStream fin = new FileInputStream(this.filePath);
ZipInputStream zin = new ZipInputStream(fin);
ZipEntry ze = null;
while ((ze = zin.getNextEntry()) != null) {
Log.v("Unzip", "Unzipping " + ze.getName());
if (ze.isDirectory()) {
_dirChecker(ze.getName());
} else {
// Read 16 k at a time
byte[] buffer = new byte[16*1024];
int read;
FileOutputStream fout = new FileOutputStream(this.location+ "/" + ze.getName());
while ((read = zin.read(buffer)) != -1)
{
fout.write(buffer, 0, read);
}
zin.closeEntry();
fout.close();
}
}
zin.close();
result = "success";
} catch (Exception e) {
Log.e("unzip", "unzip", e);
result = "failure";
}
return result;
}
Why don't you try this code.It works awesome
String zipname = "data.zip";
FileInputStream fis = new FileInputStream(zipname);
ZipInputStream zis = new ZipInputStream(new BufferedInputStream(fis));
ZipEntry entry;
while ((entry = zis.getNextEntry()) != null) {
System.out.println("Unzipping: " + entry.getName());
int size;
byte[] buffer = new byte[2048];
FileOutputStream fos = new FileOutputStream(entry.getName());
BufferedOutputStream bos = new BufferedOutputStream(fos, buffer.length);
while ((size = zis.read(buffer, 0, buffer.length)) != -1) {
bos.write(buffer, 0, size);
}
bos.flush();
bos.close();
}
zis.close();
fis.close();
}
All of the unzip code eventually ends up in zlib. There is no Java implementation of "deflate" compression in Android core libs.
The only reason java.util.Zip should be slower than a "native" unzip is if the file I/O is done badly, e.g. something is using really small buffers. Looking at the code linked from the question, this is exactly what's happening -- it's operating on individual bytes.
One of the comments on the solution provided a patch that uses a 4K buffer. Drop that in and see what happens to your performance.
Try to just write 40Mb file to SDCard and measure time spent.
(Almost) all free (or even paid) implementations of zip archive support libraries are based on the same zlib code, which takes most processing speed during the unzipping. Java code should be much slower than native one, so I'd suggest to try NDK unzipping. Also, trying to unzip archive with zero compression level will give you a guess how much time unzipping code takes and how much time is spend on just data copying.
public boolean unzip(String zipfilepath, String destinationdir) {
try {
FileInputStream fis = new FileInputStream(zipfilepath);
ZipInputStream zis = new ZipInputStream(new BufferedInputStream(fis));
ZipEntry entry;
while ((entry = zis.getNextEntry()) != null) {
System.out.println("Unzipping: " + entry.getName());
int size;
byte[] buffer = new byte[2048];
FileOutputStream fos = new FileOutputStream(destinationdir+ "/" + entry.getName());
BufferedOutputStream bos = new BufferedOutputStream(fos, buffer.length);
while ((size = zis.read(buffer, 0, buffer.length)) != -1) {
bos.write(buffer, 0, size);
}
bos.flush();
bos.close();
}
zis.close();
fis.close();
return true;
}catch(Exception e){
return false;
}
}
String zipfilepath = context.getFilesDir().getPath()+"/"+myfile.zip;
String destinationdir = context.getFilesDir().getPath();
unzip(zipfilepath, destinationdir);
Need to display images downloaded and extracted from a zip into the "/files" directory of the app. the images are getting in there properly as far as I can tell - I am able to extract them from the emulator and view/open them from my desktop. but every attempt, every variation of code I have found and tried so far has failed (Tag: skia / Text: --- decoder->decode returned false).
My latest construct, which does work for image files downloaded separately and uncompressed :
String imgFile = new File(getFilesDir(), "myImage.jpg").getAbsolutePath();
ImageView myImageView = new ImageView(this);
Bitmap bm = null;
try{
bm = BitmapFactory.decodeFile(imgFile);
myImageView.setImageBitmap(bm);
} finally{
mainLayout.addView(myImageView);
}
And here is the construct I am using to handle the zip extraction. I assume this is where the problem lies but I am clueless as to what I could possibly do differently and to what effect:
ZipInputStream zis = new ZipInputStream(fis);
BufferedInputStream in = new BufferedInputStream(zis, 8192);
ZipEntry ze;
while ((ze = zis.getNextEntry()) != null){
File dest_file = new File(getFilesDir(), ze.getName());
FileOutputStream fout = new FileOutputStream(getFilesDir() + "/" + ze.getName());
BufferedOutputStream out = new BufferedOutputStream(fout, 8192);
byte b[] = new byte[1024];
int n;
while ((n = in.read(b,0,1024)) >= 0) {
out.write(b,0,n);
}
for (int c = zis.read(); c != -1; c = zis.read()) {
fout.write(c);
}
zis.closeEntry();
fout.close();
}
zis.close();
fis.close();
At a terrible standstill here. Appreciate any solutions/suggestions.
I am trying to read a response from a server and transform it from InputStream to String but something goes wrong and i cannot see right now why.
InputStream is = entity.getContent();
FileOutputStream folder = new FileOutputStream(
Environment.getExternalStorageDirectory()
+ "/test.xml");
try {
byte[] buf = new byte[1048576];
int current = 0;
int newCurrent = 0;
while ((current = is.read(buf)) != -1) {
newCurrent = newCurrent + current;
folder.write(buf, 0, current);
}
System.out.println("returned folder" + folder);
folder.flush();
} catch (IOException e) {
System.out.println("error on reading input: " + e.getMessage());
}
This is the error:
error on reading input: Socket closed
This is the error I get and another problem that i don't understand is why it does not read the entire content from InputStream(maybe because it's all in one line?).
Thanks.
you dont need to read entire stream in one shot and put it in a byte array, in fact you read it in parts through while loop and put the contents in file stream gradually:
int count;
byte[] filebytes = new byte[1024];
while((count = is.read(filebytes)) != -1){
folder.write(filebytes, 0, count); //writing buffer into file
}
in.close();
folder.flush();
folder.close();
According to stacktrace your crash happens in readLine() while your code uses is.read(buf).
DO these match together?
I am currently writing an application that reads a zip file in my assets folder which contains a bunch of images. I am using the ZipInputStream API to read the contents and then writing each file to my: Environment.getExternalStorageDirectory() directory. I have everything working but the first time the application is run writing the images to the storage directory is INCREDIBLY slow. It takes about about 5 minutes to write my images to disc. My code looks like this:
ZipEntry ze = null;
ZipInputStream zin = new ZipInputStream(getAssets().open("myFile.zip"));
String location = getExternalStorageDirectory().getAbsolutePath() + "/test/images/";
//Loop through the zip file
while ((ze = zin.getNextEntry()) != null) {
File f = new File(location + ze.getName());
//Doesn't exist? Create to avoid FileNotFoundException
if(f.exists()) {
f.createNewFile();
}
FileOutputStream fout = new FileOutputStream(f);
//Read contents and then write to file
for (c = zin.read(); c != -1; c = zin.read()) {
fout.write(c);
}
}
fout.close();
zin.close();
The process of reading the contents of the particular entry and then writing to it is VERY slow. I am assuming it is more to do with reading than writing. I've read that you can use a byte[] array buffer to speed up the process but this does not seem to work! I tried this but it only read part of the file...
FileOutputStream fout = new FileOutputStream(f);
byte[] buffer = new byte[(int)ze.getSize()];
//Read contents and then write to file
for (c = zin.read(buffer); c != -1; c = zin.read(buffer)) {
fout.write(c);
}
}
When I do that I only get about 600-800 bytes written. Is there a way to speed this up?? Have I implemented the buffer array incorrectly??
I found a much better solution which implements the BuffererdOutputStream API. My solution looks like this:
byte[] buffer = new byte[2048];
FileOutputStream fout = new FileOutputStream(f);
BufferedOutputStream bos = new BufferedOutputStream(fout, buffer.length);
int size;
while ((size = zin.read(buffer, 0, buffer.length)) != -1) {
bos.write(buffer, 0, size);
}
//Close up shop..
bos.flush();
bos.close();
fout.flush();
fout.close();
zin.closeEntry();
I managed to increase my load time from anywhere from an average of about 5 minutes to about 5 (depending on how many images are in the package). Hope this helps!
Try use http://commons.apache.org/io/
like:
InputStream in = new URL( "http://jakarta.apache.org" ).openStream();
try {
System.out.println( IOUtils.toString( in ) );
} finally {
IOUtils.closeQuietly(in);
}