I'm writting an app that deals with Zip files of content. I have a few different unzipping methods, depending on where the content is stored.
When unzipping on the SD card, I'm using ZipFile, and that's working great, as I'm always trying to get a specific file out of the zip.
When I'm dealling with a zip that is stored in Assets, I'm resorting to getting an InputStream from the asset, and then iterating through the stream to find the specific file that I'm looking for. e.g:
ZipInputStream zipInputStream = new ZipInputStream(bufferedInputStream); // Editted following comment from Jave
ZipEntry entry;
ZipEntry targetEntry = null;
try {
while (targetEntry == null
&& (entry = zipInputStream.getNextEntry()) != null)
{
if (entry.getName().endsWith(fileName))
{
targetEntry = entry;
}
}
I'm finding that this iteration is taking way too much time. Does anyone know of a faster method to unzip an asset? (for example is there an alternate way to grab an asset, rather than getting it as a Stream?)
I can't really think of anything (other than perhaps copying the zip to the SD before I begin).
ZipInputStream inZip = new ZipInputStream(MainActivity.this.getAssets().open(_zipFile));
Like this,your can get a zipFile by name from assets.
Related
This is for an experimental Android project done in Java. What I need is to read a ZIP file from an external card (as fast as possible, if possible) and return the InputStream corresponding to an entry each time it is needed by an OSMDroid custom tiles provider.
I can do this pretty easily when serving a ZIP from the primary (or whatever it's called within their mangled filesystem) disk but when trying to read from an external card, the only option in newer versions of Android is to use their shiny SAF.
When the path is not a relative one but a URI to an external card, I use a DocumentFile to retrieve the ZIP's InputStream (apparently, the only way to do it).
DocumentFile file = DocumentFile.fromSingleUri(this, uri);
ContentResolver contentResolver = getContentResolver();
InputStream inStream = contentResolver.openInputStream(file.getUri());
Question 1:
What would be amazing at this point, where using DocumentFile already took its toll on performance due to how slower it is than the good ol File, is to be able to have that inStream converted into a ZipFile. Or somehow directly get a ZipFile from DocumentFile. Is that possible? Maybe get a File from DocumentFile then a ZipFile from that?
I didn't find any solution so I went on with the InputStream, which I converted to a ZipInputStream. The problem in my code is that I can only run through it once, when what I need is to do it every time a file's stream from the Zip is needed.
public class MyZipFileArchive implements IArchiveFile {
protected static ZipInputStream mZipInputStream;
}
public void init(InputStream inStream) {
mZipInputStream = new ZipInputStream(inStream);
}
public InputStream getInputStream(ITileSource pTileSource, long pMapTileIndex) {
final String path = pTileSource.getTileRelativeFilenameString(pMapTileIndex);
ZipEntry entry = null;
while ((entry = mZipInputStream.getNextEntry()) != null) {
if (path.equals(entry.getName())) {
return mZipInputStream;
}
}
return null;
}
Question 2:
The first time the getInputStream is called it returns what it finds but now the stream is at its end and the second time it does it, null is returned.
So, I need a way to either reset the stream, if possible, each time the function is called, or use another approach. I tried storing each ZipEntry into a List at the beginning, then loop through it. It doesn't work. Also, I did not find a way to convert ZipEntry into the InputStream required to be returned.
How would you tackle this? Basically, I need to use DocumentFile and I need to get a ZipFile from it or some fast way to find its entries.
I'm trying to populate a ListView with a mixture of files stored on the SDcard AND stored as assets in the APK. Using TraceView, I can see that the performance of AssetManager.list() is poor in comparison to File.listFiles(), even though I'm using a filename filter for the SDcard.
Here is a simple method that returns all the png files from a folder on the SDcard:
// The folder on SDcard may contain files other than png, so filter them out
private File[] getMatchingFiles(File path) {
File[] flFiles = path.listFiles(new FilenameFilter() {
public boolean accept(File dir, String name) {
name = name.toLowerCase();
return name.endsWith(".png");
}
});
return flFiles;
}
I invoke that method here and it takes about 12ms to retrieve 16 files:
final String state = Environment.getExternalStorageState();
if (Environment.MEDIA_MOUNTED.equals(state)||Environment.MEDIA_SHARED.equals(state)) {
File path = Environment.getExternalStoragePublicDirectory(getResources().getString(R.string.path_dir));
if (path.exists()){
File[] files = getMatchingFiles(path);
...
Whereas the am.list method takes 49ms to retrieve just the names of about 6 files!
// Get all filenames from specific Asset Folder and store them in String array
AssetManager am = getAssets();
String path = getResources().getString(R.string.path_dir);
String[] fileNames = am.list(path);
...
Can anyone explain why the performance would be so poor? Is the performance proportional to the number of assets stored in the APK? I'm aware that assets are compressed, but I'm only fetching the names of the assets, which I thought would be stored in a table somewhere.
Coverdriven's comment "stored in a table somewhere" inspired me to solve my own problem which I've been putting off for a while.
This doesn't answer the OP but does offer a different approach and it handles subfolders which CommonsWare's solution doesn't unless you go recursive (which of course is another possible solution). It's specifically aimed at apps which have a large number of assets in subfolders.
I added an ANT pre-build target to run this command (I'm on Windows)
dir assets /b /s /A-d > res\raw\assetfiles
This creates a recursive (/s), barebones (/b) listing of all files, excluding directory entries (/A-d) in my assets folder.
I then created this class to statically load the contents of assetfiles into a hashmap, the key of which is the filename and the value the full path
public class AssetFiles {
// create a hashmap of all files referenced in res/raw/assetfiles
/*map of all the contents of assets located in the subfolder with the name specified in FILES_ROOT
the key is the filename without path, the value is the full path relative to FILES_ROOT
includes the root, e.g. harmonics_data/subfolder/file.extension - this can be passed
directly to AssetManager.open()*/
public static HashMap<String, String> assetFiles = new HashMap<String, String>();
public static final String FILES_ROOT = "harmonics_data";
static {
String line;
String filename;
String path;
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(TidesPlannerApplication.getContext().getResources().openRawResource(R.raw.assetfiles)));
while ((line = reader.readLine()) != null) {
// NB backlash (note the escape) is specific to Windows
filename = line.substring(line.lastIndexOf("\\")+1);
path = line.substring(line.lastIndexOf(FILES_ROOT)).replaceAll("\\\\","/");;
assetFiles.put(filename, path);
}
} catch (IOException e) {
e.printStackTrace();
}
}
public static boolean exists(String filename){
return assetFiles.containsKey(filename);
}
public static String getFilename(String filename){
if (exists(filename)){
return assetFiles.get(filename);
} else {
return "";
}
}
}
To use it, I simply call AssetFiles.getFilename(filename) which returns the full path which I can pass to AssetManager.open(). Much much faster!
NB. I haven't finished this class and it's not hardened yet so you'll need to add appropriate exception catches and actions. It's also quite specific to my app in that all of my assets are in subfolders which are in turn located in a subfolder of the assets folder (see FILES_ROOT) but easy to adapt to your situation.
Note also the need to replace backslashes, since Windows generates the assetfiles listing, with forward slashes. You could eliminate this on OSX and *nix platforms.
Can anyone explain why the performance would be so poor?
Reading the contents of a ZIP archive (the APK where the assets are located) is slower than reading the contents of a directory on the filesystem, apparently. In the abstract, this is not especially surprising, as I suspect that this would be true for all major operating systems.
Read in that list() data once, then save it somewhere else for faster access (e.g., database), particularly in a form that is optimized for future lookups (e.g., where a simple database query could give you what you want, vs. having to load and "recursively search it" again).
If you have a deep tree of directories in the assets you can detect firstly if an item is file or directory and then call .list() on it (really accelerates the walking through the tree). This is my solution I've discovered for this:
try {
AssetFileDescriptor desc = getAssets().openFd(path); // Always throws exception: for directories and for files
desc.close(); // Never executes
} catch (Exception e) {
exception_message = e.toString();
}
if (exception_message.endsWith(path)) { // Exception for directory and for file has different message
// Directory
} else {
// File
}
You can approach APK package as it's a ZIP file and read all the entries using Java's builtin ZipFile. It will give you all the file names with their full paths. Perhaps it shouldn't be hard to find which directories you have.
So far this is the fastest approach I've tested.
credit goes to #obastemur's commit on jxcore-android-basics sample project
I have a big archive (zip in my case) with size ~100MB and with ~15000 files in it. I need to QUICKLY extract only one file form this archive.
I tried the next code:
final String zipPath = "archive.zip";
FileInputStream fin = new FileInputStream(zipPath);
in = new ZipInputStream(fin);
for (ZipEntry entry = in.getNextEntry(); entry != null; entry = in.getNextEntry()) {
if(entry.equals("file.name")){
//unzip this entry
break;
}
}
It works but too SLOW.
Is it some another possibility to find necessary file in archive? For example, on linux it extremally fast possible with command
unzip archive.zip myfile.name
In general, I need to find and decompress one file from some archive. It can be some another format... May be with another format it can be more easy.
You can use the libzip library.
I guess I'm a little confused as to how files are stored on an actual machine (or emulator even).
While programming, I can save my xml file in the assets folder manually, but how to write an app that will have to connect to the network and download the file,save it somewhere and then manipulate it ? where will it store said file ?
I want to create a new file, but I read on another post that the assets folder as such is not available once packaged; So where are they created and stored ? How can they be transferred. Its just, I'm new to this platform and the file system is a little confusing.
If you want to use XML that is updated, you should think of copying the file(s) from assets to device storage. You can take a look at How to copy files from 'assets' folder to sdcard? to know how this can be done.
Another alternative is to use the database where you can store the parsed data from the XML. So that you need not parse the file whenever you need to access the contents.
You have two options: call getFilesDir() from your activity to obtain a path to the internal data folder that can only be read/write from your app.
Or, you can write/read your xml file to external storage (SD Card). Use the method Environment.getExternalStorageDirectory() to get the root path of the external storage, then create your own folder as you see fit.
Note that if you write to external storage, every app in the phone will have access to it.
Even I faced this issue. Now I have a xml file which is has application properties.This is packaged in the assets folder.Once packaged we cannot edit a file in assets folder.
Now on app load I just copy this file to path returned by
context.getFilesDir().getAbsolutePath();
And the application edit it from the same place. You can see if the file is modified in the FileExplorer panel of DDMS view. The file is stored in the folder named same as your application package name for eg: com.abhi.maps
Alternatively you can also copy it to SD card.However it is risky because, sd card may bot be available all the time.
You can use the following code to copy file from assets folder:
private static void copyFile(String filename, Context context) {
AssetManager assetManager = context.getAssets();
InputStream in = null;
OutputStream out = null;
try {
in = assetManager.open(filename);
String newFileName = context.getFilesDir() + "/" + filename;
out = new FileOutputStream(newFileName);
byte[] buffer = new byte[1024];
int read;
while ((read = in.read(buffer)) != -1) {
out.write(buffer, 0, read);
}
in.close();
in = null;
out.flush();
out.close();
out = null;
} catch (Exception e) {
Log.e("tag", e.getMessage());
}
}
Hope it helps! :)
I am trying to obtain only one file (I know its name) from very large zip archive. This archive include around 100000 files because I do not want find my file in loop. I think that must be some solution for this case, something like command on Linux.
unzip archive.zip myfile.txt
I wrote following code
try {
FileInputStream fin = new FileInputStream(rootDir+"/archive.zip");
ZipInputStream zin = new ZipInputStream(fin);
ZipEntry ze = new ZipEntry("myfile.txt");
FileOutputStream fout = new FileOutputStream(rootDir+"/buff/" + ze.getName());
for (int c = zin.read(); c != -1; c = zin.read()) {
fout.write(c);
}
zin.closeEntry();
fout.close();
zin.close();
} catch(Exception e) {
Log.e("Decompress", "unzip", e);
}
This code create new file in buff directory, but this file is empty!
Please, help me with this trouble!
Thanks for your time!
I'm fairly new to Java, but the API documentation contains a pretty reasonable amount of information for the standard Java libraries, including for java.util.zip. Going from there into the ZipFile Entry, you can scroll down to the method listing to find a method called getEntry. This seems to be the route you should start with!
EDIT: bear in mind that you will probably need to include the directory (e.g.: "dir\subdirF\subdirW\FileThatYouWant.txt") when making the call, since that seems to be the way the files are named when you go through one-by-one.
EDIT 2: a considerable wealth of information is available here: Compressing and Decompressing Data Using Java APIs, if you're willing to read a bit :D.
Subject to memory constraints, the only reasonable solution for you might be to use a ZipInputStream object, which AFAIK will require you to step through each ZipEntry in the archive (on average 50,000?), but will not require you to load the entire file into memory. As far as performance, I would guess manually stepping through would be just as efficient as any current implementation of this niche functionality.