I'm trying to populate a ListView with a mixture of files stored on the SDcard AND stored as assets in the APK. Using TraceView, I can see that the performance of AssetManager.list() is poor in comparison to File.listFiles(), even though I'm using a filename filter for the SDcard.
Here is a simple method that returns all the png files from a folder on the SDcard:
// The folder on SDcard may contain files other than png, so filter them out
private File[] getMatchingFiles(File path) {
File[] flFiles = path.listFiles(new FilenameFilter() {
public boolean accept(File dir, String name) {
name = name.toLowerCase();
return name.endsWith(".png");
}
});
return flFiles;
}
I invoke that method here and it takes about 12ms to retrieve 16 files:
final String state = Environment.getExternalStorageState();
if (Environment.MEDIA_MOUNTED.equals(state)||Environment.MEDIA_SHARED.equals(state)) {
File path = Environment.getExternalStoragePublicDirectory(getResources().getString(R.string.path_dir));
if (path.exists()){
File[] files = getMatchingFiles(path);
...
Whereas the am.list method takes 49ms to retrieve just the names of about 6 files!
// Get all filenames from specific Asset Folder and store them in String array
AssetManager am = getAssets();
String path = getResources().getString(R.string.path_dir);
String[] fileNames = am.list(path);
...
Can anyone explain why the performance would be so poor? Is the performance proportional to the number of assets stored in the APK? I'm aware that assets are compressed, but I'm only fetching the names of the assets, which I thought would be stored in a table somewhere.
Coverdriven's comment "stored in a table somewhere" inspired me to solve my own problem which I've been putting off for a while.
This doesn't answer the OP but does offer a different approach and it handles subfolders which CommonsWare's solution doesn't unless you go recursive (which of course is another possible solution). It's specifically aimed at apps which have a large number of assets in subfolders.
I added an ANT pre-build target to run this command (I'm on Windows)
dir assets /b /s /A-d > res\raw\assetfiles
This creates a recursive (/s), barebones (/b) listing of all files, excluding directory entries (/A-d) in my assets folder.
I then created this class to statically load the contents of assetfiles into a hashmap, the key of which is the filename and the value the full path
public class AssetFiles {
// create a hashmap of all files referenced in res/raw/assetfiles
/*map of all the contents of assets located in the subfolder with the name specified in FILES_ROOT
the key is the filename without path, the value is the full path relative to FILES_ROOT
includes the root, e.g. harmonics_data/subfolder/file.extension - this can be passed
directly to AssetManager.open()*/
public static HashMap<String, String> assetFiles = new HashMap<String, String>();
public static final String FILES_ROOT = "harmonics_data";
static {
String line;
String filename;
String path;
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(TidesPlannerApplication.getContext().getResources().openRawResource(R.raw.assetfiles)));
while ((line = reader.readLine()) != null) {
// NB backlash (note the escape) is specific to Windows
filename = line.substring(line.lastIndexOf("\\")+1);
path = line.substring(line.lastIndexOf(FILES_ROOT)).replaceAll("\\\\","/");;
assetFiles.put(filename, path);
}
} catch (IOException e) {
e.printStackTrace();
}
}
public static boolean exists(String filename){
return assetFiles.containsKey(filename);
}
public static String getFilename(String filename){
if (exists(filename)){
return assetFiles.get(filename);
} else {
return "";
}
}
}
To use it, I simply call AssetFiles.getFilename(filename) which returns the full path which I can pass to AssetManager.open(). Much much faster!
NB. I haven't finished this class and it's not hardened yet so you'll need to add appropriate exception catches and actions. It's also quite specific to my app in that all of my assets are in subfolders which are in turn located in a subfolder of the assets folder (see FILES_ROOT) but easy to adapt to your situation.
Note also the need to replace backslashes, since Windows generates the assetfiles listing, with forward slashes. You could eliminate this on OSX and *nix platforms.
Can anyone explain why the performance would be so poor?
Reading the contents of a ZIP archive (the APK where the assets are located) is slower than reading the contents of a directory on the filesystem, apparently. In the abstract, this is not especially surprising, as I suspect that this would be true for all major operating systems.
Read in that list() data once, then save it somewhere else for faster access (e.g., database), particularly in a form that is optimized for future lookups (e.g., where a simple database query could give you what you want, vs. having to load and "recursively search it" again).
If you have a deep tree of directories in the assets you can detect firstly if an item is file or directory and then call .list() on it (really accelerates the walking through the tree). This is my solution I've discovered for this:
try {
AssetFileDescriptor desc = getAssets().openFd(path); // Always throws exception: for directories and for files
desc.close(); // Never executes
} catch (Exception e) {
exception_message = e.toString();
}
if (exception_message.endsWith(path)) { // Exception for directory and for file has different message
// Directory
} else {
// File
}
You can approach APK package as it's a ZIP file and read all the entries using Java's builtin ZipFile. It will give you all the file names with their full paths. Perhaps it shouldn't be hard to find which directories you have.
So far this is the fastest approach I've tested.
credit goes to #obastemur's commit on jxcore-android-basics sample project
Related
I am trying to use Android's internal helpers to get a path from the system for my file first and then put my files where the system wants. Because tomorrow they might change their minds.
I made a simple program to explore this subject. Here is my code;
public class MainActivity extends AppCompatActivity {
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
String path = letsMakeAfile(this, "myFile.txt");
}
private static String letsMakeAfile(Context context, String nameOfFile) {
String strOfFinalPath ="";
//ask the system what path to use...
String strOfContextPath = context.getFilesDir() + "/";
//line above doesnt work without ' + "/" ' or something on the end
//line above records this path: /data/user/0/com.example.filesexperimenting/files/
//this appears to be an invalid path unless "user" is a hidden directory
Log.d("IDIOT", "strOfContextPath: "+ strOfContextPath);
try
{
File file = new File(strOfContextPath, nameOfFile);
if (file.exists() == false) {
file.mkdirs();
//after this line "makes dirs" is file automatically still made and dropped in?
letsMakeAfile(context, nameOfFile);
//I assume not so Ive made a recursive call
}
else
;
//escape recursion....
strOfFinalPath = file.getAbsolutePath();
//Here I record the path where I hope the file is located
Log.d("IDIOT", "strOfFinalPath: "+ strOfFinalPath);
}
catch (Exception e) {
e.printStackTrace();
Log.d("IDIOT", "CATCH ERROR: "+ e.getLocalizedMessage());
}
//runs without a catch
return strOfFinalPath;
}
}
Logcat:
2019-04-09 09:59:22.901 16819-16819/? D/IDIOT: strOfContextPath: /data/user/0/com.example.filesexperimenting/files/
2019-04-09 09:59:22.901 16819-16819/? D/IDIOT: strOfFinalPath: /data/user/0/com.example.filesexperimenting/files
Ultimately I am getting a path of /data/user/0/com.example.filesexperimenting/files/ from context.getFilesDir() which appears to be an invalid path unless "user" is a hidden directory (then why can I see root?). In Device File Explorer under data the only other directories are app, data and local
What am I missing? I'll assume its something with file.makedirs()
Full disclosure, I am a student and there is not a lot out there on this so your replies, while obvious to you at your experience level, should help others. I have some experience with Java and more with C++ but Android is new to me. Thanks in advance!
So, in talking outside of StackExchange it appears that using java.io like I am trying to in the example can cause some problems because of the preset file directories that may be locked or restricted that Java io might not know about.
Android has it's own method openFileOutput(String name, int mode) that has the ability to create the app resource file and directory it belongs in.
Description copied from class: android.content.Context
Actions:
~Open a private file associated with this Context's application package for writing.
~Creates the file if it doesn't already exist.
~No additional permissions are required for the calling app to read or write the returned file.
Params:
~name – The name of the file to open; can not contain path separators.
~mode – Operating mode.
Returns: The resulting FileOutputStream.
Throws: java.io.FileNotFoundException
If you want to be able to navigate to the location of your saved files through the file explorer (either in Android Studio or the Files app on the phone) you should use Context.getExternalFilesDir().
Context.getFilesDir() returns a directory not accessible by anyone BUT the creating application. So if you would like to see what is in this file you would need to open it with the same application that wrote it. IE: Print the contents to the screen after you save it in your app.
Context.getExternalFilesDir() returns a directory completely accessible by anyone and any application. So files created and saved in this external directory can be seen by Android Studio's file explorer as the OP has screenshot or by any application installed on the phone.
What is nice about both of these methods is that as long as you are only accessing files you have created you never need to ask the user for storage permissions Read or Write. If you would like to write to someone else's external files dir then you do.
Source
Check if sdcard is mounted or not.
if (android.os.Environment.getExternalStorageState().equals(android.os.Environment.MEDIA_MOUNTED)){
///mounted
}
Get the path of sd card
File dir= new File(android.os.Environment.getExternalStorageDirectory());
walkdir(dir);
ArrayList<String> filepath= new ArrayList<String>();
//list for storing all file paths
public void walkdir(File dir) {
File listFile[] = dir.listFiles();
if (listFile != null) {
for (int i = 0; i < listFile.length; i++) {
if (listFile[i].isDirectory()) {
// if its a directory need to get the files under that directory
walkdir(listFile[i]);
} else {
// add path of files to your arraylist for later use
//Do what ever u want
filepath.add( listFile[i].getAbsolutePath());
}
}
}
}
Try using this:
context.getFilesDir().getParentFile().getPath()
I have an apk file which contains zipped database (in asset folder). I want to unzip this database during first app run. For that case I am going to use zip4j like so:
public void unzipping() {
String source = "";//here source of zipped database
String destination = "";//here where database should be after installation
String password = "mypassword";
try {
ZipFile zipFile = new ZipFile(source);
if (zipFile.isEncrypted()) {
zipFile.setPassword(password);
}
zipFile.extractAll(destination);
} catch (ZipException e) {
e.printStackTrace();
}
}
The problem is that I don't know what correct path to specify in "source" and "destination".
There is nothing wrong with using encryption to protect user data. But understand, that the attacker can always decompile/memory dump your app and grab the password you are using to read/write to the database.
I'd a SQLCipher , which makes it transparent to the other parts of the app.
I have tried to make sense of the getFileStreamPath madness for some hours now.
Could someone please explain how to test if a path = "shop/crates/fruits" exists?
In an attempt to simplify the test i have broken the path in to segments.
I thought i had it. But the test breaks when shop exists but there is no crates..Weird!
Or is it?
public static Boolean pathExists(String path, Context ctx)
{
Boolean result = false;
String[] pathSegments = path.split("/");
String pathStr = "";
for(int i = 0;i<pathSegments.length;i++ )
{
pathStr += pathSegments[i];
if(!ctx.getFileStreamPath(pathStr).exists())
{
result = false;
break;
}
pathStr += "/";
result = true;
}
return result;
}
Yeah, the crappy android APIs.
The secret sauce is to use context.getFilesDir(). That's your file:
File file = new File(context.getFilesDir() + File.separator + path);
After that, the file behaves normally.
First, the getFileStreamPath returns the absolute path on the filesystem where a file created with openFileOutput(String, int) is stored.
Are you trying to test the internal or external storage's path? If you want to use external storage use getExternalFilesDir(). If you want to use internal package embedded resources like res/raw, it's another story:
Android how to get access to raw resources that i put in res folder?
But I don't think it will work the way you are presuming to get it.
Read this http://developer.android.com/guide/topics/data/data-storage.html
carefully.
Read Using the Internal Storage chapter there.
Also see:
How to create a file on Android Internal Storage?
Read/write file to internal private storage
http://www.vogella.de/articles/AndroidFileSystem/article.html
I need the name (String) of all files in res/raw/
I tried:
File f = new File("/");
String[] someFiles = f.list();
It looks like the root directory is the root of the android emulator...and not my computers root directory. That makes enough sense, but doesn't really help me find out where the raw folder exists.
Update: Thanks for all the great replies. It appears that some of these are working, but only getting me half way. Perhaps a more detailed description will aid
I want to get all the mp3 files in the raw folder so I can get all the names, then add them to a URI to play a random MP3 in the following way...
String uriStr = "android.resource://"+ "com.example.phone"+ "/" + "raw/dennis";
Uri uri = Uri.parse(uriStr);
singletonMediaPlayer = MediaPlayer.create(c, uri);
When I put "dennis.mp3" into the assets folder, it does show up as expected, however, using the above code, I can't access that MP3 anymore, unless there is something along the line of:
String uriStr = "android.assets://"+ "com.example.phone"+ "/" + "dennis";
Uri uri = Uri.parse(uriStr);
To list all the names of your raw assets, which are basically the filenames with the extensions stripped off, you can do this:
public void listRaw(){
Field[] fields=R.raw.class.getFields();
for(int count=0; count < fields.length; count++){
Log.i("Raw Asset: ", fields[count].getName());
}
}
Since the actual files aren't just sitting on the filesystem once they're on the phone, the name is irrelevant, and you'll need to refer to them by the integer assigned to that resource name. In the above example, you could get this integer thus:
int resourceID=fields[count].getInt(fields[count]);
This is the same int which you'd get by referring to R.raw.whateveryounamedtheresource
This code will retrieve all the files from 'New Foder' of sdCard.
File sdCardRoot = Environment.getExternalStorageDirectory();
File yourDir = new File(sdCardRoot, "New Folder");
for (File f : yourDir.listFiles()) {
if (f.isFile())
{
String name = f.getName();
Log.i("file names", name);
}
}
and also make sure to add android sd card write permission in your manifest.xml file
I need the name (String) of all files in res/raw/
There are no files in res/raw/ on the device. Those are resources. There is no good way to iterate over resources, other than by using reflection to iterate over the static data members of the R.raw class to get the various ID names and values.
but doesn't really help me find out where the raw folder exists.
As a folder, it only exists on your development machine. It is not a folder on the device.
You can use AssetManager:
As far as I can remember, you will have list with (just try different paths):
final String[] allFilesInPackage = getContext().getResources().getAssets().list("");
Look at http://developer.android.com/reference/android/content/res/Resources.html
You can generally acquire the Resources instance associated with your application with getResources().
Resources class provides access to http://developer.android.com/reference/android/content/res/AssetManager.html (see to getAssets() method).
And finally obtain access to your packaged (apk) files with AssetManager.list() method.
Enjoy!
From a Context, you can call something like this:
getResources().getAssets().list("thePath");
link to documentation
i have a subfolder in the Assets folder called images where i store my images(of course) :) The things is that i want to get the name for the images which i'm getting but the problem is that i'm also getting other unknown names like: "android-logo-mask.png" which i guess are android's default images. Is there a way i can skip this "android default images" to get only the names of my images? My plan is to save this names in a database to use it as reference for showing the images later on an ImageView. Is it a good idea to use the image name to show the images? Here is some code if it needs:
Context context;
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
context = this.getApplicationContext();
ImageHelper i = new ImageHelper();
i.readImages(context);
}
public class ImageHelper {
public ImageHelper(){}
public void readImages(Context context){
AssetManager am = context.getAssets();
try {
String[] getImages = am.list("images");
for(String imgName : getImages){
Log.e("IMAGE NAME----->", imgName);
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
Thanks in advance.
Use some other name for images folder like "MyImages". It should work fine then.
"images" should be a path like "/path/assets/images/image.png"
It is very interesting. Even if you don't have an assets/images directory, when you use:
myImageList = Arrays.asList(getResources().getAssets().list("images"));
you'll see android-logo-mask.png and android-logo-shine.png listed in the results.
That led me to wondering where those actual image files live in the Android code. I don't know the code base at all, but:
https://android.googlesource.com/platform/frameworks/base/+/master/core/res/assets/images/
lists those two in an assets directory under res. (Different from the assets directory where I store my project assets, that's not under res).
Being required to provide a path to an individual file for the AssetManagers list method (which returns an array) just doesn't make any sense. And the method documentation says:
Return a String array of all the assets at the given path.
It makes me wonder if there's an error in the list method, or if there was some other reason it was designed to return particular system assets when an asset list is required.
In a similar fashion, if I use:
myAssetList = Arrays.asList(getResources().getAssets().list(""));
I see "sounds" and "webkit" included, which don't correspond to any existing subdirectory of my assets.
If it occurs that the system adds files in this "images" folder, why not create a new folder for your images that you are sure to contain only your files ?