I have an app that creates custom files that are shown in a gallery. They are not images or other standard media files. Because the files are large, I ask users for access to a folder in which the files will be stored.
So far so good.
Reading those files or file info via DocumentFile is so much slower than via File. Operations like
DocumentsContract.findDocumentPath()
documentFile.isFile()
documentFile.getName()
take up to 15ms each, which accumulates to a large time when reading a large number of files.
What's the best way to handle this? Is there another way to do this? I don't want the files to be in internal app memory or users might lose their work when they uninstall the app.
Thanks in advance! This is stressing me out a lot.
Do it asynchronously. Do you need to get all of that data immediately? Probably not. So do it on a thread/in a coroutine, and just program your app to not display the data if you do not yet have it (and refresh itself when you do).
In addition, even if you do need it up front- do you need it for all files up front? In a gallery, you can make do with just the ones that are immediately displayed, and the next few which might be. So only fetch that, and get the others when you need them (or when its likely you'll need them soon). Treat it like fetching results from a web API- you don't fetch every post ever made, you fetch a few dozen, then you fetch a few dozen more when they're getting close to the bottom
In the end, DocumentFile is a wrapper around other APIs, a wrapper that is designed to provide a convenient, File-ish API for developers to use. It is not designed for bulk use. If you want to try to get more speed, you can look at the source code to DocumentFile and related classes used in its implementation, such as DocumentContract19, and work at a lower level.
For example, getName() winds up using this queryForString() method to do the real work. Making a query using ContentResolver adds IPC overhead, so if there are other values that you need that you can get in that same query, you could do the query yourself. For example, isFile() also winds up doing a query, so you could combine those two requests into one and cut the overall time in half (roughly).
Do not use DocumentFile class.
Only use DocumentsContract functions.
Its about twenty times as fast as DocumentFile and nearly as fast as classic file operations
Related
I have an app that wants to have the ability to download files locally. A user can download a file (no restrictions to file type) and should be able to save it on the device so it can be used for other purposes. I would also like the user to be able to delete the file from the app (they would know which file is downloaded and which isn't, a ticker would indicate if it's backed up locally). Say it is a pdf file -- the user would want to open it with different apps or edit it if they have the ability to, or just share it via email. Considering we cannot opt out of scoped storage anymore (requirement to target 30), I got a couple of questions.
I've tried to use Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOWNLOADS).getAbsolutePath()) and also getExternalFilesDirPath(ctx, Environment.DIRECTORY_DOWNLOADS)). While the former is deprecated, it works for me in that the Files app can be used to navigate to the file in question (and possibly open/delete the file). On the other hand, I am not being able to delete this file from my app due to lack of permissions. Obviously, the latter path is unable to be navigated to by another app (or is it? I haven't found a way, hence the question).
The other thing I've considered using is the MediaStore API but I'm struggling to see how this is an improvement over the old ways in terms of function. Disregarding moving back to manual content resolver and cursor usage, how should arbitrary files be sorted? Should I manually sort by mime types and have different methods for saving for specific media types? It sounds exceptionally tedious and counter intuitive. If this is the way, so be it, I will implement it, but it does not sound like the way to go. On the positive side, it at least sounds like a solution due to the content resolver's CRUD abilities.
I'm working on a RN app that uses a 3rd party library for the download paths, which old/new versions, respectively, use different paths (rn fetch blob and rn fetch blob util). Additionally, MediaStore API doesn't have a RN implementation as of right now, so everything would have to be done from scratch, too.
What are my options? In the short term I'm considering disabling the erase feature from the app (at least for now). Anything I am missing and should consider? Thanks in advance.
Is there a way to use the StorageActionFramework ACTION_OPEN_DOCUMENT intent to request info for a specific file? More specifically, I have a URI for a particular file, which I obtained from MediaStore. I would like to find out if it has write access and, if so, delete it. Further, I'd like to do this without any UI (none is needed in this particular context).
The documentation says how to do it for a class of files using intent.addCategory (Intent.CATEGORY_OPENABLE), for example. And to restrict it to a certain type of file, intent.setType ("image/*"), for example. But I don't see anything to restrict it to a specific file.
Also, I realize that once you receive a set of files from the StorageActionFramework, you can view properties for individual files.
It appears that if I could use ACTION_OPEN_DOCUMENT to get the SAF's URI, I could then use DocumentsContract.deleteDocument() to delete it.
More Context
In my situation, the user has invoked the camera app from within my app and taken one or more pictures. My code then queries the MediaStore to determine the file name(s) for the new image(s). Then I desire to move the file(s) to a directory specific to my app. This works fine for files located in "internal" and "external" storage but not for removable storage.
In the long run, this solution is inadequate as clearly it will use to much permanent storage (although that is mitigated by the fast pace which storage size is increasing). However my app does need control over image files which are taken via the app and, thus, leaving them on the removable storage will potentially break the app.
Another complicating factor is the lack of an Android API for taking multiple photos and saving them to a prescribed location. There is such an API for taking a single photo but that will not work for my application. Thus, I am relegated to letting the camera app save its files where it wants to and then moving them afterward.
Some other apps simply make a copy of photos and store the copy in a private directory. I could do that but that exacerbates the storage problem even more. The long term solution will probably be using cloud storage in combination with a local private cache.
Is there a way to use the StorageActionFramework ACTION_OPEN_DOCUMENT intent to request info for a specific file?
No. For starters, the Storage Access Framework has little to do with files.
The closest thing that would fit the overall structure would be if you could supply a starting Uri, to allow users to choose and open a document nearby that one. That would be a nice feature but is not supported.
The documentation says how to do it for a class of files using intent.addCategory (Intent.CATEGORY_OPENABLE), for example.
No. CATEGORY_OPENABLE means that the Uri that you get back should work with openInputStream(), openOutputStream(), and related methods, and that a query() on the Uri should be able to return the OpenableColumns. It has little to do with files.
And to restrict it to a certain type of file, intent.setType ("image/*")
That limits the content to a particular MIME type (or wildcard). It has little to do with files.
I would like to find out if it has write access and, if so, delete it
You might have write access to change the contents, via openOutputStream(). I am not aware that you have a means of deleting the underlying content.
Further, I'd like to do this without any UI
The only reason to use ACTION_OPEN_DOCUMENT is to show a UI, to allow the user to choose a piece of content.
My code then queries the MediaStore to determine the file name for the new image
Since there is no requirement for a camera app to update the MediaStore, this does not seem like it will be especially reliable.
Another complicating factor is the lack of an Android API for taking multiple photos and saving them to a prescribed location. There is such an API for taking a single photo but that will not work for my application
Since there is no requirement for a camera app to allow the user to take multiple photos in succession, this does not seem like it will be especially reliable.
I am developing a notepad app which can store simple text files and checklists. Currently I maintain a separate file (say info.txt) that maintains information about whether a given file is a simple text file or a checklist and based on that I render my UI (for either listing all files or opening a file) to show that file in my app. However I am not very happy with this approach because is slow and does not appear to scale well.
Is there a better way to add "metadata" (e.g. if it is a simple text or cheklist, tags, etc) about a file in android?
Any help will be greatly appreciated
There are several ways of storing persistent data in Android.
The way you are currently doing it is through the device storage, and you are quite right it would probably not scale well in addition to being directly accessible to the user meaning they could edit or delete your metadata.
Using SharedPreferences would be one way of storing the metadata which has the advantage of being completely hidden from the user, as well as being relatively easy to set up. The main disadvantages I can see are that it may not scale well if a user has a large number of files, and it is much more difficult to retrieve files with certain criteria, a certain tag for instance, as you mention in the comments.
The best way to store data that will scale well, be persistent, and let you run queries on the data would be an on device SQLite database. SQLite will usually have more overhead in terms of setup time, but is far more robust and featured than any of the other options besides perhaps network based storage, which based on the information you have given is probably not something you are interested in. Based on your problem the SQLite database is probably the way to go and has the bonus of being expandable in case you ever decide to add more information, or even store the files in the SQLite database.
I'm building an app that fetches XML from a server, then parses it (it's DIDL formatted in places). The goal is to load as much into memory as possible to make it easier for users to browse (so as to enable fast scrolling through results), but I keep getting OutOfMemoryErrors. I'm kind of new to Android, and it seems like the VMs are really finicky about things. I'm loading no more than a megabyte of XML, and pretty much discarding it right away. What should I do to prevent these errors from happening?
Should I load chunk by chunk of the file over the network, write it to disk, then load chunk by chunk back into and out of memory, parsing everything into POJOs? If it can be avoided, I'd like to not have to implement some form of pagination, as per the Twitter app (it used to load more entries when you hit the bottom, it now loads a lot all at once and likewise crashes with an OOM error.) I'm running a Nexus One if that helps, CM7/Android 2.3.3.
You can read this nice article about XML parsing on Android. Using pull parser would be good choice for you, since it does not need to read complete document into the memory, which is the problem in your case. I would suggest that you store parsed results into the database, since once they are there, you can quickly do list them and page them any way you want, and performance of the DB is great. This way, you need to do loading of complete data from server to DB just once (if data is not changing on the server) or to load it once and then to get updates from time to time if it is changing (like in case of twitter).
Move all your object declarations out of loops and nullify them after the use. And use System.gc() frequently (beleive me, it works). Use class level objects as less as you can. Execute your app and keep watch on the Logcat logs.
There is a web service that provides some data that my app makes use of. This data is fairly large and only changes VERY infrequently so I thought it would be nice if the app could cache it on the SD Card and only update it as needed.
Currently I'm grabbing the data (an XML file) and parsing it into an object tree using SAX. This process takes (at most) 2-3 seconds over my WIFI. However, serializing the resulting objects to the SDCard takes significantly longer (a minute or more) and deserializing it still takes longer than just download/parsing in the first place.
Does anyone have any recommendations for improving this or alternate ideas for persisting this data (other than just saving the XML file and reparsing every time)?
UPDATE: This is more than a trivial collection of records. The object-graph is actually ridiculously complex and storing it into a database would result in dozens of tables with only a single record in each one.
Android serialization is notoriously slow. I highly suggest switching to using XML or JSON (or whatever) and writing the file out that way. Since you've already got an XML parser, it may make the most sense just to cache the original XML file you downloaded and reparse it as necessary.
I have switched from Serializable to JSON file storage in an app before and the speed increase was incredible, at least one order of magnitude.
(I may be misunderstanding your question - I assume you are using Serializable for writing to the disc. If you are reproducing the XML, then I'm not sure why it is so much slower on the SD card. Also, I agree that the SQLite database makes the most sense typically, but as you've already stated it does not fit the needs of your application.)
Also unless your data is at least 100s of Kb, I would suggest just storing it in your private data storage instead of on the SD card. Keep in mind that you can't rely on the SD card being available.
I've just been writing an android application for the last week which basically does this. It fetches some (large) XML file online, and then displays part of the data in various views.
We do it by fetching and parsing the XML using SAX, and (while parsing) writing it all to a SQLite database. And then we are just querying the database each time we need to display some view of the dataset.
Works like a charm, and is fast enough for displaying a lot of data on a google map overlay, where we are querying the database on every single call to the draw method of our map overlay.
So I would definitely suggest going for a SQLite database, if the data in the XML document is easily represented in a database.
If the web service can give you just a specified number of results(something like: requestData between index 1 and 10 or give me first 25 results) try to use that (put a simple "Load more results" button or implement an auto-loading mechanism). If the web service not provide this feature then try to save your xml on sdcard and when you need the data try to parse just a specified number of results. Hope this help!
Why don't you use database? See Android Data Storage Guide