I am trying to process multiple file writing commands (to seperate files) without hanging the UI.
To put my question in context, imagine the following
My main application looks like a file manager. It currently sees 10 files each of about 5MB in size. (Don't worry about how this list works etc.)
When I select one file item, I want it to immediately start copying/duplicating the file onto another location on the SD card. Typically this should take a few seconds
I want to be able to select a second, or a third etc. file immediately after the first. At the end of everything, all the files which I selected will be duplicated. So I could, for example click 5 files within 5 seconds, but all the copying actions takes a minute.
At this moment two options have come to my mind:
The first is to simply put the file writing commands of each file in a seperate thread. Pseudocode will look like this
Onclick
new Thread()
write file
If this works, there can be scenarios where I have 10 threads running simultaneously, writing to 10 seperate files. I would like to know if anyone has done this before, and what I should be looking out for
The second option, of course, is if there is already a certain data structure/known methods that can address this problem. Some sort of a pending queue system that gets larger as I add requests, but gets smaller as the system writes the data away.
I'm not absolutely sure how the SD-card works, but I can tell you that trying to write in parallel to a single hard disk is a bad idea because it will actually slow down the performance compared to a sequential write.
You may want to try using an ExecutorService and measuring the performance with different thread counts but I'm afraid you will end up having to implement a queue with a single thread taking the queued files and writing them one by one.
I would create an AsyncTask class that simply copies a given file over. Each time a file is selected, create a new instance of that class for the selected file. The thread management built into Android for AsyncTask is well balanced and should handle this use case nicely. It will be easy to provide feedback for progress and completion using the built-in AsyncTask methods.
I think the classes in java.util.concurrent are what you need; specifically the Executors class to create a ThreadPoolEecutor. It has the benefit of accepting as many tasks as the user clicks but limiting the number of threads to some limit you specify. (Spawning threads without limit can be a problem, slowing down not only each other but the UI as well.)
Related
I have an app that creates custom files that are shown in a gallery. They are not images or other standard media files. Because the files are large, I ask users for access to a folder in which the files will be stored.
So far so good.
Reading those files or file info via DocumentFile is so much slower than via File. Operations like
DocumentsContract.findDocumentPath()
documentFile.isFile()
documentFile.getName()
take up to 15ms each, which accumulates to a large time when reading a large number of files.
What's the best way to handle this? Is there another way to do this? I don't want the files to be in internal app memory or users might lose their work when they uninstall the app.
Thanks in advance! This is stressing me out a lot.
Do it asynchronously. Do you need to get all of that data immediately? Probably not. So do it on a thread/in a coroutine, and just program your app to not display the data if you do not yet have it (and refresh itself when you do).
In addition, even if you do need it up front- do you need it for all files up front? In a gallery, you can make do with just the ones that are immediately displayed, and the next few which might be. So only fetch that, and get the others when you need them (or when its likely you'll need them soon). Treat it like fetching results from a web API- you don't fetch every post ever made, you fetch a few dozen, then you fetch a few dozen more when they're getting close to the bottom
In the end, DocumentFile is a wrapper around other APIs, a wrapper that is designed to provide a convenient, File-ish API for developers to use. It is not designed for bulk use. If you want to try to get more speed, you can look at the source code to DocumentFile and related classes used in its implementation, such as DocumentContract19, and work at a lower level.
For example, getName() winds up using this queryForString() method to do the real work. Making a query using ContentResolver adds IPC overhead, so if there are other values that you need that you can get in that same query, you could do the query yourself. For example, isFile() also winds up doing a query, so you could combine those two requests into one and cut the overall time in half (roughly).
Do not use DocumentFile class.
Only use DocumentsContract functions.
Its about twenty times as fast as DocumentFile and nearly as fast as classic file operations
I am loading multiple photos from Dropbox using the Core API in Android and when making requests they are serialized.
I am getting the thumbnails to show in a grid view and when the user clicks on one of them it goes to another activity it fetches the full res version from the server.
It does this fine, but the high res version only starts to be downloaded when all the others async tasks fetching the thumbnails are finished.
So what I want to know is, is this a limitation of the Dropbox Core API? Or is there a way to make the high res async task have priority over others so it is immediately downloaded and then the others can resume.
This is due to how async tasks actually work. They don't run in parallel on 3.0+ (they used to until 3.0), because too many newbie developers weren't able to program in parallel without errors, so Google decided to change it. Instead, async tasks run on a single thread in FIFO order.
You can override this however. Instead of calling asynctask.execute(), call asynctask.executeOnExecutor() and use a THREAD_POOL_EXECUTOR. This will execute it in parallel on its own thread. I believe there's a thread cap, but it will at least make several run in parallel.
If the thread cap becomes an issue, you can always drop down to using threads instead of using async tasks. You need to do some work yourself to do an onPostExecute, but it isn't that hard. And if you're creeating your own threads you can make up to the OS limit.
What is the best way to upload large files in the background of an Android/Glass application? I'm currently triggering an asynch task for each file (photos / videos) I want to upload, but if the asynch task crashes or the activity that launched it hits an exception the file upload fails.
Is there an android or Glass design pattern to store files in an application-specific directory and have a task that constantly works to make sure those files are uploaded then removed from that folder?
Depending on your requirements you should either use a background service (as #straya mentioned)
https://developer.android.com/training/run-background-service/create-service.html
Or you can use a SyncAdapter:
http://developer.android.com/training/sync-adapters/creating-sync-adapter.html
The first approach you would use if you need to upload those files as soon as possible, the second approach you would use if you want to bundle up several files together and let the operating system decide when to sync them with the server (you can specify how often you'd like your app to synchronize).
I don't know the requirements of you application but sounds like the second approach is what you need. That is what google is using for many applications like gmail, etc. It is way harder to implement then the background service, but it's worth it. Android is smart about the Sync Adapters, it bundles up several adapters together when possible to preserve the battery consumption.
use a background Service to manage long-running tasks that may occur whether an Activity is created or not.
As for crashes, avoid them by programming defensively and handling possible exceptions.
I'm trying to develop an android app that takes some data from a txt file (which is about 7 million lines long close to 32 MB) processes it according to some conditions places them in an SQL database and retrieves them. As far as the database is concerned it works well since I have tested it in alternative way, but for some reason it has a hard time dealing with the file, and my code becomes very unresponsive. What is an efficient way of doing this?
If time-consuming part is the file input may be using an async task to to do the work (read the file and prepare the DB) in the background and then notify the user when it's done. This way the user can keep using the app. Here are some links:
AsyncTask
AsyncTask example
According to the documentation the asyncTask should only be used for not so long operations (a few seconds only):
AsyncTasks should ideally be used for short operations (a few seconds
at the most.) If you need to keep threads running for long periods of
time, it is highly recommended you use the various APIs provided by
the java.util.concurrent pacakge such as Executor, ThreadPoolExecutor
and FutureTask.
run the code in separate thread (async task)
instead of loading whole file at once, process it line by line. this would help you to reduce memory consumption
I do not have not created any code yet on this topic so I will just ask a question.
I am making an application in which user will insert multiple paths, and in those paths/folders I will be searching for N biggest files in every folder separately. I was thinking that I should use new Thread for every path but I am not sure if this is a good practice or good idea in general.
One little subquestion - should I use TreeSet(RedBlack tree) to keep files in some kind of order or B-Tree would be better?
EDIT: By new Thread I mean using multiple threads with AsyncTask or something like that