Develop Reference

Can you encapsulate multiple nested transactions across different threads into an overall transaction with greenDAO?

I am working on an Android application that uses greenDAO as a data persistence layer. The application downloads data from various different sources across multiple threads (determined by a thread pool), each piece of data is inserted into the database in a transaction using insertOrReplaceInTx. This is working fine.
My question is whether it is technically possible, using greenDAO, to encapsulate these different transactions (which occur on different threads) into an overall transaction, using nested transactions. I know in theory it is possible to do this if all the transactions were taking place on a single thread, however I am unsure if this possible with the insertOrReplaceInTx calls occurring on different threads.
The reason I wish to encapsulate these into a single overall transaction is because they represent a synchronisation process within an app. In the event of any single part of the import failing, I wish to abort and rollback all of the modifications within the overall transaction.
If I begin a transaction with db.beginTransaction on the main thread where I initiate the import process, this creates a deadlock when another thread tries to insertOrReplaceInTxt.
Is the correct way to counter this to ensure that all greenDAO transactions are taking place on the same thread?

Afaik, you cannot because each thread manages its own connection.
If you have such dependency between these operations, you probably want to sync them anyways.
e.g. what if Job A finishes way before Job B and Job B's db connection fails. Your data will go out of sync again. You still need some logic for the other job.
Also, writers are mutually exclusive.
I would suggest creating a utility class that can run a list of runnables in a transaction. Each job, when finished, enqueues a Runnable to this utility. These runnables include the actual database commands.
When the last one arrives (this depends on your dependency logic), the utility will run all runnables in a transaction.
A sample implementation may look like this: (I used a simple counter but you may need a more complex logic)
class DbBundle {
AtomicInteger mLatch;
List<Runnable> mRunnables = new ArrayList();
DbBundle(int numberOfTx) {
mLatch = new AtomicInteger(numberOfTx);
}
void cancel() {
mLatch.set(-1); // so decrement can never reach 0 in submit
}
boolean isCanceled() {
mLatch.count() < 0;
}
void submit(Runnable runnable) {
mRunnables.add(runnable);
if (mLatch.decrementAndGet() == 0) {
db.beginTransaction();
try {
for (Runnable r : mRunnables) r.run();
db.setTransactionSuccessful()
} finally {
db.endTransaction();
}
}
}
}
When you create each job, you pass this shared DbBundle and the last one will execute them all.
So a job would look like:
....
try {
if (!dbBundle.isCanceled()) { // avoid extra request if it is already canceled
final List<User> users = webservice.getUsers();
dbBundle.submit(new Runnable() {
void onRun() {
saveUsers(users);//which calls db. no transaction code.
});
});
} catch(Throwable t) {
dbBundle.cancel();
}

How many async task can be used in a class android?

Recently I attended an interview in which I was asked a question: How many asynctask can be used in a class? By using execute method you will use by calling asynctask. So maximum limit of asynctask in a class is the question thrown to me.
What is the answer for this? Can someone please explain how many and why?

The question itself does not make any sense. You can use as many AsyncTask in a class as you want, if there were a restriction on that it would be ridiculous. I assume he meant how many AsyncTask can be executed at the same time and how they are executed and the answer to that would be: It depends.
AsyncTasks can be executed either in series or in parallel. The default behaviour depends on the API level of the device. The documentation of execute() of AsyncTask says:
Note: this function schedules the task on a queue for a single
background thread or pool of threads depending on the platform
version. When first introduced, AsyncTasks were executed serially on a
single background thread. Starting with DONUT, this was changed to a
pool of threads allowing multiple tasks to operate in parallel.
Starting HONEYCOMB, tasks are back to being executed on a single
thread to avoid common application errors caused by parallel
execution. If you truly want parallel execution, you can use the
executeOnExecutor(Executor, Params...) version of this method with
THREAD_POOL_EXECUTOR; however, see commentary there for warnings on
its use.
Having said that you can choose whether you want to execute them in parallel or in series like this:
// Executes the task in parallel to other tasks
asyncTask.executeOnExecutor(AsyncTask.THREAD_POOL_EXECUTOR);
// Adds the task to a queue and executes one at a time.
asyncTask.executeOnExecutor(AsyncTask.SERIAL_EXECUTOR);
However even if you run the tasks in parallel there is a limit to how many can run at the same time. To find out where that limit is you have to look into the source code of AsyncTask.
Up until Android 4.3 (Jelly Bean) the limits were hardcoded to those values:
private static final int CORE_POOL_SIZE = 5;
private static final int MAXIMUM_POOL_SIZE = 128;
private static final int KEEP_ALIVE = 1;
But with Android 4.4 that was changed and the limits are calculated depending on the used processor in the device:
private static final int CPU_COUNT = Runtime.getRuntime().availableProcessors();
private static final int CORE_POOL_SIZE = CPU_COUNT + 1;
private static final int MAXIMUM_POOL_SIZE = CPU_COUNT * 2 + 1;
private static final int KEEP_ALIVE = 1;
The implementation of the ThreadPoolExecutor remained the same in both cases:
public static final Executor THREAD_POOL_EXECUTOR
= new ThreadPoolExecutor(CORE_POOL_SIZE, MAXIMUM_POOL_SIZE, KEEP_ALIVE,
TimeUnit.SECONDS, sPoolWorkQueue, sThreadFactory);
So that should pretty much answer your question. But if you really want to find out how the AsyncTask works then you should study the source code yourself! This link leads to the AsyncTask implementation on Android 4.4.

How can I start two asynchronous threads which join to common execution once both are complete?

I'm looking for a design pattern or approach for the following scenario. I wish to kick off two separate background threads for data retrieval from different sources. I then want one method (on the UI thread) to be called once both background threads have completed their work. As the data from the two sources must be combined to be useful, I must wait until both have finished retrieving before manipulating the data. How can I achieve this on the Android platform?

Edit: My first version has been bothering me, and I didn't like the necessary added boolean with it, so here's another version. Call it with this from onPostExecute of each added task.
ArrayList<AsyncTask> tasks;
public void doStuffWhenDone(AsyncTask finishedTask)
{
tasks.remove(finishedTask);
if(tasks.size() > 0)
return;
... do stuff
}
I'll keep the older one up also, since they both work, but I think the above is much cleaner. Now to go tidy up one of my earlier projects.
ArrayList<AsyncTask> tasks;
boolean hasBeenDone = false;
public void doStuffWhenDone()
{
for(int i=0;i<tasks.size();i++)
if(hasBeenDone || (tasks.get(i).getStatus() != AsyncTask.Status.FINISHED))
return;
hasBeenDone = true;
... do stuff
}
It's easily extendable to however many tasks you have, and there's no need for a thread to handle the threads. Just call the method at the end of each task. If it's not the last one done, nothing happens.
Edit: Good point, but I don't think it needs to be atomic. Since both AsyncTasks' onPostExecute methods run on the UI thread, they'll be called one after the other.

Use a CountDownLatch, like this:
CountDownLatch barrier = new CountDownLatch(2); // init with count=2
startWorkerThread1(barrier);
startWorkerThread2(barrier);
barrier.await(); // it will wait here until the count is zero
doStuffWithTheResult();
when a worker thread finishes, call barrier.countDown() from it.

You can use AsyncTask and an int to know if both jobs are finished...

Elegant way to transform Multi-thread into Single-thread

1.
I have main UI thread and it uses library functions.
MainUI {
library.funcA(1);
library.funcB(2,3);
}
I don't want Main-Thread to wait for the end of each call. Also creating new thread for each call is ineffective and not thread-safe so I want them to run on another same thread.
So I created another thread with a single message queue / handler.
MainUI {
handler.post(new Message(FUNC_A,[1]));
handler.post(new Message(FUNC_B,[2,3]));
}
but it seems having overhead that I need to create new Message object every time and decode it again in handleMessage(). Furthermore It looks ugly to handle the various number of arguments packing/unpacking. I thought making all functions as Runnable and post them to handler, but functions have different arguments and I need to make new classes extending Runnable class!
my current plan is to insert synchronized keywords into every function in library so threads run one by one,
but I dislike many threads creating/disappering so making system heavy (also not thread-safe). I want to hear opinions.
2.
if the library is called from many threads (not only Main-thread), what's the best way to convert calls to run on the library's own single thread?

In my opinion, your current approach is feasible option to make thread-safe. However, if you want to go for having Runnable passed to handler, you could consider overloading your Runnable class. Here is pseudo-code
private Class MyRunner extends Runnable{
public MyRunner(){
//This is without any argument
}
public MyRunner(arg1){
//This is with 1 argument
}
public MyRunner(arg1,arg2){
//This is with 2 arguments
}
}
Still, you will have many instances of MyRunner class created.

You can use a Runnable like this:
int x = 1;
int y = 2;
// "a" and "b" are here to show how variable values can be sent to funcB.
// You could skip this and just have "funcB( 1, 2 )", of course.
final int a = x;
final int b = y;
handler.post( new Runnable() {
public void run() { library.funcB( a, b ); }
} );
The handler bit is a bit trickier. It has to run a looping thread to run each Runnable from the queue. Then it has to have a post method to put things on the queue. Some form of blocking queue would be the simplest way to link the two, saving you from having to do your own Object.wait/Object.notify. Synchronizing the post method would be the easiest way to keep multiple threads from tripping over each other.

How AsyncTask works in Android

I want to know how AsyncTask works internally.
I know it uses the Java Executor to perform the operations but still some of the questions I am not understanding. Like:
How many AsyncTask can be started at a time in an Android app?
When I start 10 AsyncTask, will all tasks will run simultaneously or one by one?
I have tried with 75000 AsyncTask to test the same. I don't get any problem and seems like all the tasks will be pushed to stack and will run one by one.
Also when I start 100000 AsyncTasks, I start getting OutOfMemoryError.
So is there any limit of no of AsyncTask which can be run at a time?
Note: I have tested these on SDK 4.0

AsyncTask has a rather long story.
When it first appeared in Cupcake (1.5) it handled background operations with a single additional thread (one by one). In Donut (1.6) it was changed, so that a pool of thread had begun to be used. And operations could be processed simultaneously until the pool had been exhausted. In such case operations were enqueued.
Since Honeycomb default behavior is switched back to use of a single worker thread (one by one processing). But the new method (executeOnExecutor) is introduced to give you a possibility to run simultaneous tasks if you wish (there two different standard executors: SERIAL_EXECUTOR and THREAD_POOL_EXECUTOR).
The way how tasks are enqueued also depends on what executor you use. In case of a parallel one you are restricted with a limit of 10 (new LinkedBlockingQueue<Runnable>(10)). In case of a serial one you are not limited (new ArrayDeque<Runnable>()).
So the way your tasks are processed depends on how you run them and what SDK version you run them on.
As for thread limits, we are not guaranteed with any, yet looking at the ICS source code we can say that number of threads in the pool can vary in range 5..128.
When you start 100000 with default execute method serial executor is used.
Since tasks that cannot be processed immediately are enqueued you get OutOfMemoryError (thousands of tasks are added to the array backed queue).
Exact number of tasks you can start at once depends on the memory class of the device you are running on and, again, on executor you use.

let us dive deep into the Android’s Asynctask.java file to understand it from a designer’s perspective and how it has nicely implemented Half Sync-Half Async design pattern in it.
In the beginning of the class few lines of codes are as follows:
private static final ThreadFactory sThreadFactory = new ThreadFactory() {
private final AtomicInteger mCount = new AtomicInteger(1);
public Thread newThread(Runnable r) {
return new Thread(r, "AsyncTask #" + mCount.getAndIncrement());
}
};
private static final BlockingQueue<Runnable> sPoolWorkQueue =
new LinkedBlockingQueue<Runnable>(10);
/**
* An {#link Executor} that can be used to execute tasks in parallel.
*/
public static final Executor THREAD_POOL_EXECUTOR
= new ThreadPoolExecutor(CORE_POOL_SIZE, MAXIMUM_POOL_SIZE, KEEP_ALIVE,
TimeUnit.SECONDS, sPoolWorkQueue, sThreadFactory);
The first is a ThreadFactory which is responsible for creating worker threads. The member variable of this class is the number of threads created so far. The moment it creates a worker thread, this number gets increased by 1.
The next is the BlockingQueue. As you know from the Java blockingqueue documentation, it actually provides a thread safe synchronized queue implementing FIFO logic.
The next is a thread pool executor which is responsible for creating a pool of worker threads which can be taken as and when needed to execute different tasks.
If we look at the first few lines we will know that Android has limited the maximum number of threads to be 128 (as evident from private static final int MAXIMUM_POOL_SIZE = 128).
Now the next important class is SerialExecutor which has been defined as follows:
private static class SerialExecutor implements Executor {
final ArrayDeque<Runnable> mTasks = new ArrayDeque<Runnable>();
Runnable mActive;
public synchronized void execute(final Runnable r) {
mTasks.offer(new Runnable() {
public void run() {
try {
r.run();
} finally {
scheduleNext();
}
}
});
if (mActive == null) {
scheduleNext();
}
}
protected synchronized void scheduleNext() {
if ((mActive = mTasks.poll()) != null) {
THREAD_POOL_EXECUTOR.execute(mActive);
}
}
}
The next important two functions in the Asynctask is
public final AsyncTask<Params, Progress, Result> execute(Params... params) {
return executeOnExecutor(sDefaultExecutor, params);
}
and
public final AsyncTask<Params, Progress, Result> executeOnExecutor(Executor exec,
Params... params) {
if (mStatus != Status.PENDING) {
switch (mStatus) {
case RUNNING:
throw new IllegalStateException("Cannot execute task:"
+ " the task is already running.");
case FINISHED:
throw new IllegalStateException("Cannot execute task:"
+ " the task has already been executed "
+ "(a task can be executed only once)");
}
}
mStatus = Status.RUNNING;
onPreExecute();
mWorker.mParams = params;
exec.execute(mFuture);
return this;
}
AS it becomes clear from the above code we can call the executeOnExecutor from exec function of Asynctask and in that case it takes a default executor. If we dig into the sourcecode of Asynctask, we will find that this default executor is nothing but a serial executor, the code of which has been given above.
Now lets delve into the SerialExecutor class. In this class we have final ArrayDeque<Runnable> mTasks = new ArrayDeque<Runnable>();.
This actually works as a serializer of the different requests at different threads. This is an example of Half Sync Half Async pattern.
Now lets examine how the serial executor does this. Please have a look at the portion of the code of the SerialExecutor which is written as
if (mActive == null) {
scheduleNext();
}
So when the execute is first called on the Asynctask, this code is executed on the main thread (as mActive will be initialized to NULL) and hence it will take us to the scheduleNext() function.
The ScheduleNext() function has been written as follows:
protected synchronized void scheduleNext() {
if ((mActive = mTasks.poll()) != null) {
THREAD_POOL_EXECUTOR.execute(mActive);
}
}
So in the schedulenext() function we initialize the mActive with the Runnable object which we have already inserted at the end of the dequeue. This Runnable object (which is nothing but the mActive) then is executed on a thread taken from the threadpool. In that thread, then "finally "block gets executed.
Now there are two scenarios.
another Asynctask instance has been created and we call the execute method on it when the first task is being executed.
execute method is called for the second time on a same instance of the Asynctask when the first task is getting executed.
Scenario I : if we look at the execute function of the Serial Executor, we will find that we actually create a new runnable thread (Say thread t) for processing the background task.
Look at the following code snippet-
public synchronized void execute(final Runnable r) {
mTasks.offer(new Runnable() {
public void run() {
try {
r.run();
} finally {
scheduleNext();
}
}
});
As it becomes clear from the line mTasks.offer(new Runnable), every call to the execute function creates a new worker thread. Now probably you are able to find out the similarity between the Half Sync - Half Async pattern and the functioning of SerialExecutor. Let me, however, clarify the doubts. Just like the Half Sync - Half Async pattern's Asynchronous layer, the
mTasks.offer(new Runnable() {
....
}
part of the code creates a new thread the moment execute function is called and push it to the queue (the mTasks). It is done absolutely asynchronously, as the moment it inserts the task in the queue, the function returns. And then background thread executes the task in a synchronous manner. So its similar to the Half Sync - Half Async pattern. Right?
Then inside that thread t, we run the run function of the mActive. But as it is in the try block, the finally will be executed only after the background task is finished in that thread. (Remember both try and finally are happening inside the context of t). Inside finally block, when we call the scheduleNext function, the mActive becomes NULL because we have already emptied the queue. However, if another instance of the same Asynctask is created and we call execute on them, the execute function of these Asynctask won’t be executed because of the synchronization keyword before execute and also because the SERIAL_EXECUTOR is a static instance (hence all the objects of the same class will share the same instance… its an example of class level locking) i mean no instance of the same Async class can preempt the background task that is running in thread t. and even if the thread is interrupted by some events, the finally block which again calls the scheduleNext() function will take care of it.what it all means that there will be only one active thread running the task. this thread may not be the same for different tasks, but only one thread at a time will execute the task. hence the later tasks will be executed one after another only when the first task complets. thats why it is called SerialExecutor.
Scenario II: In this case we will get an exception error. To understand why the execute function cannot be called more than once on the same Asynctask object, please have a look at the below code snippet taken from executorOnExecute function of Asynctask.java especially in the below mentioned portion:
if (mStatus != Status.PENDING) {
switch (mStatus) {
case RUNNING:
throw new IllegalStateException("Cannot execute task:"
+ " the task is already running.");
case FINISHED:
throw new IllegalStateException("Cannot execute task:"
+ " the task has already been executed "
+ "(a task can be executed only once)");
}
}
AS from the above code snippet it becomes clear that if we call execute function twice when a task is in the running status it throws an IllegalStateException saying “Cannot execute task: the task is already running.”.
if we want multiple tasks to be executed parallely, we need to call the execOnExecutor passing Asynctask.THREAD_POOL_EXECUTOR (or maybe an user defined THREAD_POOL as the exec parameter.
You can read my discussion on Asynctask internals here.

AsyncTasks has a fixed size queue internally for storing delayed tasks. The queue size by default is 10. For example if you start 15 your tasks in a row, then first 5 will enter their doInBackground(), but the rest will wait in the queue for free worker thread. As one of the first 5 finishes, and thus releases the worker thread, a task from the queue will start execution. In this case at most 5 tasks will run together.
Yes, there is limit of how many tasks can be run run at a time. So AsyncTask uses thread pool executor with limited max number of the worker threads and the delayed tasks queue use fixed size 10. Max number of worker threads is 128. If you try to execute more than 138 custom tasks your application will throw the RejectedExecutionException.

How many AsyncTask can be started at a time in an Android app?
AsyncTask is backed by a LinkedBlockingQueue with a capacity of 10 (in ICS and gingerbread). So it really depends on how many tasks you are trying to start & how long they take to finish - but it's definitely possible to exhaust the queue's capacity.
When I start 10 AsyncTask, will all tasks will run simultaneously or one by one?
Again, this depends on the platform. The maximum pool size is 128 in both gingerbread and ICS - but the *default behavior* changed between 2.3 and 4.0 - from parallel by default to serial. If you want to execute in parallel on ICS you need to call [executeOnExecutor][1] in conjunction with THREAD_POOL_EXECUTOR
Try switching to the parallel executor and spam it with 75 000 tasks - the serial impl. has an internal ArrayDeque that has no upper capacity bound (except OutOfMemoryExceptions ofc).