I have a small Android application which does a server call to post some User data to a server.
Following is the code :
private boolean completed = false;
public String postData( Data data){
new Thread(new Runnable() {
#Override
public void run() {
try{
String response = callApi(data);
completed = true;
}catch(Exception e){
Log.e("API Error",e.getMessage());
completed = true;
return;
}
}
}).start();
while(!completed){
// Log.i("Inside loop","yes");
}
return response.toString();
}
The above method calls the API to post data and returns the response received which works fine.
The loop at the bottom is a UI blocking loop which blocks the UI until a response is received or an error.
The problem :
I tried the same code for Marshmallow and Oreo device and the results were different.
For Marshmallow : Things moved in line with my expectation. :)
For Oreo (8.1.0) :
The very first API call works good enough after I open the App. However, the subsequent API calls after, cause the UI to block forever although an Error or Response is received from the Server(verified by logging and debugging).
However, on setting breakpoints(running in Debug mode) the App moves with much less trouble.
It seems the system is unable to exit the UI blocking loop although the condition is met.
The second behavior which was noticed is when I log a message in the UI blocking thread, the System is able to exit the loop and return from the Method though the API response is not logged.
Could someone help understand such inconsistency across these two flavors of Android and what could be the change introduced causing such a behavior for Oreo but not for Marshmallow?
Any insight would be extremely helpful.
It's more likely to be differences in the processor cache implementation in the two different hardware devices you're using. Probably not the JVM at all.
Memory consistency is a pretty complicated topic, I recommend checking out a tutorial like this for a more in-depth treatment. Also see this java memory model explainer for details on the guarantees that the JVM will provide, irrespective of your hardware.
I'll explain a hypothetical scenario in which the behavior you've observed could happen, without knowing the specific details of your chipset:
HYPOTHETICAL SCENARIO
Two threads: Your "UI thread" (let's say it's running on core 1), and the "background thread" (core 2). Your variable, completed, is assigned a single, fixed memory location at compile time (assume that we have dereferenced this, etc., and we've established what that location is). completed is represented by a single byte, initial value of "0".
The UI thread, on core 1, quickly reaches the busy-wait loop. The first time it tries to read completed, there is a "cache miss". Thus the request goes through the cache, and reads completed (along with the other 31 bytes in the cache line) out of main memory. Now that the cache line is in core 1's L1 cache, it reads the value, and it finds that it is "0". (Cores are not connected directly to main memory; they can only access it via their cache.) So the busy-wait continues; core 1 requests the same memory location, completed, again and again, but instead of a cache miss, L1 is now able to satisfy each request, and need no longer communicate with main memory.
Meanwhile, on core 2, the background thread is working to complete the API call. Eventually it finishes, and attempts to write a "1" to that same memory location, completed. Again, there is a cache miss, and the same sort of thing happens. Core 2 writes a "1" into appropriate location in its own L1 cache. But that cache line doesn't necessarily get written back to main memory yet. Even if it did, core 1 isn't referencing main memory anyway, so it wouldn't see the change. Core 2 then completes the thread, returns, and goes off to do work someplace else.
(By the time core 2 is assigned to a different process, its cache has probably been synchronized to main memory, and flushed. So, the "1" does make it back to main memory. Not that that makes any difference to core 1, which continues to run exclusively from its L1 cache.)
And things continue in this way, until something happens to suggest to core 1's cache that it is dirty, and it needs to refresh. As I mentioned in the comments, this could be a fence occurring as part of a System.out.println() call, debugger entry, etc. Naturally, if you had used a synchronized block, the compiler would've placed a fence in your own code.
TAKEAWAYS
...and that's why you always protect accesses to shared variables with a synchronized block! (So you don't have to spend days reading processor manuals, trying to understand the details of the memory model on the particular hardware you are using, just to share a byte of information between two threads.) A volatile keyword will also solve the problem, but see some of the links in the Jenkov article for scenarios in which this is insufficient.
Related
When I have a small single block of code that should not get executed in parallel in my application (eg. when an Activity gets killed and restarted when the user turns the device), would it be possible to put it into a code block synchronized to the Application Context?
As far as I understood, this should work as long as the application context object stays the same over the full lifecycle of the app – but does it? I haven't found anything about its lifecycle.
synchronized (getApplicationContext()) {
if (piCacheFileDoesNotExist() == true) {
calculatePIandStoreToCacheFile(); // This is an example only...
}
}
Would that example be a possible way (especially when the task is too small to create an extra Service for it) or would it be a bad idea because of anything I've missed?
I believe you're trying to solve the wrong problem with the wrong tools.
syncrhonization: This is often use to make some code "thread-safe-ish" by ensuring an "object" (any) can act as a gatekeeper. If you can guarantee that there's only one instance of some object, then multiple threads could ask said unique object "are you free so I can run this code?" if the answer is yes, then the code is executed on that thread, otherwise, the thread must wait until the object frees it. Think of a semaphore.
synchronized(XXX) { ... } is doing that. It ensures ... is not executed until XXX says so. (note: this is an oversimplification).
It's often used like so:
private val lock = Object()
fun doSomething() {
synchronized(lock) { ... }
}
This means the code ... will only be executed by one thread at a time...
Now this has nothing to do with Android Lifecycle, App Lifecycle, and more specifically, Android Activity Lifecycle.
Your problem is an architecture problem. You have a UI (Activity) that can and will be destroyed by the Android OS at will (or when your users leave your app either explicitly or by performing a different action like turning the screen off).
You also have a block of code that -and I quote- "should not get executed in parallel in my application". Now this is a bit misleading. Do you not want this code to be run multiple times at the same time (in parallel, that is) or do you only want to run this code "once" in your app and never do it again until certain condition is fulfilled?
Depending on which case you need, the solutions may be different, but in all cases, the actual code that needs execution should not be part of an activity, because as we know, Activities are meant to be created and destroyed at all times. This then raises the question of: Where do I put this information that I want to persist for LONGER than what my activity does?
And here you have many options... from using a ViewModel (that can live a bit longer) to using a use-case/interactor/repository/etc. that can store this "information" in a more persistent way, and be able to provide it to the (new) activity when it is restored.
I suggest you spend some time reading about MVVM, Android Jetpack, Android Lifecyle, and, because your block of code appears to be "long work" you execute in a background thread, invest time in learning the basics of Kotlin Coroutines, LiveData and/or Flow/StateFlow, as they will make your modern Android life easier.
when I turn the device and back, each time the current Activity gets ended and a new one starts
This means the data that would let the code/activity decide what to do, should not live in the activity, otherwise it would get lost when the inevitable Activity Destroy/Recreate happens. Think of this: You wouldn't store your car keys inside the car... would you? ;)
This is why ViewModels, Respositories, etc. can help you overcome this. In truth, you may or may not like the extra complexity, code, boiler-plate stuff, etc, that comes with this, but when you're developing an Android app, you have to abide by its idiosyncrasies (and bad decisions sometimes). Try telling an iOS dev that you don't want to use ViewControllers and that you're gonna build your own... they are going to scream at you: "Don't fight the framework!!!!!!!"...
Well, this is similar. Do your best to learn the Android way.
Curl version: 7.71.0 with c-ares
Background
We are building a library that's being integrated into mobile apps. We are targeting both iOS and Android.
Curl initialisation happens in a static block inside the library.
The iOS version of the library is bundled into a framework, which is loaded at app startup, if i'm not mistaken.
The Android version of the library is bundled in a module, which is lazily loaded. (I know this is an issue, especially since we link against OpenSSL, but it's probably important for context).
We built a small HTTP client with curl, that allows use to download some data blob from trusted servers.
Quick architecture review
The HTTP client is running on its own thread. It holds a curl_multi_handle, and any transfer started append a curl_easy_handle to it, and return a handle to a Response that contains a buffer to read the received bytes from, and is used to control the transfer if needed.
Since cURL handles are not thread safe, any action (referred to as Tasks from now on) to the handle is dispatched to the HTTP client's thread, and a boost::shared_future is returned (we might want to block or not depending on the use case).
Here is a rough idea of how the main loop is structured:
while (!done) {
deal_with_transfer();
check_transfer_status();
cleanup_any_orphan_transfer();
execute_all_queue_tasks();
curl_multi_poll(multi, nullptr, 0, very_large_number, nullptr);
}
Appending to the task queue also performs a curl_multi_wakeup(multi) to make sure that task is executed (e.g. adding a new download is also a dispatched task).
The issue
We've only thus far tested on Android, and we've seen in some cases, HTTP client tasks that are blocking are sometimes never returning.
Logs and stacktraces show that we wait on a task being executed on by the HTTP client, but the client is still polling. Everything seems to indicate that it was't woken up when appending a task.
I can't seem to replicate the issue locally, on a device, but it happens often enough to be a blocker issue.
I'm a bit at a loss here, and I don't really know where to start looking to find a way to reproduce the issue, let alone fixing it.
I hope I gave enough context to start making educated guess, or even find a the source of error!
Thanks for reading!
Limitations on network activities for background processes
Mobile operating systems such as Android and iOS have a different scheduling strategies for background processes compared to traditional *nix operating systems. The mobile OS tends to starve the background processes in order to preserve the battery time. This is know as background process optimization and will be applied to the processes/threads of the application the moment application enters in background.
As of Android 7, the background processes are no longer informed about the network events with the CONNECTIVITY_ACTION broadcasts unless they register in the manifest that they want to receive the events.
Although the lobcurl is used in android native code, the threads created from the native library will be subject of the entitlements that the application declared in the manifest (which need to be granted).
Workaround to try
I know how frustrating a blocking issue can be so I can offer you a quick workaround to try until the problem is resolved.
curl_multi_poll() can receive a timeout that in your code is set to a very_large_number. In addition, the last parameter of the function call is a pointer to an integer numfds that will be populated with the number of file descriptors on which an event occurred while the curl_multi_pool() was pooling.
You can use this in your favor to construct a workaround in the following way:
Make the very_large_number a reasonably_small_number
replace the nullptr with &numfds
Surround the curl_multi_poll with a do ... while loop
So you will have something like this:
int numfds;
while (!done) {
deal_with_transfer();
check_transfer_status();
cleanup_any_orphan_transfer();
execute_all_queue_tasks();
numfds = 0;
do {
curl_multi_poll(multi, nullptr, 0, reasonably_small_number, &numfds);
numfds += check_for_other_conditions();
} while ( numfds == 0 );
}
Select the timeout to be a reasonable number (ex. 1s, 10s, 60s ...) that will allow you to break the pooling forcefully, at the same time, not to drain the battery.
I am adding the check_for_other_conditions() so you can use is to check additional conditions. By example the size of the task queue, assuming that there are situations in which the curl_multi_poll() can miss an event, although the event occurred, this extra check can help to break the loop and start executing the tasks.
I am building an application for both iOS and Android using Xamarin cross platform. I have one project called "Core", and then one for iOS and one for Android.
In the shared "Core" project I want to show a loading animation that I made, while fetching the current location from the device. I can get my locationWatcher, start it, and call LocationWatcher.CurrentPosition directly, but since it has just started watching for location updates, it hasn't yet had time to get any location.
My solution to this is to wait 2 seconds after starting it and then get the location. However I don't want to wait asynchroneously, I want the UI to be locked while waiting. This is because the user should not be able to do anything until current position os known.
I currently achieve this by having a loop:
DateTime startTime = DateTime.Now();
while ( (DateTime.Now() - startTime).TotalMilliseconds() < 2000) {
//Locking the UI here, but it's what I want so it's ok.
}
This seems like an ugly 'hack' for something that in Java would simply be "Thread.sleep(2000)"
Is there a better way of doing this?
The multiplatform way with Xamarin Forms would be to
await Task.Delay(ms);
Edit:
After reading that you really want to block the mainthread heres the multiplatform way to block a Thread:
Task.Delay(ms).Wait()
Blocking the UI thread is bad practice, since among other things it could cause the device to mark the app as "not responding". You are also not guaranteed that the position will have been determined in two seconds, so in the best case your suggestion is only a partial fix.
What you'll really want to do is to block UI input by disabling the possibility to give input (there should be methods to disable your UI elements) and then see if there's a callback from the geolocation service when it has determined your location. In the callback, re-enable the UI elements. If such a callback does not exist, perform the location lookup synchronously on the UI thread with RunOnUIThread/InvokeOnMainThread, so that the thread is blocked only until it returns.
If you want to go ahead with your solution anyway,
RunOnUiThread(() => Thread.Sleep(2000));
should do the trick, but I really recommend against doing that. On iOS, you'll have to do
InvokeOnMainThread(() => Thread.Sleep(2000));
I'm using an app that uses async tasks to do short term background calculation jobs. These seem to end OK (go through onPostExecute() etc), but in the Eclipse debugger, one or more still hangs around.
Then I found this link - AsyncTask threads never die - so OK, it's about a thread pool and in theory not an issue.
Problems is however, I am also trying to use Google in-app billing code V3, and that appears to throw an exception whenever you carry out a purchase and there's already an AsyncTask thread hanging around. Catching the exception won't help - it still won't do anything.
How can I get around this? What do I need to do to guarantee the old calculation thread(s) have gone?
Found out what is going on here, and it wasn't what I thought. I'll detail it here as it may be useful to somebody. Has nothing to do with other AsyncTask threads and thread pooling.
In the IabHelper class are two functions flagStartAsync() and flagEndAsync(). The aim of these is to produce a single pass gate (bit like wait() and signal() in traditional multi-threading) so only one async operation (that is, communications with Google Play and the server) can occur at a time. If flagStartAsync() get called while something is already going on, it produces an exception. Not terribly graceful, but effective I guess.
The flagStartAsync() 'test and set' gets called at the start of launchPurchaseFlow() among other places, and flagEndAsync gets called in handleActivityResult() - again - among other places. So providing the purchase flow does something that always produces a result, no problem. The problem is - it doesn't always.
If you look at launchPurchaseFlow() there are several paths out that will not kick off the async operation, and if one of those get taken, mAsyncInProgress (the relevant flag) gets left set.
What blew it in my case was that I hadn't checked that the item was already purchased, and 'already purchased' is one of the paths out. Mea culpa, but the problem is that I cannot convince myself that there aren't several other paths that you just cannot avoid at times. What if operation is slow and the 'purchase' button gets pressed twice, for instance? I bet there are others as well. One could catch the exception, and that would stop a crash, but it wouldn't really help if nothing came along to clear the flag in the end. I guess the exception handler could call flagEndAsync() but it has an uncomfortable 'sledgehammer to crack a nut' feel.
It strikes me that this is probably a non-robust piece of code. What I've done for now is call flagEndAsync() in the various ways out of launchPurchaseFlow(), but that is just a temporary fix. I don't know enough about the IabHelper code, but I think it needs more careful thought and I need to analyse it to see everything it does.
I am working on an image processing Android application. Suppose you have a C++ singleton object that provides some time-consuming functions and allocates its own memory. Furhtermore, the C++ library will provide some other functions that will do some time-consuming work as well. This functions will be called by the singleton object. They can allocate their own temporary memory (that will be freed on function termination) and need to exchange data with the singleton object. The workflow is the following:
the native C++ library is loaded, the singleton object created (it will allocate memory and load data from the asset directory).
the user, using the application interface, select an image and loads it
the image is passed to the singleton object that will computes some informations
the user can request a particular image processing algorithm, the singleton object is asked to call the corresponing function
repeat from 4 or go to 2 if the user load another image (the singleton object will be resetted (the memory allocated on step 1 is retained until the application is tereminated)).
Step 2 and 3 are the most time consuming part of the app. I would like the user to be able to stop the current processing if too much time is passed and the application to remain responsive during the time consuming processing algorithms. The most simple way to do this app is to call the native functions and wait the, but this will probably block the UI. Another way is to design those functions to check a flag every N processed pixels to know if the function must stop (this would allow me to free memory when it happens). A third option could be to use java threads, but how?
You will have to run the time consuming task off the UI thread. You could do this with a native thread, but it would be simpler to call the native function from a background thread in java - there are several ways you can do that, such as an async task, etc which you can read about.
When you start the time consuming operation, you'll want the UI to display some sort of busy indicator to the user. The UI thread will have to remain responsive (ie, the user can 'back' or 'home') but you can disable most of your other controls if you wish.
Your native operation in the background thread would, as you suggested, periodically check a stop request flag. You will probably find it easiest to make that a native flag and set it with another (brief) native function called from the UI thread; there's the option of making it a java flag and calling java from C to check it, but that seems more complicated.
If your processing is going to be especially lengthy, arguably you should do the work not only in the background, but in the context of an Android service rather than that of an activity. To a first approximation, native code will not care about the difference, however there are potential implications for what happens if the activity goes to the background during processing - if the work is being done in a service (or more specifically, if the process contains a service which is active), Android will try to let it keep running if possible. In contrast, if the process only has an activity which is now not active because something else is in the foreground, Android is more likely to kill it or throttle its available CPU. Ultimately, whatever you do your native code will need to deal with the possibility of its process being killed before the work is done - ie, you have to be able to recover from such a state when a new process is created as the user returns your activity to the foreground. Having your flag also able to notify the native code of an onDestroy() call as an alert to save its work could be a help, but it will still need to be able to recover (at least cleanly re-do) from being killed without the courtesy of that notification.