We have an android app that uses soap to communicate with an API implemented in C++ with gsoap. The android application uses the ksoap2-android library for soap communications.
When the android app runs on an actual device and communicates over wifi everything seems to work as it should. However when run in the emulater (virtualized x86 image) requests with longer replies often fail with a ProtocolException unexpected end of stream.
This error and because it only tends to happen with long responses let me to the believe that on the gsoap side the socket gets closed to soon. I have tried somethings.
default accept_flags on the gsoap side didn't work
explicitly remove SO_LINGER flags from accept_flags in gsoap, didn't solve the issue (I was worried it might set linger with a timeout of zero which would result in exactly my problem)
explicitly specify the SO_LINGER flag in the accept flags and a timeout of 10, didn't solve the issue
Then i tried adding a sleep for 1 second right before shutdown in gsoaps tcp_disconnect function. That solved the problem but sleeping is no good ofcourse.
As sleeping is not a good solution I ended up using select which seems to work ok but I'm not 100% confident in this solution.
Here is the actual code change. These are the last lines of the tcp_disconnect function in stdsoap2.cpp. The tcp_select call is what I have added.
if (soap_valid_socket(soap->socket) && !(soap->omode & SOAP_IO_UDP))
{
tcp_select(soap, soap->socket, SOAP_TCP_SELECT_RCV | SOAP_TCP_SELECT_ERR, 5);
soap->fshutdownsocket(soap, soap->socket, SOAP_SHUT_RDWR);
soap->fclosesocket(soap, soap->socket);
soap->socket = SOAP_INVALID_SOCKET;
}
return SOAP_OK;
}
Is this the right solution?
Related
Curl version: 7.71.0 with c-ares
Background
We are building a library that's being integrated into mobile apps. We are targeting both iOS and Android.
Curl initialisation happens in a static block inside the library.
The iOS version of the library is bundled into a framework, which is loaded at app startup, if i'm not mistaken.
The Android version of the library is bundled in a module, which is lazily loaded. (I know this is an issue, especially since we link against OpenSSL, but it's probably important for context).
We built a small HTTP client with curl, that allows use to download some data blob from trusted servers.
Quick architecture review
The HTTP client is running on its own thread. It holds a curl_multi_handle, and any transfer started append a curl_easy_handle to it, and return a handle to a Response that contains a buffer to read the received bytes from, and is used to control the transfer if needed.
Since cURL handles are not thread safe, any action (referred to as Tasks from now on) to the handle is dispatched to the HTTP client's thread, and a boost::shared_future is returned (we might want to block or not depending on the use case).
Here is a rough idea of how the main loop is structured:
while (!done) {
deal_with_transfer();
check_transfer_status();
cleanup_any_orphan_transfer();
execute_all_queue_tasks();
curl_multi_poll(multi, nullptr, 0, very_large_number, nullptr);
}
Appending to the task queue also performs a curl_multi_wakeup(multi) to make sure that task is executed (e.g. adding a new download is also a dispatched task).
The issue
We've only thus far tested on Android, and we've seen in some cases, HTTP client tasks that are blocking are sometimes never returning.
Logs and stacktraces show that we wait on a task being executed on by the HTTP client, but the client is still polling. Everything seems to indicate that it was't woken up when appending a task.
I can't seem to replicate the issue locally, on a device, but it happens often enough to be a blocker issue.
I'm a bit at a loss here, and I don't really know where to start looking to find a way to reproduce the issue, let alone fixing it.
I hope I gave enough context to start making educated guess, or even find a the source of error!
Thanks for reading!
Limitations on network activities for background processes
Mobile operating systems such as Android and iOS have a different scheduling strategies for background processes compared to traditional *nix operating systems. The mobile OS tends to starve the background processes in order to preserve the battery time. This is know as background process optimization and will be applied to the processes/threads of the application the moment application enters in background.
As of Android 7, the background processes are no longer informed about the network events with the CONNECTIVITY_ACTION broadcasts unless they register in the manifest that they want to receive the events.
Although the lobcurl is used in android native code, the threads created from the native library will be subject of the entitlements that the application declared in the manifest (which need to be granted).
Workaround to try
I know how frustrating a blocking issue can be so I can offer you a quick workaround to try until the problem is resolved.
curl_multi_poll() can receive a timeout that in your code is set to a very_large_number. In addition, the last parameter of the function call is a pointer to an integer numfds that will be populated with the number of file descriptors on which an event occurred while the curl_multi_pool() was pooling.
You can use this in your favor to construct a workaround in the following way:
Make the very_large_number a reasonably_small_number
replace the nullptr with &numfds
Surround the curl_multi_poll with a do ... while loop
So you will have something like this:
int numfds;
while (!done) {
deal_with_transfer();
check_transfer_status();
cleanup_any_orphan_transfer();
execute_all_queue_tasks();
numfds = 0;
do {
curl_multi_poll(multi, nullptr, 0, reasonably_small_number, &numfds);
numfds += check_for_other_conditions();
} while ( numfds == 0 );
}
Select the timeout to be a reasonable number (ex. 1s, 10s, 60s ...) that will allow you to break the pooling forcefully, at the same time, not to drain the battery.
I am adding the check_for_other_conditions() so you can use is to check additional conditions. By example the size of the task queue, assuming that there are situations in which the curl_multi_poll() can miss an event, although the event occurred, this extra check can help to break the loop and start executing the tasks.
I am deploying my Nodejs sample app to Google App Engine Flexible env and when I am using google app engine URL which is in the form appspot.com to hit my API, it is taking around 11 secs to send response from my mobile data, but other APIs are sending response in milisecs.
Also, the time delay is only happening when I am opening my android app and sending request to the server after that all requests are taking normal time, and again delay is coming when I again open the app and send request to the server.
Edit - I found that
This can be a caused when your application is still booting up or warming up instances to serve the request and can be called as loading latency. To avoid such scenarios you can implement health check handler like readiness check so that your application will only receive traffic when its ready
That's why I checked in my Logs that readiness check is performed sometimes around 1 sec
and sometimes around 200 ms
Can anyone please tell me is there anything wrong in warming up my instances because I don't think cold boot time is causing this problem.
Edit 2
I have also tried to set min_num_instances: 2 so that once loaded atleast my 2 instances will again not get boot up, but the thing is delay is again same.
Edit 3
runtime: nodejs
#vm: true
env: flex
automatic_scaling:
min_num_instances: 2
max_num_instances: 3
Edit 4
I am noticing a strange behaviour that when I am using this app Packet Capture to capture traffic, then all https requests (if I am not enabling SSL Proxying) and all Http requests are executing in milisecs whereas without using this app all Http/Https requests are taking 11-16 secs of delay.
I don't know how but is there any certificate kind of issue here?
Edit 5
Below I have attached Network Profiler where delay is coming 15 secs
Please Help
Depends on which App Engine you are using and how you setup the scaling, there's always a loading time if you don't have a ready instance to serve a request. But if you have readiness check to ensure your instance is ready (and not cold started for the request), then there shouldn't be a problem.
Can you find a loading request or any corresponding slow request in your logs? If not, then it's likely an issue with the app. If possible, instead of calling this API on your app, do it from two apps (one is already open, one is not). So you make calls from both apps and if you notice that the one that's already open is getting a response faster than the other one, that means that's a problem with the app itself. App Engine can't determine whether or not your app is pre-opened so any difference would be client side.
=== Additional information ===
In the your logs, there's no delay at all. The request enter Google and was processed within a few milliseconds. I am sure there's something application-side. Maybe your app is constructing the request URL (first request) from some other source that results in the delay? App Engine has no knowledge of whether or not your app is opened or not or whether it's sending a first request after being opened, it cannot act differently based on it. As long as your App Engine instance is ready and available, it will treat your request the same way regardless of whether or not it's your first request after the app is opened.
The issue is resolved now, it was happening because of network service provider which is Bharti Airtel, their DNS lookup was taking the time to resolve the hostname. After explicitly using alternative DNS like Google 8.8.8.8 the issue got completely resolved. Maybe it's a compatibility issue of Airtel with Google Cloud.
Last time I checked I remember having to put a warmup request handler so that Google would know that the instance is up and running and can be used to answer calls. Keep in mind that code has to be EXACTLY under the endpoint you specify in the handler under the yaml file. (Wouldn't be the first time someone forgets that)
Here are the docs https://cloud.google.com/appengine/docs/standard/python/configuring-warmup-requests this is python specific, but you can also check other languages like Go, Java, and such in the docs.
If the problem is client dependant (each time a new clients spawns and makes a call it gets the latency) then it is most likely, either a problem with the client app itself or with initialization, registration or DNS resolution.
You could also try to reproduce the requests with CURL or similar, and see if also with those you see the mentioned delay.
I want to develop an application in QtCreator using Qt 5.4.
I'm trying to synchronize a communication with a QMutex and a QWaitCondition. The communication runs in a separate thread that requests data and then waits for the answer using QWaitCondition, if the host is not available, the wait conditions returns after the time specified in timeout_ms.
mtWait.lock();
bool ret = wcWait.wait(&mtWait, timeout_ms);
mtWait.unlock();
The application works well when I run it on my PC (compiled with minGW), but when I use the Build-kit for android (compiler is gcc 4.9), the wait condition returns immediately with a timeout (ret == false).
Only if there is no timeout specified (i.e. the wait condition waits forever), it works as expected. (But the timeout is needed, as the connection may break during runtime.)
Does anybody know this problem or has even a solution for this?
Thank you in advance and best wishes,
Thomas.
Not sure but maybe this is related
QTBUG-46047
?
If so you have to wait for Qt 5.4.2 or build Qt with patch (see QTBUG-44386)
I've recently taken over an Android project. We're looking to try to speed up a sync process we have. This is in the code currently:
System.setProperty("http.keepAlive", "false");
I didn't write the code, so I don't really know the reason this code was added, but ive done some googling and it seems necessary to keep connections working:
http://android-developers.blogspot.com/2011/09/androids-http-clients.html
HttpUrlConnection.openConnection fails second time
When we set this to true, the sync process speeds up substantially, but I don't want to set it to true and not have a decent idea of the consequences. Does anyone know if it's still necessary to set http.keepAlive to false? If so, for all Android devices? Is there an API level where it doesnt matter anymore?
Digging through the code history, we saw where http://square.github.io/okhttp/ was integrated. Is it still necessary to keep this setting to false when we're using OkHTTP?
Thanks!
This question is rather old, but I want to point out for the sake of others that simply disabling that line is not always perfectly safe. The reason some developers do this is because the client may be talking to a server that des not correctly set the Content-Length header in the response.
If the value in the header is lower than the actual number of bytes the server sends, the extra bytes will be included as the first bytes of the response to the next request, which may make that response invalid.
Disabling keep-alive is a way to ensure that each request is self-contained, preventing errors in one response from affecting any others.
You can change that to true with no problems. http.keepAlive just tells the client that it can keep the connection to the server open, rather than renegotiating a connection every time you do something. There shouldn't be any consequences to enabling it; at least, not in my experience.
I have having problems with a WSDL/SOAP service call in an app I have built in flash builder for mobile. I have connected to the service using flash builders built in data/services functionality.
For the most part, the service call works perfectly but once in a while it will cause the app to crash - on my android device it completely locks up (spinning animation stops) and then Android informs me that the app is not responding and asks if I want to close it.
The crash appears to occur quite frequently but not with any pattern. One time it happened on my third attempt, another time it took approximately 30, a couple of times I could not get it to happen and most times it occurs somewhere in between.
It appears that the crash happens after a service call is made but before any response is received. Neither the success or the fault listeners are ever fired. I am very confident that I am sending exactly the same variables to the service every time.
I have used web service calls in other apps without trouble so I have to assume there is something in this particular build that is causing problems but I can't seem to find anything.
Any thoughts on possible causes, things to test or even a solution would be hugely appreciated.
Thank you,
Jamie
Your question lacks essential details, so now I can suggest you to setup Charles proxy and monitor you requests trough it.
If you send too many requests simultaneously, you shall not be confident in fault/result events as air runtime has limitations (in any case, it is a good practice to handle request timeout).