I receive pseudo-randomely ECONNRESET from my backend's company. I say pseudo random because althought It doesn't happend the same way I can provoque it almost every time by launching a large amount of request.
I typicaly launch downloads from activity's life cycles events and therefore I use Retrofit's Call.enqueue() to network on background. In the part of the code that seems to cause trouble, I'm launching a series of download (~15 REST routes for jsons and 5-6 files) from a background thread. With that scenario, the ECONNRESET apears 2 out of 3 try on one of the called REST route.
There is no more explanation server side the only thing that we logged was read/write ECONNRESET.
Here is what I've tried :
Update to okhttp 3.5.0 (from 3.2.0) and retrofit 2.1.0 (from 2.0.2)
I added "Connection:close" int my requests header to prevent keep alive.
I reduced my total pool to 1 :
.connectionPool(new ConnectionPool(0, 1, TimeUnit.SECONDS))
It happend on my phone (Android 6), I don't have other phone to test the code. I've some unrelated trouble with my AVD that prevent me for testing on different android versions (soon to be fixed).
Would you know what could provoque this ?
Thanks,
For the records, I was using HttpURLConnection in my file download method (whearas my REST api was questioned throught retrofit and okhttp), I changed it to okHttp and it's all good now.
Related
I have an Android app Activity which fills a RecyclerView with some content from a DB hosted in Google's GCP CloudSQL. The data is served via a python flask app run by Google's AppEngine.
My problem is that some step or steps along the way from sending the GET request to having the RecyclerView inflated and full with the content introduce a heavy delay. The process takes about 10 to 12 seconds.
I have trouble understanding where the problem lies. I have tried the following and haven't been able to isolate a candidate for the delay. Taking into account that the Android app runs on an Android Studio emulator on my localhost:
If I run my flask app locally on my computer, but still request the data from the CloudSQL DB, the process is fast. So, it would seem that the problem is neither the DB nor the Android app RecyclerView inflating step (therefore, it must be the AppEngine flask app).
But if I run both the DB and the flask web server both on GCP and request the data via my web browser, I also get the data (JSON) fast. So, it would seem that the flask app hosted on GCP's AppEngine is also fine.
So, if according to the above tests all the three individual elements, the DB, the AppEngine Flask app and the RecyclerView inflation, all seem to behave good in terms of speed, why is it the chained process so slow in my app?
Most information out there for similar problems ascribe the slow response to AppEngine's cold start, but after many attempts, I am starting to think that this might not be my problem. Aside from the fact that, when I request the DB content via a web browser I get the response decently fast, I have checked and/or tested:
Reducing number of items in the list.
Setting the minimum number of instances to 1 + have enabled warmup requests (and my app processes them).
Setting the minimum number of idle instances to 1.
Use a WSGI production server (waitress) instead of the development flask service.
the AppEngine is located in "right" GCP Zone (europe-west-3). By "right" I mean the geographically closest to me, and the same region in which the CloudSQL DB is hosted.
Have set a "keep-alive" refresh cron job every minute to ensure no instance has to start cold.
Have tried going to manual scaling with one fixed instance.
None of the above solved the problem or reduced the total waiting time in the app. According to the AppEngine Dashboard, the loading latency of many requests is taking between 2 and 4 seconds, which is not the 10-12 seconds I have to wait on the Android app, but still seems abnormally, long taking into account all the measures in place for avoiding cold start (and, again, the fact that retrieving the DB info via web browser works at normal speed). This makes me think that either I have not successfully solved the cold start thing, or the latency problem lies elsewhere.
I am lost, and I do not know where to continue looking for issues. I would appreciate getting some tips in the right direction before I have to implement an in-device DB cache.
EDIT
Below there is a summary of HTTP request latencies to the web server (/refresh is the instance keep-alive resource, and /allrecords is an actual working endpoint). As it can be seen, the latencies are quite OK (which matches the good speed when retrieving the data via web browser).
I am quite confident the problem does not have to do with AppEngine cold start, so one would think the problem must lie within the Android app, but if I do the DB request via a web server in my local machine, the Android app works at normal speed.
EDIT 2
Retrieving info from the web server in JSON format via the web browser of the emulated device also works fast. So it does not seem to be a problem of the emulated device with internet connection speed.
I am deploying my Nodejs sample app to Google App Engine Flexible env and when I am using google app engine URL which is in the form appspot.com to hit my API, it is taking around 11 secs to send response from my mobile data, but other APIs are sending response in milisecs.
Also, the time delay is only happening when I am opening my android app and sending request to the server after that all requests are taking normal time, and again delay is coming when I again open the app and send request to the server.
Edit - I found that
This can be a caused when your application is still booting up or warming up instances to serve the request and can be called as loading latency. To avoid such scenarios you can implement health check handler like readiness check so that your application will only receive traffic when its ready
That's why I checked in my Logs that readiness check is performed sometimes around 1 sec
and sometimes around 200 ms
Can anyone please tell me is there anything wrong in warming up my instances because I don't think cold boot time is causing this problem.
Edit 2
I have also tried to set min_num_instances: 2 so that once loaded atleast my 2 instances will again not get boot up, but the thing is delay is again same.
Edit 3
runtime: nodejs
#vm: true
env: flex
automatic_scaling:
min_num_instances: 2
max_num_instances: 3
Edit 4
I am noticing a strange behaviour that when I am using this app Packet Capture to capture traffic, then all https requests (if I am not enabling SSL Proxying) and all Http requests are executing in milisecs whereas without using this app all Http/Https requests are taking 11-16 secs of delay.
I don't know how but is there any certificate kind of issue here?
Edit 5
Below I have attached Network Profiler where delay is coming 15 secs
Please Help
Depends on which App Engine you are using and how you setup the scaling, there's always a loading time if you don't have a ready instance to serve a request. But if you have readiness check to ensure your instance is ready (and not cold started for the request), then there shouldn't be a problem.
Can you find a loading request or any corresponding slow request in your logs? If not, then it's likely an issue with the app. If possible, instead of calling this API on your app, do it from two apps (one is already open, one is not). So you make calls from both apps and if you notice that the one that's already open is getting a response faster than the other one, that means that's a problem with the app itself. App Engine can't determine whether or not your app is pre-opened so any difference would be client side.
=== Additional information ===
In the your logs, there's no delay at all. The request enter Google and was processed within a few milliseconds. I am sure there's something application-side. Maybe your app is constructing the request URL (first request) from some other source that results in the delay? App Engine has no knowledge of whether or not your app is opened or not or whether it's sending a first request after being opened, it cannot act differently based on it. As long as your App Engine instance is ready and available, it will treat your request the same way regardless of whether or not it's your first request after the app is opened.
The issue is resolved now, it was happening because of network service provider which is Bharti Airtel, their DNS lookup was taking the time to resolve the hostname. After explicitly using alternative DNS like Google 8.8.8.8 the issue got completely resolved. Maybe it's a compatibility issue of Airtel with Google Cloud.
Last time I checked I remember having to put a warmup request handler so that Google would know that the instance is up and running and can be used to answer calls. Keep in mind that code has to be EXACTLY under the endpoint you specify in the handler under the yaml file. (Wouldn't be the first time someone forgets that)
Here are the docs https://cloud.google.com/appengine/docs/standard/python/configuring-warmup-requests this is python specific, but you can also check other languages like Go, Java, and such in the docs.
If the problem is client dependant (each time a new clients spawns and makes a call it gets the latency) then it is most likely, either a problem with the client app itself or with initialization, registration or DNS resolution.
You could also try to reproduce the requests with CURL or similar, and see if also with those you see the mentioned delay.
I have an app that requires a lot of async requests, for which I am using the loopj asynchttpclient library.
After login to a service, many requests get made, someone loading 5-10 images at one time, and a couple JSON requests, all made around the same time. I never had an issue in all of my debug testing over the last several months. But after building a signed APK and installing, when multiple requests get ran simultaneously (in the above description) - the requests stop returning anything. None of the call backs are called. The server is receiving the requests and returning, but the async http callbacks never get called.
I have read that if all the callbacks are not implemented its possible they won't call, but I have implemented them all including the deprecated ones and still no luck.
Any reason why this would happen? I am kind of late in the game and am trying to avoid having to switch to an entirely different networking library.
Cheers
My Android app makes three distinct forms of data transfer:
Queries the local database: returns results if found (local / short timeout);
Queries a remote API: typical response size is around 1k (remote but tiny payload / medium timeout);
Downloads a remote file: size may be anywhere between 100k - 1MB (remote and large data / long timeout).
These transfers are made using extended subclasses of AsyncTask and they work pretty well, certainly for steps 1 and 2. The issue is with step 3.
If the device is on WiFi, 3G or another reasonably fast connection, the AsyncTask will not time out and the remote file downloads in time. If the device is on 2G or a slow connection, it will time out - even when the download is still progressing (albeit slowly).
Ideally, I would like to monitor the download progress at intervals, and either reset the AsyncTask's internal timeout counter or prevent the task from cancelling if progress has been made since the last check. Unfortunately, both of those options seem to require either read or write access to AsyncTask's private timeout variable.
How can I prevent AsyncTask from timing out when download progress is still continuing?
Edit: I've just noticed that StackOverflow offered me the tag of Android Download Manager. Is this the kind of thing that ADM is designed for?
I'm going to answer my own question here, or at least the edit, to say that in the end I implemented Android's DownloadManager to ensure that background downloads could continue at whatever pace the connection allowed. I then implemented a Manifest-declared Receiver which started a Service to insert the response into the database. Finally, the user is notified via the toolbar that the download has finished and that they may return to the app to view the results.
However, I'm not going to accept my own answer :-) because I haven't answered whether it's possible to stop an AsyncTask timing out in the way that I asked. But everything I found suggests that the answer is 'no'.
I have a problem, when i startup my hybrid app. It takes very long to load the first page. ~40 seconds.
I work with GWT, Google App Engine and RequestFactories. I detected, that the app makes several request to the server (~ 10 requests).
Now i wonder, how can i increase the performance of staring up my app.
Group all request into one Request, which delivers all data with a single Request.
(~300kb data)
Make a startup page, with low amount of requests and data.
(~50kb data)
Better idea?
I prefer, that i can keep the current startup page. Can you share your experiance?
You have pointed out the two main ways to reduce the start-up time.
1.- Reduce requests: With RF you can group all requests in one if you share the same service instance and call only once the .fire() method. Use Bundles for images, css, and other resources so as them are downloaded in the same request.
2.- Reduce the js size: split your code in pieces using GWT.runAsync(). So the first piece of code could be the minimum stuff for starting your app until the user interacts with it. You can use the gwt compilation reports to see whether your code is bigger. You can use the new closure compiler in gwt-2.5 to reduce and optimize the final js.
3.- Another thing you must check is that your webserver configuration is sending the appropriate headers so as the browser caches gwt fragments, as well as you should check if it is compressing the files before sending to the client. You can make gwt compiler to pre-compress those js fragments though.
4.- For successive loads, I mean the user goes to your app a second time, consider using localstorage for caching certain things which don't change instead of requesting them again. Configure your HTML5Manifest, if you are using mgwt, it is really easy since they include a linker to produce the manifest in compile time.
5.- Consider using light-weight widgets and frameworks instead of heavy widgets (js collections, elemental, gquery, mgwt, etc).
I have several recommendation:
Load data that changed rarely in a single request at start up.
Activate GZip compression on your server.
Use background call when application start up finished in user idle time.
Group related request in a single request.