I am experiencing a bad behavior of Pubnub in the following scenario:
Pubnub SDK for Python as a subscriber
Python 3+Django on Ubuntu 14.04
Ubuntu machine hosted on Azure
Android Pubnub client as a publisher
It seems to be that, at a certain point, the Pubnub connection on the server side becomes stale, that is, any message sent from the client is not received by the subscriber.
I have noticed that there are some errors on the log related to pubnub connection:
WARNING 2015-09-30 17:21:24,778 connectionpool 26551 139638563919616 Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'ReadTimeoutError("HTTPConnectionPool(host='pubsub.pubnub.com', port=80): Read timed out. (read timeout=320)",)': /subscribe/.../.../0/...?uuid=...&auth=...&pnsdk=PubNub-Python/3.7.3
After about 5 minutes, the message is received and correctly processed.
I guess that the root of the problem lays in the way Azure manages long http connections, as:
The problem does not arise on my local machine, which has the very same OS version and the same stack
There is an obscure 'azure' parameter in the Python SDK. I tried to activate it without any noticeable difference
As a test I added two more subscribers on my pc and they both receive messages instaneously
Thanks.
It seems to be that the root cause is the fact that Azure cuts HTTP connections above 4 minutes (https://azure.microsoft.com/it-it/blog/new-configurable-idle-timeout-for-azure-load-balancer/).
On the other hand, though, Pubnub creates connections with 5 minutes timeout (320 seconds, see https://github.com/pubnub/python/blob/master/pubnub.py#L1881).
Unfortunately, the Pubnub 320 seconds timeout can not be changed, whereas the Azure timeout seems to be modifiable only via Powershell scripts (uncomfortable, especially if you do not have a Microsoft Window machine).
All in all, I changed the Pubnub source code with a 120 seconds timeout and now everything is going pretty well.
It would be advisable to:
Document that on Pubnub side
Modify Pubnub so that the 320 seconds timeout can be changed
Improve the Azure interface in order to change the timeout parameter without using Powershell
If there is anything that can be done from the PubNub SDK, look at this:
Azure flag on init
https://github.com/pubnub/python/blob/master/pubnub.py#L2112
https://github.com/pubnub/python/blob/master/pubnub.py#L2141-L2146
Linux platform
And this should be getting called if Azure is running on linux:
- https://github.com/pubnub/python/blob/master/pubnub.py#L69-L77
Let me know if that helps any.
Related
I am trying to write my own AB STREAMING update based on https://android.googlesource.com/platform/bootable/recovery/+/master/updater_sample and after many days of developing and decomposing looks it good.
But I have big problem with using https protocol (which is required by default). When i try to call method applyUpdate(Context context, UpdateConfig config) from UpdateManager, I got error 9 (DOWNLOAD_TRANSFER_ERROR) immediately.
It's interesting, because all other metadata were downloaded without problems. It looks problem is only with downloading of payload.bin.
When I try to change protocol from https to http (by enabling in manifest of course and changing link in json file) no problem appeared.
So questions are:
Is it bug in android? Do you have the same problem (I found another question one year ago but without reaction). Is there any special request for https, webserver...
Yes, I can leave http protocol enable, but I am afraid of new steps from Google, they can forbid this option.
Thank you
D
I was facing a similar issue where the Streaming AB OTA update was failing with error 9 (DOWNLOAD_TRANSFER_ERROR) but on Android 10.
Here are the logs from my device
E/update_engine: [1003/030413.677875:ERROR:libcurl_http_fetcher.cc(436)] Unable to get http response code.
I/update_engine: [1003/030413.678370:INFO:libcurl_http_fetcher.cc(467)] Transfer resulted in an error (0), 46484 bytes downloaded
I/update_engine: [1003/030413.678435:INFO:libcurl_http_fetcher.cc(481)] No further proxies, indicating transfer complete
I/update_engine: [1003/030413.678483:INFO:multi_range_http_fetcher.cc(172)] Received transfer complete.
I/update_engine: [1003/030413.678528:INFO:multi_range_http_fetcher.cc(129)] TransferEnded w/ code 0
I/update_engine: [1003/030413.678572:INFO:multi_range_http_fetcher.cc(144)] Didn't get enough bytes. Ending w/ failure.
I/update_engine: [1003/030413.678675:INFO:action_processor.cc(116)] ActionProcessor: finished DownloadAction with code ErrorCode::kDownloadTransferError
I/update_engine: [1003/030413.678723:INFO:action_processor.cc(121)] ActionProcessor: Aborting processing due to failure.
I/update_engine: [1003/030413.678785:INFO:update_attempter_android.cc(454)] Processing Done.
I/update_engine: [1003/030413.678834:INFO:dynamic_partition_control_android.cc(151)] Destroying [] from device mapper
D/OTAManager: onStatusUpdate invoked, status=0, progress=0.00
D/OTAService: onProgressUpdate() called
D/OTAService: OTA Progress :0
I/OTAService: onEngineStatusUpdate - status=IDLE/0
D/OTAService: Sending Response to Client:- Msg:6 Data:0 Status:IDLE D/OTAManager: onPayloadApplicationComplete invoked, errorCode=9
D/OTAManager: setUpdaterState invoked newState=1
D/OTAService: onUpdaterStateChange state=ERROR/1
I/OTAService: onEnginePayloadApplicationComplete - errorCode=OS Update failed due to an error in fetching the payload/9 FAILURE
"libcurl_http_fetcher.cc: Unable to get HTTP response" => This is the culprit.
To know why this is happening I ran the following commands:
adb shell
curl -i https://myurl
To which the response was
curl: (60) SSL certificate problem: certificate has expired
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
I guess I need to update the SSL certificate on the server. However, your issue might not be exactly the same. But running the "curl" command from ADB shell might give you a clue.
Hope this helps.
I have a sample app that uses Parse Server. I've been testing my app on both emulator and physical devices and I noticed that the response time is much slower on the physical device.
This one is from the emulator.
and this is one is from the physical device:
I've been scratching my head regarding this one. I've also posted it on the parse community platform and followed the advice there to manually monitor the server for every change I've made until I reach my prior server configuration. I overhauled the server from scratch. Do some load testing -> Deployed the server on a replica set -> load testing. Now, I'm in the process of doing some basic queries. I used ADB logcat to see what's happening on both device and saw these errors on the physical device and not on the emulator:
E[N rtchannel]_rt_mqtt_publish_callback(356)=>Failed to publish the request over MQTT. token:be925eea-407c-417d-96c9-3af5a7e83e48, category:mcd-sync-data-task-category, requestId:54138, error:Error Domain=mqtt Code=3008
E[S sync]_createResponseError(166)=>Network Response be925eea-407c-417d-96c9-3af5a7e83e48 contains sync error Error Domain=mqtt Code=3008 - Underlying error (null): Error Domain=mqtt Code=3008
This one also shows before pulling the payload.
W[S sync-state-machine]MCDSyncStateMachineScheduleDelayedRetry(173)=>skipping the new retry with longer delay
W[S sync]_scheduleNextCursorRetry(326)=>skipping delayed retry due to state machine state decision
The number of times these errors pop up is related to the number of queries executed. I did not put any configurations on the server which use MQTT. Any help is much appreciated.
I'm trying to build a simple Qt based Bluetooth Server (rfcomm) that just prints in the console the text that is sent by the client. I've noticed that when I'm using QCoreApplication(this is what I need) instead of QGuiApplication or QApplication, I'm not able to receive any message and I'm not notified when a client is connected (SLOTS are not called).
The entire code can be found here. The server can be tested with Qt Bluetooth Chat example and the entire code of my server is extracted from the example. It could be something related to the event loop but I don't know what. I'm running the server on OS X El Capitan with Qt 5.6.1.
According to the Qt team response on this reported bug Bluetooth I/O related functionality needs a running loop which is not available through QCoreApplication.
Qt 5.7 has a workaround for this - using Core Foundation event dispatcher.
To activate this event dispatcher you'll need to set the 'QT_EVENT_DISPATCHER_CORE_FOUNDATION=1' environment variable.
I am using below code to connect to MQTT server
MqttConnectOptions options = new MqttConnectOptions();
options.setCleanSession(true);
options.setKeepAliveInterval(Constants.CLOUD_KEEP_CONN_ALIVE);
byte[] payload = String.valueOf(0).getBytes((Charset.defaultCharset());
options.setWill("willTopic", payload, 0, true);
mAndroidAsyncClient.connect(options, this);
Once connected I get a callback . Now my question is how to debug in server , that setWill is working using mosquitto_sub .
I am using setWill to know client presence.
To confirm setWil is working I wanted to debug from server by subscribing to willtopic and get logs .
I tried connecting and disconnecting many times but there is message sent over will topic.
Command I used on server is
mosquitto_sub -t appTopic --will-topic willTopic
It will be helpful if anyone share the command or way to debug Mqtt setWill.
I tried referring to How to Find Connected MQTT Client Details . But it did not workout for me.
The only real way to test that the LWT (Last Will & Testement) message gets sent is to make the broker send it.
To do this you will have to cause a none clean shutdown of the client, given this is on android the easiest way would be to run the app in the emulator, let it connect then kill the emulator (You have to do this because last time I tried to get the emulator to disable all network activity I discovered it just sent the Network Down broadcasts but left the network up). Once the keep alive period expires and the broker notices the client has gone it should publish the LWT message and you should be able to see this with mosquitto_sub.
There may be ways to query the broker for a given LWT message but this would be totally dependant on the broker you are using. I'm not aware of any brokers that make this easy, but there may be something under the '$SYS/#' topic tree on brokers that support that interface.
EDIT:
Also when using mosquitto_sub you should just be using the -t not --will-topic. --will-topic is the topic that mosquitto_sub would publish it's own LWT on if it was disconnected. You can have as many -t options as you need, and -v will print the topic name before the message body so you can tell them apart
mosquitto_sub -v -t appTopic -t willTopic
Google Cloud Messaging (Push message) problem.
I have two smartphones(HTC Sensation XE, Desire HD) and a tablet(Samsung Galaxy Note 10.1 Wifi).
Sometimes I need to wait for almost 15 minutes to receive the GCM push messages. Sometimes I receive the GCM push messages immediately(within 30 secs).
Is it normal? Or just because I'm under Wifi connected?
If it is normal, are there any other services like "long-polling" I can use for retrieving messages from server?
I'm currently using GCM+Polling(AlarmManager) to retrieve messages from remote server.
Any suggestions or better ideas?
Below is my test result:
The left side is server send time, while the right side is client mobile receive time.
All the test results are under Wifi connected environment.
==========================
HD
23:10:18, 23:24:XX
XE
23:11:21, 23:22:44
Note
23:10:20, 23:14:54
==========================
HD
00:08:12, 00:08:27
XE
00:07:55, 00:07:58
Note
00:08:04, 00:13:35
==========================
HD
00:40:21, 00:55:22
XE
00:39:56, 00:40:14
Note
00:40:13, 00:40:59
Thanks!!
GCM uses throttling when there's excessive use, see the following document for more details:
http://developer.android.com/google/gcm/adv.html#throttling
Please read my answer here:
Google Cloud Messaging - messages either received instantly or with long delay
It's basically a tcp timeout problem
You can manually trigger the heartbeat to keep the connection alive by the following code. Some network routers will automatically kill the idle socket connections if it is idle for a particular time period (sometimes 5 minutes). Execute the code in every 5 minute so that the connection will be re-established if it is closed. But, obviously reducing the interval will consume bit more battery (I don't know how much it is relevant here because nowadays the smart phones come up with high mAh battery).
getApplicationContext().sendBroadcast(new Intent("com.google.android.intent.action.GTALK_HEARTBEAT"));
getApplicationContext().sendBroadcast(new Intent("com.google.android.intent.action.MCS_HEARTBEAT"));