I have a Node.js server using socket.io to connect Android apps and I've been hosting it locally by just running it in my IDE and connecting to my local IPv4 address but I want it to work without me having to keep my PC running constantly so I've tried using Google Cloud and managed to get it mostly working but the client doesn't keep the connection and disconnects consistently.
I followed this tutorial up to step 4 after that I ran gcloud app deploy.
My Node.js server is in one file, it has these declarations at the top.
const express = require("express");
const app = express();
const http = require("http");
const server = http.createServer(app);
const { Server } = require("socket.io");
const io = new Server(server);
I then have this for the initial client connection.
io.on("connection", (socket) => {
console.log("User connected.");
Everything inside of this is just listeners for what gets emitted by the client so I don't think they're the problem.
And then outside of that I have
server.listen(8080, () => {
console.log("Server is listening");
});
I don't know if anything from the package.json file is relevant but I can provide it if need be.
After deploying to Google Cloud using the tutorial there are a few things in the logs that may be the reason behind the problem.
Waiting for network connection open. Subject:"app/invalid" Address:127.0.0.1:8080
Waiting for network connection open. Subject:"app/invalid" Address:127.0.0.1:8081
Wait successful. Subject:"app/invalid" Address:127.0.0.1:8080 Attempts:97 Elapsed:485.916418ms
App is listening on port 8080. We recommend your app listen on the port defined by the PORT environment variable to take advantage of an NGINX layer on port 8080.
These might have simple solutions but I have very little experience with server hosting.
Once my Android client connects to the server, the log outputs "User connected." as it should, then around 10 seconds later it does it again and this repeats. There's no error I can see between the connections just a few socket.io POST/GET requests.
I tried adding session affinity to the app.yaml but hasn't solved it either.
This is my app.yaml if any changes need to be made here
runtime: nodejs16
env: standard
instance_class: F1
automatic_scaling:
min_idle_instances: automatic
max_idle_instances: automatic
min_pending_latency: automatic
max_pending_latency: automatic
network:
session_affinity: true
Thanks for any help, if I need to provide any other files I can do so.
Socket communication requires persistent network connections. So, App Engine Flexible provides an option to keep the network connection alive.
I think you are deploying your app in App Engine Standard , which does not support this option.
So, you can deploy your app in App Engine Flexible and you need to include the following configuration in your app.yaml
network:
session_affinity: true
For refence on session affinity, please refer to this page https://cloud.google.com/appengine/docs/flexible/nodejs/using-websockets-and-session-affinity#session_affinity
So, your updated app.yaml can look like this
runtime: nodejs
env: flex
network:
session_affinity: true
Please refer to this page for supported yaml configuration for flexible environment - https://cloud.google.com/appengine/docs/flexible/nodejs/reference/app-yaml
Messages you have pasted are most likely not indicating a problem. Your application is running in a sandbox environment for which a container instance and a web server must be initialized the first time that your app engine service receives a request. This is also called Loading request and the messages you see indicate the startup of your container instance and your webserver.
You can take a look at Google's own documentation regarding handling requests in node.js or follow a quickstart guide.
If there are no other logs during the disconnection, I would suggest checking the quotas to see if you're not exceeding any.
Related
I'm trying to write a mobile app via Capacitor that makes use of PouchDB. When I run the app in the emulator via Android Studio the connection to the remote CouchDB instance fails. I've tracked this down to a failure in the fetch API for certain URLs when running on Android.
To debug I made a minimal web application and wrapped it using Capacitor to run on Android. The app includes the following code
const testFetch = (url) => {
console.log("Testing fetch", url)
fetch(url)
.then((response) => response.text())
.then((t) => {
console.log("Respose from fetch:", url)
console.log(t)
console.log("that was it")
})
.catch((reason) => {
console.log("FETCH FAILED", url, reason)
})
}
I then have three tests:
testFetch("https://jsonplaceholder.typicode.com/todos/1"); // just some JSON
testFetch("http://10.0.2.2:5984/simple"); // local pouchdb instance
testFetch("http://10.0.2.2:8080/sample.json"); // local http server + CORS
The second two use the IP address that is an alias for the development machine when running in the Android emulator. I confirmed that I can access all of these URLs from the browser on the emulator but the app succeeds on the first and fails on the second two (error: TypeError: Failed to fetch). When running the base web app in the browser, all succeed (with localhost instead of 10.0.2.2).
CORS headers are in place on all URLs. As far as I can see the app doesn't even try to access the two servers that fail - no HEAD requests for example. I've also tried various other URLs and can't see a pattern to the failures -- eg. it's not the port number != 80.
Any clues as to what is going on would be appreciated.
So the thing I didn't notice that the failing URLs had in common was http rather than https. It turns out that fetch silently fails to work for any http URL, just giving the error 'Failed to fetch'.
I'm not sure whether this is a feature of the Android web view or of Capacitor itself. The Capacitor docs suggest that using https is a good idea but not that http won't work.
This policy doesn't get altered by setting a Content Security Policy in the main page header.
The original goal was to connect a local PouchDB database to a remote CouchDB instance. This now works as long as the CouchDB instance is served via https. Without that you just get silent failure to sync.
I'm trying to use MQTT (Paho library on Android, mosquitto message broker on a linux server) to pass moves in a turn-based game, replacing a custom server I wrote years ago. Its simplicity and pub-sub design seem perfect: each device subscribes to a unique id as "topic" and communicates that as its "address." Then other devices can reach it by publishing to that address.
It works perfectly in my test Linux client (connecting using the mosquitto-dev library on Ubuntu). And it works perfectly on Android WHEN THE ANDROID APP IS RUNNING. In the Linux client case, if a message is sent while the app isn't running or connected the app receives the message as soon as it does connect and subscribe. On Android, however, this doesn't happen. Only messages sent (or resent) by another client while the android client is subscribed are ever delivered.
I'm new to MQTT, but it seems pretty clear that the "cleanSession" connection parameter is what controls this: unless you "clean" a session, you get everything that was published to your topic while you were not subscribed. On the Linux client side, passing "true" to mosquitto_new(..., clean_session, ...) does indeed prevent my Linux client from getting pre-connection messages. But on the Android side, calling .setCleanSession(boolean) has no effect when the MqttConnectOptions instance is passed to .connect().
I'm using 1.1.+ of paho. Per the tags in the repo at https://github.com/eclipse/paho.mqtt.android.git, v1.1.1 is the latest.
implementation "org.eclipse.paho:org.eclipse.paho.client.mqttv3:1.1.+"
implementation "org.eclipse.paho:org.eclipse.paho.android.service:1.1.+"
I suspect that this is simply a bug in the Android Paho library (which doesn't seem to have been worked on in four years.) But I hope I'm wrong! Is there a way to accomplish what I want?
Alternatively, is there a better library? The googling I've done suggests that in spite of its age Paho is still what most Android devs are using to speak MQTT.
Thanks!
About the cleanSession flag.
The Client and Server can store Session state to enable reliable messaging to continue across a sequence of Network Connections. This bit is used to control the lifetime of the Session state.
If CleanSession is set to 0, the Server MUST resume communications with the Client based on state from the current Session (as identified by the Client identifier). If there is no Session associated with the Client identifier the Server MUST create a new Session. The Client and Server MUST store the Session after the Client and Server are disconnected. After the disconnection of a Session that had CleanSession set to 0, the Server MUST store further QoS 1 and QoS 2 messages that match any subscriptions that the client had at the time of disconnection as part of the Session state. It MAY also store QoS 0 messages that meet the same criteria.
More about cleanSession on : https://docs.oasis-open.org/mqtt/mqtt/v3.1.1/csprd02/mqtt-v3.1.1-csprd02.html
If I understand correctly your requirement, then indeed you need to use clenSession=true. You can also try publishing and subscribing with QoS=0. Some brokers do not store QoS=0 messages, mosquitto as well. (as per https://mosquitto.org/man/mqtt-7.html)
I've found a workaround, and in the process confirmed my suspicion that the MqttAndroidClient class is broken in not honoring that setting. Instead of using MqttConnectOptions I tried using MqttAsyncClient. The code changes are trivial, though underneath there's considerable change as the Android client knows about and uses background Services. With the simple change of using this different class, I'm able to connect & subscribe and immediately receive all messages that were published while I was not connected.
I'm trying to create a websocket connection to my Phoenix app from an Android client. I'm trying to use this library but I'm running into this issue and I'm unable to successfully join a channel.
Upon reviewing the source code of the above java phoenix client library, it looks like the initial request from the client to connect to the socket is made with http schema and not ws (the source code explicitly changes the provided url to make sure it always uses http). It's not clear to me how this would work without additional configuration in my Phoenix app: if a socket connect request is made to http://localhost:4000/socket, the request will fail because there is no route for /socket when the schema is http.
There's nothing in the library docs that says any additional config is required in my Phoenix app to make this work, but I don't see how it could work for the reason stated above.
Does a Phoenix app have built in handling for the connection upgrade, etc, required on handshake as specified here?
As a note, I have no issues making websocket connections from my javascript web client to my Phoenix backend.
Any suggestions are appreciated!
Have you tried using the default path for a channel http://localhost:4000/socket/websocket ?
I am working on the Android client implementation that connects via websocket to a server. I've just enabled HTTPS on the server instance and now I am receiving this issue:
W/System.err: com.neovisionaries.ws.client.HostnameUnverifiedException: The certificate of the peer (CN=otherdomain.com) does not match the expected hostname (domain.com)
at com.neovisionaries.ws.client.SocketConnector.verifyHostname(SocketConnector.java:171)
at com.neovisionaries.ws.client.SocketConnector.doConnect(SocketConnector.java:126)
at com.neovisionaries.ws.client.SocketConnector.connect(SocketConnector.java:83)
at com.neovisionaries.ws.client.WebSocket.connect(WebSocket.java:2152)
at com.neovisionaries.ws.client.ConnectThread.runMain(ConnectThread.java:32)
at com.neovisionaries.ws.client.WebSocketThread.run(WebSocketThread.java:45)
It seems that the websocket client is finding mismatched server host names.
Some other related info:
otherdomain.com is a different one of our URLs. It is pointed to separate instance of the same web application running on the same server as domain.com which is the app I am currently working on.
My code is logging the URL that it is connecting to and I can see that it is correctly referencing domain.com:
connecting to: wss://domain.com?session_key=TheCorrectSessionKey
As noted otherdomain.com is another instance of the same web application. That instance already previously had HTTPS enabled and everything is working properly with that instance. The same client code I am using now is able to connect successfully to this instance.
The same web application has a front end page that also connects to the websocket via wss://. This is working properly in both Chrome and Firefox browsers for domain.com, as far as I can tell the only client having issues is the Android client, and it's only the domain.com variant of the app having issues, the otherdomain.com one seems to be working fine.
I found this issue opened on the library I am using for websockets. But I'm not 100% sure if my problem is related to it, but it may be. In particular someone states:
Not being able to connect to a WebSocket-enabled server domain unless it is also configured to be the default server on the website.
I assume only one site can be default, and perhaps otherdomain.com got set to default since it was completed first. This is all a bit over my head though to be honest, I'm not sure if this is relavent or not.
I'm writing a Lovefilm client for Android, and it's not going too badly except I keep having problems with the remote calls to retrieve data from the API.
Does anyone have any tips for debugging remote calls like this? Can I tcpdump on Android or is there a native way of doing it?
For example, I'm using the Scribe-java library for OAuth to access the Lovefilm API, I can authenticate find and retrieve a list of films on the users account fine when the device is running Gingerbread, but trying to retrieve the accessToken on Froyo causes a blank response & and apparent response code of -1, I'd like to be able to see what's going on under the cvers their.
Another example I'd like to be able to the raw http for is trying to run a search, I get and IOError that says "Received authentication challenge is null"
I've used Fiddler (http-proxy for debugging http calls) with the android emulator in these cases. Just start the proxy, and start the emulator with the correct proxy address (-http-proxy ).
Fiddler is the most useful option. On the emulator #Scythe answer will work, but on a real device you will need to set the proxy in the Apache Http Client. The following code will do that:
HttpHost proxy = new HttpHost("youripaddr", 8888);
params.setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
If you are using https, fiddler is not so useful. In that case can enable the build in logging support in Apache Http Client. The following code does that:
Headers only:
java.util.logging.Logger apacheHeaderLog = java.util.logging.Logger.getLogger("org.apache.http.headers");
apacheHeaderLog.setLevel(java.util.logging.Level.FINEST);
Headers & Wire:
java.util.logging.Logger apacheWireLog = java.util.logging.Logger.getLogger("org.apache.http.wire");
apacheWireLog.setLevel(java.util.logging.Level.FINEST);
Note that this will have to have a java.util.logging Handler configured at finest level and the default handler is configured to log to logcat, which will filter DEBUG (finest) entries by default.
If your system can share the wi-fi connection you should be able to route packets from any device through your system and then using wireshark you can get monitor your calls or get a tcpdump.
Also , and more importantly , it would be best if you log your network calls and responses as suggested by #Matthew
Windows 7 wi-fi connection sharing : http://www.winsupersite.com/article/faqtip/windows-7-tip-of-the-week-use-wireless-hosted-networking-to-share-an-internet-connection-wirelessly.aspx
Since I always run into similar troubles and it seems a lot of people having the same issues over and over again I wrote up a quick tutorial for debugging client-server communication by using netcat and cURL.
That of course only works for the simplified case that you always 'fake' on side of the connection.
For eavesdropping you can use tools like tcpdump or Wireshark. Which will definitely be easier if you're able to run the server instance directly on your local machine.
Stetho is a great tool from FB which helps in debugging android Apps. You can have access to local data and have a check on your network using this.
http://facebook.github.io/stetho/