We have discovered a strange bug in on of our Android apps in that the app cannot connect to the internet over cellular on certain Android Models/Mobile Networks (connecting via WiFi works perfectly). The URL which the app is trying is to reach is however accessible using the device's normal browser, which rules out the URL being blocked upstream. Any ideas what could be causing this would be appreciated please.
which rules out the URL being blocked upstream
Not necessarily. As one of the commenters pointed out, you do not indicate how you are using this URL in your app. If this is an HTTP request, and it works in a browser but not your app, try changing the User-Agent HTTP header of your request to match the one from the browser.
You have provided no error log, but from experience I have run into a few inconsistencies when connecting to a server.
Using HttpsUrlConnection a SNI header is set on HoneyComb and above but not on previous versions, which can alter how the server responds. To add to this there are some general SSL handling inconsistencies between API levels, such as handling of wildcard domains in a certificate is buggy in some api levels.
Second some phones/api levels add a header to HttpUrlConnection requests that specifies time the request was sent like so:
X-Android-Sent-Millis=1353085024868
Some servers seem to use these headers to detect mobile traffic and alter the response.
I had problems when using a dual-simcard. Do you have more devices connected? Try to shut them down.
Related
I'm developing an Android application, it is like a browser, has a web-view and sends http(s) requests. For debugging purpose, I want to be able to see the requests sent and received by the application (in particular the headers), but I'm unable to do so.
What i tried
I mainly debug the app with Chrome on PC, and use the Network Inspector in Chrome. The problem is that my app uses the ShouldInterceptRequest to intercept the requests an then manually sends a request with cronet. Chrome, in this case, shows some "provisional headers" that are from the original request, an not the headers of the actual request i sent manually.
I tried to use Fiddler and HTTP Toolkit, but the server I'm communicating with, doesn't like the certificate they use, so they can monitor correctly, but, if active, i cannot reach the page i need to monitor.
I also tried the Android Studio network inspector, but seems it work only for HttpURLConnection and Okhttp(1)
Thanks for your time.
If you're using a WebView, set a custom WebViewClient and override shouldInterceptRequest. That will pass in a WebResourceRequest object that will include all headers.
I am working on the Android client implementation that connects via websocket to a server. I've just enabled HTTPS on the server instance and now I am receiving this issue:
W/System.err: com.neovisionaries.ws.client.HostnameUnverifiedException: The certificate of the peer (CN=otherdomain.com) does not match the expected hostname (domain.com)
at com.neovisionaries.ws.client.SocketConnector.verifyHostname(SocketConnector.java:171)
at com.neovisionaries.ws.client.SocketConnector.doConnect(SocketConnector.java:126)
at com.neovisionaries.ws.client.SocketConnector.connect(SocketConnector.java:83)
at com.neovisionaries.ws.client.WebSocket.connect(WebSocket.java:2152)
at com.neovisionaries.ws.client.ConnectThread.runMain(ConnectThread.java:32)
at com.neovisionaries.ws.client.WebSocketThread.run(WebSocketThread.java:45)
It seems that the websocket client is finding mismatched server host names.
Some other related info:
otherdomain.com is a different one of our URLs. It is pointed to separate instance of the same web application running on the same server as domain.com which is the app I am currently working on.
My code is logging the URL that it is connecting to and I can see that it is correctly referencing domain.com:
connecting to: wss://domain.com?session_key=TheCorrectSessionKey
As noted otherdomain.com is another instance of the same web application. That instance already previously had HTTPS enabled and everything is working properly with that instance. The same client code I am using now is able to connect successfully to this instance.
The same web application has a front end page that also connects to the websocket via wss://. This is working properly in both Chrome and Firefox browsers for domain.com, as far as I can tell the only client having issues is the Android client, and it's only the domain.com variant of the app having issues, the otherdomain.com one seems to be working fine.
I found this issue opened on the library I am using for websockets. But I'm not 100% sure if my problem is related to it, but it may be. In particular someone states:
Not being able to connect to a WebSocket-enabled server domain unless it is also configured to be the default server on the website.
I assume only one site can be default, and perhaps otherdomain.com got set to default since it was completed first. This is all a bit over my head though to be honest, I'm not sure if this is relavent or not.
I'm working on a multilayer Android app where I have to intercept HTTP requests, modify one of the headers, execute and return responses. I can't change a way of creating requests, but I can change the API base URL used for requests. The approach I chose was writing a proxy listening on a specific port and doing a man-in-the-middle job:
taking requests,
applying required changes to HTTP requests,
executing it and
returning response.
I applied SparkJava as an HTTP server and OkHttpClient as an HTTP client. It worked like a charm until I found it works only on Android 8 and newer. There is a problem with missing ThreadLocal<S> ThreadLocal.withInitial (Supplier<? extends S> supplier) in API prior to 26 that SparkJava's backend - Jetty, uses).
I guess I will try to look for other HTTP server framework working fine in older API and reimplement my proxy.
In the meantime, does this approach seem to be ok? Or maybe there is another way of proxying HTTP requests in Android app that would allow modifying headers having only an option of changing base API URL?
All ideas much appreciated.
PS. I know that in a typical app this kind of proxying would make a security issue, but this is not a concern in this case.
I'm debugging an API on my home development environment with the IP range 192.168.0.0-255.
I'm in Australia (relevant for reasons below), the API's written in PHP, and I'm running Apache.
My server IP = 192.168.0.20
My router is configured to forward HTTP requests to this IP.
My router's internal IP is 192.168.0.1 and external IP is, for this example, 123.123.123.123
For the following cases:
Laptop, all browsers, over wi-fi
Android tablet, all browsers, over wifi
Android phone, using Internet browser, over wifi
the server detects $_SERVER['REMOTE_ADDR'] = 123.123.123.123.
The latency is effectively zero and no caching occurs. So far, all as expected.
However, when I submit the same query from an Android mobile phone (assigned the IP of 192.168.0.10) in Chrome over the same wi-fi connection, the server detects:
$_SERVER['HTTP_X_FORWARDED_FOR']: 123.123.123.123
$_SERVER['HTTP_FORWARDED']: 123.123.123.123
$_SERVER['REMOTE_ADDR'] : 66.249.84.217`
The 'REMOTE_ADDR' also takes on the values 66.249.84.223 and 66.249.84.229
There is a delay of ~400ms and my API is being cached - hence not returning correct/latest values.
Relevant parts of HTTP header:
Forwarded: for=123.123.123.123
Scheme: http
Via: 1.1 Chrome-Compression-Proxy
I see 66.249.84.0-255 belongs to Google, and understand why compression may often be useful in mobile context. But in my case I'd rather avoid the extra latency of a round-world-trip.
When I run the same queries over HTTPS, no re-routing through Googles's servers occurs.
Is there any way to avoid Chrome mobile re-routing my packets?
Your request is, presumably, being routed through Google's Data Compression Proxy.
Since Google does not try to hijack your SSL certificates, no such rerouting is being done for SSL connections.
On the client side, a user can simply disable this in settings with Settings > Bandwidth Management > Reduce data usage.
On the server side, it's too late to "undo" the routing, though you can indicate with a Cache-Control: no-transform header that you don't want your response transcoded.
From the Google Developer Docs:
As a site owner, how do I opt-out from content optimization?
Data compression proxy respects the standard Cache-Control: no-transform directive. Site owners can mark individual resources with this directive and the proxy will pass them through directly to the mobile browser.
https://developer.chrome.com/multidevice/data-compression
Yes, same answer for me. I have a network camera in New Zealand I access from Canada. I noticed a bunch of suspicious access in the Access Log in the range 66-xx-xx-xx, and believing the camera had been hacked I immediately added an access denial for that range (at the time I knew the range was owned by google ... but as far as I was concerned at the time it could have been a cloud machine of unknown purpose.)
Then noticing I could no longer access the netcam on chrome on my mobile (but could on the default browser, and any windows browser) It became clear that there was some kind of proxy in the middle. Further research led me here.
Here's some more info -
https://developer.chrome.com/multidevice/data-compression
Turning off Reduce Data Usage in chrome settings immediately restored access to my netcam
I have to wonder what's in it for google ? what valuable info are the mining from my traffic as it passes through their proxy?
I'm writing a Lovefilm client for Android, and it's not going too badly except I keep having problems with the remote calls to retrieve data from the API.
Does anyone have any tips for debugging remote calls like this? Can I tcpdump on Android or is there a native way of doing it?
For example, I'm using the Scribe-java library for OAuth to access the Lovefilm API, I can authenticate find and retrieve a list of films on the users account fine when the device is running Gingerbread, but trying to retrieve the accessToken on Froyo causes a blank response & and apparent response code of -1, I'd like to be able to see what's going on under the cvers their.
Another example I'd like to be able to the raw http for is trying to run a search, I get and IOError that says "Received authentication challenge is null"
I've used Fiddler (http-proxy for debugging http calls) with the android emulator in these cases. Just start the proxy, and start the emulator with the correct proxy address (-http-proxy ).
Fiddler is the most useful option. On the emulator #Scythe answer will work, but on a real device you will need to set the proxy in the Apache Http Client. The following code will do that:
HttpHost proxy = new HttpHost("youripaddr", 8888);
params.setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
If you are using https, fiddler is not so useful. In that case can enable the build in logging support in Apache Http Client. The following code does that:
Headers only:
java.util.logging.Logger apacheHeaderLog = java.util.logging.Logger.getLogger("org.apache.http.headers");
apacheHeaderLog.setLevel(java.util.logging.Level.FINEST);
Headers & Wire:
java.util.logging.Logger apacheWireLog = java.util.logging.Logger.getLogger("org.apache.http.wire");
apacheWireLog.setLevel(java.util.logging.Level.FINEST);
Note that this will have to have a java.util.logging Handler configured at finest level and the default handler is configured to log to logcat, which will filter DEBUG (finest) entries by default.
If your system can share the wi-fi connection you should be able to route packets from any device through your system and then using wireshark you can get monitor your calls or get a tcpdump.
Also , and more importantly , it would be best if you log your network calls and responses as suggested by #Matthew
Windows 7 wi-fi connection sharing : http://www.winsupersite.com/article/faqtip/windows-7-tip-of-the-week-use-wireless-hosted-networking-to-share-an-internet-connection-wirelessly.aspx
Since I always run into similar troubles and it seems a lot of people having the same issues over and over again I wrote up a quick tutorial for debugging client-server communication by using netcat and cURL.
That of course only works for the simplified case that you always 'fake' on side of the connection.
For eavesdropping you can use tools like tcpdump or Wireshark. Which will definitely be easier if you're able to run the server instance directly on your local machine.
Stetho is a great tool from FB which helps in debugging android Apps. You can have access to local data and have a check on your network using this.
http://facebook.github.io/stetho/