I'm debugging an API on my home development environment with the IP range 192.168.0.0-255.
I'm in Australia (relevant for reasons below), the API's written in PHP, and I'm running Apache.
My server IP = 192.168.0.20
My router is configured to forward HTTP requests to this IP.
My router's internal IP is 192.168.0.1 and external IP is, for this example, 123.123.123.123
For the following cases:
Laptop, all browsers, over wi-fi
Android tablet, all browsers, over wifi
Android phone, using Internet browser, over wifi
the server detects $_SERVER['REMOTE_ADDR'] = 123.123.123.123.
The latency is effectively zero and no caching occurs. So far, all as expected.
However, when I submit the same query from an Android mobile phone (assigned the IP of 192.168.0.10) in Chrome over the same wi-fi connection, the server detects:
$_SERVER['HTTP_X_FORWARDED_FOR']: 123.123.123.123
$_SERVER['HTTP_FORWARDED']: 123.123.123.123
$_SERVER['REMOTE_ADDR'] : 66.249.84.217`
The 'REMOTE_ADDR' also takes on the values 66.249.84.223 and 66.249.84.229
There is a delay of ~400ms and my API is being cached - hence not returning correct/latest values.
Relevant parts of HTTP header:
Forwarded: for=123.123.123.123
Scheme: http
Via: 1.1 Chrome-Compression-Proxy
I see 66.249.84.0-255 belongs to Google, and understand why compression may often be useful in mobile context. But in my case I'd rather avoid the extra latency of a round-world-trip.
When I run the same queries over HTTPS, no re-routing through Googles's servers occurs.
Is there any way to avoid Chrome mobile re-routing my packets?
Your request is, presumably, being routed through Google's Data Compression Proxy.
Since Google does not try to hijack your SSL certificates, no such rerouting is being done for SSL connections.
On the client side, a user can simply disable this in settings with Settings > Bandwidth Management > Reduce data usage.
On the server side, it's too late to "undo" the routing, though you can indicate with a Cache-Control: no-transform header that you don't want your response transcoded.
From the Google Developer Docs:
As a site owner, how do I opt-out from content optimization?
Data compression proxy respects the standard Cache-Control: no-transform directive. Site owners can mark individual resources with this directive and the proxy will pass them through directly to the mobile browser.
https://developer.chrome.com/multidevice/data-compression
Yes, same answer for me. I have a network camera in New Zealand I access from Canada. I noticed a bunch of suspicious access in the Access Log in the range 66-xx-xx-xx, and believing the camera had been hacked I immediately added an access denial for that range (at the time I knew the range was owned by google ... but as far as I was concerned at the time it could have been a cloud machine of unknown purpose.)
Then noticing I could no longer access the netcam on chrome on my mobile (but could on the default browser, and any windows browser) It became clear that there was some kind of proxy in the middle. Further research led me here.
Here's some more info -
https://developer.chrome.com/multidevice/data-compression
Turning off Reduce Data Usage in chrome settings immediately restored access to my netcam
I have to wonder what's in it for google ? what valuable info are the mining from my traffic as it passes through their proxy?
Related
I'm new to web-crawlers, trying to crawl ridership data of metro from the cellphone maps app(www.amap.com) with Fiddler, but I got this HTTP connect method, which is not viewable. There are icons of locks next to the URL and in 'Response' it says this:
'Encrypted HTTPS traffic flows through this CONNECT tunnel. HTTPS Decryption is enabled in Fiddler, so decrypted sessions running in this tunnel will be shown in the Web Sessions list.'
I found a solution suggesting that customizing rules in fiddler may help, so I followed and added this to its script:
if (oSession.oRequest[‘User- Agent’].IndexOf("Android") > -1 && oSession.HTTPMethodIs("connect")) {
oSession.oResponse.headers["Connection"]="Keep-Alive";
}
The changes to Fiddler Script
But of course, it didn't work, I've tried both iPhone and android and changed the header in the script respectively, none of them helped.
So is this app and HTTP connect method crawlable? The data is constructively helpful to my research, instead, it is not provided in website 'amap', so it has to be done through a cellphone.
If you have HTTPS decryption enabled in Fiddler but you see (mostly) only CONNECT requests this means that the apps on the device try to open a connection but do not trust the Fiddler root certificate.
If you try to use the apps on-device you will notice that there is currently no working network connection available (requests just don't work as the apps don't accept the server certificate created by Fiddler).
On Android devices since Andorid 6 you need root permissions to instal the Fiddler rot certificate or alternatively if you want to monitor a single app you can try to modify and re-sign the app. All details are described in this question and answer:
Some androids apps won't connect through fiddler
I have a Rails application running in my laptop and an android application on phone. I want to call apis of rails application where routes are namespace with sub-domain.
My api look like below:
http://api.localhost:3000/login (With sub-domain)
When I replace localhost with ip address. It will work in browser but in android application it gives me Android java.net.SocketTimeoutException: failed to connect. But when I namespace without sub-domain it works.
http://192.168.1.101:3000/api/login (Works fine)
What is the correct URI with sub-domain ?
http://api.192.168.101:3000/login (Gives above error)
The easiest way to solve the problem is if you have a domain registered. You can add a DNS entry for hostname api.192.168.101.
If not, you have few options.
Setup your wireless router to resolve the hostname api.192.168.101 to 192.168.1.101. I am assuming, your phone's DNS server is your wireless router. If your wireless router does not support adding local DNS entries checkout the next option.
You have to add the entries to "/etc/hosts" file of your Android phone. It is easy for rooted phones
I suggest you consider buying a domain from a good vendor who provides you a decent DNS management console.
So this is a two part question:
Part a: I'm trying to implement a secure connection to the KMS. From the documentation, I've understood that KMS Configuration file would need to be updated with the SSL certificate and then the HTTPS connection from the client can be made. Please let me know if there are any other steps that are involved in achieving SSL security.
Part b: From a better understanding now and from comments from a previous question I posted, Kurento Utils does not connect to KMS directly (this was an fyi and a clarification I received and I wanted documented here just in case). Now I'm trying to use Kurento Client to connect to KMS and I'm trying to understand the role of ICE/TURN/STUN servers acting as negotiators in the middle. If I were to specify my own server URL, I'm assuming that I would not need to include "freeice" and "normalice" and instead specify my own server's URL. In the code snippet below taken from the tutorial on github, I'm assuming that I would need to replace the argument for ice_servers to point to the url where my server is running? Or since this is the client, do I really need an ICE server because as said from the first statement, the utils don't connect to the KMS but the client can, right? So if I were to specify the Kurento URL for "ws_uri" parameter, then I won't need to even use ICE servers...right? I don't really understand the concept of ICE/TURN servers very well in terms of how they integrate with Kurento and hence, I would like to understand in English as to what changes would I need to make in order to get this to work. I will bang my head to write the code myself! Thanks much in advance!
`
var args = getopts(location.search,
{
default:
{
ws_uri: 'ws://' + location.hostname + ':8888/kurento',
file_uri: 'file:///tmp/recorder_demo.webm', //file to be stored in media server
ice_servers: undefined
}
});`
Answer A
Only this and nothing more... at least for KMS. On the client side, you'll need to specify the WSS port and so on.
Answer B
Your client might need a STUN/TURN server, and that's independent of where KMS is located. STUN and TURN are used in the candidate harvest process, to discover the network topology of your peer. You have two peers: KMS and your Android app, and both need to have, in their SDPs and during the negotiation, a candidate that is reachable by them (app will connect with KMS and viceversa) If both peers are on the same network, you can go without using STUN/TURN. The moment you have a NAT in between, you need at least STUN for that peer to be able to harvest candidates that have the public IP on the other side of the NAT, which is not known by the peer unless STUN is used.
TURN is used as a relay server, and it is needed in a small set of cases. If you are almost certain you are going to use TURN, you need to have that in a machine different than KMS (it makes close to no sense to have both the relay server and the media server installed together)
So the answer is yes, you are most likely going to need STUN/TURN in your clients.
I've been creating an app to learn how Android works and just playing around with various features like the sensors, SMS listeners, phone listeners, wifi listeners, etc.
I recently added a bit of code from the Android Volley library to send a request to my website every time it connects to a wifi network (I don't have a cellular data plan).
It sends a GET request to a very sparse text file that returns back:
hello
The odd thing is that after connecting to open public wifi networks, oftentimes the response that the phone gets from this same request will be:
<html>
<title>Redirecting...</title>
<script language="javascript">
document.location.href="http://den-80202-7200.localdomain:8000/index.php?zone=pms&redurl=http://my-personal-server.com/hello
</script>
</html>
Which is really strange to me, as it will return this even the next day, connecting to other networks, connecting to my secured network at home, etc.
And I have no idea what this URL is:
http://den-80202-7200.localdomain:8000
It appears to me, (still learning about how this all works), that my phone is sending my GET request to my website, but somehow my website is returning back this possibly infected response, which, if I was in a browser may harm me. Maybe my website server is infected with something?
Or, my phone has malware that is sending all my requests through this unknown server?
Is that what is going on? How to prevent my phone from keep sending requests through this unknown server? I did a virus scan and it comes up clean. Is there a way to flush the phone's DNS cache or something similar?
While in general it's good to be suspicious about hijacked or modified requests on open wifi networks, in this case it appears to be expected behavior for public wifi networks.
Specifically, many public wifi networks require authentication either for paid access or to accept terms of service. To accomplish this, they will often intercept HTTP requests from unauthenticated clients and return a response which redirects them to a captive portal.
Since .localdomain is not a valid TLD, this URL will not work outside a network without a local DNS entry for it. This appears similar to the use of .local as a reserved TLD for local-network only DNS entries. Again, this is indicative of public wifi networks using .localdomain URLs to redirect to a locally hosted captive portal.
Note also that the redirect URL has the parameter "redurl=http://my-personal-server.com/hello", this specifies the URL that you'll be redirected to after your authenticate. As expected, this was the original URL you requested.
NOTE: As for this showing up on your private wifi networks afterwards, I suspect this is a caching issue. Again, since .localdomain isn't a valid TLD the request to "http://den-80202-7200.localdomain:8000" will fail outside of the public wifi network with support for this specific DNS entry.
Maybe your system is infected with Malware or SpyBots.
Check out below link:
http://www.speedguide.net/port.php?port=8000
We have discovered a strange bug in on of our Android apps in that the app cannot connect to the internet over cellular on certain Android Models/Mobile Networks (connecting via WiFi works perfectly). The URL which the app is trying is to reach is however accessible using the device's normal browser, which rules out the URL being blocked upstream. Any ideas what could be causing this would be appreciated please.
which rules out the URL being blocked upstream
Not necessarily. As one of the commenters pointed out, you do not indicate how you are using this URL in your app. If this is an HTTP request, and it works in a browser but not your app, try changing the User-Agent HTTP header of your request to match the one from the browser.
You have provided no error log, but from experience I have run into a few inconsistencies when connecting to a server.
Using HttpsUrlConnection a SNI header is set on HoneyComb and above but not on previous versions, which can alter how the server responds. To add to this there are some general SSL handling inconsistencies between API levels, such as handling of wildcard domains in a certificate is buggy in some api levels.
Second some phones/api levels add a header to HttpUrlConnection requests that specifies time the request was sent like so:
X-Android-Sent-Millis=1353085024868
Some servers seem to use these headers to detect mobile traffic and alter the response.
I had problems when using a dual-simcard. Do you have more devices connected? Try to shut them down.