I am trying to connect to a WebSocket server that my Android device connects to from an app. I captured the packets on my Android device, and the initial request headers look like this:
GET / HTTP/1.1
Pragma: no-cache
Cache-Control: no-cache
Host: example.com:80
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: ysWaBflPV9EmRaB1JpPMOQ==
Origin: http://example.com:80
Sec-WebSocket-Protocol: default-protocol
Sec-WebSocket-Extensions:
Sec-WebSocket-Version: 13
The response from the server looks like this:
HTTP/1.1 101 Switching Protocols
Date: Wed, 31 Jan 2018 02:37:20 GMT
Connection: upgrade
Set-Cookie: AWSALB=0yaRd5HOPlZSITp+bcXoZoIn/7YOOqE9o4/t/8b3kw2PTxooflm/85w+1JfudEE0Cwb1BUkWPV+t4kOnEm4FbLSWwMMFp8URbblZKj0a0kd0xB+glbLBHWxc/TPW; Expires=Wed, 07 Feb 2018 02:37:20 GMT; Path=/
Server: nginx/1.12.1
Upgrade: websocket
Sec-WebSocket-Accept: bj5wLF8vmyDrA7pqEgbHKbxqQSU=
Then, some communication begins, with lots of unrecognizable characters and some clear words in the messages. I don't have much experience with WebSockets, but I assume it is some form of compression.
I was able to send an identical request to this server using the ws module in Node.js, and I got an identical response to the one above. One notable difference was that when I set the protocol header to default-protocol, I received an error saying "Server sent no subprotocol". Without using this header, I still got the same response.
After the initial response, however, I did not receive any more messages, even though I did on my Android device. After about 30 seconds, the connection closes with code 1006 and no further information.
I tested the same request with curl and received the same headers back, but it also closed after about 30 seconds saying:
"Empty reply from server"
So my main question is obviously: What is going wrong, and how can I fix it?
More specifically, I am wondering if anyone with WebSocket experience knows if it is a problem with my client, or with the server itself.
It is possible that the server is authenticating me in some way on my Android device, but the headers that I captured are not revealing anything about that. Is it ever customary to authenticate a connection with a later message in the client-server communications? Is it possible that a separate HTTP request is authenticating me for this WebSocket server? All of these things seem unlikely to me since I found no other packets with anything related to auth requests. It seems much more likely that there is something wrong with the messages being sent.
Related
Background:
I am experiencing a very confusing behaviour with android Webviews in API 21 and up when testing in real devices.
I have a local HTML5 application (inside assets folder) with the following functionality
Login (2 steps authentication).
Show a list of items depending on the authentication.
The problem:
After doing the login requests, the server returns a cookie with the session. This cookie is not stored in the Webview when using real devices with API 21 or up. If I use emulators (Genymotion in this case), the cookies are properly stored.
More information:
The request to do the auth has the following headers:
POST http://myServer/j_spring_security_check HTTP/1.1
Proxy-Connection: keep-alive
Content-Length: 101
access-control-allow-origin: *
accept: application/json
access-control-allow-credentials: true
User-Agent: Framework/1.5.0 (Linux; U; Android 6.0.1; Nexus 5X Build/MMB29Q) App/0.1.1
Origin: file://
content-type: application/x-www-form-urlencoded
Accept-Language: en-US
X-Requested-With: app.package
Host: myServer
With the following response:
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Set-Cookie: JSESSIONID=4D169E8656DBEDFFA4D17FE8D436A5BA; Expires=Fri, 19-Feb-2016 14:27:55 GMT; Path=/; HttpOnly
Content-Type: application/json;charset=UTF-8
Content-Length: 43
Date: Fri, 19 Feb 2016 14:17:55 GMT
The cookie is not stored in devices with API 21 or more. Same request/response works fine in the rest of devices + all the emulators
Clarification:
This flags are enabled inside the app:
android.webkit.CookieManager.setAcceptFileSchemeCookies(true);
(Before CookieManager or webview is instantiated, as documentation says)
if(VERSION.SDK_INT >= 21) {
CookieManager.getInstance().setAcceptThirdPartyCookies(this.nativeWebView, true);
}
If after doing the authentication, I access the cookies datastore and
check the "hasCookies" method, I get false.
The two step auth service actually calls 3 different paths from the same endpoints. None of the cookies that the response that generate this services are stored. I don't know if this is relevant or not.
When doing simple authentication (to a different server), cookies are stored properly in all the devices emulators.
I am using Angular 1.5
I am aware that the service is using http instead of https. That will be solved in the future.
I get no error message in the consoles.
Questions:
Is there any internal security measure in the webviews that blocks the storage of the cookies? Why does it work on emulators (that are rooted devices) and not in real devices? This really bugs me.
If the network request is done using window.fetch you may need to add:
fetch('/something', { credentials: 'same-origin' }) // or 'include'
On chromium, window.fetch has the credentials flag set by default to 'omit' and no cookies are stored into the cookie storage. More details of this bug here: https://bugs.chromium.org/p/chromium/issues/detail?id=477523
I launched burp as an emulator's proxy for debugging of http requests from my application with intercepting option switched on and at the startup I found that emulator sends a GET request to google:
GET /generate_204 HTTP/1.1
User-Agent: Dalvik/1.6.0 (Linux; U; Android 4.3; sdk Build/JWR66V)
Host: 173.194.32.129
Connection: Keep-Alive
Accept-Encoding: gzip
And gets a response like:
HTTP/1.1 204 No Content
Content-Length: 0
Content-Type: text/html; charset=UTF-8
Date: Thu, 05 Sep 2013 06:56:51 GMT
Server: GFE/2.0
So I would like to know if there is some purpose for making this request to google at the startup?
It's most likely for counting things:
active developers
emulator use
framework use
generating statistics how developers are spread over the world
...
It's Android trying to tell if the Wifi (or other network connection) connection has internet. I'm testing on real devices and it does the same thing. If you don't forward the message the connection status in Android Wifi Setting will say "Connected. No internet" until you forward and it gets a success back.
When I make a request to web service over wifi everything is working well but the same when made on GPRS having WAP connection (I did not in anyway force the connection to be WAP, this might be carrier dependent. Testing done in Argentina) the server is receiving duplicate values in both Content-Type & Content-Length.
Below is the same request logged on server first over WiFi and second over GPRS.
Over WiFi:
POST /ODP/Services.asmx HTTP/1.1
User-Agent: kSOAP/2.0
SOAPAction: http://temphost.org/RetrieveConfiguration
Content-Type: text/xml
Connection: close
Content-Length: 464
Host: temp.host.com
Accept-Encoding: gzip
Over GPRS:
POST /ODP/Services.asmx HTTP/1.1
Accept-Encoding: deflate, gzip, identity
Content-Length: 464, 464
Content-Type: text/xml, text/xml
Host: temp.host.com
SOAPAction: http://temphost.org/RetrieveConfiguration
User-Agent: kSOAP/2.0
X-WAP-WTLSEncryptiontype: NONE
X-WAP-Bearerinfo: W-HTTPS=FALSE, bearertype=0
Via: W-HTTP/1.1 wgw-fe6 EMIG 5.1
x-msisdn: <User Phone number>
x-up-calling-line-id: <User Phone number>
x-technology-stack: Unknown
TE: trailers
Connection: TE
I am not able to understand how and where the multiple values are being added in the request for the Content-Type & Content-Length headers.
Can somebody enlighten me what is wrong with the WAP connection or is it being added at the carrier end while modifying the request to WAP.
Sadly there's nothing to do on the client side (customer/phone/browser), but there are some possible scenarios to get an approach to a solution.
If you're web developer (VASP side), you need to take care about the URL/URI size in order to prevent that POST needs more than one packet.
If you're the web server manager (VASP/Telco side), you can define that multiple values are going to be accepted for these specific headers (when equal).in this scenario you need to be aware that you gonna put your server under risk of HTTP response splitting attack.
If you're the proxy admin (Telco side), you can tune your GW to discard the extra header instead of merging them into one header containing multiple values but, in this scenario you need to be aware that you gonna be out of RFC recomendations.
If because of its extension a POST is truncated and requires more than one packet, content-length and content-type fields are duplicated.
WGW resolves this situation by merging them into one header containing multiple values comma separated, the problem is that this header containing multiple values generates HTTP 411 error code.
Based in latest RFC drafts, this traffic must be discarded with error code 502 but some workaround intended to remove one of the header instead of merging them in order to keep this transactions in progress is possible.
I'm trying to get Yahoo messenger notifications (comet style) to work on my Android app but I got a problem:
I have all the oauth tokens + IM cookie, I make the call to notify server but it responds with
403 Bad IM cookie or URI credential
The IM cookie which is sent is correct because it is the same as the one I got from POST /session.
The call:
POST to:http://rproxy2.messenger.yahooapis.com/v1/pushchannel/user_id?sid=T3qRkifs3BiGXCW4LOSoThi3lG2HvWNagr.mGA--&rand=-1131022177&count=10&seq=1&format=json&IM=2x5A0IOr9Js_8er8BgSYUCB9fjB.d20FbKds7NdJdkzWK4u2A92k5dMrM72OetLz9.q70NQCwZpAqafQ5.XM-|QTvOT1rY3ukBC8o4Z0zeRg--&oauth_consumer_key=dj0yJmk9cFNKb2NKbTdINEtFJmQ9WVdrOVpHMW1aVXhKTjJNbWNHbzlNamcyTnpNMU1UWXkmcz1jb25zdW1lcnNlY3JldCZ4PWJh&oauth_nonce=939bc51c9a594c7cbf903643bc10f7d1&oauth_signature_method=PLAINTEXT&oauth_timestamp=1328108171163&oauth_token=A%3DosG_yqL.hxped9sV4ZMVD2A5vugclvEOSKIw85DOg17vQa8N1jQK6UcGqeLGLXKFalrxnYA_iA_ArmhbN6oJ_CEINGwiq7MWsVjO_vQT5bwhmcBMwBLByhusduuH2BFJw1Zd2w_uDj5fEUVMK9bMDzg.fYEpdm1uK.OgwyYLPCwdkzW0w6ujPpaFAea7gwmsSBYHECzjXm8wduoe1Sw3rpJH4z9PGIe8Hkt3wrbyJpW9BuiU4g6OyE1f8nhzHVkcG_70KQiIR_oV3tP8_l4SCsbbHMvSzlyZjK1V6gJp0Q94fAcfuIWQ1Vd.iD.UZPEyzdzYK.k5Q6pE4H3At8czcXFOpmOL5eUwfbeQEtXePCL1b2v9oY4BxtH_dCUVTMhzQsV.YpKZV.ZRZzK8ThveL5gfGuqYcBZLHwVilQnQWo477XwT10GbvWUzJ2EH4v5m9q.sGMViSIKWjbE2O6YfwhYXAgQMA3d8wQLL7IoY4QC0.IR4T8M_KnZVlKoi_EiED8VuK9h7h3emBZeXoM15rqGk2..HUJQEuIe1d0YbTPEGKdM.fbCcT7_Y0Szzw3abf9b1Xjzm8VkNSKL.LXAmy8c9PyNatUT_hKFmzmWMd1RRYfd8rF7i6i7Ec_cVv06QhwCQyKlEsCQh7UChUe9JykUmiA6JOm4JOpsbTQWS1Iec35zrltc4YwouUlMx.y6JbzB_VYL9jkzeuu&oauth_version=1.0&oauth_signature=2cd720831677df11508571218aed6c36c9cf2b51%26e57b0e111092b000a43b4ae9c928178fd5da4b9f
connection: keep-alive
cookie: IM=2x5A0IOr9Js_8er8BgSYUCB9fjB.d20FbKds7NdJdkzWK4u2A92k5dMrM72OetLz9.q70NQCwZpAqafQ5.XM-|QTvOT1rY3ukBC8o4Z0zeRg--
User-Agent: Dalvik/1.2.0 (Linux; U; Android 2.2; sdk Build/FRF91)
Host: rproxy2.messenger.yahooapis.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 0
The server responds with:
HTTP/1.1 403 bad IM cookie or URI credentials
Connection: close
I followed all the instrunctions from docs, not sure how can make it to work.
Thanks.
Your HTTP POST is empty (Content-Length: 0), the docs state:
Note: even though all inputs are optional, the server
still expects an empty POST body (JSON: "{}")
so, change your Content-Type to "application/json;charset=utf-8" and include
{}
as the body of the HTTP POST.
Long lived connections need to be GET calls; see "Notification Management (Comet-Style Push) API", here.
I am creating a Webapplication - Food Review site for which we are also developing an android application.
I want to know how to couple the application session of a particular user over the Internet with the application.
When the user logs in thru the application how can i serve them content related to the user alone. How can i know which user is requesting.
I am from a PHP background so i am a little confused how its done in the mobile application
You can maintain state in the same way a web application does. When a browser stores a cookie all that means is that requests sent to a matching domain get the cookie in the HTTP header. When you construct your HTTP request in Android you have complete control of the headers so it's easy to add your cookie value.
E.G. Post to your authenticator, sending client details. The authenticator would normally respond including the cookie in its response headers, here's a sanitised version of google's cookie setting response headers:
Cache-Control:private, max-age=0
Content-Encoding:gzip
Content-Length:173
Content-Type:text/html; charset=UTF-8
Date:Fri, 04 Mar 2011 12:24:32 GMT
Expires:Fri, 04 Mar 2011 12:24:32 GMT
Location:http://www.google.co.uk/
Server:GSE
Set-Cookie:SID=UNIQUESIDGOESHERE;Domain=.google.co.uk;Path=/;Expires=Mon, 01-Mar-2021 12:24:32 GMT
HSID=SOMETHINGELSEUNIQUEHERE;Domain=.google.co.uk;Path=/;Expires=Mon, 01-Mar-2021 12:24:32 GMT;HttpOnly
You can read that value and attach it to any http requests you make and the server would be non the wiser that you aren't a normal web browser.
Android is pretty much stateless. (I mean even if it can use states, I haven't come across any such example so far)
So I don't think you can have a session as you have on the web.
You can have a token method to authenticate users for a particular time period. like the OAuth implemented web services do.
Your android app sends the token for each request and your server manages the generation and life of that token along with the permissions.
How does this sound?