I am Android developer, my application uses a bunch of http REST calls and it gets responses from servers. I use Charles to inspect the data (so I have to install Charles certificate onto my device so that https traffic can be read by myself).
Are there any tools out there like Charles that will allow me to modfiy that response packet before sending to the client ??
Depending on what exactly you want to modify, Charles' inbuilt Rewrite Tool might be what you are looking for. You find it in the Tools menu. You can specify which requests shall be modified by protocol, host, port, path and query, and you have the following modification options:
Add, modify, remove headers (request and response)
Modify Host, Path, URL, Response Status
Add, modify, remove Query Parameters
Modify body (request and response)
Another option is Fiddler. Like Charles it can be configured as a proxy for android, decrypt HTTPS traffic and modify request and response.
Charles itself has the functionality. Follow the steps:
enable break points by right click on individual request or a path
Before sending the request, Charles will give you a change to edit it. See below. Click "edit request" to fill in whatever you want and click "execute" to send the request.
Before posting the result back to your mobile phone, you have a change to edit the content. See below.
You may try OWASP ZAP or Burp Suite. OWASP ZAP is completely free and provides a number of features.
See also Android : Capturing HTTP Requests with non-rooted android device.
I've recently tested HTTP Toolkit on Android emulator. It works and allows to capture and edit response from a server. Some functions are paid (in Pro version). Requires root priviledges on real devices.
You can also use Burp Suite or Fiddler.
Related
I'm new to web-crawlers, trying to crawl ridership data of metro from the cellphone maps app(www.amap.com) with Fiddler, but I got this HTTP connect method, which is not viewable. There are icons of locks next to the URL and in 'Response' it says this:
'Encrypted HTTPS traffic flows through this CONNECT tunnel. HTTPS Decryption is enabled in Fiddler, so decrypted sessions running in this tunnel will be shown in the Web Sessions list.'
I found a solution suggesting that customizing rules in fiddler may help, so I followed and added this to its script:
if (oSession.oRequest[‘User- Agent’].IndexOf("Android") > -1 && oSession.HTTPMethodIs("connect")) {
oSession.oResponse.headers["Connection"]="Keep-Alive";
}
The changes to Fiddler Script
But of course, it didn't work, I've tried both iPhone and android and changed the header in the script respectively, none of them helped.
So is this app and HTTP connect method crawlable? The data is constructively helpful to my research, instead, it is not provided in website 'amap', so it has to be done through a cellphone.
If you have HTTPS decryption enabled in Fiddler but you see (mostly) only CONNECT requests this means that the apps on the device try to open a connection but do not trust the Fiddler root certificate.
If you try to use the apps on-device you will notice that there is currently no working network connection available (requests just don't work as the apps don't accept the server certificate created by Fiddler).
On Android devices since Andorid 6 you need root permissions to instal the Fiddler rot certificate or alternatively if you want to monitor a single app you can try to modify and re-sign the app. All details are described in this question and answer:
Some androids apps won't connect through fiddler
I'm using Charles proxy to fetch all the requests coming from my Android app to a webservice.
The thing is Charles shows me the complete request, meaning I can see the whole URL, headers and body so I can see www.example.com/rest/resource/param1/param2,
the JSON I send with it and also the authentication header.
After reading several posts like this and this one I thought the good part of working with the TLS was that one could only get the domain name from the URL, in this case www.example.com
To make sure it's not the client's fault, I requested the webservice resource with Retrofit and HttpsURLConnection and I could see the whole request both times.
I guess also the certificate is properly installed because it is shown in the browser every time an https request is made. Am I missing something else here or is this the normal behaviour?
So far I couldn't find a reason for this to happen so any help will be appreciated.
To debug with Charles proxy you must install a certificate on your browser (client).
With https the URL is encrypted.
But because you choose to use that proxy, your browser establish a secure connection to that proxy, and the proxy to the website. So, only 1) you, 2) the proxy 3)the website can decrypt the https traffic.
By installing a CA certificate on your browser, you allow the person detaining the corresponding private key (in your case, your proxy) to impersonate (so, decrypt with a MITM) any website.
Is there a way to make a faked http post request from desktop, but looks like from a real iOS/Android hardware(i.e. iPhone 5s 6.7 square inches, Samsung Galaxy 5.1-inch)?
I have no clue what’s the difference between the requests from deferent device(header,body...), how can a server know where the request from?
Thanks a lot. Merry Christmas.
The only difference is User-Agent.
You should easily be able to change it in your request, whatever the request is.
When your browser makes a request, among different headers there is a "User-Agent" header containing details of your browser and your OS. Take a look here and here.
I would use a REST Client like Postman for Chrome to send a POST Request along with the "User-Agent" string of my choice.
You can also use Chrome Developer Tools (opens with ctrl+shift+i in Windows) for the emulation feature that lets you emulate among a variety of devices. More info about that here.
With the emulation method in Chrome you can easily hit your server/webpage like you would normally by typing the URL in your browser (GET requests) from any of the available devices.
We have discovered a strange bug in on of our Android apps in that the app cannot connect to the internet over cellular on certain Android Models/Mobile Networks (connecting via WiFi works perfectly). The URL which the app is trying is to reach is however accessible using the device's normal browser, which rules out the URL being blocked upstream. Any ideas what could be causing this would be appreciated please.
which rules out the URL being blocked upstream
Not necessarily. As one of the commenters pointed out, you do not indicate how you are using this URL in your app. If this is an HTTP request, and it works in a browser but not your app, try changing the User-Agent HTTP header of your request to match the one from the browser.
You have provided no error log, but from experience I have run into a few inconsistencies when connecting to a server.
Using HttpsUrlConnection a SNI header is set on HoneyComb and above but not on previous versions, which can alter how the server responds. To add to this there are some general SSL handling inconsistencies between API levels, such as handling of wildcard domains in a certificate is buggy in some api levels.
Second some phones/api levels add a header to HttpUrlConnection requests that specifies time the request was sent like so:
X-Android-Sent-Millis=1353085024868
Some servers seem to use these headers to detect mobile traffic and alter the response.
I had problems when using a dual-simcard. Do you have more devices connected? Try to shut them down.
I'm writing a Lovefilm client for Android, and it's not going too badly except I keep having problems with the remote calls to retrieve data from the API.
Does anyone have any tips for debugging remote calls like this? Can I tcpdump on Android or is there a native way of doing it?
For example, I'm using the Scribe-java library for OAuth to access the Lovefilm API, I can authenticate find and retrieve a list of films on the users account fine when the device is running Gingerbread, but trying to retrieve the accessToken on Froyo causes a blank response & and apparent response code of -1, I'd like to be able to see what's going on under the cvers their.
Another example I'd like to be able to the raw http for is trying to run a search, I get and IOError that says "Received authentication challenge is null"
I've used Fiddler (http-proxy for debugging http calls) with the android emulator in these cases. Just start the proxy, and start the emulator with the correct proxy address (-http-proxy ).
Fiddler is the most useful option. On the emulator #Scythe answer will work, but on a real device you will need to set the proxy in the Apache Http Client. The following code will do that:
HttpHost proxy = new HttpHost("youripaddr", 8888);
params.setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
If you are using https, fiddler is not so useful. In that case can enable the build in logging support in Apache Http Client. The following code does that:
Headers only:
java.util.logging.Logger apacheHeaderLog = java.util.logging.Logger.getLogger("org.apache.http.headers");
apacheHeaderLog.setLevel(java.util.logging.Level.FINEST);
Headers & Wire:
java.util.logging.Logger apacheWireLog = java.util.logging.Logger.getLogger("org.apache.http.wire");
apacheWireLog.setLevel(java.util.logging.Level.FINEST);
Note that this will have to have a java.util.logging Handler configured at finest level and the default handler is configured to log to logcat, which will filter DEBUG (finest) entries by default.
If your system can share the wi-fi connection you should be able to route packets from any device through your system and then using wireshark you can get monitor your calls or get a tcpdump.
Also , and more importantly , it would be best if you log your network calls and responses as suggested by #Matthew
Windows 7 wi-fi connection sharing : http://www.winsupersite.com/article/faqtip/windows-7-tip-of-the-week-use-wireless-hosted-networking-to-share-an-internet-connection-wirelessly.aspx
Since I always run into similar troubles and it seems a lot of people having the same issues over and over again I wrote up a quick tutorial for debugging client-server communication by using netcat and cURL.
That of course only works for the simplified case that you always 'fake' on side of the connection.
For eavesdropping you can use tools like tcpdump or Wireshark. Which will definitely be easier if you're able to run the server instance directly on your local machine.
Stetho is a great tool from FB which helps in debugging android Apps. You can have access to local data and have a check on your network using this.
http://facebook.github.io/stetho/