Can we check which data is sent to the server from an android app just like we can check request headers, form data etc in chrome dev tools??
There are several tools that help with that problem:
https://github.com/facebook/stetho (Here you can actually use chrome dev tools)
https://github.com/jgilfelt/chuck
https://www.charlesproxy.com/
More info: https://proandroiddev.com/various-methods-to-debug-http-traffic-in-the-android-application-8685b9183418
As I understand you need to read HTTP communication from another app. You need to set up proxy server on your local computer and then redirect traffic through that proxy on your phone. I use Proxyman (https://proxyman.io/). But remember that there are some limitations. First of all you will need to create your local SSL certificate and add it to trusted store on your phone, also you will probably need Android below 7 because they added some security configs for apps which makes it harder to sniff. Next thing is that some apps may pin their SSL certificates and despite your custom cert is in trusted store app may reject connection.
Related
I have installed and have been using charles proxy(4.5.6).
I am using Android 9.
Charles works on the Chrome pages (Instagram specifically):
but it does not work on the Instagram application:
and I always get that error when I am using the app.
Should I "Save Charles root certificate" and then install it on my phone? Also, why does Charles work on the Chrome but not on the Instagram app?
Instagram's app uses a technique known as SSL Certificate Pinning. Basically, this means they bundle information about the expected SSL certificate for the server into the app, and deny any connection that doesn't present that certificate. Because the certificate presented by Charles is not Instagram's certificate, the connection is denied by the Instagram app. In Chrome, it must rely on the browser's assessment of certificate trust, which will defer to certificates installed on the device - I assume you've already set up your device to trust the Charles Proxy SSL certificate, so because Chrome trusts it (because you told it to), the website works. There used to be a way to do key pinning in the browser, but it no longer works due to some issues that were discovered.
You can disable this certificate pinning in your Facebook account settings: https://www.facebook.com/whitehat/researcher-settings/
As for the details of their implementation, there are numerous articles online about how to disable the pinning in Instagram's app. I can't vouch for any of them in particular, as I haven't tried them, and I'm not sure how they would interact with the terms of service, but you could take a look at those for more information.
I have been working with Charles to monitor and debug network requests for my app and I would like to mention few points that would help you:
If you look at the documentation of Charles SSL Certificate for Android,
As of Android N, you need to add configuration to your app in order to have it trust the SSL certificates generated by Charles SSL Proxying. This means that you can only use SSL Proxying with apps that you control.
This restriction has been added by the Android framework itself to avoid exploits and hacks for more security. So, you cannot use Charles to montior or debug network requests of third party apps (in your case -Instagram) if you're having Android 7 (Nougat) or higher.
If you really want to do it, you can try connecting an Android phone with Android 6 (Marshmallow) or lower to see if it works for you.
If you want to debug your own app, your phone and the system running Charles should be on the same network and you need to setup proxy configuration in your phone's Wi-Fi settings.
First go to Charles -> Help -> SSL Proxying -> Install Charles Root Certificate on Mobile Device or Remote Browser
You'll see an info window like this:
Note the IP address mentioned in this window and go to your phone's Settings -> Wi-Fi -> Select your Wi-Fi network -> Edit
You'll find option called Proxy, it will be None by default, update it to Manual. You'll get two input fields for entering Hostname and Port, fill those details by looking at the IP mentioned in the Charles and Save it.
Another important point is, you'll need to install Charles Root Certificate on your phone. You can do it by visiting https://chls.pro/ssl from your phone's browser. It'll automatically download the certificate and will prompt you to install it. Make sure you're connected to Charles by following the 2nd step, otherwise it won't download the certificate automatically.
Once it's done, you're good to go!
There may be two reason for this
1) The instagram app is using network security configuration file which disabled the proxy servers
for more information read this website https://developer.android.com/training/articles/security-config
2) For checking payload of your own application you need to set proxy server
as https://community.tealiumiq.com/t5/Tealium-for-Android/Setting-up-Charles-to-Proxy-your-Android-Device/ta-p/5121
On my server, I need to ensure that I receive connections only from android devices.
Is it possible? To store client cert for this in custom android assembling that is resistant to rooting ? Or I can hide absolutely nothing on rooted device? And can not avoid rooting by any means(for ex providing my own android assembled firmware)? Cause even if I set program root checking - the app can be rebuilt by 3d party to avoid this check. Any usefull ideas appreciated. Thx in advance.
1) OAuth2 is authentication and authorization protocol which is broadly used by largest and even smaller companies. Think of Facebook API. If a user is not authentication nor authorized to make that call, you can drop the inbound request. That's one method.
2) Second method would be to add your own user agent to your HTTP header and other custom HTTP headers. If your server checks on these headers, then you can drop the inbound requests.
You don't have to store the SSL certificate on the client as the client would initiate a secure connection with the server that has the SSL certificate.
Anyhow, using a certificate client side could be okay to encrypt data but I don't believe Android Java has support for that. Correct me if I'm wrong. If you do happen to encrypt data with that key, you could encrypt a certain String or bytes that you can parse into one of your custom HTTP headers but if someone finds out what the encrypted String would be, he/she can still fake a connection. However I do not recommend to store your SSL certificate on the client's device.
Regardless of what you might do, there's always a way to fake a HTTP/HTTPS connection like it's coming from an Android device but you can narrow down the incoming HTTPS requests using these two methods and make it much harder. An example would be Pokémon GO. There are plenty of unofficial APIs on GitHub who can fake a connection like it's coming from the official app.
the older pre-3.10 versions of Charles allow users to install a root certificate on their phones to help apps to allow SSL connections but the newer version has removed this feature. I am not sure if this is the reason why I wasn't able to POST successfully to the native app server.
For example I will get messages like SSLHandshake: Remote host closed connection during handshake
I suspect the root certificate is for the app on my phone to accept SSL connections from the server, but not for me to POST messages to the server.
Anyways, is there a method for me to set up SSL connections to POST?
P.S. I have added the server's url in my SSL list, and also enabled "transparent HTTP proxy." (I noticed that is not HTTPS, so perhaps Charles doesn't have transparent HTTPS feature?)
Update: I tried using mitmproxy and it worked. It looks like installing cer file to the phone is the right way to go but I am wondering why Charles removed this feature. I also think I might have missed something in the documentation. Perhaps Charles did generate a cer file in my system for me to download to the phone. If so, where can I find this file?
Yeah, all you need to do is to click help on the menu bar and then select save SSL or install SSL on mobile device and then browse to the url it gives you on your mobile browser to download the certificate.
I'm using getUserMedia() in my web app which works fine when I test my app on localhost. But if I treat my laptop as server and launch app in Google Chrome browser of my android phone, it gives me the error:
getUserMedia() no longer works on insecure origins. To use this
feature, you should consider switching your application to a secure
origin, such as HTTPS. See https://goo.gl/rStTGz for more details.
When I checked [https://goo.gl/rStTGz][1] I got to know that getUserMedia() is deprecated on insecure origins. It is written that for development mode,
You can run chrome with the
--unsafely-treat-insecure-origin-as-secure="example.com" flag (replacing "example.com" with the origin you actually want to test)
How and where can I set this flag? Is there any other alternative?
This can be done from chrome://flags/ or about://flags.
Go to about://flags, search for unsafely-treat-insecure-origin-as-secure flag, and enable it. You will have to provide the origin which you want to be treated as secure.
Multiple origins can be entered as comma-separated values.
Relaunch your browser after making this change.
Note that the protocol part is also important, and specifying the IP address, or the domain name isn't enough. eg. http:// in http://192.168.43.45. If you are not using port 80, then you may have to specify that too.
The following is a screenshot from my mobile phone.
Mobile: Samsung Galaxy S10e
Android version: 10 (Android 10)
Google Chrome version: 79.0.3945.136
For local testing of a website I am building, geolocation was needed.
Geolocation is allowed in secure locations. I do have a production server with HTTPS certificate, but the development and the debugging process would become too slow if I have to upload content to it every time.
More info
https://www.chromium.org/Home/chromium-security/prefer-secure-origins-for-powerful-new-features
Move localhost to the device
One method is to run an HTTP server on your Android device. The consensus in answers to this question is that NanoHTTPD is worth trying. If you want a ready-made application, a web search for http server for android turned up Simple HTTP Server on Google Play Store. After copying the client side of your web application to the device and starting the server, you should be able to open http://localhost:12345 in Chrome for Android.
Or make your test server secure
You can test secure-context-only features without using --unsafely-treat-insecure-origin-as-secure by turning your existing test server into a potentially trustworthy origin. Follow these steps:
If you do not already own a domain at a registrar that bundles DNS hosting compatible with the dehydrated ACME client, register one. This incurs a fee, which recurs as long as you keep the domain active.
Point a subdomain at your test web server's internal IP address. It need not be reachable from the Internet.
Configure your test web server to respond to HTTPS on port 443 of this subdomain, using NameVirtualHost or the like.
Use the dehydrated ACME client with the appropriate dns-01 hook for your DNS host to obtain a certificate from Let's Encrypt for your test web server.
Install this certificate into your test web server.
I faced with this problem too, but in Chromium, Ubuntu. I solved the problem with running this command in console:
chromium-browser --unsafely-treat-insecure-origin-as-secure="http://localhost.dev:3000" --user-data-dir=~/.config/chromium/Profile 1
where localhost.dev:3000 is your website.
For other systems information there:
where is data directory
how to launch chrome and set keys
Short information about --unsafely-treat-insecure-origin-as-secure flag:
Treat given (insecure) origins as secure origins. Multiple origins can
be supplied. Has no effect unless --user-data-dir is also supplied.
Example:
--unsafely-treat-insecure-origin-as-secure=http://a.test,http://b.test --user-data-dir=/test/only/profile/dir
I didn't check, but for android you maybe can also set flags on chrome://flags page.
I want to monitor HTTPS traffic from my application to remote server. I am trying to follow this instruction and it works for HTTP (without s), but not for HTTPS.
What is wrong? Should I write some custom code in my application to use https-proxy ?
The easiest way to do this is to use CharlesProxy to proxy your device or emulator traffic for you. The only extra step you need to do is to install the CharlesProxy SSL certificate on your device/emulator which is very straight forward:
Download the certificate from Charles Proxy (it's in their help menu) and place it on your device, then install via security settings on your device.
You then configure your device or emulators network connection to use a manual proxy and set it to the Charles Proxy address and port. Enable SSL proxying and your SSL connections will be securely routed end-to-end via Charles and Charles will be able to show you the content of requests and responses in the clear.
I'm using WireShark for sniffing, it allow you to monitor and filter raw data. But because you using https and all transactions encrypted i suppose it can't help you. May be you can switch from https to http for debug, and later when all will be works fine change protocol back to https
Do you mean you can't see the traffic at all or do you get it encrypted? Is this a web application or native application? which Android version are you using? phone or emulator?
Normally, if you set up the proxy properly, you will get the traffic, but encrypted so you can't read it. In order to see the actual content in Fiddler you would need your device to trust Fiddler's root certificate (used to create fake certificates on the fly). See this:
http://www.fiddler2.com/fiddler/help/httpsdecryption.asp
Unfortunately, I have not found a way to add root certificates to an android device other than
rooting it and replacing the certificate store (like this)
https means http secure, so it obviously can't be sniffed so easily. what would be the point if it would be the same unsecure thing as normal http?
you have to learn a bit more about secure network comunications. or, long story short, at least you will have to learn how to use a specilly devised http proxy like charles http://www.charlesproxy.com/documentation/welcome/ so you will be able to monitor you own https traffic in a clear form.