Develop Reference

Charles works on browser, but not on application - Android

I have installed and have been using charles proxy(4.5.6).
I am using Android 9.
Charles works on the Chrome pages (Instagram specifically):
but it does not work on the Instagram application:
and I always get that error when I am using the app.
Should I "Save Charles root certificate" and then install it on my phone? Also, why does Charles work on the Chrome but not on the Instagram app?

Instagram's app uses a technique known as SSL Certificate Pinning. Basically, this means they bundle information about the expected SSL certificate for the server into the app, and deny any connection that doesn't present that certificate. Because the certificate presented by Charles is not Instagram's certificate, the connection is denied by the Instagram app. In Chrome, it must rely on the browser's assessment of certificate trust, which will defer to certificates installed on the device - I assume you've already set up your device to trust the Charles Proxy SSL certificate, so because Chrome trusts it (because you told it to), the website works. There used to be a way to do key pinning in the browser, but it no longer works due to some issues that were discovered.
You can disable this certificate pinning in your Facebook account settings: https://www.facebook.com/whitehat/researcher-settings/
As for the details of their implementation, there are numerous articles online about how to disable the pinning in Instagram's app. I can't vouch for any of them in particular, as I haven't tried them, and I'm not sure how they would interact with the terms of service, but you could take a look at those for more information.

I have been working with Charles to monitor and debug network requests for my app and I would like to mention few points that would help you:
If you look at the documentation of Charles SSL Certificate for Android,
As of Android N, you need to add configuration to your app in order to have it trust the SSL certificates generated by Charles SSL Proxying. This means that you can only use SSL Proxying with apps that you control.
This restriction has been added by the Android framework itself to avoid exploits and hacks for more security. So, you cannot use Charles to montior or debug network requests of third party apps (in your case -Instagram) if you're having Android 7 (Nougat) or higher.
If you really want to do it, you can try connecting an Android phone with Android 6 (Marshmallow) or lower to see if it works for you.
If you want to debug your own app, your phone and the system running Charles should be on the same network and you need to setup proxy configuration in your phone's Wi-Fi settings.
First go to Charles -> Help -> SSL Proxying -> Install Charles Root Certificate on Mobile Device or Remote Browser
You'll see an info window like this:
Note the IP address mentioned in this window and go to your phone's Settings -> Wi-Fi -> Select your Wi-Fi network -> Edit
You'll find option called Proxy, it will be None by default, update it to Manual. You'll get two input fields for entering Hostname and Port, fill those details by looking at the IP mentioned in the Charles and Save it.
Another important point is, you'll need to install Charles Root Certificate on your phone. You can do it by visiting https://chls.pro/ssl from your phone's browser. It'll automatically download the certificate and will prompt you to install it. Make sure you're connected to Charles by following the 2nd step, otherwise it won't download the certificate automatically.
Once it's done, you're good to go!

There may be two reason for this
1) The instagram app is using network security configuration file which disabled the proxy servers
for more information read this website https://developer.android.com/training/articles/security-config
2) For checking payload of your own application you need to set proxy server
as https://community.tealiumiq.com/t5/Tealium-for-Android/Setting-up-Charles-to-Proxy-your-Android-Device/ta-p/5121

How to enable Install Via USB in Xiaomi Redimi Note 4?

I want to test the android app using android studio on Xiaomi Redimi Note 4 (MIUI Global 8.5) Phone. When i try to click the Install Via USB, it says the device is temporarily restricted 3-3 error. I have enabled USB Debugging.
I tried using VPN from google but it didn't worked. I also installed Plex VPN and tried to connect to China - Shanghai Server, but it says VIP plan and purchase it. I couldn't purchase it.
I couldn't find any solution to solve this error. Excepting solution from here..

Automated way
first of all download this .ovpn file.
Then download on your phone OpenVpn app.
Open the app, click on OVPN Profile, select the file previously downloaded and press import, then press add.
Now you have added the vpn connection with his relative settings, now switch on the button referred to the vpn connection in the section "OVPN profiles".
Manual way Open your vpn settings -> add a new vpn.
Fill these fields:
-Name: something
-Type: L2TP/IPSec PSK
-Server address: 49.4.1.77
-Pre-shared key: vpn
-Select show advanced options
-Forwarding routes: 0.0.0.0/0
-Username: vpn
-Password: vpn
The .ovpn file and the server address points to a chinese vpn that you can find on https://www.vpngate.net/en/
Please remember to tell to the other users if this answer helped you!

Remotely debugging service worker on chrome android with self signed certificate

I was trying to debug a service worker served from my dev machine and running inside chrome android on the smartphone.
I was using the usb connection and remotely debugging from my dev machine. The service worker works only in https and I can live with that, both chrome and firefox where refusing to register the SW.
Then I used a self signed certificate on my dev machine, which obviously was not trusted by the mobile browsers. They warned the certificate was not good but gave the option to proceed with that certificate. I clicked yes and, while firefox from that point was able to register the SW chrome was still compaining that the certificate was not good and refused to register the SW.
I tried to include the self signed CA in the smartphone trust list but didn't find a way to do that. In the end I had to serve the app from a production server with a regular certificate, which I didn't like at all, but I could not waste a whole day on this issue.
How is it possible to remote debug a service worker on android served from a dev machine in a LAN? localhost cannot be used because they are 2 separate devices. I found a setting to ignore the https waring for chrome but desktop version, I am using android chrome, how can I disable this boring blocking error?
Thanks

I was also facing same issue. I have written service worker for push notification for my personal use. It was running perfectly on my desktop browser as I was running it on localhost. But I wanted notification on mobile so I tried many options to accomplish same, Running remote IP as localhost on mobile, USB debugging, adding ssl certificates and many other hacks but nothing worked for me. But then "Kick Web Server" app for android mobile came to my rescue, It took only few minute to run my application on mobile browser with localhost and register service worker. So no need of SSL and USB debugging any more.
Steps
Goto playstore and search for "Kick Web Server" app (Note: after serching results will show "Web Server PHPMyAdmin/MySql" with icon like this).
Do not afraid you do not need PHP or Mysql with this app, you can run your web site with ease written in any technology stack.
Open app it will ask you for permission to access storage, you have to grant permission to it.
Get back to app and click on stopped button to start web server.
It will create directory/folder named htdocs in your mobile storage.
Delete existing content(two existing files) from htdocs directory.
Copy content of your build with index.html in htdocs directory.
Visit http://localhost:8080 in your mobile browser and you will see your site is up and running with service worker registered.

Trying to monitor HTTPs traffic from an Android App using HoneyProxy, figure out the certificates.

I am on a Windows PC and would like to monitor HTTPs traffic from my Android phone from a particular app. I installed Honeyproxy and set a proxy address on my android device using Proxy Settings app. I am able to see the GET traffic of my phone. Next, to get the HTTPS traffic, I installed the user trusted certificates as described here: http://wiki.cacert.org/FAQ/ImportRootCert#Android_Phones_.26_Tablets
I can confirm that I can see both certificates (root and class3) in the "user trusted certificates" list in Settings> Security. Next I ran the honeyproxy expecting to see the HTTPS traffic this time but there is none.
Further, no HTTPs link from the device opens (I get a certificate untrusted). What is the step I am missing? Do let me know your thoughts!

How to capture all network data in a android device like fiddler in pc?

Is there a way to analyse network traffic in a android device and log it in SD card? For example this app will run in background as a proxy making all network pass through it and based on some filters (eg: facebook data, whatsapp data) inspect data that is being sent and received and log it in a text file.

I don't know of a way to save the data to an SD card but you can capture all traffic from an Android device by configuring Fiddler on a PC that's in the same network.
In Fiddler, on your PC, go to Tools > Fiddler Options > Connections
Check Allow remote computers to connect
Check the Fiddler server's IP
In the Android device's WiFi settings, configure a proxy. Use your Fiddler server's IP and port as the Proxy hostname and Proxy port respectively.
This should make the traffic from your Android device appear in Fiddler.
Check the link at the beginning of this answer for a more detailed description and screenshots.

I think you are looking into TCPDump. But this cannot be run as a regular user, you will need to become root to executing it.
Note also that the tcpdump binary is not even published with stock images (my Cyanogenmod does not even contain this).
Here are some examples:
http://danielmiessler.com/study/tcpdump/

Some applications on Google Play allow you to capture traffic without rooting your device. One of these is Grey Shirts' Packet Capture.
They work by running on the device as a VPN service. Forcing all traffic to go through it while logging it as well. You can then view the logs on the app's UI (It might be possible to pull the session log from the SD card to open it on Wireshark. Haven't tried that).
The app also allows you to install its own root CA certificate so you can view SSL encrypted packets decrypted (just remember to delete the certificate once you're done capturing traffic). This works as longs as the app does not use customized trusted CAs.
Note that you will not be able to capture all traffic with an app running as a VPN service. Name resolutions made by apps for example will not pass through the VPN.