I'm using Charles proxy to fetch all the requests coming from my Android app to a webservice.
The thing is Charles shows me the complete request, meaning I can see the whole URL, headers and body so I can see www.example.com/rest/resource/param1/param2,
the JSON I send with it and also the authentication header.
After reading several posts like this and this one I thought the good part of working with the TLS was that one could only get the domain name from the URL, in this case www.example.com
To make sure it's not the client's fault, I requested the webservice resource with Retrofit and HttpsURLConnection and I could see the whole request both times.
I guess also the certificate is properly installed because it is shown in the browser every time an https request is made. Am I missing something else here or is this the normal behaviour?
So far I couldn't find a reason for this to happen so any help will be appreciated.
To debug with Charles proxy you must install a certificate on your browser (client).
With https the URL is encrypted.
But because you choose to use that proxy, your browser establish a secure connection to that proxy, and the proxy to the website. So, only 1) you, 2) the proxy 3)the website can decrypt the https traffic.
By installing a CA certificate on your browser, you allow the person detaining the corresponding private key (in your case, your proxy) to impersonate (so, decrypt with a MITM) any website.
Related
I'm working on android app which is communicating with REST API on my server, which is self-signed. My API is running on node, using framework Loopback.
Right now, my API is not secured, so I can send requests from my android app and it works fine. Only my domain is secured. API running on port 3000. And now, the problem comes.
I have created a web form for password reset, so when user is in the app and wants to reset password for his account, he gets email. He can open this email with reset link either in the app or in a browser.
1. case
When user opens the reset link in browser, he is on HTTPS URL. When he enters new password and hits button to change the password, he is sending request from HTTPS to unsecured API on HTTP.
This gives me error:
The page at xxx was loaded over HTTPS, but requested an insecure XMLHttpRequest endpoint xxx. This request has been blocked; the content must be served over HTTPS.
Here is my function:
function post() {
const urlParams = new URLSearchParams(window.location.search);
const token = urlParams.get('access_token');
console.log("TOKEN:", token);
const user = {
newPassword: document.querySelector('#input-password-check').value,
}
const http = new XMLHttpRequest()
http.open('POST', 'http://www.example.com:3000/api_path/reset-password')
http.setRequestHeader('x-access-token', token)
http.send(JSON.stringify(user))
http.onload = function() {
alert(http.responseText)
}
}
2. case
When I secure my API so both, my domain and API are HTTPS, I could reset passwort from web, but when I send request, I get ERR_CERT_AUTHORITY_INVALID. And also from android APP, since it doesn't trust my certificate. And I was unable to find a solution how to make my app to trust it.
I should also say that I am parsing token from url and when reseting password in app, that token is sent in header and I send request from my app in java... Not opening website. All goes through retrofix + rxjava.
Any ideas how to solve this? I don't want to go around it, I want to do it right, but also not willing to pay for SSL certificate which will be trusted...
I was able to solve this problem by using Lets Encrypt certificates. Previously, I had lets Encrypt installed on my server but I created new, self-signed certificates for Loopback API, which was stupid and I did not know what am I doing.
So don't forget to set certificate path for loopback api to Lets encrypt ones.
I am Android developer, my application uses a bunch of http REST calls and it gets responses from servers. I use Charles to inspect the data (so I have to install Charles certificate onto my device so that https traffic can be read by myself).
Are there any tools out there like Charles that will allow me to modfiy that response packet before sending to the client ??
Depending on what exactly you want to modify, Charles' inbuilt Rewrite Tool might be what you are looking for. You find it in the Tools menu. You can specify which requests shall be modified by protocol, host, port, path and query, and you have the following modification options:
Add, modify, remove headers (request and response)
Modify Host, Path, URL, Response Status
Add, modify, remove Query Parameters
Modify body (request and response)
Another option is Fiddler. Like Charles it can be configured as a proxy for android, decrypt HTTPS traffic and modify request and response.
Charles itself has the functionality. Follow the steps:
enable break points by right click on individual request or a path
Before sending the request, Charles will give you a change to edit it. See below. Click "edit request" to fill in whatever you want and click "execute" to send the request.
Before posting the result back to your mobile phone, you have a change to edit the content. See below.
You may try OWASP ZAP or Burp Suite. OWASP ZAP is completely free and provides a number of features.
See also Android : Capturing HTTP Requests with non-rooted android device.
I've recently tested HTTP Toolkit on Android emulator. It works and allows to capture and edit response from a server. Some functions are paid (in Pro version). Requires root priviledges on real devices.
You can also use Burp Suite or Fiddler.
I have two domains: foo.net and bar.com. They both have SSL certificates, and they work well in all desktop and mobile browsers. They are hosted on the same server configured with nginx.
However, when I make a request to a domain from within a native android app, it somehow gets the certificate from the wrong domain! This results in an IO Exception:
request = new HttpPost("https://foo.net/api/v1/baz");
request.setHeader("Authorization", "user:pass");
response = httpClient.execute(request);
...
javax.net.ssl.SSLException: hostname in certificate didn't match: <foo.net> != <bar.com> OR <bar.com> OR <www.bar.com>
What would cause android/java to try using the certificate from bar.com when every other measure seems to indicate that the server is correctly configured? Nothing appears in the nginx access or error log. There is no mention of bar.com anywhere in my android project.
Edit: I'm not sure why, but it appears that the server is using the certificate for bar.com for the server IP https://198.245.xx.xxx
The most likely cause for this problem is that the server uses Server Name Indication to choose which certificate to send. If the client doesn't support SNI, the server cannot choose which certificate to send during the SSL/TLS handshake (before any HTTP traffic is sent). SNI is required when you want to use multiple certificates on the same IP address and port, but not all clients support it (notoriously, IE on any version of Windows XP, and a number of mobile browsers).
You're also visibly using the Apache HTTP Client library (not HttpsURLConnection, for which there can be SNI support with some Android versions.
Support for SNI in the Apache HTTP Client library is quite recent, and certainly hasn't made it into the Android stack.
You may find the workaround described in this article useful (although it seems only to work for Android 4.2+).
Another two options would be:
to use a distinct IP address for each host (so as not to need SNI), if you're in control of server, or
to use another HTTP Client library (e.g. HttpsURLConnection).
A solution for Apache, more like a trick:
the SSL certificates are loaded based on the vhost name from /etc/apache2/sites-enabled. So, to trick that check make sure the problematic certificate is loaded first (remember that the vhosts are loaded by name).
It looks like the certificate of foo.net is misconfigured, and is using the same hostname as bar.com
Try to run an online certificate validation tool, like https://www.digicert.com/help/ on foo.net, just to be sure.
I think that you need to regenerate the certificate of foo.net with the right hostname, or reconfigure ngix to make sure that nginx serve the right certificate for the right host.
Scenario: I am trying to debug an Android app by proxying requests through Fiddler.
I got FiddlerRoot certificate installed on the Android device, and the SSL decryption works for most requests, but for other requests I can only see the HTTPS Connect, and nothing else in the Fiddler log. I think it might be image requests over SSL that fails to decrypt.
I have double-checked that "Hide images" is off, etc. Images retrieved are hosted on another domain than the main API the app talks to.
What could cause this behaviour ? And how do I get the image requests to show in Fiddler ?
I am using the latest Fiddler4.
There are plenty of tutorials on how you can intercept HTTP(s) traffic from Android using Fiddler.
Try this one: http://docs.telerik.com/fiddler/configure-fiddler/tasks/configureforandroid
However, it will fail when you try to intercept and decrypt Android SSL traffic coming from an application, and not from a browser.
It might be that the application uses a certificate pinning – and you are probably cannot decipher this connection. Lost cause!
But more probably, the reason is a bug in the HttpsUrlConnection pipeline implementation.
To solve the issue, please proceed with the following steps:
In Fiddler click "Rules->Customize Rules";
Find function OnBeforeResponse in the script
Add following code to the function body:
if (oSession.oRequest["User-Agent"].indexOf("Dalvik") > -1 &&
oSession.HTTPMethodIs("CONNECT")) {
oSession.oResponse.headers["Connection"] = "Keep-Alive";
}
Save the file and restart Fiddler.
I have a ASP.NET website deployed to IIS with a couple of ashx that returns JSONs to be consumed by an Android application.
I have implemented an authentication logic using Basic Authentication.
The problem: When accessed from Android, the server response is a 400 Bad request. The httperr log file says "400 - Hostname -".
It works when I try it out on localhost from Android emulator
It works when accessing the ashx file on the server from a browser
It works when replicating the call in Fiddler
(If I use Fiddler with the Android Emulator, the Response will be -1 (and looking in Fiddler at the raw data sent, it seems to loose the host from the url) - but this is another issue so don't dwell on that, i just thought I would mention it...)
Turns out my problem (and solution) was the same as in this thread:
HTTP POST request with authorization on android