Following this issue (that raised this question), and since we didn't get a good answer, we're looking into finding an alternative HTTPS client, one that does not use the Native_SSL library (as we think that the bug is there).
The Problem
We are attempting to use HTTPS with Client Certificate and perform upload of files larger then 44K. During the upload (checked with WireShark) at a cretin point, the server does not get any packets for 2 minutes, and closes the connection, causing a "Connection Reset By Peer".
Alternatives checked / Additional Research
We've checked the Apache Http Client port for android, but it uses the Native SSL library internally.
We've also checked the Chilkat http client, but it also didn't work.
One of the comments (By #DArkO) made to this question caused me to think perhaps the OOB Android HTTP client attempts to perform a multipart upload and fails - and that could be the root cause of all my problems.
So...
What alternative HTTP client would you recommend?
This is a work around - not a solution.
We just could not afford spending any more resources on this issue...
We've dodged the bullet by splitting the upload procedure to 2 steps: First step (which requires the Client-certificate) takes all the metadata and returns an upload token (expires in 30 seconds). Second step does not require the certificate, but uses the upload token to perform upload (still over SSL).
Related
I'm using the TRESTClient, TRESTRequest, TRESTResponse component stack for communication with the https REST API service. The Server authenticates the client by sending the request using the client certificate.
In my Android application, after calling the RestRequest.Execute (rmPost) method, a form appears asking for the selection of the client certificate. I would like to automatically assign a client certificate and avoid this additional question, but the RESTClient.OnNeedClientCertificate event is not fired.
I checked RESTLibrary, and I separately checked TNetHttpClient (the event OnNeedClientCertificate is also not fired), after several hours of reading blogs and performing dozen of tests, I am so frustrated as I have no idea how to solve the problem.
How can I avoid this additional question about choosing a client certificate and set the client certificate automatically?
Why is the TRESTClient.OnNeedClientCertificate event is not fired?
Thank you for any suggestions and help.
I am working on the Android client implementation that connects via websocket to a server. I've just enabled HTTPS on the server instance and now I am receiving this issue:
W/System.err: com.neovisionaries.ws.client.HostnameUnverifiedException: The certificate of the peer (CN=otherdomain.com) does not match the expected hostname (domain.com)
at com.neovisionaries.ws.client.SocketConnector.verifyHostname(SocketConnector.java:171)
at com.neovisionaries.ws.client.SocketConnector.doConnect(SocketConnector.java:126)
at com.neovisionaries.ws.client.SocketConnector.connect(SocketConnector.java:83)
at com.neovisionaries.ws.client.WebSocket.connect(WebSocket.java:2152)
at com.neovisionaries.ws.client.ConnectThread.runMain(ConnectThread.java:32)
at com.neovisionaries.ws.client.WebSocketThread.run(WebSocketThread.java:45)
It seems that the websocket client is finding mismatched server host names.
Some other related info:
otherdomain.com is a different one of our URLs. It is pointed to separate instance of the same web application running on the same server as domain.com which is the app I am currently working on.
My code is logging the URL that it is connecting to and I can see that it is correctly referencing domain.com:
connecting to: wss://domain.com?session_key=TheCorrectSessionKey
As noted otherdomain.com is another instance of the same web application. That instance already previously had HTTPS enabled and everything is working properly with that instance. The same client code I am using now is able to connect successfully to this instance.
The same web application has a front end page that also connects to the websocket via wss://. This is working properly in both Chrome and Firefox browsers for domain.com, as far as I can tell the only client having issues is the Android client, and it's only the domain.com variant of the app having issues, the otherdomain.com one seems to be working fine.
I found this issue opened on the library I am using for websockets. But I'm not 100% sure if my problem is related to it, but it may be. In particular someone states:
Not being able to connect to a WebSocket-enabled server domain unless it is also configured to be the default server on the website.
I assume only one site can be default, and perhaps otherdomain.com got set to default since it was completed first. This is all a bit over my head though to be honest, I'm not sure if this is relavent or not.
I have two domains: foo.net and bar.com. They both have SSL certificates, and they work well in all desktop and mobile browsers. They are hosted on the same server configured with nginx.
However, when I make a request to a domain from within a native android app, it somehow gets the certificate from the wrong domain! This results in an IO Exception:
request = new HttpPost("https://foo.net/api/v1/baz");
request.setHeader("Authorization", "user:pass");
response = httpClient.execute(request);
...
javax.net.ssl.SSLException: hostname in certificate didn't match: <foo.net> != <bar.com> OR <bar.com> OR <www.bar.com>
What would cause android/java to try using the certificate from bar.com when every other measure seems to indicate that the server is correctly configured? Nothing appears in the nginx access or error log. There is no mention of bar.com anywhere in my android project.
Edit: I'm not sure why, but it appears that the server is using the certificate for bar.com for the server IP https://198.245.xx.xxx
The most likely cause for this problem is that the server uses Server Name Indication to choose which certificate to send. If the client doesn't support SNI, the server cannot choose which certificate to send during the SSL/TLS handshake (before any HTTP traffic is sent). SNI is required when you want to use multiple certificates on the same IP address and port, but not all clients support it (notoriously, IE on any version of Windows XP, and a number of mobile browsers).
You're also visibly using the Apache HTTP Client library (not HttpsURLConnection, for which there can be SNI support with some Android versions.
Support for SNI in the Apache HTTP Client library is quite recent, and certainly hasn't made it into the Android stack.
You may find the workaround described in this article useful (although it seems only to work for Android 4.2+).
Another two options would be:
to use a distinct IP address for each host (so as not to need SNI), if you're in control of server, or
to use another HTTP Client library (e.g. HttpsURLConnection).
A solution for Apache, more like a trick:
the SSL certificates are loaded based on the vhost name from /etc/apache2/sites-enabled. So, to trick that check make sure the problematic certificate is loaded first (remember that the vhosts are loaded by name).
It looks like the certificate of foo.net is misconfigured, and is using the same hostname as bar.com
Try to run an online certificate validation tool, like https://www.digicert.com/help/ on foo.net, just to be sure.
I think that you need to regenerate the certificate of foo.net with the right hostname, or reconfigure ngix to make sure that nginx serve the right certificate for the right host.
I am confronting myself with a problem that I do not know how to interpret. I am doing a project using Android and Google App Engine. When I am trying to save information in Google App Engine's Big Table, directly from the servlet (hardcoded) - the save is performed with no problem. But when I am trying to save data from my Android device, the save is not performed and the log indicates Apache-HttpClient/UNAVAILABLE error. To be more specific:
405 55ms 0kb Apache-HttpClient/UNAVAILABLE (java 1.4)
82.155.246.249 - - [10/Jun/2013:05:20:59 -0700] "POST /servletnamehere HTTP/1.1" 405 306 - "Apache-HttpClient/UNAVAILABLE (java 1.4)" "appnamehere.appspot.com" ms=56 cpu_ms=21 cpm_usd=0.000034 app_engine_release=1.8.0 instance=00c61b117cede3f754aa1ece730dc88287a20199
I have seen that 405 HTTP error appears in the context of a POST method ( "405 errors often arise with the POST method. You may be trying to introduce some kind of input form on the Web site, but not all ISPs allow the POST method necessary to process the form." ) => indeed, I am trying to perform add (the object is a JPA Entity) in the database using a POST method. The data I am receiving from my Android device is serialized, through an input stream (in my case, working with JSON is not an option, these are the specifications).
Also, 306 HTTP error reffers to switch proxy. While the porevious error might be a bit intuitive, this one is beyond my student knowledge.
I have followed the instructions of this tutorial (http://trumpy.cs.elon.edu/joel/sigcse2011/), which is indeed what I need, but I really cannot figure out what I did wrong. I took again the procedure, stept by step, but I don't see the flaw.
Any help will be indeed appreciated. If I should post some code, I'll gladly do it.
Best wishes,
Cropcircles
LATER EDIT:
Now I get 417 HTTP error expectation failed. I have seen that I am supposed to set a certain parameter on false, but this workaround was available only for .NET. Is there anyone who knows what's the correspondent of the following, in Java? I am really really confused.
<system.net>
<settings>
<servicePointManager expect100Continue="false" />
</settings>
</system.net>
This is not an answer, but is too long for a comment and may be helpful, so here goes...
First, it's hard to tell what's going on here because there is limited info. Try to post more of the logging either on the client or on the server. Go to the server console and get more info, for instance, or add debugging in the client. I'd start by trying to make the POST from a debug/testing tool like hurl.it: http://www.hurl.it/ (hurl is a nice web front end to a command line HTTP client named curl, see curl itself if you want a more advanced peek). That way you can test and poke around and make SURE your server side works as you expect with a generic client. Then build other clients (like Android).
Second, "Apache-HttpClient/UNAVAILABLE" is not an error, it's just the user-agent String -- so ignore that part. (UNAVAILABLE is where the version typically is, but some impls don't have access to the version sometimes, it seems.)
Third, the 405 response code IS an error, it means POST is not allowed at the server you are trying to POST to. That can either be because POST is not allowed at all on said server, or you're violating some security policy (such as same origin).
If it's AppEngine, first check the APP you are using and make sure it supports POST (look for info on how to do POST at AppEngine to solve, for instance: google app engine: 405 method not allowed). When you say you can do it directly from a "servlet," do you mean that a POST from a different client works? (Servlet is a server side technology, so that's a little confusing.) If so then make sure your Android app is doing the client part the same way to the same host (multipart vs urlencoded, etc).
For a little more on the security stuff, which could be involved, see this post: Google App Engine + jQuery Ajax = 405 Method Not Allowed.
I've had the same trouble and in my case It was an error due to no write permission on server side area.
I was following an android test to write on a file a value transmitted in async way via POST method.
Apache received the POST request but was not able to write the data on the file due to security permission on it.
Scenario: I am trying to debug an Android app by proxying requests through Fiddler.
I got FiddlerRoot certificate installed on the Android device, and the SSL decryption works for most requests, but for other requests I can only see the HTTPS Connect, and nothing else in the Fiddler log. I think it might be image requests over SSL that fails to decrypt.
I have double-checked that "Hide images" is off, etc. Images retrieved are hosted on another domain than the main API the app talks to.
What could cause this behaviour ? And how do I get the image requests to show in Fiddler ?
I am using the latest Fiddler4.
There are plenty of tutorials on how you can intercept HTTP(s) traffic from Android using Fiddler.
Try this one: http://docs.telerik.com/fiddler/configure-fiddler/tasks/configureforandroid
However, it will fail when you try to intercept and decrypt Android SSL traffic coming from an application, and not from a browser.
It might be that the application uses a certificate pinning – and you are probably cannot decipher this connection. Lost cause!
But more probably, the reason is a bug in the HttpsUrlConnection pipeline implementation.
To solve the issue, please proceed with the following steps:
In Fiddler click "Rules->Customize Rules";
Find function OnBeforeResponse in the script
Add following code to the function body:
if (oSession.oRequest["User-Agent"].indexOf("Dalvik") > -1 &&
oSession.HTTPMethodIs("CONNECT")) {
oSession.oResponse.headers["Connection"] = "Keep-Alive";
}
Save the file and restart Fiddler.