Develop Reference

Android BouncyCastle (SpongyCastle) HTTPS POST request

I am trying to login to my TLS1.2 server on an old Android device running Jellybean 4.1.2, with the BouncyCastle (SpongyCastle) library, but it doesn't work. That version of Android does not have TLS enabled by default, so I needed the 3rd party lib to use it.
There are two problems.
1. I get a HTTP 302 instead of a JSON response.
2. I don't know how to send a JSON payload (for other endpoints I'll be using
With the HTTP 302, I get the following response:
Result: HTTP/1.1 302 Found
Server: Apache-Coyote/1.1
Cache-Control: private
Expires: Thu, 01 Jan 1970 00:00:00 UTC
Set-Cookie: JSESSIONID=83C535625CDEF9DEC3D7890F1A9C86B0; Path=/; Secure; HttpOnly
Location: https://www.google.com/login/auth Content-Length: 0
Date: Wed, 14 Mar 2018 15:32:19 GMT
Via: 1.1 google
Set-Cookie: GCLB=CMfzgbfeh7bLpwE; path=/; HttpOnly
Alt-Svc: clear
Connection: close
So it seems its trying to redirect to some sort of Google login, which is weird.
Also, with number 2 above, where I'm trying to send a payload, do I just add another output.write(myJSONPayload); or do I have to do something else?
My code is as follows:
{
java.security.SecureRandom secureRandom = new java.security.SecureRandom();
Socket socket = new Socket("www.myserver.com", 443);
TlsClientProtocol protocol = new TlsClientProtocol(socket.getInputStream(), socket.getOutputStream(), secureRandom);
DefaultTlsClient client = new DefaultTlsClient() {
public TlsAuthentication getAuthentication() throws IOException {
TlsAuthentication auth = new TlsAuthentication() {
// Capture the server certificate information!
public void notifyServerCertificate(Certificate serverCertificate) throws IOException {
}
public TlsCredentials getClientCredentials(CertificateRequest certificateRequest) throws IOException {
return null;
}
};
return auth;
}
};
protocol.connect(client);
java.io.OutputStream output = protocol.getOutputStream();
output.write("GET / HTTP/1.1\r\n".getBytes("UTF-8"));
//Get auth with my own class to generate base 64 encoding
output.write(("Authorization: " + BasicAuthentication.getAuthenticationHeader("myuser", "mypass")).getBytes());
output.write("Host: www.myserver.com/logon\r\n".getBytes("UTF-8"));
output.write("Connection: close\r\n".getBytes("UTF-8")); // So the server will close socket immediately.
output.write("\r\n".getBytes("UTF-8")); // HTTP1.1 requirement: last line must be empty line.
output.flush();
java.io.InputStream input = protocol.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(input));
String line;
StringBuilder sb = new StringBuilder();
try {
while ((line = reader.readLine()) != null) {
Log.d(TAG, "--> " + line);
sb.append(line).append("\n");
}
} catch (TlsNoCloseNotifyException e) {
Log.d(TAG, "End of stream");
}
String result = sb.toString();
Log.d(TAG, "Result: " + result);
}
Another question, do I have the correct HOST specified? Am I right to have the base URL in the Socket, and the full URL in the OutputStream?

The solution here was to add this method to my custom SSLSocketFactory:
private static void setupSecurityForTLS() {
if (Build.VERSION.SDK_INT <= Build.VERSION_CODES.KITKAT) {
Log.d(TAG, "Adding new security provider");
Security.insertProviderAt(new BouncyCastleJsseProvider(), 1);
Security.insertProviderAt(new BouncyCastleProvider(), 2);
}
}
and call it BEFORE I initialise the socket.
You can also add the 2 lines to a static initialiser at the top of the class, eg:
static {
Security.insertProviderAt(new BouncyCastleJsseProvider(), 1);
Security.insertProviderAt(new BouncyCastleProvider(), 2);
}

Android HttpUrlConnection: Post Multipart

I'm trying to post a test File to a spring rest servlet deployed on tomcat using Android. I'm developing on Android 4.1.2, but I have verified same problem on 4.0.3.
The problem is that the file upload requires a very long time (about 70 seconds for a 4MB file), also in local network. The time is equiparable using a 3g connection. I've excluded that it could be a server problem: executing the same call with curl it takes 1 / 2 seconds, and using apache as backend results are the same.
Using HttpClient works fine.
I'm using Spring Android RestClient 1.0.1.RELEASE and, given Android version and the fact that I'm not overriding default behaviour, it uses HttpUrlConnection instead of HttpClient to make http requests.
I have also implemented my custom ClientHttpRequestFactory in order to manipulate some details of SSL connection and I have defined my own implementation of ClientHttpRequestInterceptor in order to modify authentication header.
I have also set setBufferRequestBody(false) in order to avoid OutOfMemoryException on big files. But this property have no effects on time required.
MyClientHttpRequestFactory:
public class MyClientHttpRequestFactory extends SimpleClientHttpRequestFactory{
#Override
protected void prepareConnection(HttpURLConnection connection, String httpMethod) throws IOException {
super.prepareConnection(connection, httpMethod);
connection.setConnectTimeout(240 * 1000);
connection.setReadTimeout(240 * 1000);
if ("post".equals(httpMethod.toLowerCase())) {
setBufferRequestBody(false);
}else {
setBufferRequestBody(true);
}
}
#Override
protected HttpURLConnection openConnection(URL url, Proxy proxy) throws IOException {
final HttpURLConnection httpUrlConnection = super.openConnection(url, proxy);
if (url.getProtocol().toLowerCase().equals("https")
&&
settings.selfSignedCert().get())
{
try {
((HttpsURLConnection)httpUrlConnection).setSSLSocketFactory(getSSLSocketFactory());
((HttpsURLConnection)httpUrlConnection).setHostnameVerifier(new NullHostnameVerifier());
} catch (Exception e) {
MyLog.e(LOG_TAG, "OpenConnection", e);
}
}
return httpUrlConnection;
}
MyClientHttpRequestInterceptor:
public class MyClientHttpRequestInterceptor implements ClientHttpRequestInterceptor {
#Override
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
final HttpHeaders headers = request.getHeaders();
headers.setAuthorization(new HttpBasicAuthentication( settings.username().get(), settings.password().get()));
if (settings.enable_gzip().get()) {
headers.setAcceptEncoding(ContentCodingType.GZIP);
}
return execution.execute(request, body);
}
}
And here my Rest call:
List<ClientHttpRequestInterceptor> interceptors = Arrays.asList((ClientHttpRequestInterceptor)myClientHttpRequestInterceptor);
RestTemplate restTemplate = new RestTemplate();
restTemplate.getMessageConverters().add(new FormHttpMessageConverter());
restTemplate.getMessageConverters().add(new StringHttpMessageConverter());
restTemplate.getMessageConverters().add(new MappingJackson2HttpMessageConverter());
restTemplate.setInterceptors(interceptors);
MultiValueMap<String, Object> parts = new LinkedMultiValueMap<String, Object>();
parts.add("file", new FileSystemResource("/sdcard/test/4MB_file"));
HttpEntity<MultiValueMap> requestEntity = new HttpEntity<MultiValueMap>(parts);
restTemplate.exchange(myUrl, HttpMethod.POST, requestEntity, Integer.class).getBody();
}
Looking at Spring Android source code, the next lines of code my request is passing through are:
public ClientHttpRequest createRequest(URI uri, HttpMethod httpMethod) throws IOException {
HttpURLConnection connection = openConnection(uri.toURL(), this.proxy);
prepareConnection(connection, httpMethod.name());
if (this.bufferRequestBody) {
return new SimpleBufferingClientHttpRequest(connection);
} else {
return new SimpleStreamingClientHttpRequest(connection, this.chunkSize);
}
}
Because of this.bufferRequestBody is false, return new SimpleStreamingClientHttpRequest(connection, this.chunkSize); is executed (with chunkSize = 0)
SimpleStreamingClientHttpRequest(HttpURLConnection connection, int chunkSize) {
this.connection = connection;
this.chunkSize = chunkSize;
// Bugs with reusing connections in Android versions older than Froyo (2.2)
if (olderThanFroyo) {
System.setProperty("http.keepAlive", "false");
}
}
and then:
ClientHttpRequest delegate = requestFactory.createRequest(request.getURI(), request.getMethod());
delegate.getHeaders().putAll(request.getHeaders());
if (body.length > 0) {
FileCopyUtils.copy(body, delegate.getBody());
}
return delegate.execute();
From here is all android subsystem I think..
I have dumped tcp traffic and analyzed it:
POST /urlWherePost HTTP/1.1
Content-Type: multipart/form-data;boundary=nKwsP85ZyyzSDuAqozCTuZOSxwF1jLAtd0FECUPF
Authorization: Basic xxxxxxxxxxxxxxxxxxxxxxxxxx=
Accept-Encoding: gzip
User-Agent: Dalvik/1.6.0 (Linux; U; Android 4.1.2; sdk Build/MASTER)
Host: 192.168.168.225:8080
Connection: Keep-Alive
Content-Length: 4096225
--nKwsP85ZyyzSDuAqozCTuZOSxwF1jLAtd0FECUPF
Content-Disposition: form-data; name="file"; filename="4MB_file"
Content-Type: application/octet-stream
Content-Length: 4096000
I've tryed to re-create similar request with curl:
curl --verbose
-H "Connection: Keep-Alive"
-H "Content-Type: multipart/form-data"
-H "Accept-Encoding: gzip"
-H "Content-Disposition: form-data; name=\"file\"; filename=\"4MB_file\""
-H "Content-Type: application/octet-stream"
--user xxx:xxx
-X POST
--form file=#4MB_file
http://192.168.168.225:8080/urlWherePost
but with curl the post is ok.
Posting json data is not a problem (maybe small body size). But when I try to send "big" files the time increase.
Looking in DDMS shell, on Network Statistics I've also found that the network throughput is never over 250kb in TX. There seems to be a bootleneck, but how to investigate it? Where I can look, which parameter can I change?
Thank you for any suggestion!

Have you tried using the MultipartEntity method? I had the same problem when downloading big amounts of JSON data from the server, but I switched to this method and caught all the data that the server provided me.
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost("http://myurl.com");
try {
MultipartEntity entity = new MultipartEntity();
entity.addPart("type", new StringBody("json"));
entity.addPart("data", new JSONObject(data));
httppost.setEntity(entity);
HttpResponse response = httpclient.execute(httppost);
} catch (ClientProtocolException e) {
} catch (IOException e) {
} catch (JSONException e){
}

To upload big files, You can use this library android-async-http

For simple and easy to use, I recommend this lib https://github.com/koush/ion.
I use it on my project and it works perfectly.

Android SSL Certificate Authentication (with the Android KeyChain API) to a WCF Service

I have big problems with getting the certificates running under android.
I have an android client which connects to a WCF-Service.
I think the problem is, that the certificates are not transfered. I get an error message:
403 forbidden (in the response). I really hope you can help me.
=> in my android client
In internet explorer, it works just fine => status 200
I found this article:
http://android-developers.blogspot.de/2012/03/unifying-key-store-access-in-ics.html
"A common use of the private key is for SSL client authentication. This can be implemented by using an HttpsURLConnection with a custom X509KeyManager that returns the PrivateKey retrieved from the KeyChain API. The open source Email application for ICS uses KeyChain with an X509ExtendedKeyManager. To learn more, have a look at the source code (in SSLUtils.java)."
I have checked out the SSLUtils class and I am trying to use it.
Here is some code:
private void setHttpsAdvanced() {
HostAuth ht = new HostAuth();
ht.mPort = 443;
ht.mClientCertAlias = "jensZert";
HttpParams params = getHttpParams();
MyThreadSafeClientConnManager ccm = MyThreadSafeClientConnManager
.newInstance(params, true, 443);
try {
MyThreadSafeClientConnManager.makeScheme(true, false,
ht.mClientCertAlias);
ccm.registerClientCert(getApplicationContext(), ht);
// checkCertificate(ht.mClientCertAlias);
} catch (CertificateException e) {
Log.d(TAG, e.getMessage());
e.printStackTrace();
}
this.httpclient = new DefaultHttpClient(ccm, params);
connectionInfo = this.getConnectionInfo();
this.url = String.format("%1$s://%2$s/%3$s/%4$s",
connectionInfo.Protocol, connectionInfo.ServerName,
connectionInfo.WebserviceName, connectionInfo.Path);
httpGet = new HttpGet(url);
}
private String callTheWebserviceCertificate() {
this.setupClient();
String result = "";
HttpResponse response = null;
try {
response = (HttpResponse) this.httpclient.execute(httpGet);
result = EntityUtils.toString(response.getEntity());
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
result = e.getMessage() + "\n";
for (StackTraceElement el : e.getStackTrace()) {
result += el.toString() + "\n";
}
Log.d(TAG, result);
}
return result;
}
greetings,
jens

Android Client Connecting to Jersey Rest - Post gives 405

Im writing an android app that connects to my own Jersey rest client. HTTP get commands work fine, but im having trouble with my POSTs where im trying to send something to the server. I get a 405 sent back, so it seems like the server cannot match the request up with the resource methods. Any thoughts? Test code below...
REST SERVER
#PUT
#Consumes(MultiPartMediaTypes.MULTIPART_MIXED)
public Response putResponse(MultiPart multiPart) {
System.out.println(multiPart.getBodyParts());
return null;
}
ANDROID CLIENT
HttpClient httpclient = new DefaultHttpClient();
HttpPost request = new HttpPost(URL + "responses");
request.addHeader("Content-Type", "multipart/mixed");
MultipartEntity entity = new MultipartEntity(
HttpMultipartMode.BROWSER_COMPATIBLE);
entity.addPart("Testpart1", new StringBody("<testxml></testxml>"));
entity.addPart("image1", new StringBody("imagedata1"));
request.setEntity(entity);
request.addHeader("deviceId", deviceId);
ResponseHandler<String> handler = new BasicResponseHandler();
try {
String result = httpclient.execute(request, handler);
Log.i("tag", result);
return result;
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
httpclient.getConnectionManager().shutdown();
}
return null;
TCPMon Traffic shows the following
POST /Maintenance_Server/rest/responses HTTP/1.1
Content-Type: multipart/mixed
deviceId: xxxxx
Content-Length: 244
Host: 127.0.0.1:12345
Connection: Keep-Alive
User-Agent: Apache-HttpClient/UNAVAILABLE (java 1.4)
--jju2JFDOlzJ4LQo7YkrJYLuwDUHmB5b7
Content-Disposition: form-data; name="Testpart1"
<testxml></testxml>
--jju2JFDOlzJ4LQo7YkrJYLuwDUHmB5b7
Content-Disposition: form-data; name="image1"
imagedata1
--jju2JFDOlzJ4LQo7YkrJYLuwDUHmB5b7--
Thanks
Mark

You are sending HTTP POST, but on the server side you declare a handler for HTTP PUT only. So, it's not able to match POST to any method hence 405. Change the annotation on your resource method from #PUT to #POST or send HTTP PUT instead of POST by the client.

HttpGet - weird encoding/characters

I have a following class
public class MyHttpClient {
private static HttpClient httpClient = null;
public static HttpClient getHttpClient() {
if (httpClient == null)
httpClient = new DefaultHttpClient();
return httpClient;
}
public static String HttpGetRequest(String url) throws IOException {
HttpGet request = new HttpGet(url);
HttpResponse response = null;
InputStream stream = null;
String result = "";
try {
response = getHttpClient().execute(request);
if (response.getStatusLine().getStatusCode() != 200)
response = null;
else
stream = response.getEntity().getContent();
String line = "";
StringBuilder total = new StringBuilder();
BufferedReader rd = new BufferedReader(new InputStreamReader(stream));
while ((line = rd.readLine()) != null) {
total.append(line);
}
// Return full string
result = total.toString();
} catch (ClientProtocolException e) {
response = null;
stream = null;
result = null;
} catch (IllegalStateException e) {
response = null;
stream = null;
result = null;
}
return result;
}
}
and a web-service which response header is (I can't provide direct link because of privacy)
Status: HTTP/1.1 200
OK Cache-Control: private
Content-Type: application/json;
charset=utf-8
Content-Encoding: gzip
Server: Microsoft-IIS/7.5
X-AspNetMvc-Version: 3.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Sun, 03
Jul 2011 11:00:43 GMT
Connection: close
Content-Length: 8134
In the end I get as a result series of weird, unreadable characters (I should get regular JSON like I get in regular desktop browser).
Where is the problem? (The same code for ex. google.com works perfectly and I get nice result)
EDIT: Solution (see below for description)
Replace
HttpGet request = new HttpGet(url);
with
HttpUriRequest request = new HttpGet(url);
request.addHeader("Accept-Encoding", "gzip");
and replace
stream = response.getEntity().getContent();
with
stream = response.getEntity().getContent();
Header contentEncoding = response.getFirstHeader("Content-Encoding");
if (contentEncoding != null && contentEncoding.getValue().equalsIgnoreCase("gzip")) {
stream = new GZIPInputStream(stream);
}

The problem is here:
Content-Encoding: gzip
This means the weird characters you are getting are the gzip compressed version of the expected JSON. Your browser does the decmpression so there you see the decoded result. You should have a look at your request header and at the configuration of your server.

Well, gzip encoding is generally good practice - for JSON data (especially big one) it can actually get between 10x and 20x decrease in the amount of data to transfer (which is a GOOD THING).. So better is to let HttpClient to handle GZIP compression nicely. For example here:
http://forrst.com/posts/Enabling_GZip_compression_with_HttpClient-u0X
BTW. It seems wrong however on the server side to provide GZIP compressed data when the client does not say "Accept-Encoding: gzip", which seems to be the case... So some things have to be corrected on the server as well. The example above adds Accept-Encoding header for you.