Develop Reference

how to force OkHttp on Android 11 to send ssl client cert auth with TLS 1.3

We need to perform certificate based client authentication to a server accepting only TLS 1.3 connections.
The server is using Apache 2 and HTTP 1.1 and is configured to allow client cert auth but not to enforce it because some resources require client auth while others don't.
We're using OkHttp 4.9.1 on Android 11 to perform the call and we're following the standard docs on how to perform client auth: https://github.com/square/okhttp/tree/master/okhttp-tls
The server however replies with: 403 (auth renegotiation not allowed)
This is in line with TLS 1.3 specification which does not allow auth renegotation unless agreed upon during the initial handshake.
So far we've debugged the connection and inside OkHttp the class RealConnection and it actually performs the Handshake without negotiating a client cert.
Our research so far indicates that this might be due to the server using OPTIONAL ssl client auth but this is not somethign we can change so....
Is there any other option we can pass during OkHttp initialization to force it to perform the initial handshake using the client cert?
in case OkHttp is not a viable option is there any other HTTP client implementation which would allow us to force such authetnication during the initial handshake?

Create a HandshakeCertificates object using its builder. You'll need your private key, client cert and any intermediates on the client side.
The client will also need a root certificate for your server to trust it. The builder has a function to use the built in certificate authorities for that if you like.
private OkHttpClient buildClient(
HeldCertificate heldCertificate, X509Certificate... intermediates) {
HandshakeCertificates.Builder builder = new HandshakeCertificates.Builder()
.addTrustedCertificate(serverRootCa.certificate());
if (heldCertificate != null) {
builder.heldCertificate(heldCertificate, intermediates);
}
HandshakeCertificates handshakeCertificates = builder.build();
return clientTestRule.newClientBuilder()
.sslSocketFactory(
handshakeCertificates.sslSocketFactory(), handshakeCertificates.trustManager())
.build();
}
https://github.com/square/okhttp/blob/1ce86f35a9d957bae711fb81cec60abe9f43dda0/okhttp/src/test/java/okhttp3/internal/tls/ClientAuthTest.java#L126

Optional client certificate authentication is no different from required client certificate authentication from the perspective of the client. All what the client sees is a CertificateRequest and as a response it will send some certificates (leaf and chain), which might also be an empty list of certificates. It is up to the server to decide if this is acceptable, i.e. optional simply means that the server accepts that the client sends an empty list of certificates.
The problem thus is not optional or required client certificates, but that the server has not requested a certificate at all in the initial TLS handshake and likely only requests a certificate when accessing a specific path (the requested path can only be seen after the initial handshake). With Apache this is typically the case if client certificate is not requested in the server global configuration but instead in a path specific .htaccess file. Fix would be to move the requirement at the level of the domain and not the path only.

Volley trust additional root cert

I have a problem with volley and SSL and some old Android devices.
Problem is that the root certificate we use that is publicly trusted is not in place in these old devices and the OS is no longer updated so I thought I would add the CA as a file. I found lots of examples on how to trust a single certificate but I also want to keep trusting the existing root certificates. I just want to add a few to the trust store used by our app.
Is this possible? Is there a code example of this somewhere?
EDIT:
I have checked the following links, all deal with self-signed or a fully custom CA store or simply disable the checks which I do not want to do. I want to keep the default CA store but add one or two additional CAs
How to import self-signed SSL certificate to Volley on Android 4.1+
How can I make Android Volley perform HTTPS request, using a certificate self-signed by an Unknown CA?
Kind regards
Jens

What you can do is include the certificate within your app and load it programatically and supply it to your volley client.
The links you have shared gives me the idea that it should be possible, however I have never used volley and cannot confirm if it will actually work. So I will do an attempt and hopefully you can test it our and share your results here.
KeyStore baseTrustStore = KeyStore.getInstance(KeyStore.getDefaultType());
baseTrustStore.load(null, null);
// get your custom certificate(s) from your android device our within your app
// and load it as a certificate object
Certificate myTrustedCertificate = // your additional trusted certificate
baseTrustStore.setCertificateEntry("my-trusted-certificate", myTrustedCertificate);
int counter = 0;
KeyStore systemTrustStore = KeyStore.getInstance("AndroidCAStore");
Enumeration<String> aliases = systemTrustStore.aliases();
while (aliases.hasMoreElements()) {
String alias = aliases.nextElement();
if (systemTrustStore.isCertificateEntry(alias)) {
Certificate certificate = systemTrustStore.getCertificate(alias);
baseTrustStore.setCertificateEntry("" + counter++, certificate);
}
}
TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
trustManagerFactory.init(baseTrustStore);
TrustManager[] trustManagers = trustManagerFactory.getTrustManagers();
SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(null, trustManagers, null);
SSLSocketFactory socketFactory = sslContext.getSocketFactory();
HurlStack hurlStack = new HurlStack(null, socketFactory);
RequestQueue queue = Volley.newRequestQueue(this, hurlStack);
So I first create an empty trust store where I will add all the certificates which I want to trust. First add the custom root certificate which you have somewhere and map it to an instance of java.security.cert.Certificate. Afterwords get the android CA store and extract all the trusted certificates. Afterwords add all these certificates to your base trust store which you can use to create a TrustManagerFactory, SSLContext and SSLSocketFactory. The counter within the example is just a way to generate some alias, but you can specify or generate your own.

Message Security with Mutual Certificates for Android & iOS

I would like to ask the following. We have a mobile app both for Android & iOS that exchanges data with a .NET server.
For Android the ksoap2 library is used, while for iOS the Alamofire with AEXML libraries are used.
We would like to enable encryption for the communication between the server and the apps, specifically Message Security with Mutual Certificates (https://learn.microsoft.com/en-us/dotnet/framework/wcf/feature-details/message-security-with-mutual-certificates)
I am not able to find any information how either the Android or the iOS client could encrypt/decrypt the requests/responses.
Can you please provide any relative information?
Thanks in advance!

For the iOS Part.
By default, Alamofire will evaluate the certificate chain provided by the server using Apple's built in validation provided by the Security framework.
While this guarantees the certificate chain is valid, it does not prevent man-in-the-middle (MITM) attacks or other potential vulnerabilities.
In order to mitigate MITM attacks, applications dealing with sensitive customer data or financial information should use certificate or public key pinning provided by the ServerTrustPolicy.
ServerTrustPolicy
The ServerTrustPolicy enumeration evaluates the server trust generally provided by an URLAuthenticationChallenge when connecting to a server over a secure HTTPS connection.
let serverTrustPolicy = ServerTrustPolicy.pinCertificates(
certificates: ServerTrustPolicy.certificates(),
validateCertificateChain: true,
validateHost: true
)
There are many different cases of server trust evaluation giving you complete control over the validation process:
performDefaultEvaluation: Uses the default server trust evaluation while allowing you to control whether to validate the host provided by the challenge.
pinCertificates: Uses the pinned certificates to validate the server trust. The server trust is considered valid if one of the pinned certificates match one of the server certificates.
pinPublicKeys: Uses the pinned public keys to validate the server trust.
The server trust is considered valid if one of the pinned public keys match one of the server certificate public keys.
disableEvaluation: Disables all evaluation which in turn will always consider any server trust as valid.
customEvaluation: Uses the associated closure to evaluate the validity of the server trust thus giving you complete control over the validation process. Use with caution.
AlamoFire documentation
For Android part, i am not experienced with but i came across someone asking about the same thing and got an answer with
what you need to do is only to install your certificate into the webserver and call the webservice URL like https://my.webservice.url/ instead of http://my.webservice.url/.
If your certificate is a self-signed certificate, means you did not
bought it from a certificate authority, you will need to set the
SSLSocketFactory. You can check the project wiki on how to do that:
http://code.google.com/p/ksoap2-android/wiki/CodingTipsAndTricks#How_to_set_the_SSLSocketFactory_on_a_https_connection__in_order
Check Here.
This might be Helpful too
UPDATE: i've found this framework SOAPEEngine
this one.
Supports Basic, Digest and NTLM Authentication, WS-Security, Client side Certificate and custom security header.
you check its example for more clarifications too.

Message encryption with WCF is done through the WS-Security protocol, by setting the security attribute mode to Message. As you have undoubtedly realized by now, WS-Security is not exactly popular on the Android and iOS platforms, mostly due to it having been superseded by other technologies (like HTTPS,) so your choices in terms of existing libraries are not abundant. The fact that not even Microsoft-owned Xamarin supports it says a lot.
First, a word on WS-Security, This protocol provides three main means of enhancing message security:
Authentication through security tokens
Signing SOAP messages
Encryption of SOAP messages
So a conforming implementation should really provide all three of these functions, but we are mostly interested in encryption here, as from the question and comments it seems like you have the authentication part working.
Therefore, assuming we are looking for a mobile platform library providing minimal WCF compatibility with WS-Security signing and encryption:
Android
On Android, the closes to your needs is WSS-Client for Android. This project:
... implements the OASIS Web Service Security (WSS) standard for
Android platforms and makes XML Encryption and XML Signature available
for tablets and smartphones.
Note that this is GPL-licensed software. The readme says to contact the author for commercial license details. However, it seems to do what you're looking for. Once you have negotiated the key exchange with the server, to encrypt a previously constructed SOAP message using the SOAPUtil class you would do something like:
SOAPMessage message = SOAPUtil.createSOAPMessage();
//... Populate the message (possibly with another library)
SecCrypto serverCrypto = new SecCrypto(serverCertificate, null);
SecCrypto clientCrypto = new SecCrypto(clientPublicKey, clientPrivateKey);
SOAPMessage securedMessage = SOAPUtil.secureSOAPMessage(message, new HashMap<String,String>(SecConstants.REQ_ENCRYPT_SIGN, "yes"), clientCrypto, serverCrypto);
//...
SOAPMessage returnedSecuredMessage = SOAPUtil.sendSOAPMessage(context, securedMessage, endpoint, cryptoParams);
SOAPMessage returnedMessage = SOAPUtil.validateSOAPMessage(returnedSecuredMessage, new HashMap<String,String>(SecConstants.RES_DECRYPT_VERIFY, "yes", decCrypto);
Nevertheless, be prepared to do quite a bit of configuration work and debugging to make it match your server's needs.
If you are looking for a more current and actively developed product, Quasar Development offers a WS-Security implementation for Android.
iOS
Things look a lot more bleak on the iOS side. There are a few libraries claiming varying degrees of support for WSS, but none of them seem to match your needs:
At first SOAPEngine looks the most promising, as it claims
support for WS-Security. However, in a footnote it says that it has a
limitation that it only supports the WCF basicHttpBinding. This
would actually be OK if true. The binding used in the sample code you
linked to in the question is wsHttpBinding however it's important
to note that both wsHttpBinding and basicHttpBinding have support
for encryption though WS-Security. The difference is that
wsHttpBinding supports WS-Security by default (whereas it needs to
be enabled with basicHttpBinding) and it also supports
WS-ReliableMessaging and some other features you may or may not
care about. But the basicHttpBinding is the one intended for
compatibility with other technologies. So in order to have
WS-Security encryption on your WCF server and maximize compatibility
with other technologies at the same time, it would be OK to use
basicHttpBinding and enable WS-Security signing and encryption by
setting the mode security attribute to Message. With the
Message attribute, from the docs:
Security is provided using SOAP message security. By default, the body
is encrypted and signed. For this binding, the system requires that
the server certificate be provided to the client out of band. The only
valid ClientCredentialType for this binding is Certificate.
But this is of no use as SOAPEngine does not have any support for
encrypting messages (or at least I could not find any support for it
in the API). The only WS-Security function it supports is
authentication. So the claim that it supports WS-Security seems
misleading as the support is quite limited.
ServiceNow offers very limited support for WS-Security. It only
supports verifying server signatures. No encryption or signing on the
client side.
Chilkat has some rudimentary XML support and there is sample
code for WS-Security authentication. I didn't see any support or
sample code for WS-Security encryption.
Therefore for iOS, to the best of my knowledge, your two options are:
Pick one of the existing libraries that best matches your other needs
and reach out to the developer and see if you can get them to add the
WS-Security features you need.
Implement the bare minimum features
you need yourself. The spec is actually not that complicated and
there is sample code out there (for non-mobile platforms) that you
can use as guide, like WSS4J for example.

In Android:
I use kasoap2 to call the web services, but before the call, to enable mutual authentication with client certificate you need to initialize a SSLContext with the client authentication keys (KeyManager).
To do that you have to load your certificate and the corresponding password in a KeyStore object, my certificate is a *.pfx file. I cerate a "PKCS12" KeyStore instance. Then you need a KeyManagerFactory object to obtain the KeyManager array. I use a "PKIX" KeyManagerFactory instance. The KeyManager array is needed to init the SSLContext.
Here is an example:
public void enableMutualAuthentication(String filename, String password) {
try {
// InputStream to read the certificate file
FileInputStream cert = new FileInputStream(filename);
char[] pass = password.toCharArray();
KeyStore keystore = KeyStore.getInstance("PKCS12");
keystore.load(cert ,pass);
cert.close();
KeyManagerFactory keymanagerfactory = javax.net.ssl.KeyManagerFactory.getInstance("PKIX");
keymanagerfactory.init(keystore, pass);
KeyManager[] keymanagers = keymanagerfactory.getKeyManagers();
// This is not for the mutual authentication.
// Create a trust manager that does not validate certificate chains
TrustManager[] trustAllCerts = new TrustManager[] { new X509TrustManager() {
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return null;
}
public void checkClientTrusted(X509Certificate[] certs, String authType) {
}
public void checkServerTrusted(X509Certificate[] certs, String authType) {
}
} };
// Install the mutual authentication manager
// Install the all-trusting trust manager
SSLContext sc = SSLContext.getInstance("SSL");
sc.init(keymanagers, trustAllCerts, new java.security.SecureRandom());
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
// Create all-trusting host name verifier
HostnameVerifier allHostsValid = new HostnameVerifier() {
public boolean verify(String hostname, SSLSession session) {
return true;
}
};
// Install the all-trusting host verifier
HttpsURLConnection.setDefaultHostnameVerifier(allHostsValid);
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
} catch (KeyManagementException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (CertificateException e) {
e.printStackTrace();
} catch (UnrecoverableKeyException e) {
e.printStackTrace();
} catch (KeyStoreException e) {
e.printStackTrace();
}
}
Check those links, was what help me most.
https://chariotsolutions.com/blog/post/https-with-client-certificates-on/
http://callistaenterprise.se/blogg/teknik/2011/11/24/android-tlsssl-mutual-authentication/

okhttp doesn't validate pins correctly

Using com.squareup.retrofit2:retrofit:2.0.1 with com.squareup.okhttp3:okhttp:3.2.0 on an AVD with Android 6.0.
I'm trying to implement public key pinning using a self signed certificate that is signed by a Root CA. That Root CA is in the system CA trust store.
Using the example provided by okhttp wiki with some small changes:
OkHttpClient client = new OkHttpClient.Builder().certificatePinner(
new CertificatePinner.Builder()
.add(pinningUrl, "sha256/invalidPIN")
.build()).build();
Request request = new Request.Builder()
.url(pinningUrl)
.build();
Response response = client.newCall(request).execute();
if (!response.isSuccessful()) throw new IOException("Unexpected code " + response);
for (Certificate certificate : response.handshake().peerCertificates()) {
System.out.println(CertificatePinner.pin(certificate));
}
What happens is that response.isSuccessful returns true, no exception is thrown, although the pin isn't correct. The only thing that is done correctly is the validation of the certificate with the Root CAs in systems CA trust store.
What I've found to be working, is adding this line before the for loop. But that isn't the right approach because the request is already sent, the pinning should work before TLS negotiation is finished. Also this line isn't mentioned in any sample code I've found.
client.certificatePinner().check(pinningUrl, response.handshake().peerCertificates());
throws
javax.net.ssl.SSLPeerUnverifiedException: Certificate pinning failure!
Is there a bug in the sample code provided by okhttp or am I doing something wrong?

You’re configuring it incorrectly. Replace pinningUrl with the hostname of the pinning URL. For example, you want example.com instead of http://example.com/. If you’d like to send a PR to make hostname validation more strict, it would be quite welcome.

Two way SSL authentication on android

I am trying to get two way SSL authentication working between a Python server and an Android client application. I have access to both the server and client, and would like to implement client authentication using my own certificate. So far I have been able to verify the server certificate and connect without client authentication.
What sort of certificate does the client need and how do I get it to automatically send it to the server during the handshake process? Here is the client and server side code that I have so far. Is my approach wrong?
Server Code
while True: # Keep listening for clients
c, fromaddr = sock.accept()
ssl_sock = ssl.wrap_socket(c,
keyfile = "serverPrivateKey.pem",
certfile = "servercert.pem",
server_side = True,
# Require the client to provide a certificate
cert_reqs = ssl.CERT_REQUIRED,
ssl_version = ssl.PROTOCOL_TLSv1,
ca_certs = "clientcert.pem", #TODO must point to a file of CA certificates??
do_handshake_on_connect = True,
ciphers="!NULL:!EXPORT:AES256-SHA")
print ssl_sock.cipher()
thrd = sock_thread(ssl_sock)
thrd.daemon = True
thrd.start()
I suspect I may be using the wrong file for ca_certs...?
Client Code
private boolean connect() {
try {
KeyStore keystore = KeyStore.getInstance("BKS"); // Stores the client certificate, to be sent to server
KeyStore truststore = KeyStore.getInstance("BKS"); // Stores the server certificate we want to trust
// TODO: change hard coded password... THIS IS REAL BAD MKAY
truststore.load(mSocketService.getResources().openRawResource(R.raw.truststore), "test".toCharArray());
keystore.load(mSocketService.getResources().openRawResource(R.raw.keystore), "test".toCharArray());
// Use the key manager for client authentication. Keys in the key manager will be sent to the host
KeyManagerFactory keyFManager = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
keyFManager.init(keystore, "test".toCharArray());
// Use the trust manager to determine if the host I am connecting to is a trusted host
TrustManagerFactory trustMFactory = TrustManagerFactory.getInstance(TrustManagerFactory
.getDefaultAlgorithm());
trustMFactory.init(truststore);
// Create the socket factory and add both the trust manager and key manager
SSLCertificateSocketFactory socketFactory = (SSLCertificateSocketFactory) SSLCertificateSocketFactory
.getDefault(5000, new SSLSessionCache(mSocketService));
socketFactory.setTrustManagers(trustMFactory.getTrustManagers());
socketFactory.setKeyManagers(keyFManager.getKeyManagers());
// Open SSL socket directly to host, host name verification is NOT performed here due to
// SSLCertificateFactory implementation
mSSLSocket = (SSLSocket) socketFactory.createSocket(mHostname, mPort);
mSSLSocket.setSoTimeout(TIMEOUT);
// Most SSLSocketFactory implementations do not verify the server's identity, allowing man-in-the-middle
// attacks. This implementation (SSLCertificateSocketFactory) does check the server's certificate hostname,
// but only for createSocket variants that specify a hostname. When using methods that use InetAddress or
// which return an unconnected socket, you MUST verify the server's identity yourself to ensure a secure
// connection.
verifyHostname();
// Safe to proceed with socket now
...
I have generated a client private key, a client certificate, a server private key, and a server certificate using openssl. I then added the client certificate to keystore.bks (which I store in /res/raw/keystore.bks) I then added the server certificate to the truststore.bks
So now when the client tries to connect I am getting this error server side:
ssl.SSLError: [Errno 1] _ssl.c:504: error:140890C7:SSL routines:SSL3_GET_CLIENT_CERTIFICATE:peer did not return a certificate
And when I try to do this in the android client
SSLSession s = mSSLSocket.getSession();
s.getPeerCertificates();
I get this error:
javax.net.ssl.SSLPeerUnverifiedException: No peer certificate
So obviously the keystore I am using doesn't appear to have a correct peer certificate in it and thus isn't sending one to the server.
What should I put in the keystore to prevent this exception?
Furthermore, is this method of two way SSL authentication safe and effective?

The server needs to trust the client certificate. The usual way to do this is to create a CA, then have it sign a server certificate and a client certificate. Each one would have the CA certificate in their respective trust stores. Then you need to initialize the SSLContext with something like this:
KeyStore trustStore = loadTrustStore();
KeyStore keyStore = loadKeyStore();
TrustManagerFactory tmf = TrustManagerFactory
.getInstance(TrustManagerFactory.getDefaultAlgorithm());
tmf.init(trustStore);
KeyManagerFactory kmf = KeyManagerFactory
.getInstance(KeyManagerFactory.getDefaultAlgorithm());
kmf.init(keyStore, KEYSTORE_PASSWORD.toCharArray());
SSLContext sslCtx = SSLContext.getInstance("TLS");
sslCtx.init(kmf.getKeyManagers(), tmf.getTrustManagers(), null);
You can then use the SSLContext to create socket factories as needed. They will be initialized with the proper keys and certificates.

update
I'm having trouble getting a peer certificate added to the keystore,
and having that certificate sent to the server.
Done a little research that may aid your journey. mvsjes2 reported that incorrect ports can cause SSLPeerUnverifiedException to be thrown. Verify you are using port 443 as I can't see in your code where you set's the port.
Also check out emmby answer which I found also insightful.
https://stackoverflow.com/a/6378872/821312
https://stackoverflow.com/a/12012622/821312
Original post
So obviously the keystore I am using doesn't appear to have a correct
peer certificate in it and thus isn't sending one to the server.
Try looking at this article about using the connect to Unknown Certificates authority, which will let you use certificate that is not default with Android.. You might have to include a public/intermediate certificate in your application to ensure that existing/older devices will be able to connect with your server. Follow the article and see if that solved the problem of yours.
Furthermore, is this method of two way SSL authentication safe and effective?
Only if you can get it work first! As long as your code can checks the validity of the SSL certificate it should be effective enough. Safety wise... well as long the ssl certificate itself is created with a strong RSA signature/key it will make it more secure than just plain ol' http. Let me know if you continue to have issues.