I am trying to get two way SSL authentication working between a Python server and an Android client application. I have access to both the server and client, and would like to implement client authentication using my own certificate. So far I have been able to verify the server certificate and connect without client authentication.
What sort of certificate does the client need and how do I get it to automatically send it to the server during the handshake process? Here is the client and server side code that I have so far. Is my approach wrong?
Server Code
while True: # Keep listening for clients
c, fromaddr = sock.accept()
ssl_sock = ssl.wrap_socket(c,
keyfile = "serverPrivateKey.pem",
certfile = "servercert.pem",
server_side = True,
# Require the client to provide a certificate
cert_reqs = ssl.CERT_REQUIRED,
ssl_version = ssl.PROTOCOL_TLSv1,
ca_certs = "clientcert.pem", #TODO must point to a file of CA certificates??
do_handshake_on_connect = True,
ciphers="!NULL:!EXPORT:AES256-SHA")
print ssl_sock.cipher()
thrd = sock_thread(ssl_sock)
thrd.daemon = True
thrd.start()
I suspect I may be using the wrong file for ca_certs...?
Client Code
private boolean connect() {
try {
KeyStore keystore = KeyStore.getInstance("BKS"); // Stores the client certificate, to be sent to server
KeyStore truststore = KeyStore.getInstance("BKS"); // Stores the server certificate we want to trust
// TODO: change hard coded password... THIS IS REAL BAD MKAY
truststore.load(mSocketService.getResources().openRawResource(R.raw.truststore), "test".toCharArray());
keystore.load(mSocketService.getResources().openRawResource(R.raw.keystore), "test".toCharArray());
// Use the key manager for client authentication. Keys in the key manager will be sent to the host
KeyManagerFactory keyFManager = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
keyFManager.init(keystore, "test".toCharArray());
// Use the trust manager to determine if the host I am connecting to is a trusted host
TrustManagerFactory trustMFactory = TrustManagerFactory.getInstance(TrustManagerFactory
.getDefaultAlgorithm());
trustMFactory.init(truststore);
// Create the socket factory and add both the trust manager and key manager
SSLCertificateSocketFactory socketFactory = (SSLCertificateSocketFactory) SSLCertificateSocketFactory
.getDefault(5000, new SSLSessionCache(mSocketService));
socketFactory.setTrustManagers(trustMFactory.getTrustManagers());
socketFactory.setKeyManagers(keyFManager.getKeyManagers());
// Open SSL socket directly to host, host name verification is NOT performed here due to
// SSLCertificateFactory implementation
mSSLSocket = (SSLSocket) socketFactory.createSocket(mHostname, mPort);
mSSLSocket.setSoTimeout(TIMEOUT);
// Most SSLSocketFactory implementations do not verify the server's identity, allowing man-in-the-middle
// attacks. This implementation (SSLCertificateSocketFactory) does check the server's certificate hostname,
// but only for createSocket variants that specify a hostname. When using methods that use InetAddress or
// which return an unconnected socket, you MUST verify the server's identity yourself to ensure a secure
// connection.
verifyHostname();
// Safe to proceed with socket now
...
I have generated a client private key, a client certificate, a server private key, and a server certificate using openssl. I then added the client certificate to keystore.bks (which I store in /res/raw/keystore.bks) I then added the server certificate to the truststore.bks
So now when the client tries to connect I am getting this error server side:
ssl.SSLError: [Errno 1] _ssl.c:504: error:140890C7:SSL routines:SSL3_GET_CLIENT_CERTIFICATE:peer did not return a certificate
And when I try to do this in the android client
SSLSession s = mSSLSocket.getSession();
s.getPeerCertificates();
I get this error:
javax.net.ssl.SSLPeerUnverifiedException: No peer certificate
So obviously the keystore I am using doesn't appear to have a correct peer certificate in it and thus isn't sending one to the server.
What should I put in the keystore to prevent this exception?
Furthermore, is this method of two way SSL authentication safe and effective?
The server needs to trust the client certificate. The usual way to do this is to create a CA, then have it sign a server certificate and a client certificate. Each one would have the CA certificate in their respective trust stores. Then you need to initialize the SSLContext with something like this:
KeyStore trustStore = loadTrustStore();
KeyStore keyStore = loadKeyStore();
TrustManagerFactory tmf = TrustManagerFactory
.getInstance(TrustManagerFactory.getDefaultAlgorithm());
tmf.init(trustStore);
KeyManagerFactory kmf = KeyManagerFactory
.getInstance(KeyManagerFactory.getDefaultAlgorithm());
kmf.init(keyStore, KEYSTORE_PASSWORD.toCharArray());
SSLContext sslCtx = SSLContext.getInstance("TLS");
sslCtx.init(kmf.getKeyManagers(), tmf.getTrustManagers(), null);
You can then use the SSLContext to create socket factories as needed. They will be initialized with the proper keys and certificates.
update
I'm having trouble getting a peer certificate added to the keystore,
and having that certificate sent to the server.
Done a little research that may aid your journey. mvsjes2 reported that incorrect ports can cause SSLPeerUnverifiedException to be thrown. Verify you are using port 443 as I can't see in your code where you set's the port.
Also check out emmby answer which I found also insightful.
https://stackoverflow.com/a/6378872/821312
https://stackoverflow.com/a/12012622/821312
Original post
So obviously the keystore I am using doesn't appear to have a correct
peer certificate in it and thus isn't sending one to the server.
Try looking at this article about using the connect to Unknown Certificates authority, which will let you use certificate that is not default with Android.. You might have to include a public/intermediate certificate in your application to ensure that existing/older devices will be able to connect with your server. Follow the article and see if that solved the problem of yours.
Furthermore, is this method of two way SSL authentication safe and effective?
Only if you can get it work first! As long as your code can checks the validity of the SSL certificate it should be effective enough. Safety wise... well as long the ssl certificate itself is created with a strong RSA signature/key it will make it more secure than just plain ol' http. Let me know if you continue to have issues.
Related
I have a problem with volley and SSL and some old Android devices.
Problem is that the root certificate we use that is publicly trusted is not in place in these old devices and the OS is no longer updated so I thought I would add the CA as a file. I found lots of examples on how to trust a single certificate but I also want to keep trusting the existing root certificates. I just want to add a few to the trust store used by our app.
Is this possible? Is there a code example of this somewhere?
EDIT:
I have checked the following links, all deal with self-signed or a fully custom CA store or simply disable the checks which I do not want to do. I want to keep the default CA store but add one or two additional CAs
How to import self-signed SSL certificate to Volley on Android 4.1+
How can I make Android Volley perform HTTPS request, using a certificate self-signed by an Unknown CA?
Kind regards
Jens
What you can do is include the certificate within your app and load it programatically and supply it to your volley client.
The links you have shared gives me the idea that it should be possible, however I have never used volley and cannot confirm if it will actually work. So I will do an attempt and hopefully you can test it our and share your results here.
KeyStore baseTrustStore = KeyStore.getInstance(KeyStore.getDefaultType());
baseTrustStore.load(null, null);
// get your custom certificate(s) from your android device our within your app
// and load it as a certificate object
Certificate myTrustedCertificate = // your additional trusted certificate
baseTrustStore.setCertificateEntry("my-trusted-certificate", myTrustedCertificate);
int counter = 0;
KeyStore systemTrustStore = KeyStore.getInstance("AndroidCAStore");
Enumeration<String> aliases = systemTrustStore.aliases();
while (aliases.hasMoreElements()) {
String alias = aliases.nextElement();
if (systemTrustStore.isCertificateEntry(alias)) {
Certificate certificate = systemTrustStore.getCertificate(alias);
baseTrustStore.setCertificateEntry("" + counter++, certificate);
}
}
TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
trustManagerFactory.init(baseTrustStore);
TrustManager[] trustManagers = trustManagerFactory.getTrustManagers();
SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(null, trustManagers, null);
SSLSocketFactory socketFactory = sslContext.getSocketFactory();
HurlStack hurlStack = new HurlStack(null, socketFactory);
RequestQueue queue = Volley.newRequestQueue(this, hurlStack);
So I first create an empty trust store where I will add all the certificates which I want to trust. First add the custom root certificate which you have somewhere and map it to an instance of java.security.cert.Certificate. Afterwords get the android CA store and extract all the trusted certificates. Afterwords add all these certificates to your base trust store which you can use to create a TrustManagerFactory, SSLContext and SSLSocketFactory. The counter within the example is just a way to generate some alias, but you can specify or generate your own.
I could ask user to install my PKCS12 CA certificate using createInstallIntent of Android keychain. Now I am trying to fetch back that installed certificate from keychain through KeyChain.choosePrivateKeyAlias. Here when a user selects a cert, I could get a call back to alias. With that alias I could get the private key and cert chain programatically. Now I want to know; by user selecting this cert does it means that selected cert will be added (by OS) to all subsequent network calls made by the app? or do I still need to add that cert manually to my network requests?
I use a Volley for network operations and until now I have been adding a PKCS12 cert through the following code:
KeyStore keyStore = KeyStore.getInstance("PKCS12");
keyStore.load(
new ByteArrayInputStream(Base64Helper.decodeIntoByteArray(PKCS12_CERT_AS_STRING),
PKCS12_PASSWORD.toCharArray());
KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
keyManagerFactory.init(keyStore, PKCS12_PASSWORD.toCharArray());
SSLContext sslContext = SSLContext.getInstance("SSL");
sslContext.init(keyManagerFactory.getKeyManagers(), null, null);
HurlStack hurlStack = new HurlStack(null, sslContext.getSocketFactory());
requestQueue = Volley.newRequestQueue(MyApplication.getInstance().getAppContext(), hurlStack);
So after installing My CA cert to android keychain and making user to choose that cert in the system prompt; do I still need to do the above process to add the cert to my volley request or will this be taken care by Android OS?
I'm writing a small SSL proxy server and keep getting ssl.SSLError: [SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert certificate unknown (_ssl.c:661) from an android app client but not a browser. I did set ssl.CERT_NONE. Here is my test code:
SSLcontext = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
SSLcontext.load_cert_chain('server.crt', 'server.key')
SSLcontext.verify_mode = ssl.CERT_NONE
SSLcontext.check_hostname = False
s = socket.socket()
s.bind(('127.0.0.1', 443))
s.listen(5)
c = s.accept()[0]
c = SSLcontext.wrap_socket(c, server_side = True)
print c.recv(1024)
Is this because of certificate pinning on the android app or I'm doing something wrong ?
I did set ssl.CERT_NONE
This does not affect how the client verifies the server certificate at all. The server can not instruct the client to not verify the certificate and it would be a serious security issue if the server could do this.
SSLV3_ALERT_CERTIFICATE_UNKNOWN ... from an android app client but not a browser.
It is unknown what kind of certificate you use here. If this is a self-signed one you have probably added it once as trusted to the browser or added an explicit exception - but you did not do this for the Android app. If this is a certificate issued by a public CA then you are probably missing the chain certificates. Desktop browsers often work around this server side problem while most other clients don't.
My android application connects to an URL provided by the user. In case of HTTPS connections, if the server's certificate is issued by a CA that already exists in the Android's TrustManager, everything is fine.
But if the server uses a self-signed certificate how can I obtain that certificate and store it in the TrustManager on first connection?
I am using OkHttp library for performing network tasks. The solution that I have currently forces me to add the certificate in the application's raw folder during development but this will not work for the above mentioned scenario. The code that I am using currently is as below:
private KeyStore readKeyStore() throws KeyStoreException, CertificateException, NoSuchAlgorithmException, IOException {
KeyStore ks = KeyStore.getInstance(KeyStore.getDefaultType());
String password = "testPass";
InputStream is = null;
try {
is = activity.getApplicationContext().getResources().openRawResource(R.raw.server_key);
ks.load(is, password.toCharArray());
} finally {
if (is != null)
is.close();
}
return ks;
}
private OkHttpClient getOkHttpClient() throws CertificateException, IOException, KeyManagementException, UnrecoverableKeyException, NoSuchAlgorithmException, KeyStoreException {
SSLContext sslContext = SSLContext.getInstance("SSL");
TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
trustManagerFactory.init(readKeyStore());
KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
keyManagerFactory.init(readKeyStore(), "testPass".toCharArray());
sslContext.init(keyManagerFactory.getKeyManagers(), trustManagerFactory.getTrustManagers(), new SecureRandom());
return new OkHttpClient().setSslSocketFactory(sslContext.getSocketFactory());
}
The simple solution
Here's a code example of a trust manager callback. From the callback you can either store the self-signed certificates or just accept them right away. But you SHOULD NOT do neither. By accepting self-signed certs, you are subject to connecting to fake or malicious servers which can steal personal data, install malware, and do other nasty things.
The better-than-simple solution
If a server offers a self-signed certificate that you, the developer, trust at compile time, you can bundle the cert into the app like you're already doing. It would be much better if the server had a CA-signed certificate, but if you really need to connect to a server that offers a self-signed cert, this will (have to) do.
A nicer solution
Never accept self-signed certificates in runtime. If you want to allow your users to connect to these kind of servers, you can show a warning message like "proceed at your own risk" before accepting the self-signed cert.
You should not add it on first connection. Ideally they should just pay to get a proper certificate signed by a CA.
But even then you might find popular new CAs like Let's Encrypt that are not supported by the installed Java version.
In these cases you can either add the individual cert or the new CA manually using keytool for a JVM, or load it via standard Android mechanisms.
Or you can bundle those with the app. This example code shows how to load bundles certificates in addition to existing system certs using a Merged TrustStore https://github.com/yschimke/oksocial/blob/master/src/main/java/com/baulsupp/oksocial/security/CertificateUtils.java
I know there are many questions regarding pinning certificates in Android but I can't find what I am looking for...
I subclass SSLSocketFactory and override the checkServerTrusted() method. In this method, I do the following:
CertificateFactory cf = CertificateFactory.getInstance("X.509");
X509Certificate ca = (X509Certificate) cf.generateCertificate(new ByteArrayInputStream(PUB_KEY.getBytes("UTF-8")));
for (X509Certificate cert : chain) {
// Verifing by public key
cert.verify(ca.getPublicKey());
}
One of the items in the chain verifies and the other doesn't (which throws an Exception). I guess I can't get a grasp of how certificate chains work.
Should the same public certificate verify with all certificates in the chain?
The easiest way I found to implement certificate pinning on Android is to use the OkHttp library.
Here is an excerpt from the documentation:
By default, OkHttp trusts the certificate authorities of the host platform. This strategy maximizes connectivity, but it is subject to certificate authority attacks such as the 2011 DigiNotar attack. It also assumes your HTTPS servers’ certificates are signed by a certificate authority.
Use CertificatePinner to constrain which certificate authorities are trusted. Certificate pinning increases security, but limits your server team’s abilities to update their TLS certificates. Do not use certificate pinning without the blessing of your server’s TLS administrator!
public CertificatePinning() {
client = new OkHttpClient();
client.setCertificatePinner(
new CertificatePinner.Builder()
.add("publicobject.com", "sha1/DmxUShsZuNiqPQsX2Oi9uv2sCnw=")
.add("publicobject.com", "sha1/SXxoaOSEzPC6BgGmxAt/EAcsajw=")
.add("publicobject.com", "sha1/blhOM3W9V/bVQhsWAcLYwPU6n24=")
.add("publicobject.com", "sha1/T5x9IXmcrQ7YuQxXnxoCmeeQ84c=")
.build());
}
public void run() throws Exception {
Request request = new Request.Builder()
.url("https://publicobject.com/robots.txt")
.build();
Response response = client.newCall(request).execute();
if (!response.isSuccessful()) throw new IOException("Unexpected code " + response);
for (Certificate certificate : response.handshake().peerCertificates()) {
System.out.println(CertificatePinner.pin(certificate));
}
}
And if you need to support a self-signed certificate, the answer to Does OkHttp support accepting self-signed SSL certs? will guide you.
Should the same public certificate verify with all certificates in the
chain?
Answer:- No.
Most public CAs don't sign server certificates directly. Instead, they use their main CA certificate, referred to as the root CA, to sign intermediate CAs. They do this so the root CA can be stored offline to reduce risk of compromise. However, operating systems like Android typically trust only root CAs directly, which leaves a short gap of trust between the server certificate—signed by the intermediate CA—and the certificate verifier, which knows the root CA.
To solve this, the server doesn't send the client only it's
certificate during the SSL handshake, but a chain of certificates from
the server CA through any intermediates necessary to reach a trusted
root CA.
Check this link for more information. Hope this will help users.
Certificate and Public Key Pinning (a.k.a Certificate Pinning) in a
nutshell -
Normally, an app trusts all pre-installed CAs. If any of these CAs were to issue a fraudulent certificate, the app would be at risk from a man-in-the-middle attack( a.k.a eavesdropping ). Some apps choose to limit the set of certificates they accept by either limiting the set of CAs they trust or by certificate pinning. Certificate pinning is done by providing a set of certificates by hash of the public key. Certificate Pinning is a method that depends on server certificate verification on the client-side.
Below are the 3 ways to implement Certificate Pinning on Android -
The old-school way - TrustManager - https://medium.com/#ric…/ssl-pinning-on-android-8baa822e3bd5
OkHttp CertificatePinner - https://square.github.io/…/okh…/okhttp3/-certificate-pinner/
Something fresh - Network Security Configuration - https://developer.android.com/trai…/articles/security-config
In particular to your question, you could configure the certificates by hash of the public key in NSC using <pin-set> tag.
Note that, when using certificate pinning, you should always include a backup key so that if you are forced to switch to new keys or change CAs (when pinning to a CA certificate or an intermediate of that CA), your app's connectivity is unaffected. Otherwise, you must push out an update to the app to restore connectivity.
<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
<domain-config>
<domain includeSubdomains="true">example.com</domain>
<pin-set expiration="2018-01-01">
<pin digest="SHA-256">7HIpactkIAq2Y49orFOOQKurWxmmSFZhBCoQYcRhJ3Y=</pin>
<!-- backup pin -->
<pin digest="SHA-256">fwza0LRMXouZHRC8Ei+4PyuldPDcf3UKgO/04cDM1oE=</pin>
</pin-set>
</domain-config>
</network-security-config>
Your Questions and Doubts
Should the same public certificate verify with all certificates in the chain?
No, because each certificate in the chain (root, intermediate and leaf certificate)) was signed with a different private/public key pair.
One of the items in the chain verifies and the other doesn't (which throws an Exception). I guess I can't get a grasp of how certificate chains work.
That's because your certificate is the leaf one, thus you can only verify your public key against it, not against the root and intermediate certificate(s).
A Code Approach
I subclass SSLSocketFactory and override the checkServerTrusted() method.
If you really want to code it yourself I would suggest you to use instead the built-in OkHttp Ceritficate Pinner, that you can build like this:
import okhttp3.CertificatePinner;
public class OkHttpPinnerService {
// true if the Approov SDK initialized okay
private boolean initialized;
// cached OkHttpClient to use or null if not set
private OkHttpClient okHttpClient;
public synchronized OkHttpClient getOkHttpClient() {
if (okHttpClient == null) {
// build a new OkHttpClient on demand
if (initialized) {
// build the pinning configuration
CertificatePinner.Builder pinBuilder = new CertificatePinner.Builder();
Map<String, List<String>> pins = YourConfig.getPins("public-key-sha256");
for (Map.Entry<String, List<String>> entry : pins.entrySet()) {
for (String pin : entry.getValue())
pinBuilder = pinBuilder.add(entry.getKey(), "sha256/" + pin);
}
// build the OkHttpClient with the correct pins preset and ApproovTokenInterceptor
Log.i(TAG, "Building new Approov OkHttpClient");
okHttpClient = okHttpBuilder.certificatePinner(pinBuilder.build()).build();
} else {
// if the Approov SDK could not be initialized then we can't pin or add Approov tokens
Log.e(TAG, "Cannot build Approov OkHttpClient due to initialization failure");
okHttpClient = okHttpBuilder.build();
}
}
return okHttpClient;
}
}
The code was not tested for syntax errors or logical correctness. I just copied it from this repo and slightly adapted it.
A Codeless Approach
Since Android API 24 it is possible to implement certificate pinning to the public key hash via the built-in security config file, that doesn't require any code to be written, just a properly configured network_security_config.xml file added to your project.
To avoid mistakes while building the network_security_config.xml file I recommend you to use the Mobile Certificate Pinning Generator to extract the live pin being used by the domain you want to pin against and to build for you the correct configuration. For example:
Now just copy paste the generated configuration the network_security_config.xml file in your project and add this same file to the AndroifManifest.xml. Just follow the instructions on the page.