Develop Reference

Openssl Generated Client Certificates for Cloudflare mTLS Show as invalid or corrupted [duplicate]

When trying to import a pkcs12 certificate file into android for use with the openvpn connect app, I am prompted to input a password. This is the password relevant to this pkcs12 file. I proceed to input the correct password and am met with a "incorrect password" message.
To confirm that it is not the file that is faulty, I then tried to install the same certificate on a windows computer, where the same password was accepted and the certificate was installed without issue.
This was tested on two different smartphones running android 11 security update 2022-02-05.
Has anyone seen this issue before? I can only find similar issues online with no resolution.

I had the same issue. It took me about a month to figure it out.
The tl;dr is this:
$ openssl pkcs12 -nodes < your.p12 > /tmp/certbag.pem
$ openssl pkcs12 -export -legacy -in /tmp/certbag.pem > /tmp/legacy.p12
Then use legacy.p12.
Apparently Android cannot import newer pkcs12 files. I tried this on Android 12 and Android 13. This is what man openssl-pkcs12 says for -legacy:
In the legacy mode, the default algorithm for certificate encryption is RC2_CBC or 3DES_CBC depending on whether the RC2 cipher is enabled in the build. The default algorithm for private key encryption is 3DES_CBC. If the legacy option is not specified, then the legacy provider is not loaded and the default encryption algorithm for both certificates and private keys is AES_256_CBC with PBKDF2 for key derivation.
Using openssl pkcs12 -info in my case I see this on the original .p12 file, which was created using Python's PyCryptography PKCS12 support:
MAC: sha256, Iteration 1
MAC length: 32, salt length: 8
PKCS7 Encrypted data: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 20000
And using openssl pkcs12 -info -legacy on the converted .p12 file I see this:
MAC: sha1, Iteration 2048
MAC length: 20, salt length: 8
PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 2048
The original one fails to import while the converted (legacy one) imports perfectly well.

PKCS12 is a encrypted container format for certificates and cryptographic keys. For encrypting the contained data multiple algorithms exists. Unfortunately not all systems processing PKCS#12 files do support all possible encryption algorithms.
When reading a PKCS#12 file by a system/program and it encounters an unsupported cryptographic algorithm you would expect an error message like "unable to read file: unknown or unsupported algorithm". Unfortunately in reality most implementations just output the generic error message "incorrect password".
Detecting the used encryption algorithm:
For detecting the used encryption algorithm execute
openssl pkcs12 -info -in example.p12
After entering the password(s) you will see the decoded data of the PKCS12 file, the encryption type can be seen by certain lines in the output.
The most recent encryption format (that is not yet supported by all programs) is used if you find a line like:
Shrouded Keybag: PBES2, PBKDF2, AES-256-CBC, Iteration 10000, PRF hmacWithSHA256
The older often called "legacy" encryption format is used if you find a line like:
Shrouded Keybag: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 1
A third even older algorithm exists. I have not found an example PKCS#12 file, but it should be output as pbeWithSHA1And40BitRC2-CBC.
Converting a PKCS#12 file to the old encryption format
Changing the encryption type used by a PKCS#12 file is pretty complicated as you have to extract all the contained keys and certificates and the reassemble everything into a new file. The necessary openssl commands are denoted here:
https://help.globalscape.com/help/archive/secureserver3/Converting_an_incompatible_PKCS_12_format_file_to_a_compatible_PKCS_12_.htm

I ran into the problem that the above solution with -legacy option did not work on an actual ubuntu/openssl with my new email certificate.
Little additional problem: I had a .pfx file not a .p12 not knowing if this is the same container format with other ending?
The following workflow was a succes:
$ openssl pkcs12 -nodes < your.pfx > /home/ubuntu/certbag.pem
$ openssl pkcs12 -keypbe PBE-SHA1-3DES -certpbe PBE-SHA1-3DES -export -in /home/ubuntu/certbag.pem -out /home/ubuntu/new.pfx -name "SMIME-Cert"
Delete certbag.pem afterwards! It contains your private key without encryption!
Certificate imports now flawlessly on android 10.
Thanks to the above solution and the provided links!

In case anyone is struggling with GnuTLS certtool...
TL;DR this should work with both Android 9 & Android 12:
certtool --load-privkey client.key --load-certificate client.crt \
--load-ca-certificate ca.crt \
--to-p12 --outder --outfile client.p12 \
--p12-name "A Friendly Name" \
--hash SHA1 --pkcs-cipher 3des-pkcs12 --password YourPassword
Explanation
When creating PKCS#12 files, you have to choose MAC hash algorithm (--hash=xxx) and cipher algorithm (--pkcs-cipher=xxx). From my test, Android support is as below.
Hash Algorithm
Cipher Algorithm
Android 9
Android 12
(any)
aes-128, aes-192, aes-256
no
no
SHA384, SHA512
3des-pkcs12
no
no
SHA256
3des-pkcs12
yes
no
SHA1
3des-pkcs12
yes
yes
SHA256
rc2-40
yes
no
SHA1
rc2-40
yes
yes
As can be seen above, Android 9 actually supports both SHA256 and SHA1 as MAC, but Android 12 somehow only supports SHA1.
In certtool, the default MAC hash algorithm is SHA256 even if you choose --pkcs-cipher=3des-pkcs12. Therefore you have to explicitly specify --hash=SHA1, otherwise the p12 file won't work for Android 12.
Other comments
Tested phones are Xperia XZ1 Compact (G8441, Android 9) and Xperia 10 ii (XQ-AU52, Android 12).
certtool default MAC iteration is 600000 (compared to openssl's 2048). This paranoid setting results several seconds slowness when installing .p12 files on phones & PCs. I haven't found a parameter to change this iteration (openssl 3.x specifies by -iter).

One self-signed cert to rule them all? Chrome, Android, and iOS

Yet another self-signed cert question, but I've tried for several days to find the best/correct way to create a self-signed cert that will work in my development environment for the latest versions of Chrome, Android, and iOS.
The instructions I've found here and elsewhere are outdated for at least one of these platforms.
Here is the best I've found, but it only works with Chrome and Android.
openssl req -new -newkey rsa:2048 -days 3650 -nodes -x509 -subj "/C=US/ST=Oklahoma/L=Stillwater/O=My Company/OU=Engineering" -keyout ca.key -out ca.crt
openssl genrsa -out "test.key" 2048
openssl req -new -key test.key -out test.csr -config openssl.cnf
openssl x509 -req -days 3650 -in test.csr -CA ca.crt -CAkey ca.key -CAcreateserial -extensions v3_req -extfile openssl.cnf -out test.crt
openssl x509 -inform PEM -outform DER -in test.crt -out test.der.crt
Contents of openssl.cnf:
[req]
default_bits = 2048
encrypt_key = no # Change to encrypt the private key using des3 or similar
default_md = sha256
prompt = no
utf8 = yes
# Specify the DN here so we aren't prompted (along with prompt = no above).
distinguished_name = req_distinguished_name
# Extensions for SAN IP and SAN DNS
req_extensions = v3_req
# Be sure to update the subject to match your organization.
[req_distinguished_name]
C = US
ST = Oklahoma
L = Stillwater
O = My Company
OU = Engineering
CN = test.com
# Allow client and server auth. You may want to only allow server auth.
# Link to SAN names.
[v3_req]
basicConstraints = CA:TRUE
subjectKeyIdentifier = hash
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth, serverAuth
subjectAltName = #alt_names
# Alternative names are specified as IP.# and DNS.# for IP addresses and
# DNS accordingly.
[alt_names]
DNS.1 = test.com
After installing test.crt and test.key on my development server, this method works great for Chrome: just added test.crt to my Mac's keychain and turned on "Always Trust" for it.
It also works great for Android: emailed test.der.crt to the device and tapped it to install. Most important: it showed up in the "USER" tab under Settings / Encryption & credentials / Trusted credentials. This is essential for using networkSecurityConfig in my Android app.
Unfortunately it didn't work for iOS:
I installed it in an Xcode simulator by dragging the certificates to it.
I had to install both test.crt and ca.crt. If I just installed test.crt, it remained in "unverified" status, which makes sense since the ca.crt is the root certificate.
It did not show up under Settings / About / Certificate Trust Settings which is required for me to turn it on.
When my app tries to access my server with NSMutableURLRequest, it gets a "TIC SSL Trust Error" with 10 key-value pairs, including:
NSURLErrorFailingURLPeerTrustErrorKey=
_kCFStreamErrorDomainKey=3
_kCFStreamErrorCodeKey=-9813
NSErrorPeerCertificateChainKey=1 element, and NSLocalizedDescription=The certificate for this server is invalid. You might be connecting to a server that is pretending to be “test.com” which could put your confidential information at risk.
Any idea how to change what I did so I can turn it on for iOS under "Certificate Trust Settings"?
Note 1: Since other answers to other questions about the -9813 error code suggested there may be a missing intermediate certificate, I added ca.crt to my Apache configuration for the SSLCaCertificateFile setting. It still worked fine for Chrome and Android, but had exactly the same error in iOS.
Thanks!

This answer has been updated (and simplified) to be compatible with iOS 13 and Android 8. Credit now goes to https://discussions.apple.com/thread/250666160 answer by user:fixitnowyes on October 6, 2019.
Just one openssl command works to create a self-signed certificate that works in Chrome, Android, and iOS:
openssl req -config openssl.cnf -new -x509 -days 825 -out ca.crt
This outputs both ca.crt and ca.key. Note that 825 days is the maximum duration allowed by iOS 13+, and it must be specified in the openssl command. The days setting in openssl.cnf does not do anything that I can tell.
Check information about the certificate with:
openssl x509 -in ca.crt -text -noout
Contents of openssl.cnf:
[ req ]
default_bits = 2048
default_keyfile = ca.key
default_md = sha256
default_days = 825
encrypt_key = no
distinguished_name = subject
req_extensions = req_ext
x509_extensions = x509_ext
string_mask = utf8only
prompt = no
# The Subject DN can be formed using X501 or RFC 4514 (see RFC 4519 for a description).
# Its sort of a mashup. For example, RFC 4514 does not provide emailAddress.
[ subject ]
countryName = US
stateOrProvinceName = Oklahoma
localityName = Stillwater
organizationName = My Company
OU = Engineering
# Use a friendly name here because it's presented to the user. The server's DNS
# names are placed in Subject Alternate Names. Plus, DNS names here is deprecated
# by both IETF and CA/Browser Forums. If you place a DNS name here, then you
# must include the DNS name in the SAN too (otherwise, Chrome and others that
# strictly follow the CA/Browser Baseline Requirements will fail).
commonName = test.com
emailAddress = me#home.com
# Section x509_ext is used when generating a self-signed certificate. I.e., openssl req -x509 ...
[ x509_ext ]
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer
# You only need digitalSignature below. *If* you don't allow
# RSA Key transport (i.e., you use ephemeral cipher suites), then
# omit keyEncipherment because that's key transport.
basicConstraints = critical, CA:TRUE
keyUsage = critical, digitalSignature, keyEncipherment, cRLSign, keyCertSign
subjectAltName = DNS:test.com
extendedKeyUsage = serverAuth
# RFC 5280, Section 4.2.1.12 makes EKU optional
# CA/Browser Baseline Requirements, Appendix (B)(3)(G) makes me confused
# In either case, you probably only need serverAuth.
extendedKeyUsage = TLS Web Server Authentication
# Section req_ext is used when generating a certificate signing request. I.e., openssl req ...
[ req_ext ]
subjectKeyIdentifier = hash
basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment
subjectAltName = DNS:test.com
nsComment = "OpenSSL Generated Certificate"
# RFC 5280, Section 4.2.1.12 makes EKU optional
# CA/Browser Baseline Requirements, Appendix (B)(3)(G) makes me confused
# In either case, you probably only need serverAuth.
# extendedKeyUsage = serverAuth, clientAuth
# [ alternate_names ]
# DNS.1 = example.com
# DNS.2 = www.example.com
# DNS.3 = mail.example.com
# DNS.4 = ftp.example.com
# Add these if you need them. But usually you don't want them or
# need them in production. You may need them for development.
# DNS.5 = localhost
# DNS.6 = localhost.localdomain
# DNS.7 = 127.0.0.1
# IPv6 localhost
# DNS.8 = ::1
After creating the certificates...
Server installation:
Install ca.crt and ca.key in your server.
Restart server.
Chrome / Safari installation:
Add ca.crt to your Mac's KeyChain Access in the System keychain (or PC equivalent).
Set it to "Always Trust" (in Mac) so that it works in Chrome and Safari.
iOS device installation:
Send ca.crt to your device and download it to Files
Go to Files and open ca.crt
Go to General / VPN & Device Management, find your cert (listed by domain), and install it
Go to General / About / Certificate Trust Settings and turn on the certificate
iOS Xcode simulator installation:
Drag ca.crt to the simulator. Note that there is no confirmation that anything happened.
There should be no need to go to Settings / General / About / Certificate Trust Settings and enable it. It should be already enabled.
Android installation:
Email ca.crt to your Gmail account, then log into Gmail in your
Android simulator and tap to install it.
It should appear in the "USER" tab under Settings / Lock screen & security / Encryption & credentials / Trusted credentials.

I'll share my batch file for creating self signed certificates that work on Windows, iOS, Android, Chrome, FireFox and Safari. I've tested on the latest versions of Chrome, Firefox and the native browsers on mobile (Android 8.1, Latest iOS).
I found the accepted answer wasn't quite covering some scenarios - I still had to accept an exception on some mobile versions of Firefox. Although accepting the exception shows the lock icon this would cause the url to show in PWAs when installed as fullscreen or standalone modes. My goal was to support an intranet with no exceptions in order to have proper PWA installations on a private local network.
What I found covered all scenarios (real devices) tested was;
creating a CA
creating a signing request
creating a CERT with the signing request
binding the CERT to the website
installing the CA on all devices.
Note: The batch file creates a lot of files but many of them are just different formats to support different scenarios. The batch file will ask for a password several times. Don't get confused by entering multiple passwords - as the batch file progresses through steps it's always verifying the initial password created on step one. If you run into an error through a pause at the end of the batch file.
SelfCert.bat
:: Please install openssl in c:\openssl
:: The path to config should be C:\openssl\openssl.cnf
:: Otherwise you will need to add -config "path to openssl.cnf" when making requests
:: Change to current directory of batch file
cd %~dp0
:: GENERATING CA
:: 1.Generate RSA
openssl genrsa -aes256 -out ca-key.pem 4096
:: 2.Generate a public CA Cert
openssl req -new -x509 -sha256 -days 365 -key ca-key.pem -out ca.pem
:: GENERATING CERTIFICATE
:: 1. Create a RSA key
openssl genrsa -out cert-key.pem 4096
:: 2. Create a Certificate Signing Request (CSR)
openssl req -new -sha256 -subj "/CN=yourcngoeshere" -key cert-key.pem -out cert.csr
:: 3. Create a extfile with all the alternative names
:: OBSOLETE HANDLED IN C#
:: echo "subjectAltName=DNS:your-dns.record,IP:XXX.XXX.XXX.XXX" >> extfile.cnf
:: 4. Create the Certificate
openssl x509 -req -sha256 -days 365 -in cert.csr -CA ca.pem -CAkey ca-key.pem -out cert.pem -extfile extfile.cnf -CAcreateserial
:: At this point the chian is completed.Now we will create different formats for uses cases and add to trusted root.
:: 5. Create a crt for Android
openssl x509 -outform der -in ca.pem -out ca.crt
:: 6. Create a pfx for IIS, Kestrel
openssl pkcs12 -export -out cert.pfx -inkey cert-key.pem -in cert.pem -certfile ca.pem
:: 7. Add the certification to our trusted root (iOS can add crt or pem)
certutil.exe -addstore root ca.pem
:: Remember in iOS to enable the certificate after installing it
:: Settings > General > About > Certificate Trust Settings
extfile.cnf
I create this file before running the batch file but you could uncomment step 3 under GENERATING CERTIFICATE.
subjectAltName=DNS:your-dns.record,IP:XXX.XXX.XXX.XXX
Server / Client - Windows
On Windows the CA is added to the Trusted Root on the last line of the batch file certutil.exe -addstore root ca.pem.
On IIS and Kestrel I assign the cert.pfx to the website.
builder.WebHost.ConfigureKestrel(opt => { opt.ListenAnyIP(6001, listOpt => { listOpt.UseHttps(#"path to cert.pfx", "pfxpassword");});});
Client - Android
On Android I email the ca.crt. Some versions of Android seem to be picky about the type. Clicking the attachment from the email is enough to install it.
Verify it's installed by nagivating to Settings > Security & Location > Encryption & credentials > Trusted Credentials > User.
If not you may need to go to Settings > Security & Location > Encryption & credentials > Install from SD Card > Downloads
Client - iOS
On iOS I email the ca.crt or the ca.pem. Be sure to follow the prompted instructions for enabling the CA after installation.
Usually after downloading you need to go to Settings > General > Profiles > Install
After installing it you need to go to Settings > General > About > Certificate Trust Settings > Enable Certificate
Two days ago I dove into certification chains and having no experience in cryptography really hurt my brain. Major credits go to Christian Lempa. I just put his commands into a batch file and converted them to different formats at the end. Hopefully this helps someone out.

Where does Android store the platform, shared, media and release keys in the final Android OS image

Does anyone have any idea where the public keys used for signing (platform, shared, media and release key ) stored in the final generated Android OS image?
The 'Signing Builds for Release' ( https://source.android.com/devices/tech/ota/sign_builds ) page provides information on how Android OS images are signed.
The standard Android build uses four keys, all of which reside in build/target/product/security:
testkey: Generic default key for packages that do not otherwise specify a key. Used for development builds
releasekey: Generic default key for packages that do not otherwise specify a key.Used for release builds
platform: Test key for packages that are part of the core platform.
shared: test key for things that are shared in the home/contacts process.
media: Test key for packages that are part of the media/download system.
The public keys (releasekey.x509.pem, platform.x509.pem, shared.x509.pem, media.x509.pem) associated with the above private keys need to be included as part of the Android image.
These are provided as part of the build process and generally stored in build/target/product/security on the host used to build the Android OS image
However, what is not provided is where the public keys used for signing are located in the generated OS image.
For example when dm-verity is used, the RSA-2048 key in libmincrypt-compatible format is stored in the /boot partition at /verity_key.

They are not stored directly, but are stored as part of signed apk which are already part of system image. PackageManager parses them and store them in
/data/system/packages.xml.
In that xml you see tags like:
public-key identifier
Which contains public key of all apks.
In case you already have some apk which is also on device, you can unzip it.
// To get public key from apk
openssl pkcs7 -inform DER -print_certs -out cert.pem -in CERT.RSA
openssl x509 -in cert.pem -pubkey -noout
This will be same as one of public keys stored in packages.xml
Apart from this in device at /etc/security/mac_permissions.xml there are signatures which tell that app with certain signature below to certain SE context.
You can read its details at
http://androidxref.com/7.1.1_r6/xref/system/sepolicy/README
OTA certificates are stored at /etc/security/otacerts.zip which is used by recovery system.

Trying to import the private key into a PKCS12 format key store, but instead I get usage note

I am following this tutorial
http://ankitagarwal.com/wordpress/2014/05/08/https-communication-between-an-android-app-and-tomcat7-using-self-signed-certificates/
and I am currently on "Create the Server’s Key Store containing its Self-Signed Digital Certificate" section.
When I do
openssl pkcs12 –export –inkey web_server_private_key.pem –in web_server_ssl_certificate.pem –out web_server_key_store.p12
I get usage information...why?
No keys are imported/generated.

Your copy/pasted example uses "fancy hyphens". Here's what happens when I run that locally:
openssl pkcs12 –export –inkey web_server_private_key.pem –in web_server_ssl_certificate.pem –out web_server_key_store.p12
(prints usage information, as in your screenshot)
And with ascii hyphens:
openssl pkcs12 -export -inkey web_server_private_key.pem -in web_server_ssl_certificate.pem -out web_server_key_store.p12
Error opening input file web_server_ssl_certificate.pem
web_server_ssl_certificate.pem: No such file or directory
So, that explains the usage information.

JKS, BKS and PKCS12 file formats

I am in the process of setting up a headless server that builds Phonegap hybrid apps for Android using data - JS, CSS, HTML + a keystore - provided by the user. I want to institute some basic client side checks to ensure that the keystore being uploaded is valid. For JKS files I have found that I can do a rudimentary check by ensuring that the first four bytes of the supplied file are the MAGIC number 0xFEEDFEED as specified here. I realize that this does not eliminate the possibility that the user supplies garbage but it does help as a preliminary client-side screen. I would like to implement similar screening for the PKCS12 and BKS keystores but have been unable to find any explanations for those file formats. I'd be most grateful to anyone who might be able to provide some information on the subject.

First, two things to take into consideration:
JCEKS is missing in your list (more secure version of JKS, magic number is 0xCECECECE).
There are two incompatible versions of BKS. The newer version was introduced with Bouncy Castle 1.47, replacing the older version completely. Therefore BKS keystores that were generated with BC 1.47 or newer cannot be read with BC 1.46 or older. In BC 1.49 a new keystore type "BKS-V1" has been added, that is compatible with the older format (see BC Release Notes).
BKS format starts with a version number in the first 4 bytes and ends with a null byte and a SHA-1 hash (20 bytes).
PKCS#12 is not so easy to detect. You will have to parse it as an ASN.1 structure (see RFC 7292):
PFX ::= SEQUENCE {
version INTEGER {v3(3)}(v3,...),
authSafe ContentInfo,
macData MacData OPTIONAL
}
If the file cannot be parsed as the above ASN.1 structure, it's not PKCS#12.
For a more accessible explanation of the PKCS12 format check here.