I am in the process of setting up a headless server that builds Phonegap hybrid apps for Android using data - JS, CSS, HTML + a keystore - provided by the user. I want to institute some basic client side checks to ensure that the keystore being uploaded is valid. For JKS files I have found that I can do a rudimentary check by ensuring that the first four bytes of the supplied file are the MAGIC number 0xFEEDFEED as specified here. I realize that this does not eliminate the possibility that the user supplies garbage but it does help as a preliminary client-side screen. I would like to implement similar screening for the PKCS12 and BKS keystores but have been unable to find any explanations for those file formats. I'd be most grateful to anyone who might be able to provide some information on the subject.
First, two things to take into consideration:
JCEKS is missing in your list (more secure version of JKS, magic number is 0xCECECECE).
There are two incompatible versions of BKS. The newer version was introduced with Bouncy Castle 1.47, replacing the older version completely. Therefore BKS keystores that were generated with BC 1.47 or newer cannot be read with BC 1.46 or older. In BC 1.49 a new keystore type "BKS-V1" has been added, that is compatible with the older format (see BC Release Notes).
BKS format starts with a version number in the first 4 bytes and ends with a null byte and a SHA-1 hash (20 bytes).
PKCS#12 is not so easy to detect. You will have to parse it as an ASN.1 structure (see RFC 7292):
PFX ::= SEQUENCE {
version INTEGER {v3(3)}(v3,...),
authSafe ContentInfo,
macData MacData OPTIONAL
}
If the file cannot be parsed as the above ASN.1 structure, it's not PKCS#12.
For a more accessible explanation of the PKCS12 format check here.
Related
When trying to import a pkcs12 certificate file into android for use with the openvpn connect app, I am prompted to input a password. This is the password relevant to this pkcs12 file. I proceed to input the correct password and am met with a "incorrect password" message.
To confirm that it is not the file that is faulty, I then tried to install the same certificate on a windows computer, where the same password was accepted and the certificate was installed without issue.
This was tested on two different smartphones running android 11 security update 2022-02-05.
Has anyone seen this issue before? I can only find similar issues online with no resolution.
I had the same issue. It took me about a month to figure it out.
The tl;dr is this:
$ openssl pkcs12 -nodes < your.p12 > /tmp/certbag.pem
$ openssl pkcs12 -export -legacy -in /tmp/certbag.pem > /tmp/legacy.p12
Then use legacy.p12.
Apparently Android cannot import newer pkcs12 files. I tried this on Android 12 and Android 13. This is what man openssl-pkcs12 says for -legacy:
In the legacy mode, the default algorithm for certificate encryption is RC2_CBC or 3DES_CBC depending on whether the RC2 cipher is enabled in the build. The default algorithm for private key encryption is 3DES_CBC. If the legacy option is not specified, then the legacy provider is not loaded and the default encryption algorithm for both certificates and private keys is AES_256_CBC with PBKDF2 for key derivation.
Using openssl pkcs12 -info in my case I see this on the original .p12 file, which was created using Python's PyCryptography PKCS12 support:
MAC: sha256, Iteration 1
MAC length: 32, salt length: 8
PKCS7 Encrypted data: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 20000
And using openssl pkcs12 -info -legacy on the converted .p12 file I see this:
MAC: sha1, Iteration 2048
MAC length: 20, salt length: 8
PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 2048
The original one fails to import while the converted (legacy one) imports perfectly well.
PKCS12 is a encrypted container format for certificates and cryptographic keys. For encrypting the contained data multiple algorithms exists. Unfortunately not all systems processing PKCS#12 files do support all possible encryption algorithms.
When reading a PKCS#12 file by a system/program and it encounters an unsupported cryptographic algorithm you would expect an error message like "unable to read file: unknown or unsupported algorithm". Unfortunately in reality most implementations just output the generic error message "incorrect password".
Detecting the used encryption algorithm:
For detecting the used encryption algorithm execute
openssl pkcs12 -info -in example.p12
After entering the password(s) you will see the decoded data of the PKCS12 file, the encryption type can be seen by certain lines in the output.
The most recent encryption format (that is not yet supported by all programs) is used if you find a line like:
Shrouded Keybag: PBES2, PBKDF2, AES-256-CBC, Iteration 10000, PRF hmacWithSHA256
The older often called "legacy" encryption format is used if you find a line like:
Shrouded Keybag: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 1
A third even older algorithm exists. I have not found an example PKCS#12 file, but it should be output as pbeWithSHA1And40BitRC2-CBC.
Converting a PKCS#12 file to the old encryption format
Changing the encryption type used by a PKCS#12 file is pretty complicated as you have to extract all the contained keys and certificates and the reassemble everything into a new file. The necessary openssl commands are denoted here:
https://help.globalscape.com/help/archive/secureserver3/Converting_an_incompatible_PKCS_12_format_file_to_a_compatible_PKCS_12_.htm
I ran into the problem that the above solution with -legacy option did not work on an actual ubuntu/openssl with my new email certificate.
Little additional problem: I had a .pfx file not a .p12 not knowing if this is the same container format with other ending?
The following workflow was a succes:
$ openssl pkcs12 -nodes < your.pfx > /home/ubuntu/certbag.pem
$ openssl pkcs12 -keypbe PBE-SHA1-3DES -certpbe PBE-SHA1-3DES -export -in /home/ubuntu/certbag.pem -out /home/ubuntu/new.pfx -name "SMIME-Cert"
Delete certbag.pem afterwards! It contains your private key without encryption!
Certificate imports now flawlessly on android 10.
Thanks to the above solution and the provided links!
In case anyone is struggling with GnuTLS certtool...
TL;DR this should work with both Android 9 & Android 12:
certtool --load-privkey client.key --load-certificate client.crt \
--load-ca-certificate ca.crt \
--to-p12 --outder --outfile client.p12 \
--p12-name "A Friendly Name" \
--hash SHA1 --pkcs-cipher 3des-pkcs12 --password YourPassword
Explanation
When creating PKCS#12 files, you have to choose MAC hash algorithm (--hash=xxx) and cipher algorithm (--pkcs-cipher=xxx). From my test, Android support is as below.
Hash Algorithm
Cipher Algorithm
Android 9
Android 12
(any)
aes-128, aes-192, aes-256
no
no
SHA384, SHA512
3des-pkcs12
no
no
SHA256
3des-pkcs12
yes
no
SHA1
3des-pkcs12
yes
yes
SHA256
rc2-40
yes
no
SHA1
rc2-40
yes
yes
As can be seen above, Android 9 actually supports both SHA256 and SHA1 as MAC, but Android 12 somehow only supports SHA1.
In certtool, the default MAC hash algorithm is SHA256 even if you choose --pkcs-cipher=3des-pkcs12. Therefore you have to explicitly specify --hash=SHA1, otherwise the p12 file won't work for Android 12.
Other comments
Tested phones are Xperia XZ1 Compact (G8441, Android 9) and Xperia 10 ii (XQ-AU52, Android 12).
certtool default MAC iteration is 600000 (compared to openssl's 2048). This paranoid setting results several seconds slowness when installing .p12 files on phones & PCs. I haven't found a parameter to change this iteration (openssl 3.x specifies by -iter).
When trying to import a pkcs12 certificate file into android for use with the openvpn connect app, I am prompted to input a password. This is the password relevant to this pkcs12 file. I proceed to input the correct password and am met with a "incorrect password" message.
To confirm that it is not the file that is faulty, I then tried to install the same certificate on a windows computer, where the same password was accepted and the certificate was installed without issue.
This was tested on two different smartphones running android 11 security update 2022-02-05.
Has anyone seen this issue before? I can only find similar issues online with no resolution.
I had the same issue. It took me about a month to figure it out.
The tl;dr is this:
$ openssl pkcs12 -nodes < your.p12 > /tmp/certbag.pem
$ openssl pkcs12 -export -legacy -in /tmp/certbag.pem > /tmp/legacy.p12
Then use legacy.p12.
Apparently Android cannot import newer pkcs12 files. I tried this on Android 12 and Android 13. This is what man openssl-pkcs12 says for -legacy:
In the legacy mode, the default algorithm for certificate encryption is RC2_CBC or 3DES_CBC depending on whether the RC2 cipher is enabled in the build. The default algorithm for private key encryption is 3DES_CBC. If the legacy option is not specified, then the legacy provider is not loaded and the default encryption algorithm for both certificates and private keys is AES_256_CBC with PBKDF2 for key derivation.
Using openssl pkcs12 -info in my case I see this on the original .p12 file, which was created using Python's PyCryptography PKCS12 support:
MAC: sha256, Iteration 1
MAC length: 32, salt length: 8
PKCS7 Encrypted data: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 20000
And using openssl pkcs12 -info -legacy on the converted .p12 file I see this:
MAC: sha1, Iteration 2048
MAC length: 20, salt length: 8
PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 2048
The original one fails to import while the converted (legacy one) imports perfectly well.
PKCS12 is a encrypted container format for certificates and cryptographic keys. For encrypting the contained data multiple algorithms exists. Unfortunately not all systems processing PKCS#12 files do support all possible encryption algorithms.
When reading a PKCS#12 file by a system/program and it encounters an unsupported cryptographic algorithm you would expect an error message like "unable to read file: unknown or unsupported algorithm". Unfortunately in reality most implementations just output the generic error message "incorrect password".
Detecting the used encryption algorithm:
For detecting the used encryption algorithm execute
openssl pkcs12 -info -in example.p12
After entering the password(s) you will see the decoded data of the PKCS12 file, the encryption type can be seen by certain lines in the output.
The most recent encryption format (that is not yet supported by all programs) is used if you find a line like:
Shrouded Keybag: PBES2, PBKDF2, AES-256-CBC, Iteration 10000, PRF hmacWithSHA256
The older often called "legacy" encryption format is used if you find a line like:
Shrouded Keybag: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 1
A third even older algorithm exists. I have not found an example PKCS#12 file, but it should be output as pbeWithSHA1And40BitRC2-CBC.
Converting a PKCS#12 file to the old encryption format
Changing the encryption type used by a PKCS#12 file is pretty complicated as you have to extract all the contained keys and certificates and the reassemble everything into a new file. The necessary openssl commands are denoted here:
https://help.globalscape.com/help/archive/secureserver3/Converting_an_incompatible_PKCS_12_format_file_to_a_compatible_PKCS_12_.htm
I ran into the problem that the above solution with -legacy option did not work on an actual ubuntu/openssl with my new email certificate.
Little additional problem: I had a .pfx file not a .p12 not knowing if this is the same container format with other ending?
The following workflow was a succes:
$ openssl pkcs12 -nodes < your.pfx > /home/ubuntu/certbag.pem
$ openssl pkcs12 -keypbe PBE-SHA1-3DES -certpbe PBE-SHA1-3DES -export -in /home/ubuntu/certbag.pem -out /home/ubuntu/new.pfx -name "SMIME-Cert"
Delete certbag.pem afterwards! It contains your private key without encryption!
Certificate imports now flawlessly on android 10.
Thanks to the above solution and the provided links!
In case anyone is struggling with GnuTLS certtool...
TL;DR this should work with both Android 9 & Android 12:
certtool --load-privkey client.key --load-certificate client.crt \
--load-ca-certificate ca.crt \
--to-p12 --outder --outfile client.p12 \
--p12-name "A Friendly Name" \
--hash SHA1 --pkcs-cipher 3des-pkcs12 --password YourPassword
Explanation
When creating PKCS#12 files, you have to choose MAC hash algorithm (--hash=xxx) and cipher algorithm (--pkcs-cipher=xxx). From my test, Android support is as below.
Hash Algorithm
Cipher Algorithm
Android 9
Android 12
(any)
aes-128, aes-192, aes-256
no
no
SHA384, SHA512
3des-pkcs12
no
no
SHA256
3des-pkcs12
yes
no
SHA1
3des-pkcs12
yes
yes
SHA256
rc2-40
yes
no
SHA1
rc2-40
yes
yes
As can be seen above, Android 9 actually supports both SHA256 and SHA1 as MAC, but Android 12 somehow only supports SHA1.
In certtool, the default MAC hash algorithm is SHA256 even if you choose --pkcs-cipher=3des-pkcs12. Therefore you have to explicitly specify --hash=SHA1, otherwise the p12 file won't work for Android 12.
Other comments
Tested phones are Xperia XZ1 Compact (G8441, Android 9) and Xperia 10 ii (XQ-AU52, Android 12).
certtool default MAC iteration is 600000 (compared to openssl's 2048). This paranoid setting results several seconds slowness when installing .p12 files on phones & PCs. I haven't found a parameter to change this iteration (openssl 3.x specifies by -iter).
Does anyone have any idea where the public keys used for signing (platform, shared, media and release key ) stored in the final generated Android OS image?
The 'Signing Builds for Release' ( https://source.android.com/devices/tech/ota/sign_builds ) page provides information on how Android OS images are signed.
The standard Android build uses four keys, all of which reside in build/target/product/security:
testkey: Generic default key for packages that do not otherwise specify a key. Used for development builds
releasekey: Generic default key for packages that do not otherwise specify a key.Used for release builds
platform: Test key for packages that are part of the core platform.
shared: test key for things that are shared in the home/contacts process.
media: Test key for packages that are part of the media/download system.
The public keys (releasekey.x509.pem, platform.x509.pem, shared.x509.pem, media.x509.pem) associated with the above private keys need to be included as part of the Android image.
These are provided as part of the build process and generally stored in build/target/product/security on the host used to build the Android OS image
However, what is not provided is where the public keys used for signing are located in the generated OS image.
For example when dm-verity is used, the RSA-2048 key in libmincrypt-compatible format is stored in the /boot partition at /verity_key.
They are not stored directly, but are stored as part of signed apk which are already part of system image. PackageManager parses them and store them in
/data/system/packages.xml.
In that xml you see tags like:
public-key identifier
Which contains public key of all apks.
In case you already have some apk which is also on device, you can unzip it.
// To get public key from apk
openssl pkcs7 -inform DER -print_certs -out cert.pem -in CERT.RSA
openssl x509 -in cert.pem -pubkey -noout
This will be same as one of public keys stored in packages.xml
Apart from this in device at /etc/security/mac_permissions.xml there are signatures which tell that app with certain signature below to certain SE context.
You can read its details at
http://androidxref.com/7.1.1_r6/xref/system/sepolicy/README
OTA certificates are stored at /etc/security/otacerts.zip which is used by recovery system.
I have a file, which I encrypted on PC from command line using aes256 and a passphrase.
gpg --cipher-algo AES256 --output testfile.gpg --symmetric testfile.txt
Now I need to decrypt this file from my android application.
I had expiarance with decryption data (encrypted with aes256) in android before but then I had to provide KEY (256 bit) and a salt (128 bit) to decrypt.
How can I convert my passphrase to key/salt pair?
Or how having key/salt pair I can encrypt a file using GPG (and using the same key/salt to decrypt later)?
I decided to use AESCrypt app for encryption and then I can decrypt this file on android side using standard android API (no libraries needed).
AES crypt has documented its file format. They also has open source implementation for java which allowed me to write my code.
I'm trying to use BouncyCastle with android to implement ECDH and EL Gamal. I've added the bouncycastle jar file (bcprov-jdk16-144.jar) and written some code that works with my computers jvm however when I try and port it to my android application it throws:
java.security.NoSuchAlgorithmException: KeyPairGenerator ECDH implementation not found
A sample of the code is:
Security.addProvider(new org.bouncycastle.jce.provider.BouncyCastleProvider());
java.security.KeyPairGenerator keyGen = org.bouncycastle.jce.provider.asymmetric.ec.KeyPairGenerator.getInstance("ECDH", "BC");
ECGenParameterSpec ecSpec = new ECGenParameterSpec("prime192v1");
keyGen.initialize(ecSpec, SecureRandom.getInstance("SHA1PRNG"));
KeyPair pair = keyGen.generateKeyPair();
PublicKey pubk = pair.getPublic();
PrivateKey prik = pair.getPrivate();
I then wrote a simple program to see what encryption algorithms are available and ran it on my android emulator and on my computers jvm the code was:
Set<Provider.Service> rar = new org.bouncycastle.jce.provider.BouncyCastleProvider().getServices();
Iterator<Provider.Service> ir = rar.iterator();
while(ir.hasNext())
System.out.println(ir.next().getAlgorithm());
On android I do not get any of the EC algorithms while ran normally on my computer it's fine.
I'm also getting the following two errors when compiling for a lot of the bouncy castle classes:
01-07 17:17:42.548: INFO/dalvikvm(1054): DexOpt: not resolving ambiguous class 'Lorg/bouncycastle/asn1/ASN1Encodable;'
01-07 17:17:42.548: DEBUG/dalvikvm(1054): DexOpt: not verifying 'Lorg/bouncycastle/asn1/ess/OtherSigningCertificate;': multiple definitions
What am I doing wrong?
You probably want Spongy Castle - a repackage I made of Bouncy Castle specifically targeted for Android. As noted here:
http://code.google.com/p/android/issues/detail?id=3280
...the Android platform unfortunately incorporates a cut-down version of Bouncy Castle, which also makes installing an updated version of the libraries difficult due to classloader conflicts - even when you add your full BC jar, you don't get the additional classes you added.
Spongy Castle is a full replacement for the crippled versions of the Bouncy Castle cryptographic libraries which ship with Android. There are a couple of small changes to make it work on Android:
all package names have been moved from org.bouncycastle.* to org.spongycastle.* - so no classloader conflicts
the Java Security API Provider name is now SC rather than BC
I don't know if this answers your question, but some of the BouncyCastle libraries are already in the Android SDK. Perhaps the error about ambiguous class is because BouncyCastle is already included in the emulator.
It seems you can use it via the javax.crypto.Cipher class.
If you browse the Android code you will see not all BouncyCastle functionalities are included.
see libcore/security/src/main/java/org/bouncycastle/jce/provider/BouncyCastleProvider.java
In particular the output of ECDH is commented out, which means at compilation it will be completely left out from the android jar file.