My team needs to develop a solution to encrypt binary data (stored as a byte[]) in the context of an Android application written in Java. The encrypted data will be transmitted and stored in various ways, during which data corruption cannot be ruled out. Eventually another Android application (again written in Java) will have to decrypt the data.
It has already been decided that the encryption algorithm has to be AES, with a key of 256 bits.
However I would like to make an informed decision about which AES implementation and/or "mode" we should use. I have read about something called GCM mode, and we have done some tests with it (using BouncyCastle/SpongyCastle), but it is not entirely clear to me what exactly AES-GCM is for and what it "buys" us in comparison to plain AES - and whether there are any trade-off's to be taken into account.
Here's a list of concerns/requirements/questions we have:
Padding: the data we need to encrypt will not always be a multiple of the 128 bits, so the AES implementation/mode should add padding, yet only when necessary.
I was under the impression that a plain AES implementation, such as provided by javax.crypto.Cipher, would not do that, but initial tests indicated that it does. So I'm guessing the padding requirement in itself is no reason to resort to something like GCM instead of "plain" AES. Is that correct?
Authentication: We need a foolproof way of detecting if data corruption has occurred. However, ideally we also want to detect when decryption is attempted with an incorrect key. Hence, we want to be able to differentiate between both of these cases. The reason I ended up considering GCM in the first place was due to this Stackoverflow question, where one of the responders seems to imply that making this distinction is possible using AES-GCM, although he does not provide a detailed explanation (let alone code).
Minimise overhead: We need to limit overhead on storage and transmission of the encrypted data. Therefore we wish to know whether, and to what extent, the choice for a specific AES implementation/mode influences the amount of overhead.
Encryption/decryption performance: Although it is not a primary concern we are wondering to what extent the choice of a specific AES implementation/mode influences encryption and decryption performance, both in terms of CPU time and memory footprint.
Thanks in advance for any advice, clarification and/or code examples.
EDIT: delnan helpfully pointed out there is no such thing as "plain AES". So to clarify, what I meant by that is using Java's built-in AES support.
Like so: Cipher localCipher = Cipher.getInstance("AES");
In 2012 the answer is to go for GCM, unless you have serious compatibility issues.
GCM is an Authenticated Encryption mode. It provides you with confidentiality (encryption), integrity, and authentication (MAC) in one go.
So far, the normal modes of operation have been ECB (which is the default for Java), CBC, CTR, OFB, and a few others. They all provided encryption only. Confidentiality by itself is seldom useful without integrity though; one had to combine such classic modes with integrity checks in an ad-hoc way. Since cryptography is hard to get right, often such combinations were insecure, slower than necessary or even both.
Authenticated Encryption modes have been (fairly recently) created by cryptographers to solve that problem. GCM is one of the most successful: it has been selected by NIST, it efficient, it is is patent free, and it can carry Additional Authenticated Data (that is, data which stays in the clear, but for which you can verify authenticity). For a description of other modes see this excellent article of Matthew Green.
Coming to your concerns:
Padding: by default, Java uses PKCS#7 padding. That works, but it is often vulnerable to padding oracle attacks which are best defeated with a MAC. GCM embeds already a MAC (called GMAC).
Authentication: AES-GCM only takes one AES key as input, not passwords. It will tell you if the AES key is wrong or the payload has been tampered with, but such conditions are treated as one. Instead, you should consider using an appropriate Key Derivation Algorithm like PBKDF2 or bcrypt to derive the AES key from the password. I don't think it is always possible to tell if the password is incorrect or if the payload has been modified, because the data necessary to verify the former can always be corrupted. You can encrypt a small known string (with ECB AES), send it along, and use it to verify if the password is correct.
Minimise overhead: At the end of the day, all modes leads to the same overhead (around 10-20 bytes) if you want authentication. Unless you are working with very small payloads, this point can be ignored.
Performance: GCM is pretty good in that it is an online mode (no need to buffer the whole payload, so less memory), it is parallelizable, and it requires one AES operation and one Galois multiplication per plaintext block. Classic modes like ECB are faster (one AES operation per block only), but - again - you must also factor in the integrity logic, which may end up being slower than GMAC.
Having said that, one must be aware that GCM security relies on a good random number generation for creation of the IV.
Related
i created an Android Application that reads data from sensors mounted onto smartphone and sent this data in JSON format to an application server.
I configured server and android app to work over https protocol and to authenticate themselves with a mutual autentication (made up with self-signed certificates).
Now, I was asked to add, in the Android application, the encryption before the client and the server start the authentication process.
In this regard, I want to understand what is the best algorithm between DES and AES from resource consumption point of view.
I don't find anything on web. Can you redirect me to some resource where i can understand more about this argument?
DES is deprecated in almost all legitimate uses of symmetric encryption (it is obsolete and easily brute-forceable). AES (Rijndael with block size of 128 bits) is the standard symmetric encryption solution, while Blowfish, Twofish, RC4 (also not suggested), and 3DES/TDES are other options. AES is universally supported and will be your best solution in this case. Android has support for AES encryption through native Java JCA (formerly JCE) libraries as well as SpongyCastle -- a modified version of BouncyCastle designed for Android).
If you need something more substantial to back this up, there are plenty of resources:
Thomas Pornin discusses the differences between DES and AES
Thomas Pornin again
Christian Dobre
Thomas Pornin on speed differences between AES and 3DES
Summary (and additional links) of AES performance
You should use a strong (128 or 256 bit) key generated from a CSPRNG (or securely derived from a strong passphrase via PBKDF2, bcrypt, or scrypt with high work factor/iterations), a unique and non-predictable IV for each encryption operation, and (preferably) an authenticated encryption with associated data (AEAD) cipher block mode of operation like GCM or EAX, or failing that, a HMAC-based message authentication code (MAC) over the cipher text and verify it with a constant-time equality check before performing any decryption.
Don't ever use DES. It's positively antique (over 40 years old) and the short key length (56 bits) means it can nowadays be cracked quite quickly with modest hardware.
Its weakness was recognized some time ago, which is why Triple DES (3DES) was introduced to offer better security (increasing the key length to 112 bits). However, this came at the expense of increased computation times, since the algorithm has to do considerably more work to encrypt each block of data.
AES beats 3DES in every respect. By design, it is able to operate at high speed with low memory requirements. Furthermore, the latest processors (including Intel x86 and ARM architectures) have built-in AES instructions that allow encryption and decryption to be performed in hardware, resulting in speeds that are orders of magnitude higher than would be possible with 3DES.
I have to agree with the other answers as regards DES - it provides little defense against a motivated attacker.
In terms of RSA (the other algorithm you mention in your title), public-key encryption algorithms are generally considerably slower (by a magnitude of about 1000 based on what I've read, although I've never personally timed it).
It's also arguably less secure to use public-key cryptography for exchanging long messages.
As some background, public-key cryptography generally depends on some kind of a trap-door function (i.e. a function that is relatively easy to compute but is difficult to find the inverse for). It turns out that those functions are remarkably difficult to find; one of the most common ones now (which is what RSA is based on) is integer factorization, which is NP-Intermediate on "standard" computers (but is broken for quantum computers).
First, the fact that integer factorization is NP-intermediate is at least a theoretical weakness in RSA - technically, no one's actually proven that NP-intermediate problems are intrinsically more "difficult" than polynomial-time algorithms (although it's widely believed that they are) because that would entail solving the P vs. NP problem, which is one of the major outstanding questions in computer science.
It turns out that many of the trap-door functions aren't quite as difficult to find inverses for as it is to break a good symmetric-key algorithm like AES or Twofish - i.e. the best public cryptanalysis for public-key encryption algorithms tend to be at least somewhat more feasible than the ones for symmetric-key algorithms. (There's a good article here on why it's completely infeasible to "break" AES with brute force, and the known attacks against it aren't even close to feasible either).
For this reason, public-key cryptography's often used for things like key exchange, at which point both parties switch to symmetric-key cryptography.
All that to say that the other people are right - use AES :).
When searching through sites/blogs and articles about secure key storage on Android, I've found that hardware key storage isn't explained consistent. In the sense that some say that the keys are stored at the Trusted Execution Environment (TEE), while others say only the master key derived from a hardware key (baked in the TEE) is used to encrypt the encryption keys and is thus stored in the normal world.
So in summary, secure hardware backed key storage:
Keys stored in the TEE?
Keys stored in the normal world but protected with a hardware derived key?
Which one is implemented in Android? Or are both possible and is the implementation dependent on the processor manufacturers?
The litaturate is quite inconsistent.
Thanks in advance,
Gilles Callebaut
Generally, these two solutions are both possible, and also it is the recommended solution for TEE OS. You can just read the ARM and Global Platform's TEE whitepaper in detail.
I am working on a Android application related to secure data communication. I am using a few constant values in my application, and am saving them in constant.java class. I don't want these values to be reverse engineered, but even though I am using ProGuard for Android, for experts it's easy to reverse the code. These constants are very secure. I can use properties file or any file in res folder but this approach is not at all secure.
Can anybody can tell me how to proceed? Is there any file format I can save my constants or prevent properties file from reverse engineered. Is there any option such as saving it in .py python file format and reading it from Android code?
It is fundamentally impossible to securely store secret constants on a device, since hackers can reverse engineer them through static and dynamic analysis. You can only make it a bit more difficult, by obfuscating the values:
Compute them with some algorithm, instead of storing them literally. Even a trivial algorithm may increase the time needed to extract the constants.
Distribute the components of the values throughout the code.
Use native code. It is generally more difficult to reverse engineer, at least if the code and its API are sufficiently large and complex.
Maybe look into whitebox cryptography, which tries to weave constant keys into the implementations of cryptographic algorithms, in such a way that the constant keys can't be extracted. This is still the realm of research and high-end commercial solutions.
You might get some ideas that you can apply yourself from my presentation and from Scott Alexander-Bown's presentation at Droidcon in London.
You can also use a commercial obfuscator like the extended version of ProGuard, DexGuard, to harden code for you, with techniques like string encryption and class encryption.
How effective the protection is depends on the time and effort that you can invest, on the value of your product, on the time and effort that hackers are willing to spend, on their expertise, etc.
Similar question: Best Practice for storing private API keys in Android
(I am the developer of ProGuard and DexGuard)
The answer is Dont do it!. Secret constants are never secret. You should always assume your opponent is smart enough to reconstruct what you've hidden behind your smokescreen.
And anyway, you don't need to do it. For secure communications, instead use a public key infrastructure. Heres roughly how this works.
Your server generates a private and public key, and then you include the public key with your apps installation. It doesn't matter if the attacker finds this. All it allows is for your app to securely send a message to the server, and ONLY the server can decrypt it because only they have the private key.
So first thing your app should do is generate private and public key. Use whatever secure storage locker your OS provides to keep the private key safe. Its not invulnerable, but it's a damn lot more secure than anything you'll come up with. And then send the public key to your server.
Now you can securely send messages to the server using the servers public key, and the server can securely send messages to you using your public key.
Don't try reinventing the wheel here. Security researchers with serious qualifications in hard math and comp sci spend lifetimes coming up with these systems, and if you blow your implentation you leave it open for hackers to break in and steal your stuff. Use a widely trusted off the shelf PKI encryption library like OpenSSL and keep abreast of whatever source of security alerts covers that library.
I would like to encrypt data in my Android app. I have little experience in what the current state of encryption is, either for Android specifically or in general. I remember years ago that the US had laws that prevented software companies from exporting strong encryption technology. I'm not sure if any of that is applicable today considering that Android is open source code.
What I want to accomplish is to allow a user to encrypt data using only a password. I prefer to avoid using private/public keys because this probably requires having to enter in those two keys by the user. In my app, the user should be able to encrypt/decrypt data using a password. Their data will be sent from one mobile device to another and it should not be possible to decrypt their data on the receiving end without knowing the password. I do use SSL when sending the data but that isn't good enough because the data needs to remain encrypted on the server before it gets relayed to the receiving device. If a hacker had access to my server, they could potentially read the data. For this reason, I want to encrypt it on the sending device and only decrypt it on the receiving device.
A number of issues I need to resolve:
Is there an encryption API that will let me encrypt just using a password but if not, then I'll consider using private/public keys.
What are the current encryption algorithms that I can use and are they available internationally or does the US put some restriction on using them?
Would I be better off creating my own custom encryption algorithm and modify it from time to time to prevent hackers from easily breaking it? Or would you discourage this? If so, why? If I change algorithm regularly, I will need to include an algorithm ID in the data in order for the decryption code to recognize whether it can decrypt that current version of the algorithm.
I'm not looking for any "extremely hard to break" algorithm but something that should be adequate. The kind of data being stored are images, videos, audio and GPS data. It would be nice if the solution worked on Android 2.2 and above. I don't see why any algorithm shouldn't work on these versions since algorithms should be independent of an OS. Nevertheless, maybe Android does use a built-in algorithm for only certain versions?
Edit:
Code I am using now for some basic internal encryption in my app looks like this, but I have my doubts it's sufficient enough:
import java.security.SecureRandom;
import javax.crypto.Cipher;
import javax.crypto.KeyGenerator;
import javax.crypto.SecretKey;
import javax.crypto.spec.SecretKeySpec;
SecretKeySpec skeySpec = new SecretKeySpec(raw, "AES");
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.ENCRYPT_MODE, skeySpec);
byte[] encrypted = cipher.doFinal(clear);
Originally, the above line was:
SecureRandom.getInstance("SHA1PRNG")
but Google changed this in 4.0 and this caused my code to break. Anything encrypted with my previous code could no longer be decrypted with the default API for 4.0. I personally found that a major issue with Google because they broke compatibility. What's to stop them from repeating this again in the future? My encryption/decryption cannot be dependent upon Google's choice of breaking compatibility.
I am also forced to use 128 bit encryption because apparently support for 192 and 256 bits is not necessarily available on all devices, persumably because of local government laws.
Is there an encryption API that will let me encrypt just using a
password but if not, then I'll consider using private/public keys.
I've written an open source Java library that encrypts data with a password, using 256-bit AES. You can this find this on GitHub: JNCryptor. This is compatible with the RNCryptor project for iOS.
This may work for you on Android, or at least you can see how the code works and adapt it as necessary. (If you don't find it works as you'd like, feel free to raise an issue and I will adjust the library).
What are the current encryption algorithms that I can use and are they
available internationally or does the US put some restriction on using
them?
There are a myriad of algorithms available, but you will be fine if you stick something very standard, such as AES.
As far as I know, the US still forbids exporting software that uses 256-bit AES key sizes. There may be other restrictions in other countries too. But I am not a lawyer.
Would I be better off creating my own custom encryption algorithm and
modify it from time to time to prevent hackers from easily breaking
it? Or would you discourage this?
Do not do this. Always use a well-known algorithm that has been subjected to extensive peer review. There should be no need to regularly change your choice of algorithm, unless it has been known to be compromised.
I noticed that a DropBoxManager has been introduced in Android API since FroYo (API 8).
It looks like an alternative logger capable of logging not only text but also files or byte arrays, but I could not find any detailed doc anywhere about how and when we should use it.
The latest android dev blog post introducing StrictMode talks about it, StrictMode can append data to the DropBox, and we are given a shell command to retrieve these data.
Please share here your knowledge about this! Why has it been implemented in addition to the usual logcat? Can we use this to share data across apps? What kind of apps use it?
There are basically three logs on the system:
Log:
for short, textual data
in-memory ringbuffer, fast
ephemeral (you'll lose it on a crash, or the ringbuffer scrolls)
intended for app developers
EventLog is:
for short, binary data
in-memory ringbuffer, fast
ephemeral (you'll lose it on a crash, or the ringbuffer scrolls)
intended for platform developers to collect statistics
DropBox:
for long text or binary data
persistent, written to disk
kinda slow (disk)
meant for platform developers too, mostly to collect crashes & large statistics
subject to limits, deleted by tag if a tag's count and/or size get too large
DropBox is what we used during development to capture all the StrictMode violations in Gingerbread.
You can use DropBox for one-off debugging, but it's not really recommended. It's definitely not recommended as a way to share data between apps. It's not reliable enough, and you can't put permissions on the data. You should just use a shared userid and use the normal filesystem with appropriate permissions.