I'm trying to build a secure system for transmitting data from a client Android app to a web server running PHP.
What I want to do is ensure that the system is cryptographically secure in such a way that the messages from the app can be verified as being actually from the app itself, rather than being from a devious user who may have written a custom script or perhaps using cURL in order to game the system.
There are a number of use cases for this kind of verification, for example:-
If an app contains an advert from which you gather metrics, you would want to verify that the click data is being sent from the app rather than from a malicious user who has figured out your API and is sending dummy data.
The app might have a multiple-choice survey and again, you would want to ensure that the survey results are being collected from the app.
The app is collecting GPS traces and you want to ensure that the data is being sent from the app itself.
In each of these cases, you would want to ensure that the origin of the messages is the app itself, and not just a user who is running a simple script to fake the data.
Some ideas I've considered:-
SSL - Works well for securing the channel and preventing tampering (which fulfils some of the requirements) but still cannot ensure the integrity of the source of the data.
Public-key cryptography - The client app could encrypt the data with a private key and then transmit it to the server where it can be decoded. The problem is that the private key needs to be hardcoded within the app -- the app could be decompiled and the private key extracted and then used to send fake data.
Home-made algorithms - A very similar question to this is asked here where the solutions only work until "someone figures out your algorithm" -- i.e. not a great solution!
Hash chain - This seemed like a really interesting way of using one-time keys to verify each data payload from the client to server, but again it relies on the app itself not being decompiled, because the password still needs to be stored by the app.
My limited knowledge of cryptography makes me think that it's actually theoretically impossible to build a system that would be totally verifiable in this manner, because if we cannot trust the end client or the channel, then there is nothing on which to base any trust... but maybe there's something I've overlooked!
It's not that hard, you just need to authenticate the app. You can do this with a simple user and password (over SSL) or use client authentication. In both cases, the credentials need to be in the app, and an attacker can extract them and impersonate the app. You have to leave with it and maybe implement some methods to mitigate it.
You can also authenticate the messages, by having them signed either with an asymmetric key (RSA, etc.) or a symmetric one (HMAC, etc.). A nonce helps against replays, where someone captures a validly signed messages and sends it to your server over and over again. Depending on your protocol, the overhead of using one might be too much.
To protect the credentials, you can have the client generate them and save them in the system KeyStore, although it is not quite supported by a public API, see here for some details. This, of course, requires an extra step where you need to send the generated credentials (say, public key) to your server securely which might be tricky to implement properly.
Whatever you do, don't try to invent your own cryptographic algorithm or protocol, use an established one.
Related
We are getting started with developing an android app and the corresponding REST APIs and I need to figure out a security model for the same. I've close to zero experience with designing secure systems and would like some expert opinion on the loopholes of a first draft we've come up with.
I've been all over the web for the past few days and everyone seems to suggest HTTPS and OAUTH as the proven answer. Since our app doesn't deal with anyone's bank account, I think we can live with less than DoD grade security (although even they get hacked often!). And we don't want to spend the effort for OAUTH unless there really is no other reasonable alternative.
We're trying to avoid HTTPS because the app will, at times, be polling the server every few seconds and we thought it'd be too expensive to use it for all REST calls. Also, the payload for some of those API calls can be too big (2-4 KBytes) for asymmetric encryption.
Here's what we've lined up so far:
User creates an account by entering a unique 'username' and a 'password' on the registration page in the app
The 'username' is stored in plaintext in SharedPreferences using MODE_PRIVATE
The SHA-256 of the 'password' is also stored in SharedPreferences using MODE_PRIVATE
The user credentials ('username' and hashed 'password') are sent to the server using https://
The server creates an authentication "token" (a random AES key, really, using a CSPRNG), stores it in its DB and also sends it back to the client (using https, of course)
The AES-256 key is then stored by the app in the SharedPreferences using MODE_PRIVATE
All further communication between the app and the server is done over http:// with encrypted (payload (json/xml) + timestamp + checksum/hash) (CBC with random IV)
The AES key is only updated if the user changes his password
For actions that require additional security, the app asks the user to re-enter his password which is verified against the stored hash
The app should be usable offline (It can talk to pre-registered embedded devices over a WiFi connection. Security over WiFi is another story!)
I know some of the pitfalls of the system already:
Storing the key on the phone isn't safe: If a hacker gets access to the user's phone, the user just needs to change his password and everything will be safe.
Storing keys on the server is bad: A lot of people seem to say if you really have to store the keys, at least store them on a separate server. But that adds an extra round trip between the servers for every REST call. And there can potentially be many of them when the app is polling.
Keys without expiry are bad: I can't think of another way to let the app function offline.
The real questions now are:
What are the other loopholes that I've missed so far?
What kind of effort would it take for someone to break into the system?
Most important of all, how can we improve overall security to some "reasonable" standard without overdoing it?
This is not DoD security!
You really do need to use https and insure it is setup for TLS 1.2 and Perfect Forward Secrecy. Additionally the app needs to pin the certificate.
Section 1:
3: Do not use SHA256, use PBKDF2, crypt of another hash that has an increased work factor.
4: Send the password, not the hashed password to the server, the server does the hashing.
7: When using https there is no need to encrypt the payload, that is what https does.
Section 2:
2: When storing keys on the server keep them out of any http accessible directory. This is a weak point and needs to be addresses with server security.
Section 3:
Use two factor authentication for administration of the server. Have a good scheme to control the 2nd factor, I like hardware tokens and keep track of them by their serial numbers. That way there is a limited number and they can be recovered when someone is no longer should have administrator access. They can also be loaned for short periods of time.
You also need to have disaster plans for various contingencies, do not wait for an incidence and try to deal with it on the fly. Some times appropriate immediate action is required.
All of this is basic.
You need to evaluate potential threats, attackers and the value to an attacker or user.
If you care about security and are not a domain expert hire one for advice and review, I do.
Aside: DoD security: Two guard stations, two overhead passages between buildings, the last building has one door that is a huge safe door and there are no windows. Ceiling bubblegum lights rotating when there are un-cleared personal in the building, one escort per un-cleared person who follows you everywhere including into the bathroom, multiple sensors in the ceiling, tempest shielding.
I'm building an Android app that contains sensitive chat messages.
I'd really appreciate some help regarding an encryption workflow that allows me to encrypt these messages, store them in a remote database, query for them via Angular.JS and finally decrypt them and present them to the user.
The server must not be able to decrypt the messages. Only both Android and Angular.JS clients should be able to encrypt and decrypt the data, and the encryption key should be unique for each of my users. Both clients can send messages, so both need the ability to encrypt and decrypt.
Is there any way to get this done without requiring the user to enter a custom "Encryption Key" in both clients? Is there any way for this to be automatic in some way, and without involving the server? If not, what are the best-practices in this condition? I wasn't able to find any example of this kind of encryption in any wide-known service as of today.
Thanks!
You're asking about how to do key exchange without revealing the key to the network, right?
Diffie-Hellman key exchange is one well known algorithm for doing this. The important high level properties are that the two parties, in the end, agree on a shared secret that a passive eavesdropper can't get. However, the parties don't authenticate each other, so they can't tell if they're running the algorithm with a man in the middle (e.g., the server in your question).
I've seen products use password-authenticated key exchange. As the name suggests, these algorithms require that both parties (in this case, the same user, but on different devices) know a password. So ultimately, going with this approach requires the user to enter a password on one of the devices (the other can generate it and display it to the user). It's a little less troublesome than entering an entire encryption key into both devices, right?
As for technical implementations, it's still probably going to involve the server (or a server, if not the database server) just to relay messages, but these key exchange algorithms should keep the shared secret confidential.
I wasn't able to find any example of this kind of encryption in any wide-known service as of today.
One great resource I've found is a page from Mozilla's wiki on how they implemented key exchange in their Firefox Sync product. They use this when you set up Sync on multiple devices, which requires the second device to get the key from the first device.
This is a messaging app. It has no login (username or password). This means it has to send messsages to a server, and the server must trust it is coming from the phone number it says it is coming from.
How do you do that?
1) Send token to phones with text message service
Unfortunately, you're not going to be able to guarantee with 100% certainty that the phone number reported to the server is the one it's coming from. The reason is that the client code can be reverse engineered, regardless of what you do. There are some things you can do however to make it a little more difficult, although it is important that you don't depend on this to be 100% secure. If you need 100% assurance, then you need to have the app authenticate to the server through traditional means.
The way that I would do this, is embed a token in each copy of the client that gets released, such that each client has a different token. The token should have extremely large entropy (such as a 128-bit or greater integer), and you should keep a list on the server of tokens you've issued with the phone number of the device, so you can check them for validity. Require re-installs of the app to use a new token, and blacklist the previous token so it can no longer be used. The UID can be used, but be advised that it can easily be spoofed by a rooted device.
Submit this token to the server each time and ensure that the phone number the app claims to have never changes. To make it harder for reverse engineers to find the token in your code, you can xor it one or more times with additional tokens, and you can also lay out a bunch of fake tokens throughout the code that are blacklisted on the server.
You can also encrypt it with a secret key that is retrieved from the server so that the embedded token must be retrieved by an RE during live interaction with the server. Again this in no way guarantees that the token won't be found and changed/stolen, but it raises the bar for potential reverse engineers.
To prevent someone from sniffing the wire and obtaining and/or tampering with your token, or from using a proxy like Burp Suite to capture/tamper with it, you should use an encrypted HMAC. If you've never used an HMAC before, be advised that unless you encrypt it, it only provides authentication and integrity, not confidentiality.
EDIT:
Should also add, that you should run your code through an Obfuscator before you deploy it. This won't obfuscate the token, but it will obfuscate the decompiled code so it looks like gibberish to the RE. This forces the RE to use the byte code/assembly code from your app, which is much, much harder.
Associate each phone number with a unique device identifier
Get the udid and send it with each request
Create a signature that follow each request to your server. The signature shoukd be something like: secret key1+msg+phone number+udid+secret key2, then SHA1 the string and attach to th request.
On server calculate the signature and compare to original that followed the message. If they match, ok, else don't send it.
Use strong keys, and use two, to make brue force extraction, almost impossible.
Suppose you have a mobile application (Windows Phone or Android) that connects yo your back-end using SOAP.
For making it easy, let's say that we have a Web Service implemented in C#. The server exposes the following method:
[WebMethod]
public string SayHallo() { return "Hallo Client"; }
From the server perspective, you can't tell if the caller is your mobile application or a developer trying to debug your web service or a hacker trying to reverse engineer/exploit your back-end.
How can one identify that the origin of the web service call is THE application? as anyone with the WSDL can invoke the WS.
I know I can implement some standard security measures to the web service like:
Implement HTTPS on the server so messages travel encrypted and the danger of eavesdropping is reduced.
Sign the requests on the client-side using a digest/hashing algorithm, validate the signature in the server and reject the messages that have not been signed correctly.
Write custom headers in the HTTP request. Anyways headers can be simulated.
However, any well experienced hacker or a developer who knows the signing algorithm, could still generate a well signed, well, formatted message. Or a really good hacker could disassemble the application and get access to the hidden know-how of my "top secret" communications protocol.
Any ideas how to make the SayHallo() method to answer ONLY to request made from my mobile application?
We are running in the context of a mobile application, with hardware access, there could be something that can be done exploiting the hardware capabilities.
If someone wonders, I'm thinking on how to make a mobile app secure enough for sensitive applications like banking, for example.
Thanks for your ideas.
What you are describing is bi-directional authentication. It can only be done by storing a signed public key (certificate) on boths sides of the communication. Meaning that each app would need to authenticate your server with your servers public key and the server would need to authenticate each instance of your app. The app's public key would need to be produced and stored on the server at the deployment time with each instance of your app. Think of this as 2 way HTTPS, in general the only authentication that needs to be done is one direction, with the browser authenticating the server with a trusted signing key. In your case this would need to be done on both sides. Normally you would have a service like VeriSign sign each instance of a public key, this can get quite spendy with multiple deployments of an app. In your case you could just create an in house signing application using something like OPENSSL to sign your app every time it is distributed. This does not mean that someone still could not hack your code and extract the signing key on the app side. In general any code can be hacked, it's just a question of how hard can you make it before they give up? If you were to go the custom hardware route, there are things such as crypto chips and TMP's that can serve as a key sotre and make it harder for people to gain access to the private keys on the device.
A quick google search turned up the following:
http://www.codeproject.com/Articles/326574/An-Introduction-to-Mutual-SSL-Authentication
If you are thinking about using rewards points, and are really worried about someone gaming the system from the outside a better solution is to have each person make an account that is stored securely on the server and their points are saved and tallied there. This centralizes all the data and allows you complete control over it with out worrying about a malicious app reporting non-existent points. (this is how banks work)
If you want to verify that a user is both mobile and who they say they are then the best way is to leverage the network. Send a push notification with the hashed key that you want the user to use via:
APN for iOS
something like urban airship for windows phone
GCM for Android.
In general, the model looks like:
Server authenticates itself to the many clients with a certified public key (this is the whole Public Key Infrastructure, Certificate Authorities, etc)
Each client identifies itself to the server via some other authentication system (in 99.9% of cases, this is a password)
So if you're wondering how this sort of thing works in the case of banking apps, etc that's basically how it breaks down: (1) Client and server establish a secure channel such as a shared secret key, using the server's public key, (2) Client authenticates via this secure channel using some other mechanism.
Your question specifically, however, seems more aimed at the app authenticating itself (i.e., any request from your app is authentic) with the thought that if only your app can be authenticated, and your app is well-behaved, then everything should be safe. This has a few implications:
This implies that every user of your app is trusted. In security terms, they are part of your "trusted computing base".
Attempting to achieve this sort of goal WITHOUT considering the user/user's computing platform as trusted is essentially the goal of DRM; and while it's good enough to save money for music publishers, it's nowhere close to good enough for something really sensitive.
In general:
The problem that you're specifically looking at is VERY hard to solve, if you're looking for very strong security properties.
You likely don't need to solve that problem.
If you give us some more context, we might be able to give you more specific advice.
In addition to the answers already given, how about using a login type scheme with the unique id of the phone and a password? Get the user to register with your "back-end" and every time a transaction has to be made with the back-end, require the password or have an option to automatically log in.
You can use the following way to secure and to track your requests to server.
You can force the mobile or web clients to send you a Custom Headers of the device type when accessing your webservice through REST methods
Use basic http authentication by forcing each client to have their own username and passwords provided by you as a authorized webservice provider..
If you need more advanced protection you can use OAuth 2.0 to secure your webservice.
Since your originating app is going to be Android or Windows Phone apps, either one of them will be relatively easy for the wanna be hacker to debug. in any case you're going to be running the code on a machine that you have no control over so no ssl tricks or checking signing will solve your fundamental problem.
the only way you can combat threat from it is to NOT TRUST THE CLIENT. verify that the input coming from the clients is valid before acting on it if you're making a game - that it's accompanied by a valid security token etc.
in essence build your service so that it doesn't matter if the user is using an unofficial client.
I have read a few examples on SO for securing client / data. But we have a little bit different issue, and not sure where to look.
Basically we have an android game which is a geo-location based game. We use HMAC-SHA1 to the query string to verify that the data being sent from the client is in-fact from the client. There is a small issue. The HMAC-SHA1 key. I can obfuscate till my hearts content, but the key remains in the app. Someone can easily de-compile the app, grab the key, and then send manual queries by a browser for their user account (spoofing GPS).
I saw one example where someone suggested client & server side ssl authenication. Not sure how that would work, would you not just need to attach a ssl cert to the app? Would this not be subject to de-compiling also, it would require the end user to re-compile / use the cert?
Can we some how use the package manager to get the self signed cert? I need to find out the correct way to secure our transmission so someone can't fake their transmissions for their own user account..
Thanks
To authenticate the client, it needs some form of credentials. You can either:
don't save the credentials and have the user input them every time
save them somewhere
use system credentials
use some form of an identity provider
1 is inconvenient, 2 i subject to attacks as long as someone has physical access to the device. For 3, you could use the user's Google account so you can be (pretty) sure who they are and block them if there are any problems/attacks. 4 really a variation of 3: the user will authenticate to some third-party service and it will only issue an (temporary) access token. So if the account is compromised the token will eventually expire and/or be revoked (look into OAuth). Consider the risks and amount of work to implement and take your pick.
As for using client certificates, you can store them encrypted, so you need to provide a passphrase to use them. On pre-ICS you need to implement this yourself, on ICS you can use the system key store via the KeyChain API. You will only get access to the private key after you unlock the device (uses the unlock password/PIN to protect keys) and the user explicitly grants permission.
If you want to stick to you current way of doing things, implement the HMAC part in native code (OpenSSL, etc.), and generate the key at runtime by combining bits of it. That would make it fairly hard to reverse engineer. Additionally, you might want to add some sort of a nonce, so that requests cannot be replayed.