I have developed an android app. This app sends data to my webserver(apache + mysql) using the HttpClient and HttpPost classes.
But this way a malicious atttacker could be able to send a custom post request to the webserver and corrupt my database.
1 - Is it possible to encrypt the data before the app sends it and then decrypt it in the webserver with some shared key algorithm? Or am I saying nonsense?
2 - If the previous solution is not a good one, what do I need in a summarised way to implement the ssl solution?
I have read many articles about ssl and android but I am still a bit confused. I guess I have to make code changes on both app and apache. Can anyone tell me about some tutorial to deal with this?
You don't necessarily need to encrypt your data (that's only needed if you want to prevent data from being read by an attacker).
What you need is:
Authentication: So only requests from trusted users (ie. your app) are accepted
Validation: So only "correct" requests are processed
Authentication can be as easy as setting up HTTP Basic Access Authentication on your Apache server. You'll set up a user and password, and have your app use these credentials to access the server. Any unauthenticated request will be rejected with a 403.
Unfortunately Basic Authentication is mostly insecure since anyone looking at the traffic between your app and server can grab the credentials, then forge their own requests.
OAuth would be the better option, although it is more involved. Here's a nice tutorial that covers the client side: http://nilvec.com/implementing-client-side-oauth-on-android.html
Validation means you'll need to sanitize data before using it. Your server app should assume that all data is potentially wrong or dangerous, and properly filter any input before processing it.
Related
I have to lay out a plan to develop a RESTful API (Python/Flask) that could be used by our future web app (Angularjs) and mobile apps (iOS/Android).
I have been researching for three days and have come across several scenarios:
Using HTTPS is one way on top of the methods below to keep it safer. But https is slower, which could mean we need faster and more expensive servers.
Using Basic-Http-Auth and sending username/password in plain (yet https) over the wire for every request to the API.
Using Digest-Auth, which is a hash of the password and the tracking would be automatic This would work for the web app, however I wasn't able to confirm if iPhones and Android would support this natively. If they do, that could be an easy solution!
Using a custom http header, where I would send a custom Auth string in http header upon a successful authentication. But then I have to make sure I am sending this auth code for every request that the user makes. This makes it exactly like 1) with the difference that plain passwords aren't used and the auth code can expire without any risk. Also problematic is the tracking of the auth code, which is no longer automated as in 2)
Using OAuth is an option. But its quite difficult to set up. If there is no better way, maybe thats the only way?
Securing the API like Amazon S3 as described in this great article. In short, he says that both server and client would know of a private key, which they would use to hash the communication. It will be like gangster handshake, that you only would trust the delivery boy, if he knows the gangsta handshake. Further down the comments someone asks:
How to keep the private key “secure” in a pure HTML5 app ?
You are exactly right; in a pure HTML5 (JS/CSS/HTML) app,
there is no protecting the key. You would do all communication over
HTTPS in which case you wouldn’t need a key since you could safely
identify a client using a standard API_KEY or some other friendly
identifier without the need or complexity of an HMAC.
So in other words there is even no point of using the method for an web app in first place. And honestly I don't understand how this should work on the mobile device either. A user downloads our app and how do I send the private key from the iphone to the server? The moment I transferred it, it will be compromised.
The more I am researching the more indecisive I am getting.
I was hoping to ask some pros who have done this previously and could share their experience. Many Thanks
You seem to be confusing/merging two different concepts together. We start of talking about encrypting traffic (HTTPS) and then we start talking about different ways to manage authenticated sessions. In a secure application these are not mutually exclusive tasks. There also seem to potentially be a misunderstanding how session management can impact authentication. Based on that I will provide a primer on web application/web api session management, authentication, and encryption.
Introduction
Session Management
HTTP transactions are stateless by default. HTTP does not specify any method to let your application know that a HTTP request has been sent from a specific user (authenticated or not).
For robust web applications, this is not acceptable. We need a way to associate requests and data made across multiple requests. To do this, on initial request to the server a user needs to be assigned a "session". Generally sessions have some kind of unique id that is sent to the client. The client sends that session id with every request and the server uses the session id sent in every request to properly prepare a response for the user.
It is important to remember that a 'session id' can be called many other things. Some examples of those are: session token, token, etc. For consistency I will use 'session id' for the rest of this response.
Each HTTP request from the client needs to include the session id; this can be done in many ways. Popular examples are:
It can be stored in a cookie - cookies for the current domain are automatically sent on every request.
It can be sent on the URL - each request could send the session id on the URL, not suggested since session ids will stay in the clients history
It can be sent via as a HTTP header - each request would need to specify the header
Most web application frameworks use cookies. However application that rely on JavaScript and single page designs may opt to use a HTTP header/store it in some other location that is observable by the server.
It is very important to remember that the HTTP response that notifies the client of their session id and the client's requests that contain the session id are completely plain text and 100% unsafe. To battle that, all HTTP traffic needs to be encrypted; that is where HTTPS comes in.
It is also important to point out we have not talked about linking a session to a specific user in our system. Session management is just associating data to a specific client accessing our system. The client can be in both authenticated and unauthenticated states, but in both states they generally have a session.
Authentication
Authentication is where we link a session to a specific user in our system. This is generally handled by a login process where a user supplies credentials, those credentials are verified, and then we link a session to a specific user record in our system.
The user is in turn associated with privileges for fine grained access control via access control lists and access control entries (ACL and ACE). This is generally referred to as "Authorization". Most system always have both Authentication and Authorization. In some simple systems all authenticated users are equals in which case you won't have authorization past simple authentication. Further information on this is out of scope for this question, but consider reading about ACE/ACL.
A specific session can be flagged as representing an authenticated user in different ways.
Their session data stored server side could store their user id / some other flag that denotes that the use is authenticated as a specific user
Another user token could be send to the client just like a session id (which over unencrypted HTTP is just as unsafe as sending a session id unencrypted)
Either option is fine. It generally comes down to the technology you are working in and what they offer by default.
A client generally initiates the authentication process. This can be done by sending credentials to a specific url (e.g. yoursite.com/api/login). However if we want to be 'RESTful' we generally would referencing a resource by some noun and doing the action of 'create'. This could be done by requiring a POST of the credentials to yoursite.com/api/authenticatedSession/. Where the idea would be to create an authenticated session. Most sites just POST the credentials to /api/login or the like. This is a departure from "true" or "pure" RESTful ideals, but most people find this a simpler concept rather than thinking of it as "creating an authenticated session".
Encryption
HTTPS is used to encrypt HTTP traffic between a client and server. On a system that relies on authenticated and unauthenticated users, all traffic that relies on a user being authenticated needs to be encrypted via HTTPS; there is no way around this.
The reason for this is that if you authenticate a user, share a secret with them (their session id, etc) and then begin to parade that secret in plain HTTP their session can be hijacked by man-in-the-middle attacks. A hacker will wait for for the traffic to go through an observed network and steal the secret (since its plain text over HTTP) and then initiate a connection to your server pretending to be the original client.
One way people combat this is by associating the requests remote IP address to an authenticated session. This is ineffective alone as any hacker will be able to spoof their requests remote IP address in their fake requests and then observe the responses your sever is sending back. Most would argue that this is not even worth implementing unless you are tracking historical data and using it to identify a specific user's login patterns (like Google does).
If you need to split up your site between HTTP and HTTPS sections, it is imperative that the HTTP traffic does not send or receive the session id or any token used to manage the authentication status of a user. It is also important that you do not send sensitive application data within non-HTTPs requests/responses.
The only way to secure data within web applications/APIs is to encrypt your traffic.
Your Topics One By One
Basic-Http-Auth
Authentication: YES
Session Management: NO
Encryption: NO
This is a method for authenticating by web resource only. Basic authentication authenticates uses by resource identified by URL. This was most popularly implemented by Apache HTTP Web Server with the use of .htaccess based directory/location authentication. Credentials have to be sent with each request; clients generally handled this transparently for users.
Basic authentication can be used by other systems as a mode of authentication. However, the systems that utilize Basic-Http-Auth are providing authentication and session management, not the Basic-Http-Auth itself.
This is not session management.
This is not encryption; content and credentials are nearly 100% plain text
This does not secure the contents of the application's HTTP request/responses.
Digest-Auth
Authentication: YES
Session Management: NO
Encryption: NO
This is exactly the same as Basic-Http-Auth with the addition of some simple MD5 digesting. This digesting should not be relied upon instead of using encryption.
This is not session management.
This is not encryption; the digest is easily broken
This does not secure the contents of the application's HTTP request/responses.
OAuth
Authentication: YES
Session Management: NO
Encryption: NO
OAuth just lets you have an external service validate credentials. After that it is up to you to manage/work with the result of authentication request to your OAuth provider.
This is not session management.
This is not encryption; your sites traffic is still plain text. The authentication process will be secure due to HTTPS restrictions, but your application is still vulnerable.
This does not secure the contents of the application's HTTP request/responses.
Gangster Handshake / Custom HTTP header
Authentication: YES, potentially
Session Management: YES, potentially
Encryption: NO
"Custom HTTP header" is a type of "Gangster Handshakes"; as such I will use the same section to discuss them. The only difference is that a "Custom HTTP header" is specifying where the hanshake (session id, token, user authentication toke, etc) will be stored (i.e. in a HTTP header).
It is important to note that these do not specify how authentication will be handled, nor do they specify how session management will be handled. They essentially describe how and where session ids/authentication tokens will be stored.
Authentication would need to be handled by your application or via a third party (e.g. OAuth). Session management will still need to be implemented as well. The interesting thing is you can choose the merge the two if you wish.
This is not encryption; your sites traffic is still plain text. The authentication process will be secure due to HTTPS restrictions if you use OAuth, but your application is still vulnerable.
This does not secure the contents of the application's HTTP request/responses.
What You Need To Do
...I highly suggest you make sure that you understand that a robust web application that is secure needs the following:
Encryption (HTTPS is pretty much your only choice)
Session Management
Authentication / Authorization
Authorization relies upon Authentication. Authentication relies upon Session Management and Encryption makes sure the session isn't hijacked and that the credentials are not intercepted.
Flask-Login
I think you should look into flask-login as a way to avoid re-implementing the wheel. I have personally never used it (I use pyramid for web applications in python). However, I have seen it mentioned before in web application/python boards. It handles both authentication and session management. Throw your web api/application through HTTPS and you have all three (Encryption, Session Management, and User Authentication).
If you do not / can not use flask-login, be prepared to write your own, but do research first on how to create secure authentication mechanisms.
If at all possible, if you do not understand how to write an authentication procedure please do not attempt it without first learning how hackers use pattern based attacks, timing attacks, etc.
Please Encrypt Your Traffic
...move past the idea that you can avoid using HTTPS with some "clever" token use. Move past the idea that you should avoid using HTTPS/encryption because "its slow", process intensive, etc. It is process intensive because it is an encryption algorithm. The need to ensure the safety of your user's data and your applications data should always be your highest priority. You do not want to go through the horror of notifying your users that their data was compromised.
The https it is slower, but not a not.
Only the handshaking is slower. For us the biggest problem it is to upkeep the key pair on server-mobiles side and the rights.
We have implemented a message digest too. The problem it is: is hard to set up the php-android-ios version properly. After this is done ( a parameter need to changes what is suggesting Google at first results only at android side) the problem will be with low-end devices: to much CPU usage, slow on decrypt-encrypt process, a lot slower than https, especially when you need to transform 10kb String(can take several minutes).
If I don't transfer Nasa data to Hamas, than I would go with a very simple encryption over simple HTTP: like invert the bits or so...
Go with HTTPS. It's (marginally) slower, but the security you get from it for the relatively short investment time (purchasing the SSL cert and just changing your URLs from http to https) is worth it. Without HTTPS, you run the risk of your users' sessions getting hijacked on unsecured public networks, which is extremely easy for someone to do.
I've searched everywhere, and found lots of information I cannot comprehend. I'm using a WAMP server and managed to execute a "SELECT name FROM Tablename" query and passed the data to json_encode(). I like the results so far, but now I need to protect the JSON file in the server, making it accessible only to users that run my Android app.
Through my research I found that REST might be a solution for me but I do not understand how I can implement it for my case. Is it possible to have Server-side REST Security, and Client-side as well? I understand that REST is a web service and I read a tutorial where the web service is basically a web page. My priorities are server-side json file, security and speed. The user will not be inserting any information via the Android app. I was thinking of deploying the Android application with the user and password to the specific json file (verification).
It would be helpful if you can point me to a video tutorial, or a tutorial for beginners, related to the subject.
Here are my specific questions?
Can I parse images with JSON?
Is it more efficient to mysqldump --> convert .csv file ---> SQLite? (Securely).
How big can a .CSV file get with say 1 million entries in the database?
How can I accomplish all this?
Please help, thanks.
Solution: Use HTTP Headers
Insecure solution:
use the User-Agent header.
User-Agent: MyAndroidApp/1.0
More secure solution:
First off, you'll need to use SSL so no one can just easily see your secret key.
Second, you can put a secret key in the HTTP header of the request you make from the android app:
X-Android-Secret-Key: fee400be-7d08-45c5-bf7c-ff79c35a838c
You check headers on the server and only serve the file back if the desired header is received. You keep the header somewhat secret but not impenetrable with SSL.
I'm only going to answer your original question on app authentication. Your other questions belong as separate questions.
If it's only your client and your server, you can (and should) use mutually-authenticated SSL without purchasing anything. You control the server and the client, so each should only trust one certificate, the one belonging to the other and you don't need CAs for this purpose.
Here's the high-level approach. Create a self-signed server SSL certificate and deploy on your web server. You can use the keytool included with the Android SDK for this purpose. Then create a self-signed client and deploy that within your application in a custom keystore included in your application as a resource (keytool will generate this as well). Configure the server to require client-side SSL authentication and to only accept the client certificate you generated. Configure the client to use that client-side certificate to identify itself and only accept the one server-side certificate you installed on your server for that part of it.
A step-by-step for this is a much longer answer than is warranted here. I would suggest doing this in stages as there are resources on the web about how to deal with self-signed SSL certificate in Android, both server and client side. There is also a complete walk-through in my book, Application Security for the Android Platform, published by O'Reilly.
You'll normally store that certificate/private-key in a keystore of sometype (a KeyStore if you're using Android) and that keystore will be encrypted. That encryption is based on a password, so you'll either need to (1) store that password in your client somewhere, or (2) ask the user for the password when they start your client app. What you need to do depends on your usecase. If (2) is acceptable, then you've protected your credential against reverse engineering since it will be encrypted and the password will not be stored anywhere (but the user will need to type it in everytime). If you do (1), then someone will be able to reverse engineer your client, get the password, get the keystore, decrypt the private key and certificate, and create another client that will be able to connect to the server.
There is nothing you can do to prevent this; you can make reverse engineering your code harder (by obfuscation, etc) but you cannot make it impossible. You need to determine what the risk you are trying to mitigate with these approaches is and how much work is worth doing to mitigate it.
I'm building an Android application which has to sent some information to my mysql database. The mechanism I'm trying to implement is based on JSON, php, Mysql combination. Unfortunately I'm not a veteran when it comes for those subjects. As I understand correctly the php-Mysql connection is always secure - nobody except me can see the source of php script in which I have written username and password to my database. Now the tricky part, my php script is located on Apache server and it isn't protected at all, therefore anybody can trigger it (even from the desktop browser). How can I prevent this situation? and how can I safetly trigger my php script from my Android device?
Thanks
Use SSL. This will encrypt the connection between the device and the server.
Use a client id/key for your device that is verified on the server.
In case you REALLY worry that someone will modify your app to send fake calls using such: verify the client certificate as well (piggy back). (The same way it is done with Facebook Android library and Google mobile libraries).
Use ssl, this will encrypt the connection
Set an authenticate mechanism, on your php page and you android application will send the credentials
Set a random pin code that the server side sends to the application, and he is valid only to the current session, and the application need to run a function that will generate the right answer to this current number and sends it to the server as verification, for example if the server sends me the pin number:120, and the verification function of mine is to +1 the pin number I will send the server 121, but I suggest to use a little bit more complicated algorithm.
The Android device is no different than any other HTTP client, like your browser. You need to follow the same mechanisms you will be using in order to protect a standard Web Page:
Require login to the page. The user needs to supply a valid username and password to gain access. The server returns a session, which is usually stored in a cookie. This question will help you on how to do that on Android.
To keep someone from intercepting the username and password, the log-in should be done over HTTPS
the most intuitive way is to authenticate the user (Username + Password) using an Https Connection, there is better types of secure authentication like OAuth, see this: http://code.google.com/p/oauth-signpost/
I have a server side script which gets some data that my application uses. Naturally, I don't want anyone else access the data aside from my app. I've heard it's possible to see which url's the device connects when using a certain software. How can I prevent these programs seeing the url's I'm calling to? Or is there a better way of securing the requests?
Only thing I can think of is using a password key in the url (and check if it matches on the server side):
http://example.com/getdata?key=897ihrduiuyqewudiew&get=something
but that probably isn't enough for a secure authentication. And the sniffer programs could still get that url. Any simple way of doing this more securely?
The easiest way is to use HTTPS. This way, only the server to which you connect to can be known by the sniffer.
There are other methods that use complex challenging to have a unique key only valid for a short period of time / a single request, like WSSE (see this article http://www.xml.com/pub/a/2003/12/17/dive.html )
There is very little you can do in order to protect the server requests. Someone will always be able to see the URLs your application hits and using a password in the query string won't help. In order to secure your application you need to use HTTPS and some form of authentication. The user will need to provide a username and password in order to connect.
If I understand you correctly, you should implement mutual authentication. Basically, you have certificates on both your client and server. When a request is made to the server, the server verifies that request is signed by a known client.
So, even if a sniffer knows the url and attempts to issue the same request to the server, it would be rejected since it is not signed by a known client. I am quite new to this as well, but that is the general concept. This blog has the basic steps.
http://blog.callistaenterprise.se/2011/11/24/creating-self-signed-certificates-for-use-on-android/
http://blog.callistaenterprise.se/2011/11/24/android-tlsssl-mutual-authentication/
I am in the planning phase a new project. I want to be able to control multiple relays from my android powered phone over the internet. I need to use an HTTP based server as a middleman between the phone and the relays. Django is my preferred platform because Python is my strongest skill set. This would not be a "web app" (with the exception of the admin interface for managing the user and their access to the relays). Rather, the server would simply provide an API in the form of HTTPS requests and JSON encoding. Though, I should note that I have never done any web development in my life, so I don't know best practices (yet). The authentication method should meet the following criteria:
Works over HTTPS (self-signed SSL)
Provides multi-factor authentication (in the form of something you have and something you know)
Be reasonably secure (Would be very difficult to fool, guess at. or otherwise bypass)
Is simple in implementation for the server operator and end user on the mobile client
Is lightweight in in terms of both CPU cycles and bandwidth
I plan to use the following scheme to solve this:
An administrator logs into the web interface, creates a user, and sets up his/her permissions (including a username and a password chosen by the user).
The user starts the client, selects add server, and enters the server URL and his/her credentials.
The client attempts to authenticate the the user via HTTP auth
(over SSL). If the authentication was successful, the server will generate an API key in the form of a UUID and sends it to the client. The client will save this key and use it in all API calls over HTTPS. HTTP auth is only used for the initial authentication process prior to reviving a key, as a session scheme would not be nessessary for this application. Right? The client will only work if the phone is configured to automatically lock with a PIN or pattern after a short timeout. The server will only allow one key to be generated per user, unless an administrator resets the key. Hence, simple, mobile, multifactor authentication.
Is this sound from a security standpoint? Also, can anyone point me to an example of how to use the HTTP auth that is built into Django? From a Google search, I can find a lot of snipits witch hack the feature together. But, none of them implement HTTP auth in the wayit was added to Django in 1.1. The official documentation for REMOTE_AUTH can be found here, but I am having difficulty understanding the documentation as I am very new to Django.
I'm not entirely sure of how basic auth would work on Django, but I can take a shot.
The basic auth article on wikipedia covers a pretty standard usecase for logging in. For Android I've personally skipped the first part (401) and just pass my credentials in right away.
With your auth request you will have to just grab the user credentials from the request headers (WWW-Authenticate) and then do all the necessary work for that. With the credentials you can then just use the authentication framework provided in Django to verify that the user then generate their UUID (I guess).
As for basic auth on Android it's a little bit tricky at first and may leave you pulling your hair. I've found this article on Basic HTTP auth for android which helps explain how to do it.
As for the security part of it, I'm not too sure. It's pretty simple, which I'd say is a good thing :)