I recently bought an Android device. Now I'm wondering if can I mimic protocols it uses to communicate with Google servers?
I basically want to setup some kind of "Google account", which wouldn't be served by Google, but would be fully compatible with Android devices. So, does Android use some kind of WebDAV protocol for accessing things like calendar, contacts? What kind of protocol does it use for mail (is it IMAP, as I would configure my account on a PC or some other Google-only-knows-what-is-it protocol?)
Or do I just have to mimic GData protocols?
Is there even a way to change host which Android talks to?
I know that there are things like Google Apps. They allow you to setup your own, very little part of Google, which AFAIK can be connected to Android device (you just have to create an Google account with your domain after username, I suppose), but everything's still hosted on Google servers and Android still talks to Google host.
If nothing works out, I could probably create some kind of service provider, which would act like those for Facebook, Twitter and Google, but for now I want to explore possibility of doing it on the server-side.
Not that I don't trust Google. I just don't really like someone handling valuable part of my life in files I don't own. Assume this question void if someone found a way of chowning files on Google servers ;).
No, you can not "redirect" google apps on Android to talk to your servers.
Google exposes their Apps (gmail, calendar, docs, etc..) via various APIs (GDATA), so I suppose their Android apps use those.
Even if you "mimic" those protocols, you could not redirect the apps, because AFAIK they use SSL.
Just create your own client and server software, secure the connection via SSL and you are all set. You even don't have to write the software as there are thousands open-source server apps for email, calendaring, doc sharing, etc..
If you are thinking of using Google clients without their servers and proposing a roll-your-own replacement, than you clearly do not understand the complexity of developing such a service. Do you realize there are thousands of top-notch devs working daily on this?
If you dont trust application service providers (Google, Facebook, etc..) than don't use their services. Same goes for other service providers like credit card companies, banks, mobile, telco, etc..
If you have a rooted phone it could be quite cool to mimic google server. For example by changing the calendar https url in the sqlite database to your own server.
For calendar, I guess this is Caldav, but should be written down somewhere officially.
And yes, their are caldav-sync tools for Android, but they all suck completely (Hypermatic sucks less, but it has been abandoned, and is not open source)
Related
I have a website that store private information that can be accessed by request with a secret api key.
My Android application have to access that private information, and to do that it use a proxy server that store and use the secret api key to communicate with the website
The problem is that just using Wireshark, or finding the string in the app resources file, someone can see the proxy server url and use it to get the private data from the website
How can i make this system secure? how can i be sure that none else can use the proxy except the Android app?
Thank you!
I have a website that store private information that can be accessed by request with a secret api key.
From the moment you put a secret on the client is not anymore a secret, because now is public and anyone can see it by reverse engineer the app or by intercepting the traffic with a proxy or by just watching the traffic with the tool you mention, Wireshark.
My Android application have to access that private information, and to do that it use a proxy server that store and use the secret api key to communicate with the website
Using a proxy server doesn't solve the problem, because the proxy server doesn't know WHAT is calling it. As it stands the proxy server is open to the public that knows how to call it, as you already know and pointed out:
The problem is that just using Wireshark, or finding the string in the app resources file, someone can see the proxy server url and use it to get the private data from the website.
Another approach is to use a proxy tool like the MiTM Proxy that in my opinion will allow to extract more easily the API key, as you can see in the article Steal that API Key with a Man in the Middle Attack:
While we can use advanced techniques, like JNI/NDK, to hide the API key in the mobile app code, it will not impede someone from performing a MitM attack in order to steal the API key. In fact a MitM attack is easy to the point that it can even be achieved by non developers.
ADDRESSING YOUR QUESTIONS
How can i make this system secure? how can i be sure that none else can use the proxy except the Android app?
Well you bought yourself a very hard problem to solve, that a lot will say that is only possible to make it harder to be solved by the attacker, and this is true until some degree, because a lot of developers are not aware of the Mobile App Attestation concept, that allows a back-end server to only server requests comming from a original app.
Before I can explain you the concept I would like to clarify a common misconception among developers regarding the WHO vs WHAT is accessing your back-end server.
The Difference Between WHO and WHAT is Accessing the API Server
To better understand the differences between the WHO and the WHAT are accessing an API server, let’s use this picture:
The Intended Communication Channel represents the mobile app being used as you expected, by a legit user without any malicious intentions, using an untampered version of the mobile app, and communicating directly with the API server without being man in the middle attacked.
The actual channel may represent several different scenarios, like a legit user with malicious intentions that may be using a repackaged version of the mobile app, a hacker using the genuine version of the mobile app, while man in the middle attacking it, to understand how the communication between the mobile app and the API server is being done in order to be able to automate attacks against your API. Many other scenarios are possible, but we will not enumerate each one here.
I hope that by now you may already have a clue why the WHO and the WHAT are not the same, but if not it will become clear in a moment.
The WHO is the user of the mobile app that we can authenticate, authorize and identify in several ways, like using OpenID Connect or OAUTH2 flows.
OAUTH
Generally, OAuth provides to clients a "secure delegated access" to server resources on behalf of a resource owner. It specifies a process for resource owners to authorize third-party access to their server resources without sharing their credentials. Designed specifically to work with Hypertext Transfer Protocol (HTTP), OAuth essentially allows access tokens to be issued to third-party clients by an authorization server, with the approval of the resource owner. The third party then uses the access token to access the protected resources hosted by the resource server.
OpenID Connect
OpenID Connect 1.0 is a simple identity layer on top of the OAuth 2.0 protocol. It allows Clients to verify the identity of the End-User based on the authentication performed by an Authorization Server, as well as to obtain basic profile information about the End-User in an interoperable and REST-like manner.
While user authentication may let the API server know WHO is using the API, it cannot guarantee that the requests have originated from WHAT you expect, the original version of the mobile app.
Now we need a way to identify WHAT is calling the API server, and here things become more tricky than most developers may think. The WHAT is the thing making the request to the API server. Is it really a genuine instance of the mobile app, or is a bot, an automated script or an attacker manually poking around with the API server, using a tool like Postman?
For your surprise you may end up discovering that It can be one of the legit users using a repackaged version of the mobile app or an automated script that is trying to gamify and take advantage of the service provided by the application.
Well, to identify the WHAT, developers tend to resort to an API key that usually they hard-code in the code of their mobile app. Some developers go the extra mile and compute the key at run-time in the mobile app, thus it becomes a runtime secret as opposed to the former approach when a static secret is embedded in the code.
The above write-up was extracted from an article I wrote, entitled WHY DOES YOUR MOBILE APP NEED AN API KEY?, and that you can read in full here, that is the first article in a series of articles about API keys.
First question
How can i make this system secure?
Depending on your budget and resources you may employ an array of different approaches and techniques to defend your API server, and I will start to enumerate some of the most usual ones, but before I do it so I would like to leave this note:
As a best practice a mobile app or a web app should only communicate with an API server that is under your control and any access to third party APIs services must be done by this same API server you control. This way you limit the attack surface to only one place, where you will employ as many layers of defense as what you are protecting is worth.
You can start with reCaptcha V3, followed by Web Application Firewall(WAF) and finally if you can afford it a User Behavior Analytics(UBA) solution.
Google reCAPTCHA V3:
reCAPTCHA is a free service that protects your website from spam and abuse. reCAPTCHA uses an advanced risk analysis engine and adaptive challenges to keep automated software from engaging in abusive activities on your site. It does this while letting your valid users pass through with ease.
...helps you detect abusive traffic on your website without any user friction. It returns a score based on the interactions with your website and provides you more flexibility to take appropriate actions.
WAF - Web Application Firewall:
A web application firewall (or WAF) filters, monitors, and blocks HTTP traffic to and from a web application. A WAF is differentiated from a regular firewall in that a WAF is able to filter the content of specific web applications while regular firewalls serve as a safety gate between servers. By inspecting HTTP traffic, it can prevent attacks stemming from web application security flaws, such as SQL injection, cross-site scripting (XSS), file inclusion, and security misconfigurations.
UBA - User Behavior Analytics:
User behavior analytics (UBA) as defined by Gartner is a cybersecurity process about detection of insider threats, targeted attacks, and financial fraud. UBA solutions look at patterns of human behavior, and then apply algorithms and statistical analysis to detect meaningful anomalies from those patterns—anomalies that indicate potential threats. Instead of tracking devices or security events, UBA tracks a system's users. Big data platforms like Apache Hadoop are increasing UBA functionality by allowing them to analyze petabytes worth of data to detect insider threats and advanced persistent threats.
All this solutions work based on a negative identification model, by other words they try their best to differentiate the bad from the good by identifying what is bad, not what is good, thus they are prone to false positives, despite of the advanced technology used by some of them, like machine learning and artificial intelligence.
So you may find yourself more often than not in having to relax how you block the access to the API server in order to not affect the good users. This also means that this solutions require constant monitoring to validate that the false positives are not blocking your legit users and that at same time they are properly keeping at bay the unauthorized ones.
Regarding APIs serving mobile apps a positive identification model can be used by using a Mobile App Attestation solution that guarantees to the API server that the requests can be trusted without the possibility of false positives, and I will explain it as a reply to your second question.
Second question
how can i be sure that none else can use the proxy except the Android app?
As I mentioned in the begin of my answer, the Mobile App Attestation concept may be your best option to tackle your problem.
The role of a Mobile App Attestation solution is to guarantee at run-time that your mobile app was not tampered with, is not running in a rooted device, not being instrumented by a framework like xPosed or Frida, not being MitM attacked, and this is achieved by running an SDK in the background. The service running in the cloud will challenge the app, and based on the responses it will attest the integrity of the mobile app and device is running on, thus the SDK will never be responsible for any decisions.
Frida
Inject your own scripts into black box processes. Hook any function, spy on crypto APIs or trace private application code, no source code needed. Edit, hit save, and instantly see the results. All without compilation steps or program restarts.
xPosed
Xposed is a framework for modules that can change the behavior of the system and apps without touching any APKs. That's great because it means that modules can work for different versions and even ROMs without any changes (as long as the original code was not changed too much). It's also easy to undo.
MiTM Proxy
An interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers.
On successful attestation of the mobile app integrity a short time lived JWT token is issued and signed with a secret that only the API server and the Mobile App Attestation service in the cloud are aware. In the case of failure on the mobile app attestation the JWT token is signed with a secret that the API server does not know.
Now the App must sent with every API call the JWT token in the headers of the request. This will allow the API server to only serve requests when it can verify the signature and expiration time in the JWT token and refuse them when it fails the verification.
Once the secret used by the Mobile App Attestation service is not known by the mobile app, is not possible to reverse engineer it at run-time even when the App is tampered, running in a rooted device or communicating over a connection that is being the target of a Man in the Middle Attack.
The Mobile App Attestation service already exists as a SAAS solution at Approov(I work here) that provides SDKs for several platforms, including iOS, Android, React Native and others. The integration will also need a small check in the API server code to verify the JWT token issued by the cloud service. This check is necessary for the API server to be able to decide what requests to serve and what ones to deny.
CONCLUSION
In the end, the solution to use in order to protect your API server must be chosen in accordance with the value of what you are trying to protect and the legal requirements for that type of data, like the GDPR regulations in Europe.
DO YOU WANT TO GO THE EXTRA MILE?
OWASP Mobile Security Project - Top 10 risks
The OWASP Mobile Security Project is a centralized resource intended to give developers and security teams the resources they need to build and maintain secure mobile applications. Through the project, our goal is to classify mobile security risks and provide developmental controls to reduce their impact or likelihood of exploitation.
I have been searching for a proper answer to this question, have a bit of background in front end development, but with new concepts like SaaS, PaaS, etc. want to get information from experts out there, that could help any newbie to understand what it's all about.
Say I am trying to develop eBay like an app that takes a product from a user and sells it back to the other user who needs it.
Will my app need a backend server? If so why? I am already uploading my app to Google Play Store or Apple Store.
How will the backend server like HEROKU or FIREBASE or AWS help my app?
Can I implement two different services in single app, say for eg., firebase for backend database and HEROKU for payment processing?
Thanks again for your time and information.
SaaS
Answer: SaaS stands for "Software as a Service". In layman's terms, someone developed some software and hosted it somewhere. You can use that hosted software in your software project/product as a third party service (like public API); or directly use that as individual software under some license like Firebase as mentioned.
PaaS
Answer PaaS stands for "Platform as a Service". In layman's terms, someone configured some hardware and exposed the hardware controls via some web based application or REST APIs. You can use that hardware to deploy/run/manage your application without having the actual hardware on premises.
Backend Server
Answer First of all, let me explain the server. The server is a middle-man who serves whatever is requested of it, and all browsers/mobile apps act as client. So for example, the web is all about client-server communication.
So taking the example you mentioned, an eBay-like app takes a product from a user (client action) and puts it on the server (client requests in background for server to put product on server). Then another user opens the app (client action) and searches for the product (mobile client requests server to return that product, if valid and matching search criteria), and then he can buy it (mobile client will request server to complete the purchase).
You have to understand that for any communication between web application, mobile application or desktop application, there will always be a server. Even in file sharing applications like shareit, one mobile app works as server and same mobile app elsewhere works as client.
Yes, backend servers like Heroku or Firebase or AWS will help your app to complete your application business flow.
Yes, you can implement two different services in single app, say for example, Firebase for backend database and Heroku for payment processing or hosting your application/APIs.
Unless you are experienced with building distributed applications that can persist data across multiple nodes in a consistent manner, and ensure data available, I'd say you most definitely need some kind of backend. Unless of course you only plan to have user-to-user transactions, that can rely on direct messaging between client applications - which seems pretty pointless and quite far from the requirements of an EBAY-like product.
In terms of the architecture, you can follow many different approaches, but in most of them you will require some sort of data access layer. I'd recommend looking into the three-tier software design pattern (https://en.wikipedia.org/wiki/Multitier_architecture) to better understand the way this type of software product is typically designed.
After sorting out which type of data persistence you prefer, you'll need to setup the backend where your mobile app will connect to retrieve the data from (things like products being sold, user profiles and ratings, your own history). Of course you could also connect directly to the database from the app, but that would be a big mistake - it would meaning making the DB access publicly available, and thus exposed to attack, not to mention that you would be hard-pressed to find a solution for user registration and authentication, which would have to be provided by other means anyway. Typically your backend will also manage user registration and authorisation.
Heroku, Firebase and AWS are all very different, each with their strengths and weaknesses, but there's nothing like trying them out to see what fits best. What you refer to as "Google Server" and "Apple Server" sounds like a misconception, and you probably mean the Google Play Store and the Apple Store. These are not applicational servers that you can use as a backend, and serve only as a repository for your mobile app from where users can download it, and nothing else.
Without some sort of backend mechanism, the challenge of making data available for the consumption of multiple users would be overwhelming.
I know this isn't a very specific answer, but your question is quite broad-reaching, and it seems you need to look into some basic fundaments of software engineering before going into more detail.
I am creating a mobile app that uses Google App Engine for its server side. I am quite far along in the application so I would rather not switch to another solution, and GAE is definitely the cheapest and easiest option for me. I have had very few problems with the platform. In terms of security, I have (very well) hashed user passwords, and I use SSL for all interaction between client and server. I do, however, have some security concerns.
I am not ignorant to the fact that people are perfectly capable of reverse engineering mobile applications, iOS or Android. If someone were to discover the URLs I use to access GAE, they could easily spam my application. I was wondering if there was a way to prevent people from being able to do this. I want the only entity allowed to access GAE to be the app.
I want my android app to access a specific acount [not the user's account] and download a file. Is this possible?
To use google docs you need a google account.
This does not require you to have a google mail account.
With this URL you can make yourself a google account using your own email adres:
https://accounts.google.com/NewAccount?continue=https%3A%2F%2Fdocs.google.com%2F%23&followup=https%3A%2F%2Fdocs.google.com%2F&service=writely<mpl=homepage
Good luck,
Thomas van Latum
This is generally possible using Service Accounts. Service Accounts however may not be used directly from Android applications, as Android wouldn't be able to protect your private key file.
A workaround would be to have a web application yourself that queries the data from Google using a Service Account and only connecting to your web application from Android. How you make sure that only your client can access your web application is then however in your responsibility.
What I was looking for, namely freemium cloud services, is better served by Google appEngine or Azure or Amazon or Apple cloud Freemium deals etc. Note that these are usually easy to implement but there are many scaling issues that should affect the decision. Importantly there is high cost in moving from one cloud platform to another because the code is idiosyncratic and useless outside the specific cloud.
I have developed a simple two player chess game in android to be played using Bluetooth. I want to extend it by making it possible to be played through internet. whenever a player makes a move, the move should be transferred to the other player via internet.
How to make this possible?
I have heard of C2DM mechanism.Does that suites the scenario i described and is it reliable?
Thanks:)
Yes, C2DM is ideally suited to this type of game. This is what I am using for my own game (http://www.chesspresso.net) which is a correspondence chess client for android.
Things to consider when using C2DM:
You don't send the info to the devices, you notify the devices that a move has been made. You don't use C2DM to transfer data, you use it to notify that something has changed.
Its available for 2.2+ Android, which is the majority of devices. But if you wanted to support older devices you'd have to consider an alternative. I am using polling for older devices.
You have to request for developer access, then once your app is ready you have to request production status. If you don't do this you'll hit the developer status quota very quickly once its released! They are very generous with production quota, but you have to explain what you're using it for and it also can take a few weeks to get accepted!
Your users will have to have a google account that is authorised, otherswise C2DM won't work. Most users will have an account associated with their device, but some don't so this means that you'll possibly want to validate for the presence of an account to notify the user.
Its reliable, but every now and again a device will have to wait for the message. Sometimes a few minutes. Usually its instant.
Hope that helps!
UPDATE:
C2DM has now been deprecated, and replaced by Google's GCM.
Also, I strongly suggest looking at other options as tying yourself down to a Google specific API means you won't be able to support external marketplaces. For alternatives, I am currently evaluating Amazon SNS and I will also be looking at Urban Airship. There are possibly other alternatives I have not considered evaluating yet.
UPDATE:
Evaluation update of non google based push notifications:
Amazon SNS is just not a project for this task and Urban Airship for the vast majority of apps is too expensive. Unfortunately all the other alternatives are all very expensive also, especially if your app (like mine) relies heavily on push.
A good way of doing that is using a simple direct TCP connection between the peers.
If you're new to socket programming on Java, try this:
All About Sockets
Another option is to use some sort of IM as a communication medium for app. For eg. Use Asmack to connect to XMPP Im like GTalk. Prompt user to create an account there, for your game.
And use it to send and receive commands via IM. This way you won't need to setup your mediating server.
This works if user knows who he is playing with. To collect the user data and let them search for available players, you still need to setup a server. IRC chat room may be an option to avoid this also.
GTalk was just an example. You can use any IM or IRC also.
C2DM it's not design to transfer informations, even if they are small like "horse in b4" or things like this. It's designed to inform the device of something, maybe a newer version of a document or more articles on a website.. Stuff like this.. It's not designed to communicate device to device. And also it may be not fast enough for a real time chess play.
You should look for a more traditional way of communicate via internet or to search for some libraries (I'm pretty sure that something exists..) that will help you.
IMHO, C2DM is exactly the kind of thing you would want for a chess game; to be notified when the oppo has made his turn (which may be minutes /hours / days later ?). I have discussed my game with a few google android devs and they've stated that C2DM is ideal for this. You'll need to go via a centralised server though (well, not essential but very advisable) as there may be issues with resync'ing game state etc. Worried about "hitting the limit" ? Well, for a start my c2dm acct is restricted to "just" 100,000 messages per day. I guess you're buying the drinks if you hit that !!
Chess is often played by email. You could do that.
Of course, any centralized/federated messaging system will work.
What might be better for your use is to add a jabber client to the application and have the program generate an account name that is used for automated messaging. You could host the jabber server or generate the accounts on a free provider.
Google App Engine if you know Python or Java.
Alternatively there are two web app API styles in wide use today: SOAP XML and RESTful web services.
If you know RoR I would recommend using JSON/REST, because you can just use Phusion Passenger with Apache to deploy your app. Free, extremely easy, and makes your server very reliable.
You could, and I only mention this because my friends do this all the time, use twitter as a server between the games.
I also found a lib called mages which looks quite promising.
Good luck.
I did this for my online 2D rpg: http://developingthedream.blogspot.com
Basically, use a middle-man server to co-ordinate data between all your clients.
You simply open a socket and communicate with the middle server and it takes care of passing on the information to any other connected clients.
I wouldn't recommend C2DM because of the message limit, and because the latency is still to big. Using your own server you can optimize it, plus you'll be the only one using the service so your data will be delivered faster.
I think that C2DM is not right way for playing chess because there is no warranty that messages will be delivered. You need more reliable way for data transfer