Mobile device to Cloud data scaling - android

I have a mobile app on android that needs to send small amounts of data (id, lat and long coords) every 30 seconds to be stored in a SQL Server database sitting on an amazon ec2 instance. As an example usage, say that this app has 500 current users all sending data every 30 seconds. For proof of concept I created a windows service in c# running on the database server which listens for connections on a specific tcp port; it threads, and writes the data to the database. Now this works for the 5 users I tested with but I know there are better ways and I especially do not want the insert statements to be done with a program running on the database server. So my question is, what is the correct way to handle repetitive data streams from a large user base that scales in a manageable way. I have read information regarding implementing webservices to do this but I am not sure if that is the correct solution.
Thank you for any information.

Using a web service approach is definitely more scalable than using a Windows service. If your usage grows enough to justify it, it will be also be easier to deploy your web service in a new EC2 instance (or to multiple load-balanced instances) instead of using a windows service which shares resources with your DB instance. It is also much more manageable, especially under AWS, where you can scale your infrastructure with a few clicks.
Also, under a web service (or a web application, if you prefer), it is easier to set up access control and to take other preventive security measures.

Related

How to structure an application properly on background processes

I have an application that uses a webservice to get information and save the changes made ​​by the user.
currently I have one service that runs while the application is open, and it has public methods for each operation. the trouble is that service is growing considerably and I'm thinking refactor, but the question is what is the best way?
I can think of the following options:
Deferred services the current service and that all are initialized at boot time application
Create small services and that these are initialized by local broadcast
although I have doubts about performance. Can give me some clue or example about which method is better, do not really care that changes are instantly synchronized, these are stored locally and can be synchronized when possible. Data sent are not many, so the synchronization is relatively fast
Synchronization processes are something like
Check if there is new data (I have several types of data, these are the ones that are growing)
Synchronize user preferences
Most likely there's no point of having Service running all the time. Instead, I'd go for IntentService. If possible, I'd also condifer using push notification (like GCM) so the server could let my app know that there's new data to fetch (or maybe even send it to me if you'd fit in the GCM payload limit).

Synchronizing partial database model from server to client

This is more of a conceptual question not necessarily bound to any specific technologies.
Lets say you got some database on a server, some REST/JSON API to access content in that database and some mobile client displaying data retrieved through the API.
It would be nice to have some caching mechanism on the client and also to be able to enable offline access to the data as long as the client is only reading (In my case it's fine to deny write access to offline clients to avoid having to manage all those nasty conflicts that might happen).
It appears that a nice way to solve that would be to have a subset of the servers database model present on the client and synchronizing data from the server to the client.
Access to the local database might then immediately return results but also trigger update requests to the server. In case the server returns modified data the client model then synchronizes it's local database and notifies the display of data changes.
The goal in the end is of course is that the user may browse the information regardless of the stability of his internet connection and is not annoyed by connection dialogs or similar as long as he doesn't modify any data.
Now from an implementation perspective... on one hand it seems like a bad idea to couple the server database directly to the client database as they may be from different vendors. I guess at least there would need to be a vendor independent model above both database implementations. On the other hand, transforming the data from the server database into some transport format and than putting it back into the client database seems like a lot of overhead.
Any suggestions how to solve that in an elegant and maintainable way?
I am working on an app that syncs small portions of a large database locally onto the handset. There is an initial preload that has to occur on the handset but after that the updates happen asynchronously in the background.
First of all, decoupling the server and handset using JSON or XML is highly advised. Locking into one technology always causes issues as you are forced to use the same technology regardless of the platform. That is, if you plan on expanding into other platforms (Web,iOS,etc..) you are forced to use the format dictated by the server. Choosing a generic format will make that simpler in the long run. In reality with the amount of public libraries reading/writing JSON is a trivial matter.
There are two ways that we use to sync the data;
1. AlarmManager
We schedule the AlarmManager to trigger a service to wakeup on a regular schedule (lets say every 6 hours). The wakeup starts a background service that contacts the server, downloads the changes in JSON and updates a local SQLite DB. If there is no connection, the update is skipped and scheduled for the next wakeup. We add a ConnectivityChanged receiver to automatically restart the sync when the connection is restored.
2. GCM
It's a little more work but saves a lot of battery and data usage if you only update the local database when there are changes. Google Cloud Messaging can send a wakeup message to the device and tell it to start the sync service. The sync service runs the same as the AlarmManager method above.
We do a combination of both of the methods above depending on how "fresh" you need the data and how often it changes. Something like an RSS feed should probably be updated every 30min whereas weather data may not need to be updated more than every 4 hours.
So to run the database sync we use;
Receivers -> listen for system events and trigger Service
Services -> connect to the server, download the JSON and update the SQLite providers
Providers -> insert the records into the database and broadcast content changes to ContentObservers
ContentObservers -> when the app is running, the ContentObservers update the UI with the new data
There is a lot of technical details in each of the components above but that should provide you with a very robust architecture for syncing server data with a local db.
I'm working on a project that has similar requirements. We want to have a big, available database on a server somewhere and then mobile devices that get data from it. If the devices go offline it's ok because they have saved their own copies of the data locally.
We've decided to use BigCouch (fork of Apache CouchDb that supports clustering) as the server technology and then Couchbase Mobile on the mobile devices. (As a note TouchDB for Android will replace Couchbase Mobile, but it's not stable yet.)
The reason we went with Couch* technologies is that Couch has good replication over HTTP. You can programmatically initiate a sync event on the mobile device and it will replicate all inserts, updates and deletes for you. It stores the information on it's own embedded CouchDb on the mobile device, so it can be read offline.
If you didn't want to go down the Couch road, you could simply use something like SQLlite to store the results of your REST/API calls. Then you would have to write your own replication logic for when a mobile device goes offline and then comes back. There are creative ways to do this, so maybe it's an option.

Best technology for client-server Android App

I need to make for school an app that runs on Android. Actually there are two apps, a client and a server. Ther server runs on a PC while the clients run on Android devices. I want to know what is the best technology for this. I know RMI and WebServices are not implemented in Android so what are the alternatives (besides actually communicating with sockets in the traditional way). One alternative that I did not look into is REST, but I will need to be able to notify a client once another client has done something, similar to turn base games where Player A notifies Player B that he made his move.
Like I said, sockets do the trick, but are little low-level compared to RMI and WebServices and only want to use those as a last resort.
Keep it simple. Use REST and have the clients poll for updates.
Also, if you get to a point down the road where you need to scale, this solution is actually fairly easy to scale because your servers do not need to maintain connections. Since there is no shared state between a particular server and the client (there is shared server between the application and the client), you can easily add more servers to handle the polling and put them behind a load balancer.
You can also add in caching so that the polling just gets the exact same response without causing a re-compute of the response. You would then be able to have the back-end game-state servers update the caches when the game state changes. This would let you poll much more frequently and still have a very flexible, scalable architecture.
For a turn-based game you can take a look at XMPP (e.g. Smack) that is traditionally used for instant messaging. It would also be interesting to create a game using C2DM that is used for push notifications.
You can also look into HTTP streaming which is essentially an unending HTTP response in which player moves are fed into.
Alternatively you can look into binary messaging systems that are more suited for real-time games but still applicable such as RabbitMQ (what's wrong with a smooth turn-based game?).

Using Pattern singleton on Android

I am developping an application that retrieves some data from a server.
I have two options:
Get the whole data set from the server and then work with it using the pattern 'singleton' to reduce the number of queries
For each query, get the corresponding data set from the server
What is the best choice?
In my opinion it depends.
It depends on the size of the data and if it even makes sense to return data that your user may not even need.
Personally, in my app that I am building I will be returning only the data that is required at that time. Obviously though, once I have the data I won't be fetching it again if it makes sense to keep hold of it for good or even just temporarily.
I agree with C0deAttack's answer. Your goal should be to minimize network traffic within the constraints of your app being a "good citizen" on the phone. (That means that your app does not negatively impact the user or other applications by using too many resources — including memory and file space.)
From the sound of it, I'm guessing that the data are not that voluminous. If so, I would recommend caching the response and use it locally, thus avoiding repeated queries to the server. Depending on how often the data changes, you might even consider making it persistent, so that the app doesn't have to query the server the next time it starts up. If the response includes an estimated time before it is considered outdated, that would help in establishing an update schedule. (Google's license server uses this idea.)
P.S. I don't see that this has anything (directly) to do with a singleton pattern.
How about storing your data in an sqlite database and do your queries with sql? Since you get sorting and ordering for free, it can help you writing less code. Also you have instant offline functionality if your user has no internet connection. :)

Android Remote methods (AIDL) vs Intents - performance & battery usage

My team is working on an Android project which consists of several Android applications which exchange data (on the same phone). The idea is to have several applications which are collecting some data and send this data to the main application. The main challenge here is to do the exchange as cheap as possible in terms of CPU load & battery usage.
As far as I know, there are two ways to achieve inter-process communications:
Intents & activities - one activity catches the intents of another
Remote methods (through AIDL)
I wonder which of these is more efficient in the following scenarios:
Very frequent messages/method calls with very little data sent/traffic (e.g. just passing a bunch of primitives)
Less frequent messages/method calls with large traffic chunks (e.g. collect data and periodically send a few KB/MB of data)
Very frequent messages/method calls with large data chunks exchanged
I would appreciate any help, either in terms of comparison or a reference/link to a benchmark.
I think for 1) you'd be best with a remote service and for 2) and 3) you'd be better off writing to files or a database. Intents are more for infrequent interprocess communication and starting apps and services.
You could also try to use native code to create a shared memory as an alternative option. Check out this link for details:
http://www.androidenea.com/2010/03/share-memory-using-ashmem-and-binder-in.html
I suggest you use the Unix domain sockets mechanism to address scenario 3). The high frequency will make the use of files/databases complicated, and according to this answer, using Android's IPC is not recommended performance wise, since every object has to be converted to (and back from) a Parcel which takes time.. You can also use Unix pipes but it has some restrictions:
How to create named pipe (mkfifo) in Android?
https://groups.google.com/forum/#!topic/android-ndk/lD-V7Nxe5y4
How to use unix pipes in Android

Categories

Resources