I want to create a POS application but for some reason the application should be working when there is no internet connection (and later do the synchronisation when connect to internet). the only method that came to my mind is synchronising between local db with server but I dont have idea how to or the best method to do this. I have several question about this:
is it possible to use FCM cloud messaging to do this ? I mean Client A tell FCM data has changed, and then FCM tell several clients (phone) to do sync db in the background without user's (phone owner) confirmation (automatically).
or the only way is to use this method https://github.com/codepath/android_guides/wiki/Server-Synchronization-(SyncAdapter) ? if it's so, how often we can synch db local with server? is synchronising each 3 minutes will be fine?
Yes, use FCM real-time database for this. FCM manages data for you internally on the app, as well as on the cloud. So if you write some data into your JSON tree but you don't have internet connection, the data will be pushed into the cloud once the connection is available.
The documents clearly state this (Read last section on 'Writing Offline Data': https://firebase.google.com/docs/database/android/read-and-write
Also, any data added to the database when your client was offline will be sent once the client goes online - because the real-time database keeps storage of the data as well.
Using a server-sync every 3 minutes is not feasible - and it is more likely that your app will get killed by the system at some point or the other. Use FCM and Cloud Functions along with JobSchedulers to get the best result.
Related
I am trying to make an app in which I am going to use room database for offline data caching and using NodeJs and MongoDB as a backend service.
What I am going to do is when app first opens it fetches data from server and stores in room database from where it shows in database.
My problem is whenever some new data is updated on server how would I know whether it is available in room database and when to fire server request.
Someone please let me know how can achieve this any help would be appreciated.
THANKS
You are going to need to implement some sort of routine where you can check and validate (update) the data. However, this will depend on the importance of the data and rather if it should be updated and how often.
These are some of the solutions you can look at:
Long/short polling - Client Pull
WebSockets - Server Push
Server-Sent Events - Server Push
I would probably use some sort of real-time communication (web sockets) or use a real-time database to notify the app of changes when something needs to get updated. That said it would also depend on the technology, for instance, Firebase already offers offline caching.
You can achieve this functionality in multiple ways
Configure firebase notification in your app, whenever any server side data update happen just trigger a notification, based on that you can call api and store data in room.
Maintain some version code related to updated server side data in your api, based on that version code you can write logic for storing data in room.
I'm writing an Android application that uses the Firebase RealTime Database and Firebase Cloud functions. Several of my cloud functions are triggered by writes to the RealTime database and are used further process data saved by the Android client application.
I'd like my app to handle network connectivity changes gracefully. I understand that Firebase handles loss of connection by saving changes to the RealTime database locally and then syncing the changes to the server when connectivity is restored. This is well documented in the Firebase Documentation.
In my case though, since I need the Cloud Function post-processing of data saved to some paths to have occurred before it is useful, there's no point in having this data saved if it isn't going to make it to the server (and therefore trigger a Cloud Function), in a timely manner.
I'm using the updateChildren function from DatabaseReference to save the data and have a CompletionListener attached to monitor the outcome. I thought I may be able to use the DISCONNECTED and/or NETWORK_ERROR DatabaseErrors to identify cases where my data won't be reaching the server. However, if I interrupt the network connection before updateChildren is called, there aren't any errors generated. In this case, Firebase has likely saved the data locally with plans to sync it later, so updateChildren is considered to have been completed successfully.
My questions then are:
When are the DatabaseErrors DISCONNECTED and NETWORK_ERROR actually used by Firebase? Can I use them in some way to help manage connectivity issues?
What are the best practices for handling cases where Firebase data must make it to the server in order to be useful? Should I really just be POSTing my data to the Cloud Function directly?
Does Firebase have any notion of a timeout period that can be watched and data invalidated if it isn't synced within a specific period of time?
Yes, I recognize that I can use a listener attached to /.info/connected to detect changes in the connection state, but I'd rather be able to react and gracefully handle my case as it happens. I feel that my usage of the Realtime database together with Cloud Functions is common enough that there must be generally accepted way to implement it.
Any thoughts appreciated. Many thanks.
Did you try OnDisconnect method of firebase realtime database ?
I am using kinvey as a backend to store some data. Everytime the user answers some questions on the phone(Android), the data is sent to kinvey immediately. But there can be a scenario where the user is not connected to the internet, so sending the data has failed.
I would like to know if there is any standard way to storing the data that hasnt been send in an efficient manner? Also how often should I try to resend the data that has not been sent? I would typically just put it in a db, and try to send whatever is in that table every 30 minutes or so and clear the table if the send is successful?
Would this be ideal? Also I see that Kinvey doesnt have a mechanism to handle this automatically(correct me if I'm wrong).
Thank you for any suggestions.
Local database is a good place to store your data securely. The thing is that you are sending data after every 30 min, you have to change that scenario.
Below are the steps:
Check Internet connectivity, if available than work online no need to store data locally
If internet is not available than store it in local DB
You are able to get Internet connection broadcast from system whether it is connected or not if you get connection at that time sync your data with the server (No need to check every 30 min) (System always broadcast when it get connection)
You can took a look at Firebase, feature that you're looking for is described in Offline Capabilities. If migrating from Kinvey is not an option - then store your data inside a DB and use JobScheduler to sync it to the backend when the conditions are right.
The other answers cover most of the use cases.
But if you don't want to use system broadcasts, another alternative would be for you to store the data on a database (SQLite, Firebase, etc), and try some sort of exponential backoff strategy to send your data to the server, using the AlarmManager for example. (Tutorial here).
You can try to check the internet connection from time to time, and send the data. If it fails for some reason (slow connection, connection drops, or no connection at all), try again later. Once you succeed, you can either remove the data from your data storage, or set up a flag like isSent = true.
Kinvey is rolling out offline sync capabilities to all their SDK's, it's currently in Beta. If you use iOS or Javascript for mobile development, check out the "3.0 beta" SDK's. You mentioned you use Android, support for that platform will come soon as well.
In your case, you need a cache store, it will save the data locally and sync automatically if there is no network. You don't have to worry about when or how often to sync.
http://devcenter.kinvey.com/ios-v3.0/guides/datastore#CacheStore
I have an SQLite database on Android and a MySQL database on a server. I want to synchronize these databases when a user edits data on their phone or edits data on a website.
I know how to update the MySQL database on the server when a user makes changes on their phone but I don't know how to update the Android database when a user makes changes on the website.
I have read into push notification and believe this to be a good path to follow but I have a few questions about it:
When a user updates data through a website it will send a push notification to that user's phone saying changes have been made. Can this push notification trigger to update the Android's database with the new changes made on the Server database?
What if a user turns off push notifications? Will I still be able to trigger for their Android database to be updated?
I have also read up on SQLite and MySQL database synchronization and found this post SQLite and MySQL sync but did not find the post helpful for my situation.
Are push notifications a good way to go or should I be using a different approach?
In a nutshell - I want a way for the Android device to detect changes on the MySQL database and update its SQLite database without the user initiating the synchronization.
I'm afraid I've not used push notifications. But a solution might be: You could create an early method call to an Asynchronous polling event from the launcher onCreate() that looks up the server to see if any changes have been registered (though an API of some sort) in the MySQL, and then update the SQLite that way? Since it's the first thing that happens on launch, technically the user isn't initiating it. Granted this won't update during use of the app, unless you repeat poll at regular intervals?
Token based pagination approach.
Assumptions: or calls you need to take
One of the databases will the source of truth, in case of differences in the two, which data is true, and which will be overwritten? - assuming remote database is source of truth
What's the frequency of data changes? - assuming its not realtime critical
How much stale data are we OK with dealing on the app. - assuming we're OK with a few minutes of difference
How to ensure data is consistent
Find a method to associate a token, which can be used to identify till which record is the data in sync. This is important no matter how assured you are of web requests, they will fail. So the best method is to send the last token that have stored, and the web endpoint will return data from that token to the latest value.
If the volume of data is too much here, sending chunks > sending all of it. Chunks work the same way, based on tokens.
These tokens can be simple PK auto increment column as well.
How to deal with time difference, stale data
If your application demands some data to be near realtime, better to categorize data based on a fiew screens, and whenever the user comes to the said screen, send a request in background to fetch related data columns only. Will ensure the data stays in sync w.r.t to the important columns. This is a classic push/pull approach. Can also be done on the splash screen itself.
as a rule of thumb, if you need something urgent, pull.
if it can wait, wait for a push.
Push notifications are OK as far as:
they should be silent.
there'a a limit on the number of push notifications that you can send
have costs associated
what's the fail - check mechanism? What if the requests fail?
I am working on a Point of Sale application that needs to be very good syncing mechanism. We have Magento Database.The android device have SQLite local Db. Now we need to sync in the following way:
Local ------Sync To---------------> Server (Up Sync)
Server------Sync To---------------> Locals (Down Sync)
There are 2 things:
1) write-to (How to take care??)
For every change that i do on local ,it will directly sync my local to server
2) write-back (How to take care???)
Whenever there is a change in server, we need to sync all our locals with server.
So, the task is: to identify a server update
And sync our locals.
Like there are 4 devices are running in a store and we have added one new customer through one device. Now i want that the three other devices local db also updated with the information about that customer and server also updated.
I heard about the background threads and run threads after a time interval. But what is the best way to do that which don't affect the application. Also the all Big Retail stores uses the syncing process. what they used for that ?
Any help is appreciated.
It fully depends on you database structure...
you have DATABASE in LOCAL (device) and on SERVER
NOW
You need to have TIMESTAMP fieLd added to the TABLES which actually you want to keep in SYNC.
When ever you will make any changes on server the TIMESTAMP will be updated there and same will be the case for the local database also what you have to do is now.
Run a service in the background which will keep on comparing the TIMESTAMPS of LOCAL with that of SERVER.
Now you have to put condition that if TIMESTAMP of SERVER is newer to that of LOCAL then bring changes from SERVER to LOCAL,
and vice versa will be the condition to take changes from LOCAL to SERVER.
Further you have to decide how frequently you want to run this SERVICE.
ALTERNATIVELY:
You can make the table there on SERVER which will store LAST_SYNCHED date for particular device
Whenever you will login in you device (or any other particular event on which you want it to perform this) the server will check-
when this device was LAST_SYNCHED
then it will compare it to TODAYS DATE
and will check what upadets actualy happened between these dates and
will send the changes to the LOCAL (device)
and vice versa for LOCAL (device) to SERVER
you have to play with TIMESTAMPS rest you can have your own logic how to structure the database.
I told you what I have have observed, when I have been a part of such similar project
EDIT
The above Process defines how to sync the devices with server I mean the strategy..
If you want your devices to get notified from server when to sync instead of hitting the WEB-SERVICE recurrently ..
You can make use of PUSH NOtification, GCM is one of them that send push notification to devices, you can integrate it to your project
For syncing you need to handle following two situations.
How and when to receive server updates
How to identify local non-synced data
How and when to receive server updates:
For receiving updates, we can use GCM (Google Cloud Messaging). If any updates made in server, server sends a push message to all devices. Devices will receive that push and based on the message, devices will download the data from server. (I think this is better approach than continuous hitting service for some particular intervals like polling)
For receiving only updated data from server, server maintains modified_timestamp column for all tables. First time devices will send empty timestamp, so that server sends all data to the device with server timestamp. Device receives the new data and updates local db and saves the latest server timestamp. For next time to get server updates, device will send stored server timestamp then server will send only modified data after that timestamp only. For each response server sends server timestamp, devices needs to store that timestamp and needs to use while calling service.
How to identify local non-synced data:
For sending local updates, local db needs to maintain one 'isSynced' column in tables. If any row modified in local isSynced will be false, after successful syncing local data to server isSynced will be true. so that we can handle local data up to date with server.
Updated:
You can find more information on this developer link
Have you considered using commercial solution?
http://www.mobeelizer.com/ seems like what you want to achieve. There are probably many other.
Note: no affiliation with that company.
I would say that the problem statement is incomplete. In above described setup what is missing is what actually you are going to synchronise.
Usual case in POS is that there exist few indices (id,value,...) tables that shall be distributed from central server to the client devices. In most cases it is price list, stock list, etc. Those tables should rarely be modified on client devices (actually could but then has to be redistributed from central server and acknowledged by client devices).
The other direction tend to be also pretty straightforward on client device you generate bills or invoices. These are again local stuff that shall be propagated towards server. Thus you actually store them locally and at sync point dispatch them to the server. Later on you might receive your own items back from server as an acknowledge.
EDIT: to detect changes, on-write timestamps as mentioned above is a must.
So far above described is the data flow.
Next you have to move into solution domain and implement these rules. There is couple of sync approaches (i.e.SyncML). On the other hand keeping it simple rulez. Thus the main concern should be some kind of locking and queueing that makes the thing robust.
It could also use the agent based client, in such case each device has it own agent (could be replica of last known state of the device db) but I would consider this as an advanced feature that might come in future release:-)
I am also working on the sales app in which i have to my local goals to server and server goals to my local goals
My proceder is that when ever my app is started i get the latest data from my server of my all my member and update my local data base with this data and when ever i change data in my local data base also update on sever side
also i used a sync button which will fetch latest data from the server if my team member changes its goal or profile
IF you want updated data on all the devices, why don't you use remote database only, why are you introducing local database for this.
For your case i will suggest you to work with only remote database directly so the things can be done real time.