Well, I'm building a app in both iOS and Android and both app needs to be syncronized with the cloud DB MySQL. Both the apps works offline so I need all the insert and update functionality. Delete is probably not going to be used but still I'd like to know.
Well, what I'm looking here is the solution or ideas or algo that's needs to done to achieve this.
I have kept CreatedOn and LastSync as timestamp column on each table.
Now the problem is should I always keep checking all the rows and all the columns everytime?
What I think is I should keep ModifiedOn column on all the tables and check that with LastSync for every device id. What do you guys suggest?
Maybe it's not the answer you are waiting for, but on the project I'm working on, we are using Azure Mobile Services. There is an Android and iOS SDK where they implemented synchronization between offline database and Azure (just like in Git you can call push and pull methods).
There are some limitations (TableStorage is a limited MSSQL storage), but you should check it out.
I know it's about two years overdue , but I've been doing the same thing (Android App that syncs with web api and postgresql)
What I do up to now (still researching if it's the best way, which is how I found this question) is that I keep a last_update field with a timestamp on when the data was last_updated and an altered field with a boolean on whether the data was altered (by a user action on the device)
So when I perform the sync:
I first copy the database to a backup (in case there is a problem)
I send all the rows that are marked as altered (the server handles conflicts)
then delete them
after that I find the max(last_update) value,
I subtract one minute from it
Then request all rows that have been updated after that value,(this will also also include the data I have sent because there is no way , unless there is a problem, that a row is set to altered with a last_update before the last non altered row)
It may cause some duplicate data being transfered (especially if you make A LOT of changes frequently) but in my case it's a rare phenomenon
Related
I have a question about real time notification in MySQL.
I want that when there is a change in a specific table of the MySQL db, the db notify my android app that do something.
Is it possibile to do? I have listen that trigger can help, but i didn't found anything about that in web, especially for android.
I'm creating a real-time chat app for an android project at the university. It is required to use a servlet in the project, so my servlet makes requests to the database and sends the data to the app. To do this the app makes a GET request to the servlet every second to get the new messages. I would like to avoid making these requests and updating the message list only when there is a change in the database. I know it can be done with firebase for example, but having to use the servlet I would like to know if there is a way to notify the app when this change occurs.
The messages are insert in the db with a POST request to the servlet that update db; information about messages are taken from an edit text in the app and with a send button i make the request, so the thing that i have to do is to push the android app when new messages are detected in the db.
Presumably you have some code component sending commands to the database. If yes, then that is where you could add code to post notifications to your app via websockets or something similar. Triggers in your database come at a cost and should be avoided if there is going to be a lot of them required.
As already mentioned, the app constantly polling, looking for a change is also expensive and probably best avoided.
'Is it possible?' questions are always difficult to answer. The answer is almost always 'yes', but 'is it advisable', 'is it a good idea', 'is it recommended', that is where it gets more complicated. I don't know of any way that a MySQL database change could notify individual Android app installations after a change. However, a running instance of an Android app could poll the MySQL database and look for the structure of the table, and if the structure of that table differed from the last time it polled, it could do something.
This would have a large overhead, in having the app continuously polling the MySQL database to check the structure of the table. That would add a lot of demands on your MySQL installation, eat up a lot of the app user's bandwidth and data plan.
So, 'is it possible', yes. 'is it recommended', no.
One question would be, what is the scenario where you think the table structure might change, and how often might that occur? What many people do, is check to see a database's table structures only once, when the app starts up a new session. If a change is detected on start-up, it could make the necessary local changes to its logic, then, assume the structure will remain constant through the rest of the session. This would eliminate the constant polling and other negative behaviors I mentioned before.
I'm working on a Mobile App, where the main feature has the user do a lot of CRUD (Create, Read, Update and Delete) tasks within it.
The main storage of data for the App is a local sqlite database, but the user has the option to register an account and use Cloud database to backup their data.
This App needs to be able to work both offline and online. And the user should be able to use multiple devices containing the same data.
Currently in all of my sql tables I have 3 extra columns that keep track of which entries in the database is synced: createdAt(datetime), updatedAt(datetime) and synced(boolean).
With this I am able to keep track of which entries are the most recent, and update either the local or the cloud database accordingly.
I'm using Cloud Firestore as the Cloud, and using its' Offline capabilities is not a viable option in my case.
My Question is; Which solution would be the best when keeping track of deleted entries until the App is able to sync with the cloud?
One idea for a solution for this is to have an table that contains all the ID's of deleted entries together with which table it belongs, and then when the App is able to sync; remove these entries on both the local database(on all devices) and the cloud database.
The problem I have with this solution is that, this 'deletion' table quickly will become huge, and removing entries from this table would be a problem, because of the need of all the user's devices to be up-to-date before deletion and in a scenario where the user has abandoned one of his devices, this would mean that the device would never sync, resulting in the entries not being remove from the 'deletion' table.
What would your suggestion be for a robust way of tracking deleted entries?
I don't think there's a solution that satisfies both these objectives:
Don't keep deleted items in the database forever
Make sure deletions are synchronized between all devices forever
So you will have to decide which one to give up on. Your idea satisfies 2 but not 1. A solution that would satisfy 1 but not 2 is to delete the deletion records after a period of time, maybe six months. A variation on that would be when a record is deleted, rather than actually deleting it just mark it as deleted (as well as the date when it was deleted), and if applicable remove any large pieces of data from the record. After whatever grace period you decide, the record can be actually deleted. The down side is that if a mothballed device is brought back out, it could restore previously deleted records.
I have an architecture question. If you have a web app that is storing information on a DB server, theoretically, I should be able to use the middle tier logic for a mobile app. When the mobile app starts it can connect and populate a local SQLite DB or use JSON to store information within the mobile app. What if the mobile app also needs to work in off-line mode? Do you have it sync the next time it is connected? Do you have the mobile pull down and populate a complete DB or so it available in off-line? What are the best ways to architect a mobile app that has to go from on-line to off-line?
The simplest solution would be to put a "LastEdited" column into every table in your database and then pull query all the data which has updated since the last sync ( and you can perform a check on the index to detirmine if you need to update or insert into your own local cache. )
The ability to delete rows should actually be limited to a boolean "isDeleted" flag in this case to keep the sync process nice and simple.
If you have then the ability to edit or create rows from your app then you should keep a local table of changes to sync when you can go online and may have to implement some form of "merge" logic.
Several things you need to consider.
If your app is read only, you should implement a 'delta sync' logic in your local d. Keep a timestamp of last sync and get updates from your server. Of course, you need to consider the local db size in getting too large.
If you app is read/write, when working offline, you need to consider the two way sync especially when same record can be updated in different devices/users.
I am currently developing an application that provides some temporary static, but often changing data.
You can think of it like a telephone book. Most of the entries might never change, but there might be cases where people change their telephone number or move in/out. The dataset isnt large but has to be available at any time.
My first thought was to implement a clientside-SQLDatabase which queries a backend in certain intervals asking for updates (stored in another database on the backend server). If there was an update since the last query, the updated data will be send over and stored in the local sqldatabase.
Now I stumbled over this interesting looking database model Couchbase. Altough I dont see through it completly by now, if I got it, I can use it to keep the client-side database in sync with the servers database. Is this a usecase for couchbase or should I go with another strategy?
Thanks in advance!
That's a perfect use case for Couchbase. You'll want to check out Couchbase Lite (https://github.com/couchbase/couchbase-lite-android) and the Sync Gateway (http://docs.couchbase.com/sync-gateway/).
I'm developing an Android app as a "proof of concept" for our company. If they like it and think it's worth investing, then we'll move on to bigger things. I'm trying to figure out the best/most practical approach for this.....the basics of the app will connect to our DB and display information regarding a specific customer. For now, let's say we will only pull data from 3-4 tables (but there could be 10+ in the future). If the app doesn't have an internet connection then it should use the local DB. What is the best approach for this? Here's what I was thinking and would like some input/suggestions if possible:
1.) app runs checks internet connection. If exists, check db version (how, through a web service?)..if server db is newer, get latest data. If no internet, use local db.
2.) app parses data and displays it.
If this is correct, then there could be no modifications to the web service that would add fields to a result without changing the app as well. Is there a way for an app to parse fields regardless of how many fields there are?
I've read and walked through the tutorial on google with databases and such (Notepad tutorial) but it seems like the column names are all hard-coded in the parsing class, which I was hoping to avoid.
Sorry if this is confusing but I know I need my app to use a local db to read data, I also know that the app must get data from the server when it can (via onCreate or a refresh button) and copy it locally....Copying it locally is the part I'm having trouble understanding I guess....is there no way of saying "go out and get this result and display it", knowing that those results could mean 5 fields the first time or 1 the next.
Any help/guidance is greatly appreciated!
You probably want to use a SQLLite DB to store your data locally, a ContentProvider to provide CRUD access to the db, and a SyncAdapter to sync with your server when possible. The Sync Adapter also writes to the DB via the ContentProvider. See the SampleSyncAdapter sample in the SDK for an example of how this works. You will be implementing your own ContentProvider, but the sample just uses Android's supplied Contacts ContentProvider.
http://developer.android.com/resources/samples/SampleSyncAdapter/index.html