Develop Reference

asynchronous programming between android and mysql [duplicate]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I want to sync data (such as db record, media) between an Android App and a Server. If you've seen Evernote or similar Applications, you certainly understand what I mean.
I have some question (imagine we want to sync DB records):
Every user has a part of server space for himself (such as Evernote or Dropbox). Maybe the user creates new records by cellphone and creates new records in server. How can I match these records together? If there are records with same ID What algorithms do you suggest me?
Except JSON, Are there any way for send data between cellphone device and server?
If SyncAdapter and ContentProvider can solve my problems, please explain exactly for me. (If you could offer some samples or tutorials to me OR Any advice or keywords to help broaden/guide my search would be appreciated as well).

I'll try to answer all your questions by addressing the larger question: How can I sync data between a webserver and an android app?
Syncing data between your webserver and an android app requires a couple of different components on your android device.
Persistent Storage:
This is how your phone actually stores the data it receives from the webserver. One possible method for accomplishing this is writing your own custom ContentProvider backed by a Sqlite database. A decent tutorial for a content provider can be found here: http://thinkandroid.wordpress.com/2010/01/13/writing-your-own-contentprovider/
A ContentProvider defines a consistent interface to interact with your stored data. It could also allow other applications to interact with your data if you wanted. Behind your ContentProvider could be a Sqlite database, a Cache, or any arbitrary storage mechanism.
While I would certainly recommend using a ContentProvider with a Sqlite database you could use any java based storage mechanism you wanted.
Data Interchange Format:
This is the format you use to send the data between your webserver and your android app. The two most popular formats these days are XML and JSON. When choosing your format, you should think about what sort of serialization libraries are available. I know off-hand that there's a fantastic library for json serialization called gson: https://github.com/google/gson, although I'm sure similar libraries exist for XML.
Synchronization Service
You'll want some sort of asynchronous task which can get new data from your server and refresh the mobile content to reflect the content of the server. You'll also want to notify the server whenever you make local changes to content and want to reflect those changes. Android provides the SyncAdapter pattern as a way to easily solve this pattern. You'll need to register user accounts, and then Android will perform lots of magic for you, and allow you to automatically sync. Here's a good tutorial: http://www.c99.org/2010/01/23/writing-an-android-sync-provider-part-1/
As for how you identify if the records are the same, typically you'll create items with a unique id which you store both on the android device and the server. You can use that to make sure you're referring to the same reference. Furthermore, you can store column attributes like "updated_at" to make sure that you're always getting the freshest data, or you don't accidentally write over newly written data.

If we think about today, accepted answer is too old. As we know that we have many new libraries which can help you to make this types of application.
You should learn following topics that will helps you surely:
SyncAdapter: The sync adapter component in your app encapsulates the code for the tasks that transfer data between the device and a server. Based on the scheduling and triggers you provide in your app, the sync adapter framework runs the code in the sync adapter component.
Realm: Realm is a mobile database: a replacement for SQLite & Core Data.
Retrofit Type-safe HTTP client for Android and Java by Square, Inc. Must Learn a-smart-way-to-use-retrofit
And your sync logic for database like: How to sync SQLite database on Android phone with MySQL database on server?
Best Luck to all new learner. :)

If you write this yourself these are some of the points to keep in mind
Proper authentication between the device and the Sync Server
A sync protocol between the device and the server. It will usually go in 3 phases, authentication, data exchange, status exchange (which operations worked and which failed)
Pick your payload format. I suggest SyncML based XML mixed with JSON based format to represent the actual data. So SyncML for the protocol, and JSON for the actual data being exchanged. Using JSON Array while manipulating the data is always preferred as it is easy to access data using JSON Array.
Keeping track of data changes on both client and server. You can maintain a changelog of ids that change and pick them up during a sync session. Also, clear the changelog as the objects are successfully synchronized. You can also use a boolean variable to confirm the synchronization status, i.e. last time of sync. It will be helpful for end users to identify the time when last sync is done.
Need to have a way to communicate from the server to the device to start a sync session as data changes on the server. You can use C2DM or write your own persistent tcp based communication. The tcp approach is a lot seamless
A way to replicate data changes across multiple devices
And last but not the least, a way to detect and handle conflicts
Hope this helps as a good starting point.

#Grantismo provides a great explanation on the overall. If you wish to know who people are actually doing this things i suggest you to take a look at how google did for the Google IO App of 2014 (it's always worth taking a deep look at the source code of these apps that they release. There's a lot to learn from there).
Here's a blog post about it: http://android-developers.blogspot.com.br/2014/09/conference-data-sync-gcm-google-io.html
Essentially, on the application side: GCM for signalling, Sync Adapter for data fetching and talking properly with Content Provider that will make things persistent (yeah, it isolates the DB from direct access from other parts of the app).
Also, if you wish to take a look at the 2015's code: https://github.com/google/iosched

For example, you want to sync table todoTable from MySql to Sqlite
First, create one column name version (type INT) in todoTable for both Sqlite and MySql
Second, create a table name database_version with one column name currentVersion(INT)
In MySql, when you add a new item to todoTable or update item, you must upgrade the version of this item by +1 and also upgrade the currentVersion
In Android, when you want to sync (by manual press sync button or a service run with period time):
You will send the request with the Sqlite currentVersion (currently it is 1) to server.
Then in server, you find what item in MySql have version value greater than Sqlite currentVersion(1) then response to Android (in this example the item 3 with version 2 will response to Android)
In SQLite, you will add or update new item to todoTable and upgrade the currentVersion

Look at parseplatform.org.
it's opensource project.
(As well as you can go for commercial package available at back4app.com.)
It is a very straight forward and user friendly server side database service that gives a great android client side API

one way to accomplish this to have a server side application that waits for the data. The data can be sent using HttpRequest objects in Java or you can write your own TCP/IP data transfer utility. Data can be sent using JSON format or any other format that you think is suitable. Also data can be encrypted before sending to server if it contains sensitive information. All Server application have to do is just wait for HttpRequests to come in and parse the data and store it anywhere you want.

I would suggest using a binary webservice protocol similar to Hessian. It works very well and they do have a android implementation. It might be a little heavy but depends on the application you are building. Hope this helps.

#Grantismo gives a great overview of Android sync components.
SyncManagerAndroid library provides a simple 2-way sync implementation to plug into the Android Sync framework (AbstractThreadedSyncAdapter.OnPerformSync).
https://github.com/sschendel/SyncManagerAndroid

Sync data between users in Android App without server

I am developing an Android app that has a list, I would like this list to be synced between multiple users - can it be done with out server side?

Syncing data between your webserver and an android app requires a couple of different components on your android device.
Persistent Storage:
This is how your phone actually stores the data it receives from the webserver. One possible method for accomplishing this is writing your own custom ContentProvider backed by a Sqlite database.
A ContentProvider defines a consistent interface to interact with your stored data. It could also allow other applications to interact with your data if you wanted. Behind your ContentProvider could be a Sqlite database, a Cache, or any arbitrary storage mechanism.
While I would certainly recommend using a ContentProvider with a Sqlite database you could use any java based storage mechanism you wanted.
Data Interchange Format:
This is the format you use to send the data between your webserver and your android app. The two most popular formats these days are XML and JSON. When choosing your format, you should think about what sort of serialization libraries are available. I know off-hand that there's a fantastic library for json serialization called gson: http://code.google.com/p/google-gson/, although I'm sure similar libraries exist for XML.
Synchronization Service
You'll want some sort of asynchronous task which can get new data from your server and refresh the mobile content to reflect the content of the server. You'll also want to notify the server whenever you make local changes to content and want to reflect those changes. Android provides the SyncAdapter pattern as a way to easily solve this pattern. You'll need to register user accounts, and then Android will perform lots of magic for you, and allow you to automatically sync. Here's a good tutorial: http://www.c99.org/2010/01/23/writing-an-android-sync-provider-part-1/
As for how you identify if the records are the same, typically you'll create items with a unique id which you store both on the android device and the server. You can use that to make sure you're referring to the same reference. Furthermore, you can store column attributes like "updated_at" to make sure that you're always getting the freshest data, or you don't accidentally write over newly written data..

Update local Android database

I'm currently in trouble with a SQLite database problem on Android.
My application have an local database which inside an apk file. When the application starts it will check for the new version, and download entirely new database if available (although between two database version, changes are very little). But the database is too large now. So it takes very long time when new database available. So any solution for this problem?

Basic idea
Here's how I'd do it. I'm assuming here that the client app doesn't make changes to the local database (except when it downloads a new version), so that there are only a few possible versions of the database in existence (one for every time you've made a change at the server end).
Add a column to every table called LastModified, with a default value of NOW(). That means that every time you add something to your master copy, it'll get an updated LastModified setting. You will have to make sure your updates (rather than inserts) change the LastModified field too.
Store somewhere in the database (a Settings table or something) a field that tracks the date that this version of the database was published on the server (call it PublishDate).
When the client wants to check for a new version, it sends its PublishDate to the server. The server then checks each table, and finds every row where LastModified comes after PublishDate. It sends SQL to the client to insert or update these rows on the client. It also sends the new PublishDate so that the client can update that in its local copy.
This deals with inserts and updates. It doesn't deal with deletions. It might be that they aren't an issue in your case; if they are:
Add either a separate table to log deletions, where you also track LastModified, so that you can tell the client which rows to delete; or preferably have a setup where you don't ever actually delete any rows, but just update them to be marked as "deleted".
Finally, this won't handle schema changes. Again, hopefully that isn't an issue in your case: hopefully you have a stable schema. But if you do need to add or drop tables or indexes or something, that will have to be done separately:
Create a SchemaChanges table on your master, and whenever you make structural changes, put the relevant details into the SchemaChanges table, along with a LastModified date, so that you can send this to the client on request too. If you're doing this, you'll want to send schema changes to the client first, because they might affect the meaning of other changes.
Now the nice thing about doing it this way is that you can pre-process everything on the server (because there are only a few versions in existence). For every old version, you can calculate the changes (based on the details above) that would take that old version up to the new version, and then store the resulting SQL on the server. If you do that, you avoid the need for generating the SQL on the fly: when the client sends the PublishDate, you just look up the SQL you've already calculated that transforms the version from that PublishDate to the latest version.
Alternative implementation
There is a nice and easy way of pushing the changes that the above scheme gives you, even with a slight simplification that doesn't require LastModified times, or indeed any changes to your existing structure. At the server end, where you already have the old version (because you have all the old versions) and the new version, you create an SQL dump of both databases, and then run diff over them to generate a patch file that you can send to the client app. The client app will use the same Java library to generate the SQL dump of the old version, and will then apply the diff patch to it to create a full SQL dump for the new version. At that point, it can drop the old database and create the new one from the SQL dump.
This will be very efficient if the changes aren't wholesale changes (in which case you might as well just push the new .db file).
It's fairly easy to do this by invoking the SQLite binary to create the dumps. You will need to modify the approach slightly for Android, according to this way of executing an external command.
You can use this Google library to calculate the diff patches at the server end and apply them at the client end.

Instead of getting an entire new database file, get the changes. Changes may be in the form of an SQL script. Let server generate the change script for every update, and then you can download the SQL scripts and run them on local database in sequence.

You need to create a timestamp on any changes you do to your database on your server.
When the app connects to your server, it sends the last timestamp downloaded so your server knows what is the new data to download.
Timestamps are not set on the device based on the real time, you need to download it as well to avoid timezone problems are different hour values.
You can use consecutive number versions if you prefer instead of timestamps

I have been working on a similar problem where I have to update local database every time there are any changes on server. The trick is to keep track of last updated time in local database in android and send that time to get just the updates from server in JSON format. Here, you will have to write server side code that takes input as last updated time and returns all the updates made to server side database in JSON format.
After getting the JSON data, you have to convert JSON data to SQL queries and insert or update in local database.
Have a look at this answer. And also look at Google IO 2014 code here.
I ended up using ContentProvider with schema and contract classes as explained here.
Then I created java object for every table in my database and used gson library to convert the incoming JSON updates to the java object. After that, you have to write inser/update/delete queries as given in documentation.
So basically, you have to create contract, database and provider classes for database handling. And java classes one for each table in your database and you have write code for fetching JSON updates from server, converting JSON string into java object and then insert/update/delete in database. Demo code is available in the Google IO 2014 app.

What isn't clear from your question is if you want
bidirectional sync (client and server can make data changes) or only
unidirectional sync (only server or client can make changes).
For me it sounds more like you want unidirectional sync from server to client. This is exactly what the Google I/O does with the conference data. You can found a detailed blog entry how this works here:
Conference Data Sync and GCM in the Google I/O App, just look at the chapter 'Downloading Conference Data Efficiently' which deals with manifest.json, sessions.json and speakers.json. If the data schema changes, just provide a new app which performs the schema change in the standard android way and make your json parsing routine in a way that it ignores additional fields. Just as addition:
a blogentry which provides Google I/O App Insights
The code itself on Github
If you want bidirectional sync then chiastic-security provided a good overview for the database operations. What is missing is the programmatically part of the solution:
a presentation was given at Google I/O 2010 by Virgil Dobjanschi: Developing Android REST client applications
The presentation lead to much questions since no code was presented, see for example this disscussion: Need sample Android REST Client project which implements Virgil Dobjanschi REST implementation pattern
A good picture from the architecture (SyncAdapter approach): here
and a very good blog entry about Write your own sync adapter
and you find some information in the Google blog entry (first link for the unidirectional sync)
and some code in this presentation - slide 31ff
Hope that helps.

Using greenDao you can perform such Migrations, Check it here. Good luck.

You can do this through Google Cloud Messaging API. It is a feature given to developers by Google which works on both Android and IOS devices. Google Cloud Messaging (GCM) is a service that helps developers send data between online servers and apps. Using this service you can send data to your application whenever new data is available instead of making new requests to the server in timely fashion.
A great tutorial for using GCM is this.
Hope it helps :)

Maybe you can use services for to change database version. Thus user can't notice that and User continue to use the application. When service finished it work, Application send a notification. Of course there are only approeach and advice.

Android/Web App offline/online Database setup. Creating items on both platforms

Read Update 1 for a beter question description. Sorry about that.
I'm trying to figure out how to build a Android App and Web App that syncs there data.
I know that I should make an REST API for the MySQL database to sync with the Android App.
I have made a App before that syncs data but only for retrieval (SELECT queries) on the android side.
I now want to make a Android App / Web App that should create/update items on both platforms and syncs so both have the new/updated items. The app should also work offline.
I'm used to creating id's for most database tables as key with autoincrement. Now that I'll have 2 databases i'm not sure how to create those unique IDs. Or should I ditch those id's and use a combination of columns as primary key (with a timestamp or something).
Hope it makes sence, english is not my native language.
UPDATE 1
Narowing the question:
So i have a MySQL database with an PHP API. The API will be used by the Web App and Android App.
Question is how to handle offline data creation in de android App. If you rely on a id with autoincrement of the MySQL Database.
Example: When creating a person how to get an id for that person (If the MySQL database handles ID creation)
Thanks in advance,
Otto Vanluchene

What you're asking is a very broad question and there's a lot of options, but this is how I do it.
HOW TO HANDLE NATIVE SIDE REST CALLS:
First of all I use an HTTP library to handle my GET/POST REST API calls. It handles the callbacks and can be super helpful.
This is the one I usually use:
LoopJ's Async HTTP Callback Library
This will handle GET and POST requests with a lot of cool features such as custom timeouts, JSON format, onSuccess() and onFailure() methods, etc. There's a lot of working examples of this library too. I've used it in all my apps and haven't had any problems yet!
DATABASE SETUP:
Your going to want to create a MySQL database to store all your data. You may want to read up on common DB practices.
API SETUP
I write all my APIs in PHP, but that's just my preference. The APIs you write here will be used in your web AND mobile apps to keep things consistent. Assuming you use GET, you can just pass an argument (which I usually call "callback") that will determine what chunk of information the API will return.
For example:
http://www.ottosite.com/otto_api.php?callback=get_users
So in your PHP code, you'll have it checking if callback=get_users, and if it does, it will query the database for all the users, format the json string, then simply echo it.
This may return something like:
{"status":"success","users":["user1","user2","user3"]}
So this is obviously a JSON formatted response, then in your app, when you call that API, you just parse the response into a JSONObject (JSONArray for users) and then grab whatever data you need from it.
Hopefully this helps.

approach for synching android app database with server db?

I'm developing an Android app as a "proof of concept" for our company. If they like it and think it's worth investing, then we'll move on to bigger things. I'm trying to figure out the best/most practical approach for this.....the basics of the app will connect to our DB and display information regarding a specific customer. For now, let's say we will only pull data from 3-4 tables (but there could be 10+ in the future). If the app doesn't have an internet connection then it should use the local DB. What is the best approach for this? Here's what I was thinking and would like some input/suggestions if possible:
1.) app runs checks internet connection. If exists, check db version (how, through a web service?)..if server db is newer, get latest data. If no internet, use local db.
2.) app parses data and displays it.
If this is correct, then there could be no modifications to the web service that would add fields to a result without changing the app as well. Is there a way for an app to parse fields regardless of how many fields there are?
I've read and walked through the tutorial on google with databases and such (Notepad tutorial) but it seems like the column names are all hard-coded in the parsing class, which I was hoping to avoid.
Sorry if this is confusing but I know I need my app to use a local db to read data, I also know that the app must get data from the server when it can (via onCreate or a refresh button) and copy it locally....Copying it locally is the part I'm having trouble understanding I guess....is there no way of saying "go out and get this result and display it", knowing that those results could mean 5 fields the first time or 1 the next.
Any help/guidance is greatly appreciated!

You probably want to use a SQLLite DB to store your data locally, a ContentProvider to provide CRUD access to the db, and a SyncAdapter to sync with your server when possible. The Sync Adapter also writes to the DB via the ContentProvider. See the SampleSyncAdapter sample in the SDK for an example of how this works. You will be implementing your own ContentProvider, but the sample just uses Android's supplied Contacts ContentProvider.
http://developer.android.com/resources/samples/SampleSyncAdapter/index.html